Greg Pomerantz
3c5bee96ce
Some dump files carry a trailing 'ctime' column: an older pipeline appended a FULL re-download of the symbol per download (up to ~28 vintages per date, 2019-2022). The consumer keeps the last row per date, so it silently used stale (~2021) vintages where Yahoo later revised values. scripts/clean_multivintage.py keeps, per date, the rows from the newest ctime (event files keep distinct same-day amounts; history/split keep the single newest row), drops the ctime column, sorts by date. Applied to the data root and overrides/frozen (which carried the same artifact): 1,065 files, 2,202,712 stale rows dropped. Also: stripped 3 UTF-8 BOMs (teg/lo/krft), refreshed the event backup with the cleaned files, and re-ran the double-listing scan: category C (83 symbols, 18,636 repeated within-file rows) is fully explained by the multi-vintage artifact and is now gone; A (6,589 pairs, corrected) and B (12,366 pairs, open review list) are unchanged. Whole-set structural QC after cleaning: 0 duplicate dates, 3 syms/15 rows of OHLC invariant violations, 15 syms/2,897 rows of non-positive prices (mostly long-delisted tickers), 8 unsorted files (reader sorts).
107 lines
1.7 KiB
Plaintext
107 lines
1.7 KiB
Plaintext
Date,Dividends
|
|
1989-11-22,0.05
|
|
1990-02-22,0.05
|
|
1990-05-23,0.05
|
|
1990-08-28,0.05
|
|
1990-12-03,0.05
|
|
1991-02-22,0.05
|
|
1991-05-24,0.05
|
|
1991-08-30,0.05
|
|
1991-11-22,0.05
|
|
1992-02-21,0.05
|
|
1992-06-01,0.05
|
|
1992-08-31,0.05
|
|
1992-11-30,0.05
|
|
1993-02-22,0.05
|
|
1993-05-28,0.05
|
|
1993-08-30,0.05
|
|
1993-11-19,0.075
|
|
1994-02-18,0.075
|
|
1994-05-23,0.075
|
|
1994-08-29,0.075
|
|
1994-11-21,0.15
|
|
1995-02-21,0.15
|
|
1995-05-12,0.15
|
|
1995-06-13,0.03923
|
|
1995-08-25,0.15
|
|
1995-09-13,0.03923
|
|
1995-11-16,0.18
|
|
1995-12-13,0.01962
|
|
1996-02-22,0.18
|
|
1996-05-13,0.18
|
|
1996-08-29,0.18
|
|
1996-11-13,0.21
|
|
1997-02-20,0.21
|
|
1997-05-13,0.21
|
|
1997-08-28,0.21
|
|
1997-11-12,0.25
|
|
1998-02-19,0.25
|
|
1998-05-13,0.25
|
|
1998-08-27,0.25
|
|
1998-11-10,0.26
|
|
1999-02-18,0.26
|
|
1999-05-10,0.07
|
|
1999-08-31,0.06
|
|
1999-11-09,0.06
|
|
2000-02-17,0.06
|
|
2000-05-24,0.06
|
|
2000-09-01,0.06
|
|
2000-11-21,0.06
|
|
2001-02-21,0.06
|
|
2001-05-23,0.06
|
|
2001-08-29,0.06
|
|
2001-11-21,0.06
|
|
2002-02-20,0.06
|
|
2002-05-22,0.06
|
|
2002-08-14,0.06
|
|
2002-11-13,0.06
|
|
2003-02-20,0.06
|
|
2003-05-21,0.06
|
|
2003-08-14,0.06
|
|
2003-11-12,0.06
|
|
2004-02-19,0.06
|
|
2004-05-03,0.05886
|
|
2004-05-13,0.06
|
|
2004-08-10,0.05886
|
|
2004-08-12,0.06
|
|
2004-11-10,0.06
|
|
2005-02-10,0.06
|
|
2005-05-05,0.12
|
|
2005-08-04,0.12
|
|
2005-11-22,0.12
|
|
2007-06-22,1.61858
|
|
2010-02-25,0.12
|
|
2010-05-27,0.12
|
|
2010-08-30,0.12
|
|
2010-11-29,0.12
|
|
2011-02-25,0.12
|
|
2011-05-27,0.12
|
|
2011-08-30,0.12
|
|
2011-11-29,0.12
|
|
2012-02-28,0.12
|
|
2012-05-30,0.12
|
|
2012-08-30,0.12
|
|
2012-11-29,0.12
|
|
2013-02-27,0.15
|
|
2013-05-30,0.136364
|
|
2013-08-28,0.15
|
|
2013-11-27,0.15
|
|
2014-02-27,0.15
|
|
2014-05-29,0.18
|
|
2014-08-27,0.18
|
|
2014-08-28,0.18
|
|
2014-11-26,0.18
|
|
2015-02-26,0.181818
|
|
2015-05-28,0.2
|
|
2015-08-28,0.2
|
|
2015-11-27,0.2
|
|
2016-02-26,0.22
|
|
2016-05-27,0.2
|
|
2016-08-11,0.22
|
|
2016-11-29,0.22
|
|
2017-03-01,0.27
|
|
2017-06-08,0.27
|
|
2017-08-30,0.27
|
|
2017-11-30,0.27
|
|
2018-03-06,0.234783
|