Greg Pomerantz
3c5bee96ce
Some dump files carry a trailing 'ctime' column: an older pipeline appended a FULL re-download of the symbol per download (up to ~28 vintages per date, 2019-2022). The consumer keeps the last row per date, so it silently used stale (~2021) vintages where Yahoo later revised values. scripts/clean_multivintage.py keeps, per date, the rows from the newest ctime (event files keep distinct same-day amounts; history/split keep the single newest row), drops the ctime column, sorts by date. Applied to the data root and overrides/frozen (which carried the same artifact): 1,065 files, 2,202,712 stale rows dropped. Also: stripped 3 UTF-8 BOMs (teg/lo/krft), refreshed the event backup with the cleaned files, and re-ran the double-listing scan: category C (83 symbols, 18,636 repeated within-file rows) is fully explained by the multi-vintage artifact and is now gone; A (6,589 pairs, corrected) and B (12,366 pairs, open review list) are unchanged. Whole-set structural QC after cleaning: 0 duplicate dates, 3 syms/15 rows of OHLC invariant violations, 15 syms/2,897 rows of non-positive prices (mostly long-delisted tickers), 8 unsorted files (reader sorts).
97 lines
1.6 KiB
Plaintext
97 lines
1.6 KiB
Plaintext
Date,Dividends
|
|
1982-03-08,0.00617
|
|
1982-06-28,0.00617
|
|
1982-09-27,0.00617
|
|
1982-11-29,0.00617
|
|
1983-03-07,0.00617
|
|
1983-07-01,0.00617
|
|
1983-10-07,0.00617
|
|
1983-12-12,0.00617
|
|
1984-03-08,0.00617
|
|
1984-06-07,0.00617
|
|
1984-09-13,0.00617
|
|
1984-12-04,0.00617
|
|
1985-03-05,0.00617
|
|
1985-05-30,0.00617
|
|
1985-09-10,0.00617
|
|
1985-12-09,0.00617
|
|
1986-03-12,0.0804
|
|
1986-06-02,0.0804
|
|
1986-09-15,0.0804
|
|
1986-12-02,0.0804
|
|
1987-03-09,0.0804
|
|
1987-06-01,0.0804
|
|
1987-09-16,0.0804
|
|
1988-01-12,0.0804
|
|
1988-04-13,0.0996
|
|
1988-07-11,0.0996
|
|
1988-10-07,0.0996
|
|
1989-01-09,0.0996
|
|
1989-04-10,0.12
|
|
1989-07-10,0.12
|
|
1989-10-06,0.12
|
|
1990-01-08,0.12
|
|
1990-04-06,0.1452
|
|
1990-07-09,0.1452
|
|
1990-10-05,0.1452
|
|
1991-01-14,0.1452
|
|
1991-04-08,0.1752
|
|
1991-07-08,0.1752
|
|
1991-10-07,0.1752
|
|
1992-01-06,0.1752
|
|
1992-04-06,0.21
|
|
1992-07-06,0.0525
|
|
1992-10-05,0.0525
|
|
1993-01-05,0.0525
|
|
1993-04-05,0.0624
|
|
1993-07-02,0.0624
|
|
1993-10-06,0.0624
|
|
1994-01-04,0.0624
|
|
1994-04-11,0.075
|
|
1994-07-05,0.075
|
|
1994-10-07,0.075
|
|
1995-01-03,0.075
|
|
1995-04-07,0.09
|
|
1995-07-06,0.09
|
|
1995-10-11,0.09
|
|
1995-12-29,0.09
|
|
1996-04-10,0.1101
|
|
1996-07-10,0.1101
|
|
1996-10-09,0.1101
|
|
1997-01-08,0.1101
|
|
1997-04-09,0.1326
|
|
1997-07-09,0.1326
|
|
1997-10-15,0.1326
|
|
1998-01-07,0.1326
|
|
1998-04-07,0.1575
|
|
1998-07-22,0.0535127
|
|
1998-10-07,0.0525
|
|
1999-10-07,0.05228
|
|
1999-11-12,0.21
|
|
2000-12-06,0.21
|
|
2001-12-05,0.21
|
|
2002-12-11,0.21
|
|
2003-12-10,0.21
|
|
2004-12-08,0.24
|
|
2005-12-08,0.27
|
|
2006-12-13,0.31
|
|
2007-06-13,0.0
|
|
2007-12-05,0.35
|
|
2008-12-11,0.35
|
|
2009-12-10,0.35
|
|
2010-12-09,0.4
|
|
2011-12-14,0.6
|
|
2012-12-06,0.75
|
|
2013-12-12,0.86
|
|
2014-12-11,1.15
|
|
2015-07-01,0.66
|
|
2015-12-10,0.71
|
|
2016-07-07,0.71
|
|
2016-12-08,0.78
|
|
2017-07-06,0.78
|
|
2017-12-08,0.84
|
|
2018-07-06,0.84
|
|
2018-12-07,0.88
|
|
2019-07-05,0.88
|
|
2019-12-13,0.88
|