Greg Pomerantz
3c5bee96ce
Some dump files carry a trailing 'ctime' column: an older pipeline appended a FULL re-download of the symbol per download (up to ~28 vintages per date, 2019-2022). The consumer keeps the last row per date, so it silently used stale (~2021) vintages where Yahoo later revised values. scripts/clean_multivintage.py keeps, per date, the rows from the newest ctime (event files keep distinct same-day amounts; history/split keep the single newest row), drops the ctime column, sorts by date. Applied to the data root and overrides/frozen (which carried the same artifact): 1,065 files, 2,202,712 stale rows dropped. Also: stripped 3 UTF-8 BOMs (teg/lo/krft), refreshed the event backup with the cleaned files, and re-ran the double-listing scan: category C (83 symbols, 18,636 repeated within-file rows) is fully explained by the multi-vintage artifact and is now gone; A (6,589 pairs, corrected) and B (12,366 pairs, open review list) are unchanged. Whole-set structural QC after cleaning: 0 duplicate dates, 3 syms/15 rows of OHLC invariant violations, 15 syms/2,897 rows of non-positive prices (mostly long-delisted tickers), 8 unsorted files (reader sorts).
182 lines
3.0 KiB
Plaintext
182 lines
3.0 KiB
Plaintext
Date,Dividends
|
|
2004-12-21,0.11
|
|
2005-01-14,0.088
|
|
2005-03-04,0.088
|
|
2005-04-08,0.088
|
|
2005-05-13,0.088
|
|
2005-06-10,0.088
|
|
2005-07-08,0.088
|
|
2005-08-12,0.088
|
|
2005-09-09,0.088
|
|
2005-10-14,0.088
|
|
2005-11-10,0.088
|
|
2005-12-22,0.204
|
|
2006-01-20,0.088
|
|
2006-02-16,0.088
|
|
2006-03-24,0.088
|
|
2006-04-21,0.088
|
|
2006-05-19,0.088
|
|
2006-06-23,0.093
|
|
2006-07-21,0.093
|
|
2006-08-18,0.093
|
|
2006-09-20,0.098
|
|
2006-10-18,0.098
|
|
2006-11-15,0.098
|
|
2006-12-20,0.362
|
|
2007-01-17,0.103
|
|
2007-02-14,0.103
|
|
2007-03-21,0.1
|
|
2007-04-18,0.1
|
|
2007-05-16,0.1
|
|
2007-06-20,0.098
|
|
2007-07-18,0.098
|
|
2007-08-22,0.098
|
|
2007-09-19,0.095
|
|
2007-10-17,0.095
|
|
2007-11-20,0.095
|
|
2007-12-19,0.095
|
|
2008-01-16,0.095
|
|
2008-02-20,0.095
|
|
2008-03-18,0.095
|
|
2008-04-16,0.095
|
|
2008-05-21,0.095
|
|
2008-06-18,0.095
|
|
2008-07-16,0.095
|
|
2008-08-20,0.095
|
|
2008-09-17,0.071
|
|
2008-10-22,0.066
|
|
2008-11-19,0.066
|
|
2008-12-17,0.066
|
|
2009-01-21,0.066
|
|
2009-02-18,0.066
|
|
2009-03-18,0.066
|
|
2009-04-15,0.056
|
|
2009-05-20,0.056
|
|
2009-06-17,0.056
|
|
2009-07-22,0.056
|
|
2009-08-19,0.056
|
|
2009-09-16,0.056
|
|
2009-10-21,0.056
|
|
2009-11-18,0.056
|
|
2009-12-16,0.056
|
|
2010-01-20,0.056
|
|
2010-02-17,0.056
|
|
2010-03-17,0.056
|
|
2010-04-21,0.056
|
|
2010-05-19,0.056
|
|
2010-06-16,0.056
|
|
2010-07-21,0.056
|
|
2010-08-18,0.056
|
|
2010-09-15,0.056
|
|
2010-10-20,0.056
|
|
2010-11-17,0.056
|
|
2010-12-21,0.056
|
|
2011-01-19,0.056
|
|
2011-02-16,0.056
|
|
2011-03-16,0.056
|
|
2011-04-19,0.056
|
|
2011-05-18,0.056
|
|
2011-06-15,0.058
|
|
2011-07-20,0.058
|
|
2011-08-17,0.058
|
|
2011-09-21,0.063
|
|
2011-10-19,0.063
|
|
2011-11-16,0.063
|
|
2011-12-14,0.065
|
|
2012-01-18,0.065
|
|
2012-02-15,0.065
|
|
2012-03-21,0.067
|
|
2012-04-18,0.067
|
|
2012-05-16,0.067
|
|
2012-06-20,0.07
|
|
2012-07-18,0.07
|
|
2012-08-22,0.07
|
|
2012-09-19,0.073
|
|
2012-10-17,0.073
|
|
2012-11-20,0.072
|
|
2012-12-12,0.072
|
|
2013-01-16,0.072
|
|
2013-02-13,0.072
|
|
2013-03-13,0.072
|
|
2013-04-17,0.072
|
|
2013-05-22,0.072
|
|
2013-06-19,0.072
|
|
2013-07-17,0.072
|
|
2013-08-21,0.072
|
|
2013-09-18,0.073
|
|
2013-10-16,0.072
|
|
2013-11-20,0.072
|
|
2013-12-18,0.073
|
|
2014-01-22,0.073
|
|
2014-02-19,0.073
|
|
2014-03-19,0.073
|
|
2014-04-15,0.073
|
|
2014-05-21,0.073
|
|
2014-06-18,0.073
|
|
2014-07-16,0.073
|
|
2014-08-20,0.073
|
|
2014-09-17,0.073
|
|
2014-10-22,0.073
|
|
2014-11-19,0.073
|
|
2014-12-17,0.073
|
|
2015-01-21,0.072
|
|
2015-02-18,0.072
|
|
2015-03-18,0.072
|
|
2015-04-15,0.073
|
|
2015-05-20,0.073
|
|
2015-06-17,0.073
|
|
2015-07-22,0.073
|
|
2015-08-19,0.073
|
|
2015-09-16,0.078
|
|
2015-10-21,0.078
|
|
2015-11-18,0.078
|
|
2015-12-16,0.078
|
|
2016-01-20,0.078
|
|
2016-02-17,0.078
|
|
2016-03-16,0.078
|
|
2016-04-20,0.078
|
|
2016-05-18,0.078
|
|
2016-06-15,0.078
|
|
2016-07-20,0.078
|
|
2016-08-17,0.078
|
|
2016-09-21,0.078
|
|
2016-10-19,0.078
|
|
2016-11-16,0.078
|
|
2016-12-21,0.078
|
|
2017-01-18,0.078
|
|
2017-02-15,0.078
|
|
2017-03-22,0.078
|
|
2017-04-19,0.078
|
|
2017-05-17,0.078
|
|
2017-06-21,0.078
|
|
2017-07-19,0.078
|
|
2017-08-23,0.078
|
|
2017-09-21,0.078
|
|
2017-10-19,0.078
|
|
2017-11-22,0.078
|
|
2017-12-21,0.078
|
|
2018-01-18,0.078
|
|
2018-02-15,0.078
|
|
2018-03-22,0.078
|
|
2018-04-19,0.078
|
|
2018-05-24,0.078
|
|
2018-06-21,0.078
|
|
2018-07-19,0.078
|
|
2018-08-23,0.078
|
|
2018-09-20,0.078
|
|
2018-10-18,0.078
|
|
2018-11-21,0.078
|
|
2018-12-20,0.078
|
|
2019-01-17,0.078
|
|
2019-02-14,0.078
|
|
2019-03-21,0.078
|
|
2019-04-17,0.078
|
|
2019-05-23,0.078
|
|
2019-06-20,0.078
|
|
2019-07-18,0.078
|
|
2019-08-22,0.078
|
|
2019-09-19,0.078
|
|
2019-10-17,0.078
|
|
2019-11-21,0.078
|
|
2019-12-19,0.078
|