Greg Pomerantz
3c5bee96ce
Some dump files carry a trailing 'ctime' column: an older pipeline appended a FULL re-download of the symbol per download (up to ~28 vintages per date, 2019-2022). The consumer keeps the last row per date, so it silently used stale (~2021) vintages where Yahoo later revised values. scripts/clean_multivintage.py keeps, per date, the rows from the newest ctime (event files keep distinct same-day amounts; history/split keep the single newest row), drops the ctime column, sorts by date. Applied to the data root and overrides/frozen (which carried the same artifact): 1,065 files, 2,202,712 stale rows dropped. Also: stripped 3 UTF-8 BOMs (teg/lo/krft), refreshed the event backup with the cleaned files, and re-ran the double-listing scan: category C (83 symbols, 18,636 repeated within-file rows) is fully explained by the multi-vintage artifact and is now gone; A (6,589 pairs, corrected) and B (12,366 pairs, open review list) are unchanged. Whole-set structural QC after cleaning: 0 duplicate dates, 3 syms/15 rows of OHLC invariant violations, 15 syms/2,897 rows of non-positive prices (mostly long-delisted tickers), 8 unsorted files (reader sorts).
148 lines
2.3 KiB
Plaintext
148 lines
2.3 KiB
Plaintext
Date,Dividends
|
|
1980-01-21,0.15
|
|
1980-04-24,0.15
|
|
1980-07-23,0.15
|
|
1980-10-14,0.16
|
|
1981-01-21,0.16
|
|
1981-04-22,0.16
|
|
1981-07-22,0.16
|
|
1981-10-21,0.16
|
|
1982-01-22,0.16
|
|
1982-04-23,0.16
|
|
1982-07-21,0.16
|
|
1982-10-20,0.16
|
|
1983-01-21,0.16
|
|
1983-04-22,0.16
|
|
1983-07-22,0.1
|
|
1983-10-21,0.13
|
|
1984-01-23,0.1
|
|
1984-04-26,0.1
|
|
1984-07-25,0.2
|
|
1984-10-24,0.17
|
|
1985-01-23,0.13
|
|
1985-04-25,0.11
|
|
1985-07-25,0.11
|
|
1985-10-23,0.11
|
|
1986-01-23,0.1
|
|
1986-04-17,0.08
|
|
1986-07-23,0.08
|
|
1986-10-21,0.08
|
|
1987-02-11,0.08
|
|
1987-03-17,0.08
|
|
1987-06-16,0.08
|
|
1987-09-16,0.05
|
|
1987-12-16,0.05
|
|
1988-03-23,0.05
|
|
1988-06-22,0.05
|
|
1988-09-21,0.05
|
|
1988-12-16,0.06
|
|
1989-03-22,0.08
|
|
1989-06-21,0.08
|
|
1989-09-27,0.08
|
|
1989-12-19,0.13
|
|
1990-03-28,0.09
|
|
1990-06-27,0.09
|
|
1990-09-26,0.09
|
|
1990-12-27,0.14
|
|
1991-03-27,0.07
|
|
1991-06-26,0.07
|
|
1991-09-26,0.07
|
|
1991-12-30,0.09
|
|
1992-03-31,0.07
|
|
1992-06-30,0.07
|
|
1992-09-30,0.07
|
|
1992-12-30,0.07
|
|
1993-03-31,0.07
|
|
1993-06-30,0.07
|
|
1993-09-30,0.07
|
|
1993-12-31,0.08
|
|
1994-03-31,0.07
|
|
1994-06-30,0.07
|
|
1994-09-30,0.07
|
|
1994-12-30,0.06
|
|
1995-03-31,0.07
|
|
1995-06-30,0.07
|
|
1995-09-29,0.07
|
|
1995-12-29,0.051
|
|
1996-03-29,0.05
|
|
1996-06-28,0.05
|
|
1996-09-30,0.05
|
|
1996-12-31,0.052
|
|
1997-03-31,0.05
|
|
1997-06-30,0.04
|
|
1997-09-30,0.04
|
|
1997-12-31,0.047
|
|
1998-03-31,0.03
|
|
1998-06-30,0.03
|
|
1998-09-30,0.03
|
|
1998-12-31,0.033
|
|
1999-03-31,0.03
|
|
1999-12-30,0.043
|
|
2000-12-29,0.078
|
|
2001-12-28,0.108
|
|
2002-12-30,0.153
|
|
2003-12-30,0.17
|
|
2004-12-30,0.29
|
|
2005-12-29,0.445
|
|
2006-06-28,0.125
|
|
2006-09-27,0.12
|
|
2006-12-27,0.238
|
|
2007-03-28,0.15
|
|
2007-06-27,0.13
|
|
2007-09-26,0.13
|
|
2007-12-27,0.125
|
|
2008-03-27,0.135
|
|
2008-06-26,0.116
|
|
2008-09-26,0.12
|
|
2008-12-31,0.156
|
|
2009-03-27,0.16
|
|
2009-06-26,0.12
|
|
2009-09-25,0.105
|
|
2009-12-22,0.088
|
|
2010-03-25,0.085
|
|
2010-04-09,0.019
|
|
2010-06-24,0.095
|
|
2010-09-23,0.097
|
|
2010-12-21,0.119
|
|
2011-03-24,0.1
|
|
2011-06-23,0.104
|
|
2011-09-22,0.105
|
|
2011-12-20,0.121
|
|
2012-03-22,0.103
|
|
2012-06-21,0.102
|
|
2012-09-20,0.102
|
|
2012-12-20,0.161
|
|
2013-03-21,0.092
|
|
2013-06-20,0.104
|
|
2013-09-24,0.105
|
|
2013-12-18,0.081
|
|
2014-03-20,0.115
|
|
2014-06-19,0.099
|
|
2014-09-23,0.088
|
|
2014-12-18,0.078
|
|
2015-03-30,0.103
|
|
2015-06-25,0.279
|
|
2015-09-29,0.099
|
|
2015-12-21,0.066
|
|
2016-03-30,0.133
|
|
2016-06-30,0.095
|
|
2016-09-30,0.096
|
|
2016-12-20,0.097
|
|
2017-03-29,0.096
|
|
2017-06-28,0.09
|
|
2017-09-28,0.113
|
|
2017-12-20,0.039
|
|
2018-03-26,0.103
|
|
2018-06-26,0.086
|
|
2018-09-25,0.088
|
|
2018-12-19,0.087
|
|
2019-03-26,0.105
|
|
2019-06-25,0.093
|
|
2019-09-25,0.099
|
|
2019-12-18,0.08
|
|
2020-03-25,0.088
|
|
2020-06-24,0.088
|
|
2020-09-24,0.067
|
|
2020-12-18,0.062
|
|
2021-03-24,0.046
|