f/hcp-dividend.csv at 3c5bee96ceb54e7c6d0fb87afbcec59e6b77a3f3

gmp/f

Greg Pomerantz 3c5bee96ce Clean multi-vintage 'ctime' files: 1,065 files, 2.2M stale rows

Some dump files carry a trailing 'ctime' column: an older pipeline
appended a FULL re-download of the symbol per download (up to ~28
vintages per date, 2019-2022). The consumer keeps the last row per date,
so it silently used stale (~2021) vintages where Yahoo later revised
values. scripts/clean_multivintage.py keeps, per date, the rows from the
newest ctime (event files keep distinct same-day amounts; history/split
keep the single newest row), drops the ctime column, sorts by date.
Applied to the data root and overrides/frozen (which carried the same
artifact): 1,065 files, 2,202,712 stale rows dropped.

Also: stripped 3 UTF-8 BOMs (teg/lo/krft), refreshed the event backup
with the cleaned files, and re-ran the double-listing scan: category C
(83 symbols, 18,636 repeated within-file rows) is fully explained by the
multi-vintage artifact and is now gone; A (6,589 pairs, corrected) and
B (12,366 pairs, open review list) are unchanged. Whole-set structural
QC after cleaning: 0 duplicate dates, 3 syms/15 rows of OHLC invariant
violations, 15 syms/2,897 rows of non-positive prices (mostly
long-delisted tickers), 8 unsorted files (reader sorts).

2026-08-31 22:25:02 -04:00

2.3 KiB

Raw Blame History

1	Date	Dividends
2	1985-07-29	0.18
3	1985-10-28	0.55
4	1986-01-28	0.56
5	1986-03-18	0.57
6	1986-07-28	0.58
7	1986-10-28	0.59
8	1987-01-27	0.6
9	1987-04-28	0.61
10	1987-07-28	0.62
11	1987-10-28	0.63
12	1988-02-01	0.64
13	1988-04-28	0.6528
14	1988-07-26	0.6652
15	1988-10-26	0.6776
16	1989-01-26	0.69
17	1989-04-03	0.5076
18	1989-04-26	0.1952
19	1989-07-28	0.7152
20	1989-10-30	0.7276
21	1990-01-30	0.74
22	1990-04-30	0.7528
23	1990-07-30	0.7652
24	1990-10-29	0.7776
25	1991-01-30	0.79
26	1991-04-29	0.8028
27	1991-07-30	0.8152
28	1991-10-28	0.8276
29	1992-01-21	0.84
30	1992-04-28	0.8552
31	1992-07-28	0.435
32	1992-10-28	0.4426
33	1993-01-28	0.45
34	1993-04-27	0.4576
35	1993-07-28	0.465
36	1993-10-28	0.4726
37	1994-01-28	0.48
38	1994-04-27	0.49
39	1994-04-28	0.22313
40	1994-07-28	0.5
41	1994-11-02	0.51
42	1995-01-17	0.52
43	1995-04-27	0.53
44	1995-08-01	0.54
45	1995-11-01	0.55
46	1996-02-01	0.56
47	1996-05-01	0.57
48	1996-07-31	0.58
49	1996-10-31	0.59
50	1997-01-30	0.6
51	1997-04-30	0.61
52	1997-07-31	0.62
53	1997-10-30	0.63
54	1998-02-05	0.64
55	1998-04-30	0.65
56	1998-07-30	0.66
57	1998-10-30	0.67
58	1999-02-01	0.68
59	1999-04-29	0.69
60	1999-07-29	0.7
61	1999-10-19	0.71
62	2000-02-01	0.72
63	2000-05-01	0.73
64	2000-08-01	0.74
65	2000-11-01	0.75
66	2001-02-01	0.76
67	2001-05-01	0.77
68	2001-08-01	0.78
69	2001-10-31	0.79
70	2002-01-31	0.8
71	2002-05-02	0.81
72	2002-08-01	0.82
73	2002-11-06	0.83
74	2003-02-04	0.83
75	2003-05-01	0.83
76	2003-07-31	0.83
77	2003-10-31	0.83
78	2004-02-02	0.835
79	2004-05-04	0.4175
80	2004-08-02	0.4175
81	2004-11-04	0.4175
82	2005-02-03	0.42
83	2005-05-03	0.42
84	2005-08-04	0.42
85	2005-11-03	0.42
86	2006-02-09	0.425
87	2006-05-04	0.425
88	2006-08-03	0.425
89	2006-11-01	0.425
90	2007-02-01	0.445
91	2007-05-03	0.445
92	2007-08-02	0.445
93	2007-11-01	0.445
94	2008-02-05	0.455
95	2008-05-01	0.455
96	2008-08-07	0.455
97	2008-11-06	0.455
98	2009-02-05	0.46
99	2009-05-01	0.46
100	2009-08-04	0.46
101	2009-11-05	0.46
102	2010-02-09	0.465
103	2010-04-29	0.465
104	2010-08-05	0.465
105	2010-11-04	0.465
106	2011-02-08	0.48
107	2011-05-05	0.48
108	2011-08-04	0.48
109	2011-11-03	0.48
110	2012-02-02	0.5
111	2012-05-03	0.5
112	2012-08-02	0.5
113	2012-11-01	0.5
114	2013-01-31	0.525
115	2013-05-02	0.525
116	2013-08-01	0.525
117	2013-10-31	0.525
118	2014-02-06	0.545
119	2014-05-08	0.545
120	2014-08-07	0.545
121	2014-11-06	0.545
122	2015-02-05	0.565
123	2015-05-07	0.565
124	2015-08-06	0.565
125	2015-11-05	0.565
126	2016-02-04	0.575
127	2016-05-05	0.575
128	2016-08-04	0.575
129	2016-11-08	0.37
130	2017-02-13	0.37
131	2017-05-04	0.37
132	2017-08-03	0.37
133	2017-11-03	0.37
134	2018-02-14	0.37
135	2018-05-04	0.37
136	2018-08-03	0.37
137	2018-11-02	0.37
138	2019-02-15	0.37
139	2019-05-03	0.37
140	2019-08-02	0.37
141	2019-11-01	0.37

2.3 KiB Raw Blame History

2.3 KiB

Raw Blame History