f/hcn-dividend.csv at 3c5bee96ceb54e7c6d0fb87afbcec59e6b77a3f3

gmp/f

Greg Pomerantz 3c5bee96ce Clean multi-vintage 'ctime' files: 1,065 files, 2.2M stale rows

Some dump files carry a trailing 'ctime' column: an older pipeline
appended a FULL re-download of the symbol per download (up to ~28
vintages per date, 2019-2022). The consumer keeps the last row per date,
so it silently used stale (~2021) vintages where Yahoo later revised
values. scripts/clean_multivintage.py keeps, per date, the rows from the
newest ctime (event files keep distinct same-day amounts; history/split
keep the single newest row), drops the ctime column, sorts by date.
Applied to the data root and overrides/frozen (which carried the same
artifact): 1,065 files, 2,202,712 stale rows dropped.

Also: stripped 3 UTF-8 BOMs (teg/lo/krft), refreshed the event backup
with the cleaned files, and re-ran the double-listing scan: category C
(83 symbols, 18,636 repeated within-file rows) is fully explained by the
multi-vintage artifact and is now gone; A (6,589 pairs, corrected) and
B (12,366 pairs, open review list) are unchanged. Whole-set structural
QC after cleaning: 0 duplicate dates, 3 syms/15 rows of OHLC invariant
violations, 15 syms/2,897 rows of non-positive prices (mostly
long-delisted tickers), 8 unsorted files (reader sorts).

2026-08-31 22:25:02 -04:00

2.2 KiB

Raw Blame History

1	Date	Dividends
2	1985-01-29	0.3468
3	1985-04-30	0.3468
4	1985-07-30	0.3802
5	1985-10-29	0.38
6	1986-01-28	0.38
7	1986-04-29	0.38
8	1986-07-29	0.39
9	1986-10-28	0.39
10	1987-01-27	0.39
11	1987-04-28	0.42
12	1987-07-28	0.42
13	1987-10-28	0.42
14	1988-01-27	0.44
15	1988-04-26	0.44
16	1988-07-26	0.44
17	1988-10-25	0.38
18	1989-01-25	0.38
19	1989-04-26	0.4
20	1989-07-26	0.42
21	1989-10-25	0.42
22	1990-01-25	0.43
23	1990-04-26	0.43
24	1990-07-26	0.43
25	1990-10-25	0.43
26	1991-01-25	0.435
27	1991-04-26	0.44
28	1991-07-26	0.445
29	1991-10-25	0.45
30	1992-01-24	0.455
31	1992-04-29	0.46
32	1992-07-29	0.465
33	1992-10-28	0.47
34	1993-01-26	0.475
35	1993-04-30	0.48
36	1993-07-28	0.485
37	1993-10-27	0.49
38	1994-01-31	0.495
39	1994-05-02	0.5
40	1994-08-01	0.505
41	1994-10-31	0.51
42	1995-01-30	0.515
43	1995-05-01	0.52
44	1995-08-02	0.52
45	1995-08-04	0.52
46	1995-11-01	0.52
47	1996-01-31	0.52
48	1996-05-01	0.52
49	1996-07-31	0.52
50	1996-10-30	0.52
51	1997-01-30	0.52
52	1997-05-01	0.525
53	1997-07-31	0.53
54	1997-10-30	0.535
55	1998-01-30	0.54
56	1998-05-01	0.545
57	1998-07-31	0.55
58	1998-10-30	0.555
59	1999-01-29	0.56
60	1999-04-30	0.565
61	1999-07-30	0.57
62	1999-10-29	0.575
63	2000-01-28	0.58
64	2000-04-28	0.585
65	2000-07-28	0.585
66	2000-10-27	0.585
67	2001-01-29	0.585
68	2001-04-27	0.585
69	2001-07-27	0.585
70	2001-10-29	0.585
71	2002-01-29	0.585
72	2002-04-26	0.585
73	2002-07-29	0.585
74	2002-10-29	0.585
75	2003-01-29	0.585
76	2003-04-28	0.585
77	2003-07-29	0.585
78	2003-10-30	0.585
79	2004-01-28	0.585
80	2004-04-28	0.6
81	2004-07-28	0.6
82	2004-10-28	0.6
83	2005-01-27	0.6
84	2005-04-27	0.62
85	2005-07-27	0.62
86	2005-10-27	0.62
87	2006-01-27	0.62
88	2006-04-26	0.64
89	2006-07-27	0.64
90	2006-10-27	0.64
91	2006-12-20	0.341
92	2007-01-29	0.299
93	2007-05-02	0.66
94	2007-08-01	0.66
95	2007-10-31	0.66
96	2008-01-29	0.66
97	2008-04-30	0.68
98	2008-07-30	0.68
99	2008-11-06	0.68
100	2009-02-05	0.68
101	2009-05-07	0.68
102	2009-08-06	0.68
103	2009-11-05	0.68
104	2010-02-04	0.68
105	2010-05-06	0.68
106	2010-08-05	0.69
107	2010-11-04	0.69
108	2011-02-04	0.69
109	2011-05-06	0.715
110	2011-08-05	0.715
111	2011-11-04	0.715
112	2012-02-03	0.74
113	2012-05-04	0.74
114	2012-08-03	0.74
115	2012-11-02	0.74
116	2013-02-01	0.765
117	2013-05-03	0.765
118	2013-08-02	0.765
119	2013-11-07	0.765
120	2014-02-06	0.795
121	2014-05-08	0.795
122	2014-08-07	0.795
123	2014-11-06	0.795
124	2015-02-06	0.825
125	2015-05-08	0.825
126	2015-08-07	0.825
127	2015-11-06	0.825
128	2016-02-05	0.86
129	2016-05-06	0.86
130	2016-08-05	0.86
131	2016-11-04	0.86
132	2017-02-03	0.87
133	2017-05-05	0.87
134	2017-08-04	0.87
135	2017-11-06	0.87
136	2018-02-12	0.87

2.2 KiB Raw Blame History

2.2 KiB

Raw Blame History