f/gatex-dividend.csv at 3c5bee96ceb54e7c6d0fb87afbcec59e6b77a3f3

gmp/f

Greg Pomerantz 3c5bee96ce Clean multi-vintage 'ctime' files: 1,065 files, 2.2M stale rows

Some dump files carry a trailing 'ctime' column: an older pipeline
appended a FULL re-download of the symbol per download (up to ~28
vintages per date, 2019-2022). The consumer keeps the last row per date,
so it silently used stale (~2021) vintages where Yahoo later revised
values. scripts/clean_multivintage.py keeps, per date, the rows from the
newest ctime (event files keep distinct same-day amounts; history/split
keep the single newest row), drops the ctime column, sorts by date.
Applied to the data root and overrides/frozen (which carried the same
artifact): 1,065 files, 2,202,712 stale rows dropped.

Also: stripped 3 UTF-8 BOMs (teg/lo/krft), refreshed the event backup
with the cleaned files, and re-ran the double-listing scan: category C
(83 symbols, 18,636 repeated within-file rows) is fully explained by the
multi-vintage artifact and is now gone; A (6,589 pairs, corrected) and
B (12,366 pairs, open review list) are unchanged. Whole-set structural
QC after cleaning: 0 duplicate dates, 3 syms/15 rows of OHLC invariant
violations, 15 syms/2,897 rows of non-positive prices (mostly
long-delisted tickers), 8 unsorted files (reader sorts).

2026-08-31 22:25:02 -04:00

2.3 KiB

Raw Blame History

1	Date	Dividends
2	1980-01-21	0.15
3	1980-04-24	0.15
4	1980-07-23	0.15
5	1980-10-14	0.16
6	1981-01-21	0.16
7	1981-04-22	0.16
8	1981-07-22	0.16
9	1981-10-21	0.16
10	1982-01-22	0.16
11	1982-04-23	0.16
12	1982-07-21	0.16
13	1982-10-20	0.16
14	1983-01-21	0.16
15	1983-04-22	0.16
16	1983-07-22	0.1
17	1983-10-21	0.13
18	1984-01-23	0.1
19	1984-04-26	0.1
20	1984-07-25	0.2
21	1984-10-24	0.17
22	1985-01-23	0.13
23	1985-04-25	0.11
24	1985-07-25	0.11
25	1985-10-23	0.11
26	1986-01-23	0.1
27	1986-04-17	0.08
28	1986-07-23	0.08
29	1986-10-21	0.08
30	1987-02-11	0.08
31	1987-03-17	0.08
32	1987-06-16	0.08
33	1987-09-16	0.05
34	1987-12-16	0.05
35	1988-03-23	0.05
36	1988-06-22	0.05
37	1988-09-21	0.05
38	1988-12-16	0.06
39	1989-03-22	0.08
40	1989-06-21	0.08
41	1989-09-27	0.08
42	1989-12-19	0.13
43	1990-03-28	0.09
44	1990-06-27	0.09
45	1990-09-26	0.09
46	1990-12-27	0.14
47	1991-03-27	0.07
48	1991-06-26	0.07
49	1991-09-26	0.07
50	1991-12-30	0.09
51	1992-03-31	0.07
52	1992-06-30	0.07
53	1992-09-30	0.07
54	1992-12-30	0.07
55	1993-03-31	0.07
56	1993-06-30	0.07
57	1993-09-30	0.07
58	1993-12-31	0.08
59	1994-03-31	0.07
60	1994-06-30	0.07
61	1994-09-30	0.07
62	1994-12-30	0.06
63	1995-03-31	0.07
64	1995-06-30	0.07
65	1995-09-29	0.07
66	1995-12-29	0.051
67	1996-03-29	0.05
68	1996-06-28	0.05
69	1996-09-30	0.05
70	1996-12-31	0.052
71	1997-03-31	0.05
72	1997-06-30	0.04
73	1997-09-30	0.04
74	1997-12-31	0.047
75	1998-03-31	0.03
76	1998-06-30	0.03
77	1998-09-30	0.03
78	1998-12-31	0.033
79	1999-03-31	0.03
80	1999-12-30	0.043
81	2000-12-29	0.078
82	2001-12-28	0.108
83	2002-12-30	0.153
84	2003-12-30	0.17
85	2004-12-30	0.29
86	2005-12-29	0.445
87	2006-06-28	0.125
88	2006-09-27	0.12
89	2006-12-27	0.238
90	2007-03-28	0.15
91	2007-06-27	0.13
92	2007-09-26	0.13
93	2007-12-27	0.125
94	2008-03-27	0.135
95	2008-06-26	0.116
96	2008-09-26	0.12
97	2008-12-31	0.156
98	2009-03-27	0.16
99	2009-06-26	0.12
100	2009-09-25	0.105
101	2009-12-22	0.088
102	2010-03-25	0.085
103	2010-04-09	0.019
104	2010-06-24	0.095
105	2010-09-23	0.097
106	2010-12-21	0.119
107	2011-03-24	0.1
108	2011-06-23	0.104
109	2011-09-22	0.105
110	2011-12-20	0.121
111	2012-03-22	0.103
112	2012-06-21	0.102
113	2012-09-20	0.102
114	2012-12-20	0.161
115	2013-03-21	0.092
116	2013-06-20	0.104
117	2013-09-24	0.105
118	2013-12-18	0.081
119	2014-03-20	0.115
120	2014-06-19	0.099
121	2014-09-23	0.088
122	2014-12-18	0.078
123	2015-03-30	0.103
124	2015-06-25	0.279
125	2015-09-29	0.099
126	2015-12-21	0.066
127	2016-03-30	0.133
128	2016-06-30	0.095
129	2016-09-30	0.096
130	2016-12-20	0.097
131	2017-03-29	0.096
132	2017-06-28	0.09
133	2017-09-28	0.113
134	2017-12-20	0.039
135	2018-03-26	0.103
136	2018-06-26	0.086
137	2018-09-25	0.088
138	2018-12-19	0.087
139	2019-03-26	0.105
140	2019-06-25	0.093
141	2019-09-25	0.099
142	2019-12-18	0.08
143	2020-03-25	0.088
144	2020-06-24	0.088
145	2020-09-24	0.067
146	2020-12-18	0.062
147	2021-03-24	0.046

2.3 KiB Raw Blame History

2.3 KiB

Raw Blame History