f/scripts
Greg Pomerantz 5e5a10d725 Fix Yahoo total-distribution double count (1,957 syms)
Yahoo's dividend endpoint returns the fund's TOTAL per-share distribution
(dividend + capital gain) for many share classes while the capitalGains
endpoint returns the cap-gain portion separately; summing both double-counted
cap gains and broke pre/post-tax comparability (after-tax engine could beat
the adj-based pre-tax return: impossible).

Add _fix_total_distributions to the bundle-assembly invariants
(layout-agnostic, survives re-downloads): on same-date div>capg events, the
exact Yahoo-adj implied distribution (d = P - Q*A_{t-1}/A_t) must match the
div-file amount (not the sum) on >=3 dates before rewriting div -= capg.
Self-validating: genuine separate same-date distributions are untouched.
Rewrote 17,900 cells on 1,957 symbols.

Pool check (scripts/check_adj_consistency.py): all 14 symbols now
TR(close+events) == TR(adj) to 0.01pt and post < pre with plausible drags.

Tests: rewrite case, no-rewrite case, idempotency (tests/test_data.py).
2026-09-01 16:23:03 -04:00
..
apply_dedup_corrections.py Double-listing: layout-agnostic invariants, whole-dump sweep, bulk corrections 2026-08-31 21:00:36 -04:00
audit_stale.py Freeze stale (Yahoo-empty) tickers: snapshot final series in overrides/frozen/ 2026-08-31 13:23:24 -04:00
backup_events.py Event-history protection: backup dir, empty-download fallback, docs 2026-08-31 21:59:25 -04:00
check_adj_consistency.py Fix Yahoo total-distribution double count (1,957 syms) 2026-09-01 16:23:03 -04:00
check_corrections.py Repair Yahoo adj-close/event-date misalignment (JLPSX, JLPYX) 2026-08-31 23:08:42 -04:00
clean_multivintage.py Clean multi-vintage 'ctime' files: 1,065 files, 2.2M stale rows 2026-08-31 22:25:02 -04:00
fix_adj_misalign.py Repair Yahoo adj-close/event-date misalignment (JLPSX, JLPYX) 2026-08-31 23:08:42 -04:00
scan_adj_misalign.py Repair Yahoo adj-close/event-date misalignment (JLPSX, JLPYX) 2026-08-31 23:08:42 -04:00
scan_double_listing.py Clean multi-vintage 'ctime' files: 1,065 files, 2.2M stale rows 2026-08-31 22:25:02 -04:00
verify_official.py Event-history protection: backup dir, empty-download fallback, docs 2026-08-31 21:59:25 -04:00