Scanning the verified funds for same-date div/capg pairs showed the Yahoo
double-listing mechanism in every one of the 7 'mismatch' stale funds, and
the official fiscal-year totals pin down which amount is true:
- identical-amount pairs (JLPSX, GDEUX, GSOUX, FAEVX, CVSIX, FZAGX): the
year-end distribution is in both files; keep the capitalGain copy,
remove the dividend copy (GDEUX: FY2021-08 1.73 = 1.625+0.115 and
FY2022-08 0.39 = 0.348+0.036 after; JLPSX/GSOUX/FAEVX/CVSIX same)
- differing-amount pairs (OTCRX, SHXIX, FZAGX, FGIZX, CVSIX): the
dividend-file amount is the true one in every officially-verified case
(OTCRX FY2023 0.85 = 0.852488, SHXIX FY2023 1.01 = 1.0078, FZAGX all
four FYs exact); the capitalGain row is the spurious copy
After corrections GDEUX/SHXIX/CVSIX/JLPSX verify 'ok' against their
filings. The same pattern is extended to these funds' pre-2021 history
(marked mechanism-inferred in the correction notes). Remaining residuals:
FAEVX and FGIZX are also missing regular quarterly dividend rows in the
Yahoo dump (official FY totals exceed local even after dedup) - needs the
funds' per-date distribution archives; FIKAX's official extraction is
ambiguous (systematic ~0.12/yr offset = wrong class table in the
500-fund consolidated Fidelity N-CSRS).
- EFTS queries now include 497/497K (many fund families publish their
per-fund highlights there, not in the consolidated N-CSR) and re-rank
hits by registrant name match (ticker/brand words), newest first,
capped at 2 filings per CIK
- new parse_per_share_blocks for the JPMorgan-style 'Per share operating
performance' table (per-class value blocks; dashes = zero)
- parse_highlights now tolerates row labels split across table cells
(modernized N-CSRS format, e.g. Calamos 2026)
- region finders: word-flexible name patterns (US vs U.S., class letters),
self-validating per-share regions (a candidate block must match the
local series, so a name mention in notes doesn't attribute another
fund's tables in a combined 58 MB report)
- main() keeps the best result across candidate filings (N-CSRS vs 497
can round differently) and stops early on 'ok'
- local_series applies the corrections overlay so corrected funds verify
against their filing
Results: JLPSX and CVSIX now 'ok' (all bounded fiscal years agree with
the official filings); CVSIX also gets a 2023-12-21 0.510 capital-gain
correction. bnd/pmaix still ok (no regression).
overrides/corrections/{SYM}.json (git-tracked, with as_of/source/note)
holds remove/replace/add ops on the dividend and capital-gain series.
data.py applies them on top of whatever the data root (or the frozen
snapshot) provides, in both the full build and the incremental refresh
path, and the corrections dir joins the cache manifest so a change
invalidates the cache. A goget re-download of the base CSV can never
clobber a confirmed correction. Format and usage documented in data.py.
