- EFTS queries now include 497/497K (many fund families publish their
per-fund highlights there, not in the consolidated N-CSR) and re-rank
hits by registrant name match (ticker/brand words), newest first,
capped at 2 filings per CIK
- new parse_per_share_blocks for the JPMorgan-style 'Per share operating
performance' table (per-class value blocks; dashes = zero)
- parse_highlights now tolerates row labels split across table cells
(modernized N-CSRS format, e.g. Calamos 2026)
- region finders: word-flexible name patterns (US vs U.S., class letters),
self-validating per-share regions (a candidate block must match the
local series, so a name mention in notes doesn't attribute another
fund's tables in a combined 58 MB report)
- main() keeps the best result across candidate filings (N-CSRS vs 497
can round differently) and stops early on 'ok'
- local_series applies the corrections overlay so corrected funds verify
against their filing
Results: JLPSX and CVSIX now 'ok' (all bounded fiscal years agree with
the official filings); CVSIX also gets a 2023-12-21 0.510 capital-gain
correction. bnd/pmaix still ok (no regression).
overrides/corrections/{SYM}.json (git-tracked, with as_of/source/note)
holds remove/replace/add ops on the dividend and capital-gain series.
data.py applies them on top of whatever the data root (or the frozen
snapshot) provides, in both the full build and the incremental refresh
path, and the corrections dir joins the cache manifest so a change
invalidates the cache. A goget re-download of the base CSV can never
clobber a confirmed correction. Format and usage documented in data.py.
230 tickers whose Yahoo chart responses now come back without a timestamp
array (terminated/merged funds): goget overwrites the .json on every pass
while ohlc.Conv skips the write, leaving the old CSVs as the last known
series. Snapshot them into overrides/frozen/ (git-tracked, audited in
reports/stale-funds.md) and make data.py prefer the frozen copies and
ignore any future data-root rewrite/delete for those symbols, so the
final series survives future goget runs. The cache manifest now covers
the overrides dir too; incremental refresh skips data-root files of
frozen symbols.
- data.py: non-blocking load_bundle(); background watcher thread refreshes
the parquet cache (5s scan, 30s min rebuild cadence); refresh()/
up_to_date()/generation()
- statistics tab: one table per benchmark (vs <label>), plain column names
(beta/alpha/return/vol...), selectable+reorderable stat list in
settings.json
- correlation tab: per-portfolio components-vs-benchmarks +
all-portfolios-vs-benchmarks; numbered columns
- global date range (window radio + start/end boxes) applied to all tabs;
metrics.xcorr(); equity window radio gains YTD/3M/1M
The cache now tracks every file in the data dir (mtime_ns + size) in
.cache/manifest.json. On load, a directory scan is compared against the
manifest:
- changed/added files are re-read and merged into the parquet panels
(one read + one concat + one write per touched panel; new values
win where present, old values kept where the new file is short)
- removed files drop their symbols (and names)
- an up-to-date cache is a ~30 ms memo hit
Measured on the real 4k-symbol set: full build 54 s, refresh of
5 modified + 1 added + 1 removed files 3.4 s. No scan TTL (a scan is
a few ms); a previous 5 s scan cache masked data updates.
Tests: tests/test_data.py (11 checks) added as step 1 of run_tests.sh.
- single spec grammar for symbol and benchmark fields: commas join one
portfolio (MSFT:0.6,V:0.4), spaces separate distinct symbols/portfolios;
both fields accept one or many entries
- benchmarks simulated with the same scheme/cost/tax rules; per-benchmark
beta/alpha columns; after-tax benchmark curves
- global Curve mode (pre/after/both) above the tabs; clean names in
single-curve mode
- live updates: field commits on Enter/blur, page recomputes per rerun;
portfolio+tax sims cached (st.cache_data); plotly.js from CDN (4.6MB ->
browser-cached) with F_INLINE_PLOTLY=1 offline fallback
- chart: legend underneath, solid lines, pan sticks to data edges
(width-preserving), zoom edge-clamped
- inputs persist in settings.json across reloads/restarts/devices
- tests: tests/test_app.py (AppTest) + tests/test_e2e_browser.py
(Playwright) via ./run_tests.sh
