overrides/corrections/{SYM}.json (git-tracked, with as_of/source/note)
holds remove/replace/add ops on the dividend and capital-gain series.
data.py applies them on top of whatever the data root (or the frozen
snapshot) provides, in both the full build and the incremental refresh
path, and the corrections dir joins the cache manifest so a change
invalidates the cache. A goget re-download of the base CSV can never
clobber a confirmed correction. Format and usage documented in data.py.
230 tickers whose Yahoo chart responses now come back without a timestamp
array (terminated/merged funds): goget overwrites the .json on every pass
while ohlc.Conv skips the write, leaving the old CSVs as the last known
series. Snapshot them into overrides/frozen/ (git-tracked, audited in
reports/stale-funds.md) and make data.py prefer the frozen copies and
ignore any future data-root rewrite/delete for those symbols, so the
final series survives future goget runs. The cache manifest now covers
the overrides dir too; incremental refresh skips data-root files of
frozen symbols.
- data.py: non-blocking load_bundle(); background watcher thread refreshes
the parquet cache (5s scan, 30s min rebuild cadence); refresh()/
up_to_date()/generation()
- statistics tab: one table per benchmark (vs <label>), plain column names
(beta/alpha/return/vol...), selectable+reorderable stat list in
settings.json
- correlation tab: per-portfolio components-vs-benchmarks +
all-portfolios-vs-benchmarks; numbered columns
- global date range (window radio + start/end boxes) applied to all tabs;
metrics.xcorr(); equity window radio gains YTD/3M/1M
The cache now tracks every file in the data dir (mtime_ns + size) in
.cache/manifest.json. On load, a directory scan is compared against the
manifest:
- changed/added files are re-read and merged into the parquet panels
(one read + one concat + one write per touched panel; new values
win where present, old values kept where the new file is short)
- removed files drop their symbols (and names)
- an up-to-date cache is a ~30 ms memo hit
Measured on the real 4k-symbol set: full build 54 s, refresh of
5 modified + 1 added + 1 removed files 3.4 s. No scan TTL (a scan is
a few ms); a previous 5 s scan cache masked data updates.
Tests: tests/test_data.py (11 checks) added as step 1 of run_tests.sh.
- single spec grammar for symbol and benchmark fields: commas join one
portfolio (MSFT:0.6,V:0.4), spaces separate distinct symbols/portfolios;
both fields accept one or many entries
- benchmarks simulated with the same scheme/cost/tax rules; per-benchmark
beta/alpha columns; after-tax benchmark curves
- global Curve mode (pre/after/both) above the tabs; clean names in
single-curve mode
- live updates: field commits on Enter/blur, page recomputes per rerun;
portfolio+tax sims cached (st.cache_data); plotly.js from CDN (4.6MB ->
browser-cached) with F_INLINE_PLOTLY=1 offline fallback
- chart: legend underneath, solid lines, pan sticks to data edges
(width-preserving), zoom edge-clamped
- inputs persist in settings.json across reloads/restarts/devices
- tests: tests/test_app.py (AppTest) + tests/test_e2e_browser.py
(Playwright) via ./run_tests.sh
