Answer to 'find other alpha-driven funds that complement the portfolio':
- fundlab/search.py: complementarity screen - each fund's daily total
returns vs the same 21 broad sleeve axes (BIC forward selection,
|t|>2), full + 5y; alpha (intercept t), R2, rolling 6m alpha
persistence, correlation vs the current qspnx/pmaix portfolio and the
spy/agg/tlt benchmark mix. Verdict tiers: CANDIDATE (alpha +
persistent + portfolio-corr<0.3) / semi-alpha / alpha-but-correlated /
sleeve mix / weak.
- fundlab/dbmine.py: the actual search universe - the local stocks DB
already holds ~100 US open-end alternatives (AQR, PIMCO, JPM,
Principal, Calamos, GMO, Franklin K2, ...). Name-pattern miner with
share-class family dedupe (keeps the longest-history class).
- fundlab/tickers.py + searchlist.py: external longlist resolution
(chart-API name gate + EDGAR 497 cover tickers). Finding: the famous
multi-strategy/macro names (Millennium, Balyasny, Two Sigma, Winton,
Marshall Wace, Brevan Howard, AQR Event-Driven) are private/offshore
or terminated - not US open-end accessible. Fidelity Multi-Asset
Income (FMSDX) resolved and screens as weak alpha.
- app Fund Lab: 'Alpha search - all screened funds, ranked' table
(80 funds: 13 shortlist + 59 mined + 1 external).
- results (ranked candidates, 5y alpha / t / portfolio-corr):
wmnix Westwood Alt Income +3.8% t6.5 c0.09 | pyaix Payden ARB +3.0%
t4.8 c0.13 | srdax Stone Ridge Div Alts +7.7% t4.2 c0.10 | padqx PGIM
ARB +2.3% t2.4 c0.27 | bxmdx Blackstone Alt MS +3.5% t2.4 c0.30 |
aqmix AQR Mngd Futures +8.0% t2.2 c0.21 | cmnix/gioix semi-alpha.
Key insight: AQR MN / L/S-equity / Vanguard MN show strong alpha but
corr 0.35-0.76 with the portfolio - it is already 50% market-neutral
(qspnx), so more MN is not diversifying.
- tests: 59/59 fundlab (resolver gates, query ladder, family dedupe,
ticker regex), 32/32 app, 14/14 data
- fundlab/nport.py: parse the fund's own category/percentage lines,
as-of date, net assets and dollar-valued positions from N-PORT
schedules of investments (handles per-fund and combined multi-fund
family filings; conservative keyword bucketing of positions)
- fundlab/nport_cache/<sym>.json: parsed snapshots for 15 of 16 funds
(raw SOI HTML kept locally, gitignored; source URLs + filing dates in
nport_manifest.json, md5-verified against EDGAR)
- atesx: no current SOI found (Anchor's recent filings cover the Income
fund) - listed with an honest note
- app.py: new 'Fund Lab' tab - pick a fund, see objective, reported
composition (bar + table), rough keyword buckets, top positions, and
the prospectus strategy excerpt
- tests: parser unit tests (section finding, category regex, buckets)
- extract_strategy(): finds the 'Principal Investment Strategies' /
'main investment strategies' section, scores candidates (strategy prose
+3, Q&A heading +2, TOC -5, risk subheading -5, stop-heading -2),
truncates at the next section heading; falls back to the prose after
the objective sentence when no heading exists
- fundinfo --strategy [--refresh]: populates the strategy field of
funds.json from each fund's EDGAR document
- funds.json now carries objective + strategy for 20 funds (the 9
curated index funds have no strategy: their objective is the strategy)
- ticker -> CIK scoped FTS (ciks filter) over the fund's own registrant,
word queries shed Yahoo abbreviations shortest-first
- fuzzy cover gate (SequenceMatcher, sliding window for wrapped names)
+ abbreviation table (Mgd/Glbl/Macr/Abs/Ret/Advtg/...) applied to both
sides of the match
- family-section matching inside multi-fund filings; similarity tiers:
gate (0.85) returns immediately, floor (0.70) is a CIK-scoped fallback
ranked against sibling funds' documents (best sim, then newest)
- objective phrasings: 'seeks to ...', 'seeks investment results ...',
'The Fund's investment objective is ...', '(the Fund) investment
objective is ...' (boilerplate 'is not fundamental' rejected); TOC
headings skipped by trying all heads
- sec_get retries on mid-stream connection drops
- edgar.py: SEC FTS + submissions API; strict cover-gate extraction
(name in title position or (TICKER) on the cover; underlying-reference
names like leveraged wrappers rejected); 4-pass fetch (ticker->CIK
filings, name search, annual reports, ticker search); keyword category
classifier. Returns None rather than a wrong fund's objective.
- fundinfo.py CLI: curated -> cached -> EDGAR resolution into funds.json
- funds_curated.json: human-verified objectives for 13 benchmark-pool
funds (iShares/Vanguard family-trust classes the scraper can't reach)
- tests: 26 checks incl. live EDGAR fetch of VTSAX
- data.py: non-blocking load_bundle(); background watcher thread refreshes
the parquet cache (5s scan, 30s min rebuild cadence); refresh()/
up_to_date()/generation()
- statistics tab: one table per benchmark (vs <label>), plain column names
(beta/alpha/return/vol...), selectable+reorderable stat list in
settings.json
- correlation tab: per-portfolio components-vs-benchmarks +
all-portfolios-vs-benchmarks; numbered columns
- global date range (window radio + start/end boxes) applied to all tabs;
metrics.xcorr(); equity window radio gains YTD/3M/1M
The cache now tracks every file in the data dir (mtime_ns + size) in
.cache/manifest.json. On load, a directory scan is compared against the
manifest:
- changed/added files are re-read and merged into the parquet panels
(one read + one concat + one write per touched panel; new values
win where present, old values kept where the new file is short)
- removed files drop their symbols (and names)
- an up-to-date cache is a ~30 ms memo hit
Measured on the real 4k-symbol set: full build 54 s, refresh of
5 modified + 1 added + 1 removed files 3.4 s. No scan TTL (a scan is
a few ms); a previous 5 s scan cache masked data updates.
Tests: tests/test_data.py (11 checks) added as step 1 of run_tests.sh.
- single spec grammar for symbol and benchmark fields: commas join one
portfolio (MSFT:0.6,V:0.4), spaces separate distinct symbols/portfolios;
both fields accept one or many entries
- benchmarks simulated with the same scheme/cost/tax rules; per-benchmark
beta/alpha columns; after-tax benchmark curves
- global Curve mode (pre/after/both) above the tabs; clean names in
single-curve mode
- live updates: field commits on Enter/blur, page recomputes per rerun;
portfolio+tax sims cached (st.cache_data); plotly.js from CDN (4.6MB ->
browser-cached) with F_INLINE_PLOTLY=1 offline fallback
- chart: legend underneath, solid lines, pan sticks to data edges
(width-preserving), zoom edge-clamped
- inputs persist in settings.json across reloads/restarts/devices
- tests: tests/test_app.py (AppTest) + tests/test_e2e_browser.py
(Playwright) via ./run_tests.sh
