Scanning the verified funds for same-date div/capg pairs showed the Yahoo
double-listing mechanism in every one of the 7 'mismatch' stale funds, and
the official fiscal-year totals pin down which amount is true:
- identical-amount pairs (JLPSX, GDEUX, GSOUX, FAEVX, CVSIX, FZAGX): the
year-end distribution is in both files; keep the capitalGain copy,
remove the dividend copy (GDEUX: FY2021-08 1.73 = 1.625+0.115 and
FY2022-08 0.39 = 0.348+0.036 after; JLPSX/GSOUX/FAEVX/CVSIX same)
- differing-amount pairs (OTCRX, SHXIX, FZAGX, FGIZX, CVSIX): the
dividend-file amount is the true one in every officially-verified case
(OTCRX FY2023 0.85 = 0.852488, SHXIX FY2023 1.01 = 1.0078, FZAGX all
four FYs exact); the capitalGain row is the spurious copy
After corrections GDEUX/SHXIX/CVSIX/JLPSX verify 'ok' against their
filings. The same pattern is extended to these funds' pre-2021 history
(marked mechanism-inferred in the correction notes). Remaining residuals:
FAEVX and FGIZX are also missing regular quarterly dividend rows in the
Yahoo dump (official FY totals exceed local even after dedup) - needs the
funds' per-date distribution archives; FIKAX's official extraction is
ambiguous (systematic ~0.12/yr offset = wrong class table in the
500-fund consolidated Fidelity N-CSRS).
- EFTS queries now include 497/497K (many fund families publish their
per-fund highlights there, not in the consolidated N-CSR) and re-rank
hits by registrant name match (ticker/brand words), newest first,
capped at 2 filings per CIK
- new parse_per_share_blocks for the JPMorgan-style 'Per share operating
performance' table (per-class value blocks; dashes = zero)
- parse_highlights now tolerates row labels split across table cells
(modernized N-CSRS format, e.g. Calamos 2026)
- region finders: word-flexible name patterns (US vs U.S., class letters),
self-validating per-share regions (a candidate block must match the
local series, so a name mention in notes doesn't attribute another
fund's tables in a combined 58 MB report)
- main() keeps the best result across candidate filings (N-CSRS vs 497
can round differently) and stops early on 'ok'
- local_series applies the corrections overlay so corrected funds verify
against their filing
Results: JLPSX and CVSIX now 'ok' (all bounded fiscal years agree with
the official filings); CVSIX also gets a 2023-12-21 0.510 capital-gain
correction. bnd/pmaix still ok (no regression).
overrides/corrections/{SYM}.json (git-tracked, with as_of/source/note)
holds remove/replace/add ops on the dividend and capital-gain series.
data.py applies them on top of whatever the data root (or the frozen
snapshot) provides, in both the full build and the incremental refresh
path, and the corrections dir joins the cache manifest so a change
invalidates the cache. A goget re-download of the base CSV can never
clobber a confirmed correction. Format and usage documented in data.py.
scripts/verify_official.py locates each fund's latest N-CSR/N-CSRS/10-K/10-Q
via EDGAR full-text search (full fund-name phrase first, then ticker +
name words, then bare ticker), extracts the fund's Financial Highlights
tables (both Vanguard-style and Victory-style layouts, calendar and
non-calendar fiscal years, M/D/YY and month-name headers), matches the
share class by per-share distribution series + NAV magnitude, and
compares per period window against the local Yahoo CSVs (frozen copies
for stale tickers). Per-symbol JSONs + SUMMARY.md land in
reports/xcheck_official/; results are cached per fund.
Run on the 29 curated funds: 8 ok (exact to 3dp, e.g. VTSAX 2021-2026H1),
1 mismatch (CVSIX FY2009: local 1.104 vs official 0.81 - the Yahoo
2008-12-18 row of 0.292 looks spurious), 3 weak-match (uncovered doc
formats, e.g. Leuthold), 17 not-found (mostly ETF families whose
10-K layouts aren't covered yet).
230 tickers whose Yahoo chart responses now come back without a timestamp
array (terminated/merged funds): goget overwrites the .json on every pass
while ohlc.Conv skips the write, leaving the old CSVs as the last known
series. Snapshot them into overrides/frozen/ (git-tracked, audited in
reports/stale-funds.md) and make data.py prefer the frozen copies and
ignore any future data-root rewrite/delete for those symbols, so the
final series survives future goget runs. The cache manifest now covers
the overrides dir too; incremental refresh skips data-root files of
frozen symbols.
fundlab/styletilt.py: 22 style/asset sleeves regressed on excess-of-T-bill
returns (full history + 5y); BIC forward selection identifies the tilt
stack; residual-alpha verdict ('factor exposure, not skill' when t<1.75);
data-driven English commentary with sign-specific phrasing. Rendered as a
'Style tilts' block (factor table + prose) in both the app and the HTML
report. All 24 pre-built + 8 ad-hoc fund reports rebuilt.
Any symbol that stands alone in the Symbol box (not a comma-joined
portfolio component) now gets a full report, built on demand and cached
in reports/report_data_adhoc.json. Rendered at the top of the Fund Lab
Summary as group 'A'; pointer at the top of the page. Rendering logic
extracted into render_fund_report() shared by pre-built and on-demand
entries. ~0.3-0.9 s per new fund after one-time panel warmup; instant
afterwards (memory + disk cache).
- per-fund report charts now use chart_widget.equity_chart_html
(st.iframe) instead of static st.plotly_chart: every visible window
re-bases each line to 1.0 at its left edge, so fund/reference/index
compare in any zoom level - same semantics as the Equity curves tab
- removed 'expand all' checkbox (24 expanded sections drowned the page)
- added 'rest of Fund Lab continues below' marker after the report
section; the other Fund Lab sections (alpha search, clusters, N-PORT,
drawdown, tax, CEF) were never deleted and are unchanged
- fundlab/narrative.py: data-driven English prose per fund (performance,
drivers tiered by fit, explicit 'what we do NOT know', bottom line)
- fundlab/reportdata.py: static build -> reports/report_data.json
- app.py Fund Lab Summary: at-a-glance table + per-fund expanders
(narrative, equity curve, period table with fund-ref gap, drivers,
reference mix, tax, cluster peers)
- fundlab/report.py: narrative in the HTML report; forward-selected
reference (weak-fit funds anchor to cash); SLEEVE_DESC exposure
explanations
- BUG: mix_series() never applied the betas (reference curves were raw
sleeve sums; JLPSX 'reference' +407% vs fund +123%) - fixed and all
reference curves/tables regenerated
- reports/fund_report.html + report_data.json regenerated
fundlab/report.py -> reports/fund_report.html (20 MB, plotly inlined,
opens offline). Per fund: max-history equity curve (fund vs fitted
reference vs IVV); performance table (full/5y/1y, the 5 market
episodes, calendar years) with the fund-minus-reference period-alpha
column; drivers (reference-model R²/alpha/t + 34-sleeve signature +
curated decomposition verdict and N-PORT cross-check notes); the
reference mix explained sleeve-by-sleeve (what each exposure actually
is, plus net-cash/net-levered read); tax character + taxable/IRA
placement; and a peer table of the 4 best funds in the same k=30
return-driver cluster with computed advantages/disadvantages.
Weak-fit (R²<0.5) funds anchor their tables to CASH rather than the
statistically-thin forward-selected mix (which can be an offsetting
VIX/duration spec combination whose path is meaningless); the loadings
are still shown with a 'weak fit' caveat.
Raw-intercept alphas absorbed the T-bill yield on uninvested/levered
portions (582 well-fitted funds >2%/yr off; sum-of-betas polluted by
level-matching). Now fund AND sleeves are netted against BIL daily
total return before every regression; a cash position contributes
exactly zero.
- decompose: rf_series()/excess(); shv+bil dropped from regressors
(~0 columns in excess space); FULL_WINDOW -> 2007-06-01 (BIL
inception; mixing raw pre-2007 with excess breaks the fit).
- factors: same excess treatment; shv out of DRIVERS.
- CASH axis redefined: alpha/cash_yield -> net cash position = 1 -
sum(betas) (label 'cash (net posn)').
- CANDIDATE list 250 -> 11: the old list was mostly under-invested
funds whose 'alpha' was cash yield, not skill.
- refback.py: per-fund fitted reference (forward-selected sleeves)
stored as ref_5y/ref_full in search_all.json; app alpha-search
table gains a 'reference (5y)' column - the answer to 'what is
alpha computed against' (the fund's OWN fitted sleeve mix, not one
index).
- App captions updated; raw-alpha-era results backed up as
*_rawalpha.json (not deleted).
A fund's yield is a near-constant in return space, so OLS puts it in
the intercept (alpha_ann), not in the shv/bil betas (which only
measure rate-CHANGE sensitivity). Cash funds therefore showed no
dominant driver.
- factors: cash_yield() (shv trailing-1y total return = local
stand-in for the 13-wk T-bill / 0-3m Treasury index), AXES =
DRIVERS + ['cash'] (alpha_ann / cash_yield, 1.0 = earns the cash
rate); a display/clustering axis, not a regressor.
- cluster: cash column in the loading matrix; k-means distances use
the cash-2x-emphasized matrix (a pure cash fund's level axis was
otherwise swallowed by the low-exposure cloud); labels use raw
values; label() cash-aware (0.25 threshold, 1.5x runner-up).
- app: cluster view over AXES with the emphasized distance.
The 331-fund 'no dominant driver' grab-bag now splits into
'cash (yield) +0.64' (n=187, MM + ultra-short) and 'cash (yield)
+1.26' (n=59 pure). 123/123 tests.
- cef_universe_run.py: resumable batch over all 295 CEFs ->
cef_annual_all.json (241 parsed, actual per-share character).
- cef_rank_all.json: 239-fund ranking, actual char x (t5 + 0.4 vol5);
surfaces FXBY and DPG beyond the original shortlist.
- share fractions now use the component sum as denominator (always
sum to 1); dist_tot remains the validity gate. Fixes CEE-style
111% sums when the NAV-identity fallback total is short.
- server watchdog + app CEF tab; RESEARCH.md updated.
- Fund Lab tab: CEF tax-arb shortlist table (char actual-vs-model,
discount, 5y return/vol, scenario hits, tenders, score) built from
cef_character.json + cef_annual.json.
- fundlab/server_watchdog.sh: relaunches Streamlit on :8599 if the
health endpoint stops answering (the old watchdog only covered the
overnight screen and had exited).
- fundlab/cef_universe_run.py: resumable batch extending per-share
verification to all 295 CEFs -> cef_annual_all.json.
Per-share financial-highlights parser (cef_annual.py) now reconciles
divs+gains+ROC==distributions and the full NAV chain per column for
48 of 50 shortlist funds, from three layout families (inline,
transposed, + repair rules for footnotes/split-decimals/sign-
magnitude/combined-rows). DXYZ is a correct zero-distribution result;
STEW is a documented columnar-stream exception. cef_merged.json
character column now uses actual data. 123/123 tests.
Joint backtracking placement for CEF tables that drop zero columns;
NAV-identity fallback when the printed total row is short; 4 fund-
family layouts verified (Franklin/classic/abrdn/Korea). 16/50 of the
CEF shortlist now have a verified div/gains/ROC split + FY-end
discount series. 17 new parser unit tests (123/123 green).
appreciation - a taxable-account feature, not an IRA one
Correction after user pushback. The earlier note "ROC does not help
the taxable case - the deferral replicates the IRA" was WRONG: a
traditional IRA defers to the ORDINARY rate at withdrawal, while ROC
in a taxable account defers to the LTCG rate on a >1y sale (the
distribution is basis-reducing and reappears inside the shareholder's
own capital gain). Under the premise LTCG rate < future ordinary
rate, ROC - like NAV appreciation - favors the taxable account.
Contrast ordinary income (interest, ordinary divs, STCG): taxed at
the ordinary rate in BOTH accounts, so only the IRA's deferral wins.
taxplan.py:
- _deferred_share(): per fund, share of 5y total return that defers
to the investor = (NAV change + ROC) / total return, from the
parsed per-share N-CSR table, max'd with the taxsplit appreciation
share.
- >= 50% deferred -> location "TAXABLE (defers to LTCG)" (renamed
from "TAXABLE (accrues)"); 10-50% ROC in distributions -> note.
- Merger-arb cap 0.35 -> 0.50: HMEZX's per-share table (52% NII /
30% gains / 18% ROC over 5y) refutes "mostly STCG" - HMEZX/MERVX
are now MIXED (check 1099), not clean IRA.
- RESEARCH.md: corrected placement write-up, incl. the distinction
between the tax question and the fund-quality question (heavy ROC
can mean principal erosion - PGSIX NAV -34%/5y - which affects
selection, not the optimal account).
App: order dict + filter updated to the new location name.
Tests: merger-arb expectation updated to MIXED; 2 new checks for
the ROC upgrade/note logic. 99/99 fundlab + 32/32 app.
fundlab/rocdetect.py: for a fund's latest N-CSR annual report,
parses the per-share "Financial Highlights" distribution table
(NII / gains / (tax) return of capital / total), with
whitespace normalization, dollar-basis and fiscal-year-column
rejection, per-column arithmetic placement of short rows
(NII+gains+ROC==total), and family-report disambiguation by
current-NAV match. SEC 503 backoff, per-fund cache in roc_cache/.
Verified over the 38 core funds:
- HMEZX: 18.3% of 5y distributions were return of capital
(0.62 + 0.37 per share in 2 of 5 fiscal years) - reinforces
the IRA placement.
- 17 funds verified zero ROC (AFLIX, ANGLX, ATESX, BTMIX, the EV
macro funds, FHCOX/FHMIX, LCORX/LCRIX, PMAIX/PMFKX, SVARX,
WMNUX).
- 14 unverified (non-standard report layouts - prototype parser),
PMORX/QCMMRX tickers not EDGAR-resolvable.
Also verified (form-type history on EDGAR): CVSIX, QSPNX, MBXIX,
ATRFX, ATESX, EAGMX, PBAIX, WISEX, EBSAX all file NPORT-P + N-CSR
+ 497 -> registered RICs, not pass-throughs; the whole 497-filer
universe is registered, so no candidate can pass through capital
losses. Proof case for ROC: PGSIX per-share table shows a tax
return-of-capital line and NAV eroded 11.87 -> 7.88 (-34%) over
5 fiscal years while paying out every year.
RESEARCH.md: full write-up. No app change (analysis artifact).
Follow-up to the tax-location plan: the taxplan score only measured
DISTRIBUTION character. The user rightly noted that NAV appreciation
is also a capital gain (LTCG on a >1y sale). The fund price files
carry both series - Close = raw NAV with distributions paid out,
Adj Close = total return reinvested - so the split is computable
directly per fund (5y window + most-recent-12m payout).
Findings:
- ACCUMULATORS (>=50% of 5y return is price appreciation) get a new
location "TAXABLE (accrues)": MBXIX 76% (0% payout 12m), ATESX 66%,
LAMHX 62%, CVSIX 61%, candidate PBAIX 60% (0% payout 12m). For
these the taxable account's LTCG-on-sale benefit is the dominant
tax event.
- PAY-OUT funds: HMEZX (99% of return distributed - the STCG merger-
arb case), MERVX, COSIX, PMORX, SVARX, SCFZX, DMSZX, munis, credit.
IRA placement stands.
- Data artifacts caught: JLPSX/QSPNX one-time NAV gap events ~2022
(special distribution or reorg) skew the 5y payout average; the
12m payout column reflects current behavior. QCMMRX (MMF) series
is not NAV-based - flagged.
App: tax-location expander gains 5y price / 5y payout / 12m payout
columns and the "TAXABLE (accrues)" filter. RESEARCH.md documents
the capital-loss question: registered RICs cannot distribute net
capital losses; the usable benefit is the fund's internal harvest
reserve (low capital-gain distributions after up-years), which needs
N-CSR/1099 history to verify. 97/32 suites green.
fundlab/taxplan.py categorizes the 16-fund shortlist, the 22 N-PORT
cross-checked candidates, and all 250 screened candidates by the
expected CHARACTER of their distributions, given the user's premise
that the current LTCG rate < the post-retirement ordinary rate:
qualified div + LTCG -> TAXABLE (score >= 0.60)
tax-exempt (munis) -> TAXABLE
ordinary / STCG / REIT -> IRA (score <= 0.35)
in between -> MIXED (pull the 1099-DIV)
cash -> FLEXIBLE
score = estimated share of distributions that are tax-favorable,
from three tiers of ground truth: N-PORT keyword buckets (16), SEC
assetCat/issuerCat buckets (22), sleeve loadings (250), with a
sleeve fallback when the keyword parser left >50% of a book
unclassified, and a manual override for the Leuthold wrappers
(91.7% Leuthold Core ETF, no return history yet).
Key findings:
- shortlist: TAXABLE = ATESX, JLPSX, LAMHX, LCORX, LCRIX (equity);
IRA = ATRFX, COSIX, CVSIX, PMORX, SVARX, EAGMX/EGRSX;
MIXED = MBXIX, QSPNX, PMAIX/PMFKX (same fund, two classes)
- cross-checked: 4 munis -> TAXABLE; HMEZX + MERVX are the merger-
arb trap - equity-looking books whose distributions are mostly
SHORT-TERM gains -> IRA
- candidates: 109 munis TAXABLE, 127 IRA, 6 equity TAXABLE, 7 MIXED
App: Fund Lab "Tax location" expander. Output:
fundlab/taxplan_results.json. Tests: test_taxplan() (9 checks).
97/32 suites green.
fundlab/drawdown.py detects the severe equity drawdown scenarios from
the index (IVV) rather than hard-coding them: one worst peak->trough
per calendar year since 2022, min depth 8% (a 10% floor would silently
drop the 2023 rate shock at -9.9% and the 2024 Aug-5 dip at -8.4%).
Detected: 2022 bear mkt (-24.5%), 2023 rate shock (-9.9%), 2024 vol
spike (-8.4%), 2025 tariff crash (-18.8%), 2026 Q1 drawdown (-8.9%).
For each of the 2,384 screened funds it computes that fund's own-NAV
return over each peak->trough window (first print after the peak to
the last print on/before the trough) and ranks the 250 CANDIDATEs by
# scenarios positive.
Key finding: positive in all 5 scenarios = only 7 funds, all
ultra-short/cash (BILS, QCMMRX, PULS, FHCOX, FHMIX, SAFEX, COIAX).
Drawdown resilience at the top tier is a duration property, not alpha.
The interesting tier is 4/5 WITH real 5y alpha: HMEZX merger arb
(+1.5% 2022, +3.1% 2023, t5 +7.1), MERVX, CBHCX market-neutral, SCFZX
securitized credit (t5 +8.4), ENIAX (t5 +10.1), WMNUX (t5 +6.9), RCTIX.
App: Fund Lab "Drawdown resilience" expander (scenario table +
candidate table). Output: fundlab/drawdown_results.json.
Tests: test_drawdown() added (4 checks). 88/32 suites green.
fundlab/xcheck.py - for each screen candidate, resolve the fund's OWN
registrant CIK (browse-edgar; the 497-cover CIK is the family/trust),
get the exact series name for the ticker (the only reliable
disambiguator between sibling funds), walk the 4 most recent NPORT-P
filing dates, and parse holdings from the interactive NPORT XML
(primary_doc.xml at the accession root - NOT the XSL-rendered view the
submissions API points at). Exact seriesName match > best htm exhibit
parse. Buckets from the authoritative assetCat+issuerCat codes (ABS-O,
ABS-CBDO, DBT+UST/CORP/MUN/NUSS, LON, STIV, RA, EC+RF=fund, ...), not
position-name keywords. Resumable; raw filings cached under
nport_cache/raw/ (gitignored).
nport.py - _SECTION gains the "INVESTMENT PORTFOLIO (unaudited)"
variant (NPORT-EX Sch-F files); find_section/build gain a frac
token-tolerance param (Yahoo names drift from filing names); CMBS/ABS
bucket gains CLO/CBDO/DAC terms.
app Fund Lab - "N-PORT cross-check" expander: per-candidate table
(as-of, n, t5, top code-bucket, #1 position) + per-fund holdings
detail.
RESEARCH.md - cross-check verdicts. 21/22 resolved (qcmmrx is an MMF,
no holdings). The screen's top names are REAL:
- hmezx/mervx = genuine merger arb (equity in deal targets + escrow)
- egrix = 100% wrapper in one macro managed portfolio (underlying not
NPORT-disclosed); etsix = fund of EV internal multi-strat accounts
- wmnux = discounted/zero-coupon corporate bonds + equity swaps (the
"equity names" are bond issuers/swap underlyings)
- scfzx/rctix/aflix = securitized credit/CLO/distressed/levered loans
- hicox/fhmix/usmsx/btmix (munis), aguax/femdx (EM sovereign), anglx
(agency MBS), lpxax (rotated out of prefs into bank/financial debt)
= genuine missing-factor exposures the 35-sleeve model lacks
- fhcox/dultx/safex = short-duration carry (a short-duration sleeve
would explain them)
tests/test_fundlab.py - test_xcheck (14 checks): parse_interactive,
code buckets, name-match normalization, series-name disambiguation.
Also: untrack fundlab/streamlit.log; gitignore raw/ + xcheck_run.log.
84 fundlab / 32 app / 14 data tests pass.
factors.py: full OLS of all 2,384 funds on a 35-driver basis
(overinclusive, no portfolio-corr screening - corr is a replacement
signal, not a rejection). v1's 21 + lqd/hyg/prefs/emb/tip/shy/vtv/
8 sectors/CTA/commodities. Basis fixes: drop vea/vug (dupes of
efa/qqq), drop finux (TERMINATED 2017 - silently zeroed the
complete-case mask; v1 forward selection never hit this), drop bil
(shv/bil near-null -> offsetting shv+64/bil-54 noise fits),
residualize vblix on ivv+tlt (pure vol axis), ridge 0.02.
cluster.py: hierarchical tree saved but fixed-k cuts degenerate
(most funds are blends -> one 2250-fund blob); k-means++ (deterministic)
is the useful grouping, re-run live in-app for any k.
app: Return-driver clusters expander (k slider 10-60, summary table,
member table sorted by alpha-t). Findings at k=30 in RESEARCH.md.
fundlab/RESEARCH.md - running research log: sources that work/die
(full-index = discovery workhorse; browse-edgar JS-dead;
investment-company-tickers.json nonexistent; company_tickers.json
useless for OTC; Yahoo crumb throttled but chart API fine), 13
hard-won learnings (OTC funds report exchange 'Nasdaq' -> use
instrumentType; 497 SGML cover uses UNCLOSED line-based tags ->
parse before tag-stripping; full-index columns drift -> regex the
line; one quarter != universe -> 4-qtr union; accession paths
relative to /Archives/ not /Archives/edgar/data/; family CIKs
repeat -> dedupe by series name; portfolio is 50% MN so MN alpha
funds are 'correlated', not diversifying).
fundlab/overnight.py - resumable all-stage pipeline (kill/restart safe):
verify (Yahoo chart per non-local ticker, 4-thread, 429 backoff,
local tickers measured from CSV row counts) -> select (pure
select_rows: MUTUALFUND, >=5y, one longest-history class per series
name, alpha_name as TAG not filter) -> download (goget in 200-sym
batches) -> screen (streamed, skip-already-done) -> finalize
(verdict counts + candidates the v1 name-filter would have missed).
Universe: 10,372 class tickers -> 10,260 verified -> 2,384 funds
(407 local, 1,977 external; only 54 match the alpha name pattern -
the v2 point is to screen the other 2,330).
app: alpha table now dedupes by sym with search_all.json winning
(comprehensive superset).
tests: select_rows unit tests (ETF drop, short-history drop, class
collapse, name tagging). 70/70 fundlab.
Answer to 'find other alpha-driven funds that complement the portfolio':
- fundlab/search.py: complementarity screen - each fund's daily total
returns vs the same 21 broad sleeve axes (BIC forward selection,
|t|>2), full + 5y; alpha (intercept t), R2, rolling 6m alpha
persistence, correlation vs the current qspnx/pmaix portfolio and the
spy/agg/tlt benchmark mix. Verdict tiers: CANDIDATE (alpha +
persistent + portfolio-corr<0.3) / semi-alpha / alpha-but-correlated /
sleeve mix / weak.
- fundlab/dbmine.py: the actual search universe - the local stocks DB
already holds ~100 US open-end alternatives (AQR, PIMCO, JPM,
Principal, Calamos, GMO, Franklin K2, ...). Name-pattern miner with
share-class family dedupe (keeps the longest-history class).
- fundlab/tickers.py + searchlist.py: external longlist resolution
(chart-API name gate + EDGAR 497 cover tickers). Finding: the famous
multi-strategy/macro names (Millennium, Balyasny, Two Sigma, Winton,
Marshall Wace, Brevan Howard, AQR Event-Driven) are private/offshore
or terminated - not US open-end accessible. Fidelity Multi-Asset
Income (FMSDX) resolved and screens as weak alpha.
- app Fund Lab: 'Alpha search - all screened funds, ranked' table
(80 funds: 13 shortlist + 59 mined + 1 external).
- results (ranked candidates, 5y alpha / t / portfolio-corr):
wmnix Westwood Alt Income +3.8% t6.5 c0.09 | pyaix Payden ARB +3.0%
t4.8 c0.13 | srdax Stone Ridge Div Alts +7.7% t4.2 c0.10 | padqx PGIM
ARB +2.3% t2.4 c0.27 | bxmdx Blackstone Alt MS +3.5% t2.4 c0.30 |
aqmix AQR Mngd Futures +8.0% t2.2 c0.21 | cmnix/gioix semi-alpha.
Key insight: AQR MN / L/S-equity / Vanguard MN show strong alpha but
corr 0.35-0.76 with the portfolio - it is already 50% market-neutral
(qspnx), so more MN is not diversifying.
- tests: 59/59 fundlab (resolver gates, query ladder, family dedupe,
ticker regex), 32/32 app, 14/14 data
- fundlab/nport.py: parse the fund's own category/percentage lines,
as-of date, net assets and dollar-valued positions from N-PORT
schedules of investments (handles per-fund and combined multi-fund
family filings; conservative keyword bucketing of positions)
- fundlab/nport_cache/<sym>.json: parsed snapshots for 15 of 16 funds
(raw SOI HTML kept locally, gitignored; source URLs + filing dates in
nport_manifest.json, md5-verified against EDGAR)
- atesx: no current SOI found (Anchor's recent filings cover the Income
fund) - listed with an honest note
- app.py: new 'Fund Lab' tab - pick a fund, see objective, reported
composition (bar + table), rough keyword buckets, top positions, and
the prospectus strategy excerpt
- tests: parser unit tests (section finding, category regex, buckets)
- extract_strategy(): finds the 'Principal Investment Strategies' /
'main investment strategies' section, scores candidates (strategy prose
+3, Q&A heading +2, TOC -5, risk subheading -5, stop-heading -2),
truncates at the next section heading; falls back to the prose after
the objective sentence when no heading exists
- fundinfo --strategy [--refresh]: populates the strategy field of
funds.json from each fund's EDGAR document
- funds.json now carries objective + strategy for 20 funds (the 9
curated index funds have no strategy: their objective is the strategy)
- ticker -> CIK scoped FTS (ciks filter) over the fund's own registrant,
word queries shed Yahoo abbreviations shortest-first
- fuzzy cover gate (SequenceMatcher, sliding window for wrapped names)
+ abbreviation table (Mgd/Glbl/Macr/Abs/Ret/Advtg/...) applied to both
sides of the match
- family-section matching inside multi-fund filings; similarity tiers:
gate (0.85) returns immediately, floor (0.70) is a CIK-scoped fallback
ranked against sibling funds' documents (best sim, then newest)
- objective phrasings: 'seeks to ...', 'seeks investment results ...',
'The Fund's investment objective is ...', '(the Fund) investment
objective is ...' (boilerplate 'is not fundamental' rejected); TOC
headings skipped by trying all heads
- sec_get retries on mid-stream connection drops
- edgar.py: SEC FTS + submissions API; strict cover-gate extraction
(name in title position or (TICKER) on the cover; underlying-reference
names like leveraged wrappers rejected); 4-pass fetch (ticker->CIK
filings, name search, annual reports, ticker search); keyword category
classifier. Returns None rather than a wrong fund's objective.
- fundinfo.py CLI: curated -> cached -> EDGAR resolution into funds.json
- funds_curated.json: human-verified objectives for 13 benchmark-pool
funds (iShares/Vanguard family-trust classes the scraper can't reach)
- tests: 26 checks incl. live EDGAR fetch of VTSAX
- data.py: non-blocking load_bundle(); background watcher thread refreshes
the parquet cache (5s scan, 30s min rebuild cadence); refresh()/
up_to_date()/generation()
- statistics tab: one table per benchmark (vs <label>), plain column names
(beta/alpha/return/vol...), selectable+reorderable stat list in
settings.json
- correlation tab: per-portfolio components-vs-benchmarks +
all-portfolios-vs-benchmarks; numbered columns
- global date range (window radio + start/end boxes) applied to all tabs;
metrics.xcorr(); equity window radio gains YTD/3M/1M
The cache now tracks every file in the data dir (mtime_ns + size) in
.cache/manifest.json. On load, a directory scan is compared against the
manifest:
- changed/added files are re-read and merged into the parquet panels
(one read + one concat + one write per touched panel; new values
win where present, old values kept where the new file is short)
- removed files drop their symbols (and names)
- an up-to-date cache is a ~30 ms memo hit
Measured on the real 4k-symbol set: full build 54 s, refresh of
5 modified + 1 added + 1 removed files 3.4 s. No scan TTL (a scan is
a few ms); a previous 5 s scan cache masked data updates.
Tests: tests/test_data.py (11 checks) added as step 1 of run_tests.sh.
- single spec grammar for symbol and benchmark fields: commas join one
portfolio (MSFT:0.6,V:0.4), spaces separate distinct symbols/portfolios;
both fields accept one or many entries
- benchmarks simulated with the same scheme/cost/tax rules; per-benchmark
beta/alpha columns; after-tax benchmark curves
- global Curve mode (pre/after/both) above the tabs; clean names in
single-curve mode
- live updates: field commits on Enter/blur, page recomputes per rerun;
portfolio+tax sims cached (st.cache_data); plotly.js from CDN (4.6MB ->
browser-cached) with F_INLINE_PLOTLY=1 offline fallback
- chart: legend underneath, solid lines, pan sticks to data edges
(width-preserving), zoom edge-clamped
- inputs persist in settings.json across reloads/restarts/devices
- tests: tests/test_app.py (AppTest) + tests/test_e2e_browser.py
(Playwright) via ./run_tests.sh
