Commit Graph

18 Commits

Author SHA1 Message Date
9f666538c3 Add CASH axis: money-market/cash-equivalents as a return driver
A fund's yield is a near-constant in return space, so OLS puts it in
the intercept (alpha_ann), not in the shv/bil betas (which only
measure rate-CHANGE sensitivity). Cash funds therefore showed no
dominant driver.

- factors: cash_yield() (shv trailing-1y total return = local
  stand-in for the 13-wk T-bill / 0-3m Treasury index), AXES =
  DRIVERS + ['cash'] (alpha_ann / cash_yield, 1.0 = earns the cash
  rate); a display/clustering axis, not a regressor.
- cluster: cash column in the loading matrix; k-means distances use
  the cash-2x-emphasized matrix (a pure cash fund's level axis was
  otherwise swallowed by the low-exposure cloud); labels use raw
  values; label() cash-aware (0.25 threshold, 1.5x runner-up).
- app: cluster view over AXES with the emphasized distance.

The 331-fund 'no dominant driver' grab-bag now splits into
'cash (yield) +0.64' (n=187, MM + ultra-short) and 'cash (yield)
+1.26' (n=59 pure). 123/123 tests.
2026-08-30 13:27:41 -04:00
857dfce19f CEF universe verification: 241/295 parsed + component-sum fractions
- cef_universe_run.py: resumable batch over all 295 CEFs ->
  cef_annual_all.json (241 parsed, actual per-share character).
- cef_rank_all.json: 239-fund ranking, actual char x (t5 + 0.4 vol5);
  surfaces FXBY and DPG beyond the original shortlist.
- share fractions now use the component sum as denominator (always
  sum to 1); dist_tot remains the validity gate. Fixes CEE-style
  111% sums when the NAV-identity fallback total is short.
- server watchdog + app CEF tab; RESEARCH.md updated.
2026-08-28 16:04:19 -04:00
22faeef2ec CEF stage 2b complete: 48/50 shortlist verified (47 parsed + DXYZ zero-dist)
Per-share financial-highlights parser (cef_annual.py) now reconciles
divs+gains+ROC==distributions and the full NAV chain per column for
48 of 50 shortlist funds, from three layout families (inline,
transposed, + repair rules for footnotes/split-decimals/sign-
magnitude/combined-rows). DXYZ is a correct zero-distribution result;
STEW is a documented columnar-stream exception. cef_merged.json
character column now uses actual data. 123/123 tests.
2026-08-28 14:50:46 -04:00
6f8bde4c35 CEF stage 2b: per-share financial-highlights parser (cef_annual.py)
Joint backtracking placement for CEF tables that drop zero columns;
NAV-identity fallback when the printed total row is short; 4 fund-
family layouts verified (Franklin/classic/abrdn/Korea). 16/50 of the
CEF shortlist now have a verified div/gains/ROC split + FY-end
discount series. 17 new parser unit tests (123/123 green).
2026-08-28 07:30:16 -04:00
d1e85026bf CEF pass: universe (SEC report, 973 -> 295 listed) + stage 1 screen + stage 2a character
fundlab/cef_universe.py: SEC 'Closed-End Fund Information' report
(973 active CEFs) + company_tickers CIK join -> 295 listed common
classes (preferreds and 6(c)-converted companies flagged).  Supersedes
the N-2 full-index approach (N-2/A annual updates + 404ing index
paths).  Prices via goget.

fundlab/cef_screen.py: 290 screened - t5/t12, vol5, maxdd5, the 5
crash episodes, 12m payout proxy.  Energy/midstream CEFs (EMO +305%,
SRV +234%, NML +232%) top return AND crash resilience; Voya
Dividend-Premium series (IGD maxDD -16%); EM CEFs volatile + 14-24%
dist; long-dur munis -26..-42%.

fundlab/cef_character.py: 50-fund shortlist, 35-sleeve character +
crude tax_arb = character x (upside + 0.4 x vol).

Tests: 6 new cef checks (105 total).  RESEARCH.md: CEF form facts
(N-2ASR, N-PX, N-23C-3A, BDC caveats) + remaining stage 2b work.
2026-08-27 22:24:12 -04:00
f8409fff7a RESEARCH.md: placement premise (IRA distributions taxed ordinary at withdrawal, UBTI excepted) + constrained-taxable volatility rule 2026-08-27 20:32:26 -04:00
499ae5813d RESEARCH.md: tax strategy landscape (offset/deferral/charitable taxonomy) 2026-08-27 20:15:15 -04:00
895efc9bff Correct ROC placement: ROC defers to the investor's LTCG, like
appreciation - a taxable-account feature, not an IRA one

Correction after user pushback. The earlier note "ROC does not help
the taxable case - the deferral replicates the IRA" was WRONG: a
traditional IRA defers to the ORDINARY rate at withdrawal, while ROC
in a taxable account defers to the LTCG rate on a >1y sale (the
distribution is basis-reducing and reappears inside the shareholder's
own capital gain). Under the premise LTCG rate < future ordinary
rate, ROC - like NAV appreciation - favors the taxable account.
Contrast ordinary income (interest, ordinary divs, STCG): taxed at
the ordinary rate in BOTH accounts, so only the IRA's deferral wins.

taxplan.py:
- _deferred_share(): per fund, share of 5y total return that defers
  to the investor = (NAV change + ROC) / total return, from the
  parsed per-share N-CSR table, max'd with the taxsplit appreciation
  share.
- >= 50% deferred -> location "TAXABLE (defers to LTCG)" (renamed
  from "TAXABLE (accrues)"); 10-50% ROC in distributions -> note.
- Merger-arb cap 0.35 -> 0.50: HMEZX's per-share table (52% NII /
  30% gains / 18% ROC over 5y) refutes "mostly STCG" - HMEZX/MERVX
  are now MIXED (check 1099), not clean IRA.
- RESEARCH.md: corrected placement write-up, incl. the distinction
  between the tax question and the fund-quality question (heavy ROC
  can mean principal erosion - PGSIX NAV -34%/5y - which affects
  selection, not the optimal account).

App: order dict + filter updated to the new location name.
Tests: merger-arb expectation updated to MIXED; 2 new checks for
the ROC upgrade/note logic. 99/99 fundlab + 32/32 app.
2026-08-27 17:11:55 -04:00
620e2499ac RIC-vs-pass-through verification + return-of-capital detector
fundlab/rocdetect.py: for a fund's latest N-CSR annual report,
parses the per-share "Financial Highlights" distribution table
(NII / gains / (tax) return of capital / total), with
whitespace normalization, dollar-basis and fiscal-year-column
rejection, per-column arithmetic placement of short rows
(NII+gains+ROC==total), and family-report disambiguation by
current-NAV match. SEC 503 backoff, per-fund cache in roc_cache/.

Verified over the 38 core funds:
- HMEZX: 18.3% of 5y distributions were return of capital
  (0.62 + 0.37 per share in 2 of 5 fiscal years) - reinforces
  the IRA placement.
- 17 funds verified zero ROC (AFLIX, ANGLX, ATESX, BTMIX, the EV
  macro funds, FHCOX/FHMIX, LCORX/LCRIX, PMAIX/PMFKX, SVARX,
  WMNUX).
- 14 unverified (non-standard report layouts - prototype parser),
  PMORX/QCMMRX tickers not EDGAR-resolvable.

Also verified (form-type history on EDGAR): CVSIX, QSPNX, MBXIX,
ATRFX, ATESX, EAGMX, PBAIX, WISEX, EBSAX all file NPORT-P + N-CSR
+ 497 -> registered RICs, not pass-throughs; the whole 497-filer
universe is registered, so no candidate can pass through capital
losses. Proof case for ROC: PGSIX per-share table shows a tax
return-of-capital line and NAV eroded 11.87 -> 7.88 (-34%) over
5 fiscal years while paying out every year.

RESEARCH.md: full write-up. No app change (analysis artifact).
2026-08-27 16:04:59 -04:00
9458e316cb Price-appreciation vs payout split (fundlab/taxsplit.py)
Follow-up to the tax-location plan: the taxplan score only measured
DISTRIBUTION character. The user rightly noted that NAV appreciation
is also a capital gain (LTCG on a >1y sale). The fund price files
carry both series - Close = raw NAV with distributions paid out,
Adj Close = total return reinvested - so the split is computable
directly per fund (5y window + most-recent-12m payout).

Findings:
- ACCUMULATORS (>=50% of 5y return is price appreciation) get a new
  location "TAXABLE (accrues)": MBXIX 76% (0% payout 12m), ATESX 66%,
  LAMHX 62%, CVSIX 61%, candidate PBAIX 60% (0% payout 12m). For
  these the taxable account's LTCG-on-sale benefit is the dominant
  tax event.
- PAY-OUT funds: HMEZX (99% of return distributed - the STCG merger-
  arb case), MERVX, COSIX, PMORX, SVARX, SCFZX, DMSZX, munis, credit.
  IRA placement stands.
- Data artifacts caught: JLPSX/QSPNX one-time NAV gap events ~2022
  (special distribution or reorg) skew the 5y payout average; the
  12m payout column reflects current behavior. QCMMRX (MMF) series
  is not NAV-based - flagged.

App: tax-location expander gains 5y price / 5y payout / 12m payout
columns and the "TAXABLE (accrues)" filter. RESEARCH.md documents
the capital-loss question: registered RICs cannot distribute net
capital losses; the usable benefit is the fund's internal harvest
reserve (low capital-gain distributions after up-years), which needs
N-CSR/1099 history to verify. 97/32 suites green.
2026-08-27 14:34:42 -04:00
8a9ca25750 Tax-location plan: taxable account vs IRA per fund
fundlab/taxplan.py categorizes the 16-fund shortlist, the 22 N-PORT
cross-checked candidates, and all 250 screened candidates by the
expected CHARACTER of their distributions, given the user's premise
that the current LTCG rate < the post-retirement ordinary rate:

  qualified div + LTCG      -> TAXABLE  (score >= 0.60)
  tax-exempt (munis)        -> TAXABLE
  ordinary / STCG / REIT    -> IRA      (score <= 0.35)
  in between                -> MIXED (pull the 1099-DIV)
  cash                      -> FLEXIBLE

score = estimated share of distributions that are tax-favorable,
from three tiers of ground truth: N-PORT keyword buckets (16), SEC
assetCat/issuerCat buckets (22), sleeve loadings (250), with a
sleeve fallback when the keyword parser left >50% of a book
unclassified, and a manual override for the Leuthold wrappers
(91.7% Leuthold Core ETF, no return history yet).

Key findings:
- shortlist: TAXABLE = ATESX, JLPSX, LAMHX, LCORX, LCRIX (equity);
  IRA = ATRFX, COSIX, CVSIX, PMORX, SVARX, EAGMX/EGRSX;
  MIXED = MBXIX, QSPNX, PMAIX/PMFKX (same fund, two classes)
- cross-checked: 4 munis -> TAXABLE; HMEZX + MERVX are the merger-
  arb trap - equity-looking books whose distributions are mostly
  SHORT-TERM gains -> IRA
- candidates: 109 munis TAXABLE, 127 IRA, 6 equity TAXABLE, 7 MIXED

App: Fund Lab "Tax location" expander. Output:
fundlab/taxplan_results.json. Tests: test_taxplan() (9 checks).
97/32 suites green.
2026-08-27 14:08:15 -04:00
d0ae2ec348 Drawdown-resilience screen: who was positive when equities crashed
fundlab/drawdown.py detects the severe equity drawdown scenarios from
the index (IVV) rather than hard-coding them: one worst peak->trough
per calendar year since 2022, min depth 8% (a 10% floor would silently
drop the 2023 rate shock at -9.9% and the 2024 Aug-5 dip at -8.4%).
Detected: 2022 bear mkt (-24.5%), 2023 rate shock (-9.9%), 2024 vol
spike (-8.4%), 2025 tariff crash (-18.8%), 2026 Q1 drawdown (-8.9%).

For each of the 2,384 screened funds it computes that fund's own-NAV
return over each peak->trough window (first print after the peak to
the last print on/before the trough) and ranks the 250 CANDIDATEs by
# scenarios positive.

Key finding: positive in all 5 scenarios = only 7 funds, all
ultra-short/cash (BILS, QCMMRX, PULS, FHCOX, FHMIX, SAFEX, COIAX).
Drawdown resilience at the top tier is a duration property, not alpha.
The interesting tier is 4/5 WITH real 5y alpha: HMEZX merger arb
(+1.5% 2022, +3.1% 2023, t5 +7.1), MERVX, CBHCX market-neutral, SCFZX
securitized credit (t5 +8.4), ENIAX (t5 +10.1), WMNUX (t5 +6.9), RCTIX.

App: Fund Lab "Drawdown resilience" expander (scenario table +
candidate table). Output: fundlab/drawdown_results.json.
Tests: test_drawdown() added (4 checks). 88/32 suites green.
2026-08-27 13:29:01 -04:00
a09861f39f N-PORT cross-check: verify top candidates' actual holdings
fundlab/xcheck.py - for each screen candidate, resolve the fund's OWN
registrant CIK (browse-edgar; the 497-cover CIK is the family/trust),
get the exact series name for the ticker (the only reliable
disambiguator between sibling funds), walk the 4 most recent NPORT-P
filing dates, and parse holdings from the interactive NPORT XML
(primary_doc.xml at the accession root - NOT the XSL-rendered view the
submissions API points at). Exact seriesName match > best htm exhibit
parse. Buckets from the authoritative assetCat+issuerCat codes (ABS-O,
ABS-CBDO, DBT+UST/CORP/MUN/NUSS, LON, STIV, RA, EC+RF=fund, ...), not
position-name keywords. Resumable; raw filings cached under
nport_cache/raw/ (gitignored).

nport.py - _SECTION gains the "INVESTMENT PORTFOLIO (unaudited)"
variant (NPORT-EX Sch-F files); find_section/build gain a frac
token-tolerance param (Yahoo names drift from filing names); CMBS/ABS
bucket gains CLO/CBDO/DAC terms.

app Fund Lab - "N-PORT cross-check" expander: per-candidate table
(as-of, n, t5, top code-bucket, #1 position) + per-fund holdings
detail.

RESEARCH.md - cross-check verdicts. 21/22 resolved (qcmmrx is an MMF,
no holdings). The screen's top names are REAL:
- hmezx/mervx = genuine merger arb (equity in deal targets + escrow)
- egrix = 100% wrapper in one macro managed portfolio (underlying not
  NPORT-disclosed); etsix = fund of EV internal multi-strat accounts
- wmnux = discounted/zero-coupon corporate bonds + equity swaps (the
  "equity names" are bond issuers/swap underlyings)
- scfzx/rctix/aflix = securitized credit/CLO/distressed/levered loans
- hicox/fhmix/usmsx/btmix (munis), aguax/femdx (EM sovereign), anglx
  (agency MBS), lpxax (rotated out of prefs into bank/financial debt)
  = genuine missing-factor exposures the 35-sleeve model lacks
- fhcox/dultx/safex = short-duration carry (a short-duration sleeve
  would explain them)

tests/test_fundlab.py - test_xcheck (14 checks): parse_interactive,
code buckets, name-match normalization, series-name disambiguation.
Also: untrack fundlab/streamlit.log; gitignore raw/ + xcheck_run.log.

84 fundlab / 32 app / 14 data tests pass.
2026-08-27 12:43:17 -04:00
f68b239b9a Factor screen (v2, 35 drivers) + return-driver clusters
factors.py: full OLS of all 2,384 funds on a 35-driver basis
(overinclusive, no portfolio-corr screening - corr is a replacement
signal, not a rejection). v1's 21 + lqd/hyg/prefs/emb/tip/shy/vtv/
8 sectors/CTA/commodities. Basis fixes: drop vea/vug (dupes of
efa/qqq), drop finux (TERMINATED 2017 - silently zeroed the
complete-case mask; v1 forward selection never hit this), drop bil
(shv/bil near-null -> offsetting shv+64/bil-54 noise fits),
residualize vblix on ivv+tlt (pure vol axis), ridge 0.02.

cluster.py: hierarchical tree saved but fixed-k cuts degenerate
(most funds are blends -> one 2250-fund blob); k-means++ (deterministic)
is the useful grouping, re-run live in-app for any k.

app: Return-driver clusters expander (k slider 10-60, summary table,
member table sorted by alpha-t). Findings at k=30 in RESEARCH.md.
2026-08-27 08:35:42 -04:00
4fdd5fcfd1 Overnight comprehensive screen COMPLETE: 2,384 funds, 250 candidates, RESEARCH.md results + next iterations 2026-08-27 06:56:33 -04:00
11f569b2ab Watchdog: pidfile liveness (pgrep -f was fooled by shells quoting the pattern); per-batch download fault isolation 2026-08-26 22:09:20 -04:00
eac61683f6 RESEARCH.md: live progress notes + /tmp-cleanup infra lesson 2026-08-26 21:41:07 -04:00
a94eac6545 Overnight comprehensive screen: drop the name pre-filter, screen all 2,384 funds
fundlab/RESEARCH.md - running research log: sources that work/die
  (full-index = discovery workhorse; browse-edgar JS-dead;
  investment-company-tickers.json nonexistent; company_tickers.json
  useless for OTC; Yahoo crumb throttled but chart API fine), 13
  hard-won learnings (OTC funds report exchange 'Nasdaq' -> use
  instrumentType; 497 SGML cover uses UNCLOSED line-based tags ->
  parse before tag-stripping; full-index columns drift -> regex the
  line; one quarter != universe -> 4-qtr union; accession paths
  relative to /Archives/ not /Archives/edgar/data/; family CIKs
  repeat -> dedupe by series name; portfolio is 50% MN so MN alpha
  funds are 'correlated', not diversifying).

fundlab/overnight.py - resumable all-stage pipeline (kill/restart safe):
  verify (Yahoo chart per non-local ticker, 4-thread, 429 backoff,
  local tickers measured from CSV row counts) -> select (pure
  select_rows: MUTUALFUND, >=5y, one longest-history class per series
  name, alpha_name as TAG not filter) -> download (goget in 200-sym
  batches) -> screen (streamed, skip-already-done) -> finalize
  (verdict counts + candidates the v1 name-filter would have missed).

Universe: 10,372 class tickers -> 10,260 verified -> 2,384 funds
(407 local, 1,977 external; only 54 match the alpha name pattern -
the v2 point is to screen the other 2,330).

app: alpha table now dedupes by sym with search_all.json winning
(comprehensive superset).
tests: select_rows unit tests (ETF drop, short-history drop, class
collapse, name tagging). 70/70 fundlab.
2026-08-26 21:33:11 -04:00