Financial analysis tools
Greg Pomerantz
a09861f39f
fundlab/xcheck.py - for each screen candidate, resolve the fund's OWN registrant CIK (browse-edgar; the 497-cover CIK is the family/trust), get the exact series name for the ticker (the only reliable disambiguator between sibling funds), walk the 4 most recent NPORT-P filing dates, and parse holdings from the interactive NPORT XML (primary_doc.xml at the accession root - NOT the XSL-rendered view the submissions API points at). Exact seriesName match > best htm exhibit parse. Buckets from the authoritative assetCat+issuerCat codes (ABS-O, ABS-CBDO, DBT+UST/CORP/MUN/NUSS, LON, STIV, RA, EC+RF=fund, ...), not position-name keywords. Resumable; raw filings cached under nport_cache/raw/ (gitignored). nport.py - _SECTION gains the "INVESTMENT PORTFOLIO (unaudited)" variant (NPORT-EX Sch-F files); find_section/build gain a frac token-tolerance param (Yahoo names drift from filing names); CMBS/ABS bucket gains CLO/CBDO/DAC terms. app Fund Lab - "N-PORT cross-check" expander: per-candidate table (as-of, n, t5, top code-bucket, #1 position) + per-fund holdings detail. RESEARCH.md - cross-check verdicts. 21/22 resolved (qcmmrx is an MMF, no holdings). The screen's top names are REAL: - hmezx/mervx = genuine merger arb (equity in deal targets + escrow) - egrix = 100% wrapper in one macro managed portfolio (underlying not NPORT-disclosed); etsix = fund of EV internal multi-strat accounts - wmnux = discounted/zero-coupon corporate bonds + equity swaps (the "equity names" are bond issuers/swap underlyings) - scfzx/rctix/aflix = securitized credit/CLO/distressed/levered loans - hicox/fhmix/usmsx/btmix (munis), aguax/femdx (EM sovereign), anglx (agency MBS), lpxax (rotated out of prefs into bank/financial debt) = genuine missing-factor exposures the 35-sleeve model lacks - fhcox/dultx/safex = short-duration carry (a short-duration sleeve would explain them) tests/test_fundlab.py - test_xcheck (14 checks): parse_interactive, code buckets, name-match normalization, series-name disambiguation. Also: untrack fundlab/streamlit.log; gitignore raw/ + xcheck_run.log. 84 fundlab / 32 app / 14 data tests pass. |
||
|---|---|---|
| fundlab | ||
| tests | ||
| .gitignore | ||
| app.py | ||
| chart_widget.py | ||
| data.py | ||
| families.py | ||
| metrics.py | ||
| portfolio.py | ||
| portfolios.json | ||
| portfolios.py | ||
| README.md | ||
| requirements.txt | ||
| run_tests.sh | ||
| run.sh | ||
| tax.py | ||
Stock & Portfolio Analyzer
Interactive tool for analyzing individual securities and portfolios
against local Yahoo Finance dumps (~/prog/fin/stocks, ~4k symbols).
Quick start
python3 -m venv .venv
.venv/bin/pip install -r requirements.txt
./run.sh # serves the UI on the fixed port 8599 (http://localhost:8599)
First run builds a parquet cache in .cache/ (~1 min for 4k symbols);
later runs load in well under a second. The cache tracks the data dir
per-file (mtime + size in .cache/manifest.json), so when the download
is updated, only the changed/added/removed symbols are re-read — a
partial refresh takes seconds instead of a full ~1 min rebuild.
Modules
| Module | Purpose |
|---|---|
data.py |
Ingest {sym}-history/dividend/capitalGain.csv -> cached parquet panels (date x symbol), with manifest-based incremental refresh when the data dir changes. Adj Close already includes distributions, so it drives pre-tax total returns. |
metrics.py |
Total/annualized return, vol, Sharpe, Sortino, max drawdown, Calmar, CAPM beta/alpha. Pure pandas, all transparent. |
portfolio.py |
Weighted portfolios with drift and periodic rebalancing to target weights (1W/1ME/QE/YE), one-way cost in bps. Spec grammar: commas join the elements of ONE portfolio (SYM or SYM:w, bare = equal weight), spaces separate DISTINCT symbols/portfolios (parse_items). |
tax.py |
Simplified DAS after-tax engine: FIFO lots, 365-day long/short split, separate LT/ST/dividend rates. Headline curve = what you keep if you sell everything today (unrealized gains taxed daily by lot age). |
chart_widget.py |
Self-contained plotly.js chart in an iframe: mouse zoom/pan, x clamped to the data, view edges snapped to first/last data points with day-precise labels, y tight-fit, every line re-based to 1.0 at the left edge. |
portfolios.py |
Saved portfolio definitions in portfolios.json (name, spec, scheme, cost). |
settings.json |
Persisted UI inputs (symbol/benchmark specs, scheme, costs, tax rates, period, curve/window mode) — restored on every page load and server restart; delete to reset. |
app.py |
Streamlit UI: single "symbol or portfolio" spec field (page updates as soon as the input is valid; unknown symbols get click-to-fix "did you mean" suggestions) + a benchmark box with the same grammar (one benchmark per line; a line is a single symbol or a comma-joined portfolio, simulated with the same scheme/cost/tax rules — pre- and after-tax curves, first one drives beta/alpha), scheme/costs/tax rates, save + load/compare/delete portfolios (overlaid pre/after-tax curves), curve toggle (both / pre-tax only / after-tax only), stats table, allocation, per-year tax detail. |
Development
- Run:
./run.sh→ http://localhost:8599 (fixed port; no-ops if a server is already running). The chart loads plotly.js from a CDN; for fully offline use setF_INLINE_PLOTLY=1inrun.sh. - Test:
./run_tests.shtests/test_app.py— app-level tests via Streamlit AppTest (no browser). Memory: one data bundle is ~2.3 GB, so this process keeps at most ONE AppTest alive (see its header comment).tests/test_e2e_browser.py— Playwright + headless Chromium driving the real page with real keystrokes; needs the server running on 8599. One-time setup:.venv/bin/pip install playwrightand.venv/bin/python -m playwright install chromium.
- Gotchas
- Streamlit caches imported modules per process: restart the server
after editing any
.py(kill the old one first —run.shrefuses to double-start). st.cache_datacaches the portfolio + tax simulations: they recompute only when symbols/scheme/cost/tax rates change, not on window or curve toggles.settings.json(gitignored) persists UI inputs across reloads and restarts; delete it to reset. Saved portfolios live inportfolios.json.- Data cache:
.cache/*.parquet; rebuild via the sidebar checkbox (first build ~1 min for ~4k symbols).
- Streamlit caches imported modules per process: restart the server
after editing any
Known simplifications (roadmap)
- No loss carryover or carryforward across years; no wash-sale rules.
- Distributed capital gains taxed entirely at the long-term rate.
- Single (federal-like) tax bracket; no state taxes, no AMT.
- Equal treatment of benchmark for beta/alpha (CAPM, rf = 0 by default).
Ideas: vectorbt sweeps over rebalance schemes, NiceGUI/Textual frontend, empyrical-reloaded metrics, monthly (not yearly) loss netting, tax-loss harvesting simulation.