Financial analysis tools
Greg Pomerantz
9263283995
overrides/corrections/{SYM}.json (git-tracked, with as_of/source/note)
holds remove/replace/add ops on the dividend and capital-gain series.
data.py applies them on top of whatever the data root (or the frozen
snapshot) provides, in both the full build and the incremental refresh
path, and the corrections dir joins the cache manifest so a change
invalidates the cache. A goget re-download of the base CSV can never
clobber a confirmed correction. Format and usage documented in data.py.
|
||
|---|---|---|
| fundlab | ||
| overrides | ||
| reports | ||
| scripts | ||
| tests | ||
| .gitignore | ||
| adx-split.csv | ||
| aef-split.csv | ||
| app.py | ||
| asa-split.csv | ||
| brw-split.csv | ||
| bto-split.csv | ||
| chart_widget.py | ||
| clm-split.csv | ||
| crf-split.csv | ||
| data.py | ||
| evg-split.csv | ||
| families.py | ||
| fxby-split.csv | ||
| grf-split.csv | ||
| herz-split.csv | ||
| iaf-split.csv | ||
| kf-split.csv | ||
| mci-split.csv | ||
| metrics.py | ||
| mxf-split.csv | ||
| nro-split.csv | ||
| peo-split.csv | ||
| portfolio.py | ||
| portfolios.json | ||
| portfolios.py | ||
| README.md | ||
| requirements.txt | ||
| run_tests.sh | ||
| run.sh | ||
| rvt-split.csv | ||
| saba-split.csv | ||
| swz-split.csv | ||
| tax.py | ||
| tyg-split.csv | ||
| utf-split.csv | ||
| vlt-split.csv | ||
| ztr-split.csv | ||
Stock & Portfolio Analyzer
Interactive tool for analyzing individual securities and portfolios
against local Yahoo Finance dumps (~/prog/fin/stocks, ~4k symbols).
Quick start
python3 -m venv .venv
.venv/bin/pip install -r requirements.txt
./run.sh # serves the UI on the fixed port 8599 (http://localhost:8599)
First run builds a parquet cache in .cache/ (~1 min for 4k symbols);
later runs load in well under a second. The cache tracks the data dir
per-file (mtime + size in .cache/manifest.json), so when the download
is updated, only the changed/added/removed symbols are re-read — a
partial refresh takes seconds instead of a full ~1 min rebuild.
Modules
| Module | Purpose |
|---|---|
data.py |
Ingest {sym}-history/dividend/capitalGain.csv -> cached parquet panels (date x symbol), with manifest-based incremental refresh when the data dir changes. Adj Close already includes distributions, so it drives pre-tax total returns. |
metrics.py |
Total/annualized return, vol, Sharpe, Sortino, max drawdown, Calmar, CAPM beta/alpha. Pure pandas, all transparent. |
portfolio.py |
Weighted portfolios with drift and periodic rebalancing to target weights (1W/1ME/QE/YE), one-way cost in bps. Spec grammar: commas join the elements of ONE portfolio (SYM or SYM:w, bare = equal weight), spaces separate DISTINCT symbols/portfolios (parse_items). |
tax.py |
Simplified DAS after-tax engine: FIFO lots, 365-day long/short split, separate LT/ST/dividend rates. Headline curve = what you keep if you sell everything today (unrealized gains taxed daily by lot age). |
chart_widget.py |
Self-contained plotly.js chart in an iframe: mouse zoom/pan, x clamped to the data, view edges snapped to first/last data points with day-precise labels, y tight-fit, every line re-based to 1.0 at the left edge. |
portfolios.py |
Saved portfolio definitions in portfolios.json (name, spec, scheme, cost). |
settings.json |
Persisted UI inputs (symbol/benchmark specs, scheme, costs, tax rates, period, curve/window mode) — restored on every page load and server restart; delete to reset. |
app.py |
Streamlit UI: single "symbol or portfolio" spec field (page updates as soon as the input is valid; unknown symbols get click-to-fix "did you mean" suggestions) + a benchmark box with the same grammar (one benchmark per line; a line is a single symbol or a comma-joined portfolio, simulated with the same scheme/cost/tax rules — pre- and after-tax curves, first one drives beta/alpha), scheme/costs/tax rates, save + load/compare/delete portfolios (overlaid pre/after-tax curves), curve toggle (both / pre-tax only / after-tax only), stats table, allocation, per-year tax detail. |
Development
- Run:
./run.sh→ http://localhost:8599 (fixed port; no-ops if a server is already running). The chart loads plotly.js from a CDN; for fully offline use setF_INLINE_PLOTLY=1inrun.sh. - Test:
./run_tests.shtests/test_app.py— app-level tests via Streamlit AppTest (no browser). Memory: one data bundle is ~2.3 GB, so this process keeps at most ONE AppTest alive (see its header comment).tests/test_e2e_browser.py— Playwright + headless Chromium driving the real page with real keystrokes; needs the server running on 8599. One-time setup:.venv/bin/pip install playwrightand.venv/bin/python -m playwright install chromium.
- Gotchas
- Streamlit caches imported modules per process: restart the server
after editing any
.py(kill the old one first —run.shrefuses to double-start). st.cache_datacaches the portfolio + tax simulations: they recompute only when symbols/scheme/cost/tax rates change, not on window or curve toggles.settings.json(gitignored) persists UI inputs across reloads and restarts; delete it to reset. Saved portfolios live inportfolios.json.- Data cache:
.cache/*.parquet; rebuild via the sidebar checkbox (first build ~1 min for ~4k symbols).
- Streamlit caches imported modules per process: restart the server
after editing any
Known simplifications (roadmap)
- No loss carryover or carryforward across years; no wash-sale rules.
- Distributed capital gains taxed entirely at the long-term rate.
- Single (federal-like) tax bracket; no state taxes, no AMT.
- Equal treatment of benchmark for beta/alpha (CAPM, rf = 0 by default).
Ideas: vectorbt sweeps over rebalance schemes, NiceGUI/Textual frontend, empyrical-reloaded metrics, monthly (not yearly) loss netting, tax-loss harvesting simulation.