Fix after-tax model: raw prices + reinvestment (was 2.7x over-taxing)
The old model valued holdings in ADJ (total-return index) units but computed distribution flows as raw per-share dollars — so every distribution, and its tax, was overstated by the raw/adj ratio (JLPSX: 30.10/11.25 = 2.7x; the Dec-2020 cap-gain tax showed as 12.1% of the account instead of the true 4.5%). The wiggle in the after-tax curve was this bug, not a convention issue. tax.py now: - values holdings in RAW share units on close prices (bundle.close); - receives the per-share distribution on its event date, pays the tax (recorded in TaxResult.taxes), and reinvests the after-tax remainder at the same day's raw close — the tax's effect lives in the (smaller) reinvested units and is NOT also deducted from cash (double-count caught and fixed in review); - recomputes market value after the reinvestment so equity[t] is the post-event liquidation value. With the fix, the 'as-if-liquidated' equity on JLPSX's ex-div day drops by exactly the true tax cost (4.72% vs 12.1% before); the -22.9% price drop is offset by the distribution kept. Also: - app.py passes bundle.close to the after-tax model (pre-tax portfolio_returns still uses adj); - JLPSX/JLPYX: the 2020-12-11 6.824 capital-gain distribution is moved to the true ex-div date 2020-12-14 (remove/add correction ops), so the reinvestment prices at the post-drop close; - tests/test_tax.py: 8 synthetic regression tests (tax magnitude, reinvestment MV, no double-count, ex-div equity step, per-component rates); run_tests.sh now runs it.
This commit is contained in:
parent
43ddd3ec7f
commit
576d9fba97
2
app.py
2
app.py
|
|
@ -289,7 +289,7 @@ def _compute_portfolio(w_key: tuple, scheme: str | None, cost_bps: float,
|
|||
root: str, gen: int):
|
||||
w = dict(w_key)
|
||||
r = portfolio_returns(bundle.adj, w, rebalance=scheme, cost_bps=cost_bps, start=start)
|
||||
t = after_tax_portfolio(bundle.adj, bundle.div, bundle.capg, w, rebalance=scheme,
|
||||
t = after_tax_portfolio(bundle.close, bundle.div, bundle.capg, w, rebalance=scheme,
|
||||
cost_bps=cost_bps, lt_rate=lt, st_rate=st_r,
|
||||
div_rate=div, start=start)
|
||||
return r, t
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
{
|
||||
"symbol": "jlpsx",
|
||||
"name": "JPMorgan U.S. Large Cap Core Plus Fund (I shares)",
|
||||
"note": "Yahoo listed the year-end capital-gain distribution in BOTH the dividend and capitalGains event files. Expressed as layout-agnostic invariants because a 2026-08 re-download shows Yahoo no longer returns capitalGain events for this fund at all (the dividend stream still carries the year-end rows): 'dedup' keeps at most one copy of each (date, amount) (the capitalGain copy when both files carry it; the lone dividend copy otherwise); 'drop_capg_copy' drops the spurious capitalGain row on a date where the dividend row is the true distribution (any amount). Official per-share data (JPMorgan 497, filed 2025-11-03, Class I, FYE June 30): FY2025 1.37, FY2024 2.03, FY2023 4.86, FY2022 6.78, FY2021 6.89 - local sums after dedup match every one (1.368, 2.029, 4.859, 6.785, 6.889). 2006-12-15 (div 0.207 vs capg 0.141) and 2019-08-08 (div 4.138 vs capg 4.077) keep the dividend amount per the pattern confirmed on all other verified same-date pairs; pre-2012 years have no official totals (the 2021 497 is delisted from SEC Archives) - mechanism-inferred. 'history' patches the 2020-12-11 Adj Close cell: Yahoo dated the 6.824 distribution on the 12-11 record date but the market went ex-div on 2020-12-14 (close 30.10 -> 23.22, i.e. -6.88), so the raw Adj Close column spiked +29.3% on 12-11 and snapped back -22.9% on 12-14; setting 12-11 to the 12-10 value (11.247889) removes the bogus 3-day wiggle (cumulative return unchanged). adj-close repair: Yahoo dated a distribution on 2020-12-11 (record date) but the price went ex-div on 2020-12-14 (close 30.10 -> 23.22, drop ~6.824); the raw Adj Close column spiked +29.3% on 2020-12-11 and snapped back later. Cells in [2020-12-11, 2020-12-14) rescaled by (1-f) so the adjusted path is smooth; cumulative returns are unchanged. Mechanism: scripts/fix_adj_misalign.py, scan: reports/adj_misalign/scan.md.",
|
||||
"note": "Yahoo listed the year-end capital-gain distribution in BOTH the dividend and capitalGains event files. Expressed as layout-agnostic invariants because a 2026-08 re-download shows Yahoo no longer returns capitalGain events for this fund at all (the dividend stream still carries the year-end rows): 'dedup' keeps at most one copy of each (date, amount) (the capitalGain copy when both files carry it; the lone dividend copy otherwise); 'drop_capg_copy' drops the spurious capitalGain row on a date where the dividend row is the true distribution (any amount). Official per-share data (JPMorgan 497, filed 2025-11-03, Class I, FYE June 30): FY2025 1.37, FY2024 2.03, FY2023 4.86, FY2022 6.78, FY2021 6.89 - local sums after dedup match every one (1.368, 2.029, 4.859, 6.785, 6.889). 2006-12-15 (div 0.207 vs capg 0.141) and 2019-08-08 (div 4.138 vs capg 4.077) keep the dividend amount per the pattern confirmed on all other verified same-date pairs; pre-2012 years have no official totals (the 2021 497 is delisted from SEC Archives) - mechanism-inferred. 'history' patches the 2020-12-11 Adj Close cell: Yahoo dated the 6.824 distribution on the 12-11 record date but the market went ex-div on 2020-12-14 (close 30.10 -> 23.22, i.e. -6.88), so the raw Adj Close column spiked +29.3% on 12-11 and snapped back -22.9% on 12-14; setting 12-11 to the 12-10 value (11.247889) removes the bogus 3-day wiggle (cumulative return unchanged). adj-close repair: Yahoo dated a distribution on 2020-12-11 (record date) but the price went ex-div on 2020-12-14 (close 30.10 -> 23.22, drop ~6.824); the raw Adj Close column spiked +29.3% on 2020-12-11 and snapped back later. Cells in [2020-12-11, 2020-12-14) rescaled by (1-f) so the adjusted path is smooth; cumulative returns are unchanged. Mechanism: scripts/fix_adj_misalign.py, scan: reports/adj_misalign/scan.md. Distribution date moved 2020-12-11 -> 2020-12-14 (true ex-div date, close 30.10 -> 23.22 for JLPSX / 30.29 -> 23.40 for JLPYX): Yahoo dated the 6.824 capital-gain distribution on the 12-11 record date, but the after-tax model reinvests at the event date's raw close, so the cash flow must sit on the day the price actually dropped. Ops: remove both file copies on 12-11, add the capitalGains copy on 12-14 (kept in capitalGains so it is taxed at the cap-gain rate, matching the 497's CG 6.82 / NII 0.07 split).",
|
||||
"verified": [
|
||||
"FY2021-FY2025 exact match against 497 filed 2025-11-03 (accession 0001193125-25-261054, CIK 0000763852)"
|
||||
],
|
||||
|
|
@ -42,10 +42,6 @@
|
|||
"2019-12-12",
|
||||
1.517
|
||||
],
|
||||
[
|
||||
"2020-12-11",
|
||||
6.824
|
||||
],
|
||||
[
|
||||
"2021-12-13",
|
||||
6.785
|
||||
|
|
@ -71,5 +67,27 @@
|
|||
"2020-12-11": {
|
||||
"Adj Close": 11.247889
|
||||
}
|
||||
},
|
||||
"dividends": {
|
||||
"remove": [
|
||||
[
|
||||
"2020-12-11",
|
||||
6.824
|
||||
]
|
||||
]
|
||||
},
|
||||
"capitalGains": {
|
||||
"remove": [
|
||||
[
|
||||
"2020-12-11",
|
||||
6.824
|
||||
]
|
||||
],
|
||||
"add": [
|
||||
[
|
||||
"2020-12-14",
|
||||
6.824
|
||||
]
|
||||
]
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -13,10 +13,6 @@
|
|||
"2019-12-12",
|
||||
1.517
|
||||
],
|
||||
[
|
||||
"2020-12-11",
|
||||
6.824
|
||||
],
|
||||
[
|
||||
"2021-12-13",
|
||||
6.785
|
||||
|
|
@ -38,10 +34,32 @@
|
|||
0.656
|
||||
]
|
||||
],
|
||||
"note": "Bulk correction from the double-listing scan (scripts/scan_double_listing.py): Yahoo listed this distribution in both the dividend and capitalGains event files (same date, same amount); one copy is spurious. The 'dedup' invariant keeps at most one copy, layout-agnostically. Mechanism verified against official filings for JLPSX/GDEUX/GSOUX/FAEVX/CVSIX/FZAGX; mechanism-inferred for this symbol (no per-fund official check). Revert the entry if a future official cross-check shows both rows were real. adj-close repair: Yahoo dated a distribution on 2020-12-11 (record date) but the price went ex-div on 2020-12-14 (close 30.29 -> 23.40, drop ~6.824); the raw Adj Close column spiked +29.1% on 2020-12-11 and snapped back later. Cells in [2020-12-11, 2020-12-14) rescaled by (1-f) so the adjusted path is smooth; cumulative returns are unchanged. Mechanism: scripts/fix_adj_misalign.py, scan: reports/adj_misalign/scan.md.",
|
||||
"note": "Bulk correction from the double-listing scan (scripts/scan_double_listing.py): Yahoo listed this distribution in both the dividend and capitalGains event files (same date, same amount); one copy is spurious. The 'dedup' invariant keeps at most one copy, layout-agnostically. Mechanism verified against official filings for JLPSX/GDEUX/GSOUX/FAEVX/CVSIX/FZAGX; mechanism-inferred for this symbol (no per-fund official check). Revert the entry if a future official cross-check shows both rows were real. adj-close repair: Yahoo dated a distribution on 2020-12-11 (record date) but the price went ex-div on 2020-12-14 (close 30.29 -> 23.40, drop ~6.824); the raw Adj Close column spiked +29.1% on 2020-12-11 and snapped back later. Cells in [2020-12-11, 2020-12-14) rescaled by (1-f) so the adjusted path is smooth; cumulative returns are unchanged. Mechanism: scripts/fix_adj_misalign.py, scan: reports/adj_misalign/scan.md. Distribution date moved 2020-12-11 -> 2020-12-14 (true ex-div date, close 30.10 -> 23.22 for JLPSX / 30.29 -> 23.40 for JLPYX): Yahoo dated the 6.824 capital-gain distribution on the 12-11 record date, but the after-tax model reinvests at the event date's raw close, so the cash flow must sit on the day the price actually dropped. Ops: remove both file copies on 12-11, add the capitalGains copy on 12-14 (kept in capitalGains so it is taxed at the cap-gain rate, matching the 497's CG 6.82 / NII 0.07 split).",
|
||||
"history": {
|
||||
"2020-12-11": {
|
||||
"Adj Close": 11.400895
|
||||
}
|
||||
},
|
||||
"dividends": {
|
||||
"remove": [
|
||||
[
|
||||
"2020-12-11",
|
||||
6.824
|
||||
]
|
||||
]
|
||||
},
|
||||
"capitalGains": {
|
||||
"remove": [
|
||||
[
|
||||
"2020-12-11",
|
||||
6.824
|
||||
]
|
||||
],
|
||||
"add": [
|
||||
[
|
||||
"2020-12-14",
|
||||
6.824
|
||||
]
|
||||
]
|
||||
}
|
||||
}
|
||||
|
|
|
|||
10
run_tests.sh
10
run_tests.sh
|
|
@ -9,15 +9,19 @@ cd "$(dirname "$0")"
|
|||
|
||||
fail=0
|
||||
|
||||
echo "=== 1/3 data cache tests (incremental refresh) ==="
|
||||
echo "=== 1/4 data cache tests (incremental refresh) ==="
|
||||
.venv/bin/python tests/test_data.py || fail=1
|
||||
echo
|
||||
|
||||
echo "=== 2/3 app tests (AppTest, no browser) ==="
|
||||
echo "=== 2/4 after-tax model tests (synthetic) ==="
|
||||
.venv/bin/python tests/test_tax.py || fail=1
|
||||
echo
|
||||
|
||||
echo "=== 3/4 app tests (AppTest, no browser) ==="
|
||||
.venv/bin/python tests/test_app.py || fail=1
|
||||
echo
|
||||
|
||||
echo "=== 3/3 browser e2e (Playwright, needs the server) ==="
|
||||
echo "=== 4/4 browser e2e (Playwright, needs the server) ==="
|
||||
if curl -s -m 3 -o /dev/null http://localhost:8599/healthz; then
|
||||
.venv/bin/python tests/test_e2e_browser.py || fail=1
|
||||
else
|
||||
|
|
|
|||
50
tax.py
50
tax.py
|
|
@ -4,11 +4,18 @@ Model (simplified DAS)
|
|||
----------------------
|
||||
The account starts at 1.0 (growth-ratio units; no fixed capital).
|
||||
|
||||
- Holdings are valued on *adjusted* prices, which already assume
|
||||
distributions are reinvested. Distributions therefore flow through as:
|
||||
- Holdings are valued on RAW (close) prices, in raw share units.
|
||||
Distributions (per-share dollars from the event files) are taxed on
|
||||
their event date and the after-tax remainder is REINVESTED at the same
|
||||
day's raw close (the ex-div price): a fund paying d drops by d, the
|
||||
investor keeps d(1-tau) and buys it back cheaper. The tax's effect is
|
||||
carried by the (smaller) reinvested units; it is recorded in `taxes`
|
||||
but not separately deducted from cash (that would double-count it).
|
||||
dividend income -> taxed at `div_rate`
|
||||
capital gain dist-> taxed at `lt_rate`
|
||||
The after-tax remainder increases each lot's cost basis proportionally.
|
||||
The after-tax remainder also increases each lot's cost basis
|
||||
proportionally (standard after-tax-IRR convention, so a liquidation
|
||||
doesn't re-tax the distribution).
|
||||
- On rebalance, sells are FIFO. A lot is long-term if held more than
|
||||
365 days at sale, else short-term; realized gains/losses are taxed at
|
||||
`lt_rate` / `st_rate`.
|
||||
|
|
@ -20,6 +27,12 @@ The account starts at 1.0 (growth-ratio units; no fixed capital).
|
|||
rates. This is what you would actually have in your pocket after
|
||||
selling everything and filing your taxes.
|
||||
|
||||
NOTE: distribution event dates must be EX-DIV dates for the reinvestment
|
||||
pricing to be right (the raw close on that day already reflects the
|
||||
drop). Yahoo usually dates events on the ex-div date; the record-date
|
||||
misalignments found by scripts/scan_adj_misalign.py are corrected by
|
||||
remove/add ops in overrides/corrections/ where a symbol is analyzed.
|
||||
|
||||
Deliberately NOT modeled: loss carryover, wash sales, state rates,
|
||||
brackets.
|
||||
"""
|
||||
|
|
@ -65,15 +78,17 @@ def _liquidation_tax(lt_gain: float, lt_loss: float,
|
|||
return max(st_net + lt_net, 0.0) * st_rate
|
||||
|
||||
|
||||
def after_tax_portfolio(adj: pd.DataFrame, div: pd.DataFrame, capg: pd.DataFrame,
|
||||
def after_tax_portfolio(close: pd.DataFrame, div: pd.DataFrame, capg: pd.DataFrame,
|
||||
weights: dict[str, float],
|
||||
rebalance: str | None = None, cost_bps: float = 0.0,
|
||||
lt_rate: float = 0.20, st_rate: float = 0.15,
|
||||
div_rate: float = 0.15,
|
||||
start: str | None = None, end: str | None = None
|
||||
) -> TaxResult:
|
||||
syms = [s for s in weights if s in adj.columns]
|
||||
p = adj[syms]
|
||||
"""`close` must be RAW close prices (bundle.close), not adj: the
|
||||
per-share distribution dollars are raw, so units must be raw shares."""
|
||||
syms = [s for s in weights if s in close.columns]
|
||||
p = close[syms]
|
||||
d = div[[s for s in syms if s in div.columns]].reindex(index=p.index, columns=syms).fillna(0.0)
|
||||
c = capg[[s for s in syms if s in capg.columns]].reindex(index=p.index, columns=syms).fillna(0.0)
|
||||
if start or end:
|
||||
|
|
@ -112,23 +127,30 @@ def after_tax_portfolio(adj: pd.DataFrame, div: pd.DataFrame, capg: pd.DataFrame
|
|||
|
||||
for t in range(n):
|
||||
prices = pv[t]
|
||||
market_value = float(np.dot(units, prices))
|
||||
|
||||
# --- distributions (taxed, net flows back into basis) -----------
|
||||
# --- distributions: pay tax, reinvest the after-tax remainder at
|
||||
# today's close. The tax is recorded in tax_rows but NOT also
|
||||
# deducted from cash: it is already reflected in the (smaller)
|
||||
# reinvested units, so a second deduction would double-count it.
|
||||
dinc = units * dv[t]
|
||||
cinc = units * cv[t]
|
||||
d_tax = float(dinc.sum() * div_rate)
|
||||
c_tax = float(cinc.sum() * lt_rate)
|
||||
cash -= d_tax + c_tax
|
||||
tax_rows[t, 0] = d_tax
|
||||
tax_rows[t, 1] = c_tax
|
||||
for i in range(len(syms)): # grow cost basis with reinvested net
|
||||
for i in range(len(syms)):
|
||||
net = (dv[t, i] * (1 - div_rate) + cv[t, i] * (1 - lt_rate)) * units[i]
|
||||
if net <= 0 or not lots[i]:
|
||||
if net <= 0:
|
||||
continue
|
||||
tot = sum(l.units for l in lots[i])
|
||||
for l in lots[i]:
|
||||
l.cost += net * (l.units / tot)
|
||||
px_i = float(prices[i])
|
||||
if px_i > 0:
|
||||
units[i] += net / px_i # reinvest the after-tax cash
|
||||
if lots[i]: # grow cost basis
|
||||
tot = sum(l.units for l in lots[i])
|
||||
for l in lots[i]:
|
||||
l.cost += net * (l.units / tot)
|
||||
|
||||
market_value = float(np.dot(units, prices)) # post-reinvestment
|
||||
|
||||
# --- rebalance to target weights --------------------------------
|
||||
if idx[t] in rebal:
|
||||
|
|
|
|||
97
tests/test_tax.py
Normal file
97
tests/test_tax.py
Normal file
|
|
@ -0,0 +1,97 @@
|
|||
"""Unit tests for the after-tax model (tax.py) — synthetic, no data bundle.
|
||||
|
||||
Key regression: distributions are RAW per-share dollars, so the account
|
||||
must hold RAW share units (valued on close prices). The old code valued
|
||||
units on ADJ (total-return index) prices, which overstates every
|
||||
distribution — and its tax — by the raw/adj ratio (e.g. a fund that has
|
||||
paid many distributions: adj 11 vs raw 30 -> 2.7x overcharge).
|
||||
|
||||
Run: .venv/bin/python tests/test_tax.py
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pandas as pd
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
import tax # noqa: E402
|
||||
|
||||
PASS, FAIL = 0, 0
|
||||
|
||||
|
||||
def check(name: str, ok: bool) -> None:
|
||||
global PASS, FAIL
|
||||
PASS += bool(ok)
|
||||
FAIL += not ok
|
||||
print((" ok " if ok else " FAIL ") + name)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
# One fund, raw prices 30 -> 34, ex-div 5.00 on day 6 (34 -> 29),
|
||||
# then 29.5. Account starts at 1.0 = 1/30 share.
|
||||
dates = pd.to_datetime(["2024-01-0%d" % i for i in range(1, 8)])
|
||||
close = pd.DataFrame({"f": [30.0, 31.0, 32.0, 33.0, 34.0, 29.0, 29.5]},
|
||||
index=dates)
|
||||
div = pd.DataFrame({"f": [0, 0, 0, 0, 0, 0.0, 0.0]}, index=dates)
|
||||
capg = pd.DataFrame({"f": [0, 0, 0, 0, 0, 5.0, 0.0]}, index=dates)
|
||||
|
||||
r = tax.after_tax_portfolio(close, div, capg, {"f": 1.0}, lt_rate=0.20,
|
||||
st_rate=0.15, div_rate=0.15)
|
||||
t5 = dates[5]
|
||||
|
||||
# --- the regression: tax on the distribution is 20% of 5/30 of the
|
||||
# account (the distribution as a fraction of the RAW price), i.e.
|
||||
# 3.333% — not 5/adj (which the old code implied, ~4% with this
|
||||
# adj path and up to 2.7x for high-history funds).
|
||||
check("cap-gain tax is 20% of (5/30) of the account",
|
||||
abs(r.taxes["capg_tax"][t5] - 0.2 * 5.0 / 30.0) < 1e-12)
|
||||
check("no dividend tax", r.taxes["div_tax"].abs().sum() == 0.0)
|
||||
|
||||
# --- reinvestment: the after-tax remainder (5*0.8 of the 1/30 share)
|
||||
# is bought at the ex-div close 29, so MV on the event day is
|
||||
# (1/30)*(29 + 4) = 33/30. The tax is already inside the (smaller)
|
||||
# units — it must NOT also show up in equity as a second deduction.
|
||||
mv5 = r.equity[t5] + r.liq_tax[t5] # no cash component on this day
|
||||
check("market value on ex-div day = 33/30 (after-tax reinvested)",
|
||||
abs(mv5 - 33.0 / 30.0) < 1e-9)
|
||||
|
||||
# --- liquidation value: price drop (5/30) is exactly offset by the
|
||||
# distribution kept, so equity falls by the true tax cost only.
|
||||
# Day-5 equity = MV(33/30) - liq_tax(0: basis 1+4/30 > MV, a "loss"
|
||||
# under the after-tax basis rule).
|
||||
check("equity on ex-div day = 33/30 (tax cost lives in the units)",
|
||||
abs(r.equity[t5] - 33.0 / 30.0) < 1e-9)
|
||||
|
||||
# --- pre-event day: plain ST-marked liquidation value.
|
||||
t4 = dates[4]
|
||||
check("equity day before = MV - 15% ST gain tax",
|
||||
abs(r.equity[t4] - (34.0 / 30.0) * (1 - 0.15 * (4.0 / 34.0))) < 1e-9)
|
||||
|
||||
# --- cross-day consistency: equity(5)/equity(4) from the explicit
|
||||
# values (day 4 = 34/30 MV less 15% ST gain tax; day 5 = 33/30).
|
||||
check("ex-div equity step matches explicit values",
|
||||
abs(r.equity[t5] / r.equity[t4]
|
||||
- (33.0 / 30.0) / (34.0 / 30.0 - 0.15 * 4.0 / 30.0)) < 1e-9)
|
||||
|
||||
# --- no rebalancing: one lot survives, units grew by the reinvested
|
||||
# net amount at the ex-div price.
|
||||
check("single lot outstanding", r.lots_outstanding == 1)
|
||||
|
||||
# --- dividend leg taxed at div_rate, cap-gain leg at lt_rate:
|
||||
# split a 1.00 distribution 0.4 div / 0.6 capg on day 6.
|
||||
div2 = pd.DataFrame({"f": [0, 0, 0, 0, 0, 0.4, 0.0]}, index=dates)
|
||||
capg2 = pd.DataFrame({"f": [0, 0, 0, 0, 0, 0.6, 0.0]}, index=dates)
|
||||
r2 = tax.after_tax_portfolio(close, div2, capg2, {"f": 1.0}, lt_rate=0.20,
|
||||
st_rate=0.15, div_rate=0.10)
|
||||
check("split distribution taxed by component rate",
|
||||
abs(r2.taxes["capg_tax"][t5] - 0.20 * 0.6 / 30.0) < 1e-12
|
||||
and abs(r2.taxes["div_tax"][t5] - 0.10 * 0.4 / 30.0) < 1e-12)
|
||||
|
||||
print(f"\n{PASS} passed, {FAIL} failed")
|
||||
return 1 if FAIL else 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Loading…
Reference in New Issue
Block a user