Compute all alphas in excess of the 3-mo T-bill rate (BIL)

Raw-intercept alphas absorbed the T-bill yield on uninvested/levered
portions (582 well-fitted funds >2%/yr off; sum-of-betas polluted by
level-matching). Now fund AND sleeves are netted against BIL daily
total return before every regression; a cash position contributes
exactly zero.

- decompose: rf_series()/excess(); shv+bil dropped from regressors
  (~0 columns in excess space); FULL_WINDOW -> 2007-06-01 (BIL
  inception; mixing raw pre-2007 with excess breaks the fit).
- factors: same excess treatment; shv out of DRIVERS.
- CASH axis redefined: alpha/cash_yield -> net cash position = 1 -
  sum(betas) (label 'cash (net posn)').
- CANDIDATE list 250 -> 11: the old list was mostly under-invested
  funds whose 'alpha' was cash yield, not skill.
- refback.py: per-fund fitted reference (forward-selected sleeves)
  stored as ref_5y/ref_full in search_all.json; app alpha-search
  table gains a 'reference (5y)' column - the answer to 'what is
  alpha computed against' (the fund's OWN fitted sleeve mix, not one
  index).
- App captions updated; raw-alpha-era results backed up as
  *_rawalpha.json (not deleted).

app.py

@@ -660,6 +660,7 @@ with tab_fundlab:
                               else "—"),
                     "alpha 5y": (f"{_a5*100:+.1f}% (t={_v['alpha_t_5y']:+.1f})"
                                  if isinstance(_a5, (int, float)) else "—"),
+                    "reference (5y)": _v.get("ref_5y") or "—",
                     "corr port": (f"{_v['corr_portfolio']:.2f}"
                                   if isinstance(_v.get("corr_portfolio"),
                                                 (int, float)) else "—"),
@@ -670,14 +671,18 @@ with tab_fundlab:
                 })
             st.dataframe(pd.DataFrame(_tbl), width="stretch")
             st.caption(
-                "Screen: daily total returns vs 21 broad sleeve axes (same "
+                "Screen: daily total returns IN EXCESS OF THE 3-MO T-BILL "
-                "set for every fund); 'alpha 5y' = OLS intercept over the "
+                "RATE (BIL) vs 21 broad sleeve axes (same set for every "
-                "last 5 years (t-stat); 'corr port' = correlation with your "
+                "fund); 'alpha 5y' = OLS intercept of the excess returns "
-                "current qspnx/pmaix portfolio; '6m + %' = share of rolling "
+                "over the last 5 years (t-stat) — i.e. outperformance vs "
-                "6-month windows where the fund beat its fitted sleeve mix. "
+                "the fund's OWN fitted sleeve mix (the 'reference' column), "
-                "CANDIDATE = R²5y < 0.6 (or < 0.85 with strong residual "
+                "not vs one index; a cash position earns exactly the t-bill "
-                "alpha), t5y ≥ 2, t-full ≥ 1.25, ≥ 45% positive windows, "
+                "rate and adds zero alpha. 'corr port' = correlation with "
-                "portfolio correlation < 0.3.")
+                "your current qspnx/pmaix portfolio; '6m + %' = share of "
+                "rolling 6-month windows where the fund beat its fitted "
+                "sleeve mix. CANDIDATE = R²5y < 0.6 (or < 0.85 with strong "
+                "residual alpha), t5y ≥ 2, t-full ≥ 1.25, ≥ 45% positive "
+                "windows, portfolio correlation < 0.3.")
 
     # --- return-driver clusters: k-means on 35-sleeve loading vectors ---
     with st.expander(
@@ -771,11 +776,14 @@ with tab_fundlab:
                 r.pop("_t")
             st.dataframe(pd.DataFrame(_rows), width="stretch")
             st.caption(
-                "Each fund's daily returns are regressed on 35 sleeve axes "
+                "Each fund's daily total returns IN EXCESS OF THE 3-MO "
-                "(equity styles/sizes/intl/EM, the duration ladder, IG/HY/"
+                "T-BILL RATE (BIL) are regressed on 34 sleeve axes (equity "
-                "muni/MBS/preferred/EM-debt credit, sectors, gold/oil/"
+                "styles/sizes/intl/EM, the duration ladder, IG/HY/muni/MBS/"
-                "commodities, CTA); the 35-d loading vector is the fund's "
+                "preferred/EM-debt credit, sectors, gold/oil/commodities, "
-                "'return-driver signature' and k-means groups similar "
+                "CTA); the loading vector is the fund's 'return-driver "
+                "signature' — the betas are its NET INVESTED mix — plus a "
+                "virtual CASH axis = 1 − Σβ (net cash/T-bill position; 1 = "
+                "fully in cash, < 0 = levered). k-means groups similar "
                 "signatures. 'corr port' is shown for reference only - "
                 "high-corr funds are REPLACEMENTS for current holdings, "
                 "not rejections.")
@@ -1096,9 +1104,10 @@ with tab_fundlab:
         if _dr.get("note"):
             st.markdown("**Holdings cross-check:** " + _dr["note"])
         st.caption(
-            "Method: daily total returns; greedy forward selection on BIC "
+            "Method: daily total returns IN EXCESS OF THE 3-MO T-BILL RATE "
-            "(add only if ΔBIC ≥ 2 and |t| > 2); candidate sleeves curated per "
+            "(BIL); greedy forward selection on BIC (add only if ΔBIC ≥ 2 and "
-            "fund from the strategy text + N-PORT. β = exposure, not a literal "
+            "|t| > 2); candidate sleeves curated per fund from the strategy "
+            "text + N-PORT. β = exposure, not a literal "
             "holding weight; 'drift' = mean |rolling 1y β − full β| relative to "
             "the full β. High-R² + low drift ≈ static mix; low R² with positive "
             "alpha ≈ market-neutral/alpha strategy.")

fundlab/RESEARCH.md

@@ -981,3 +981,95 @@ kills nohup'd processes (the CEF batch and the first watchdog both
 died that way; the batch was resumable so no data was lost).
 server_watchdog.sh had to be relaunched manually; consider a
 boot-time starter (systemd/cron @reboot) if reboots repeat.
+- 2026-08-30 15:09 === stage screen ===
+- 2026-08-30 15:09 screen: 2384 funds, 2384 to do
+- 2026-08-30 15:10 screen: 100/2384
+- 2026-08-30 15:11 screen: 200/2384
+- 2026-08-30 15:11 screen: 300/2384
+- 2026-08-30 15:12 screen: 400/2384
+- 2026-08-30 15:12 screen: 500/2384
+- 2026-08-30 15:13 screen: 600/2384
+- 2026-08-30 15:13 screen: 700/2384
+- 2026-08-30 15:14 screen: 800/2384
+- 2026-08-30 15:14 screen: 900/2384
+- 2026-08-30 15:15 screen: 1000/2384
+- 2026-08-30 15:15 screen: 1100/2384
+- 2026-08-30 15:16 screen: 1200/2384
+- 2026-08-30 15:17 screen: 1300/2384
+- 2026-08-30 15:17 screen: 1400/2384
+- 2026-08-30 15:18 screen: 1500/2384
+- 2026-08-30 15:18 screen: 1600/2384
+- 2026-08-30 15:19 screen: 1700/2384
+- 2026-08-30 15:19 screen: 1800/2384
+- 2026-08-30 15:20 screen: 1900/2384
+- 2026-08-30 15:20 screen: 2000/2384
+- 2026-08-30 15:21 screen: 2100/2384
+- 2026-08-30 15:21 screen: 2200/2384
+- 2026-08-30 15:22 screen: 2300/2384
+- 2026-08-30 15:22 screen done: 2384 funds
+- 2026-08-30 15:22 === stage finalize ===
+- 2026-08-30 15:22 finalize: 2384 funds screened
+- 2026-08-30 15:22    1339  sleeve mix
+- 2026-08-30 15:22    1001  weak/unstable alpha
+- 2026-08-30 15:22      17  no 5y window
+- 2026-08-30 15:22      11  CANDIDATE
+- 2026-08-30 15:22      10  alpha, but correlated with current portfolio
+- 2026-08-30 15:22       6  alpha in 5y window, but not persistent
+- 2026-08-30 15:22   candidates v1 name-filter would have missed: ['aguax', 'etsix', 'femdx', 'hicox', 'prfrx', 'puls', 'rctix', 'rpifx', 'scfzx']
+- 2026-08-30 15:22 === overnight run finished in 0.2h ===
+
+### Alphas recomputed in EXCESS of the 3-mo T-bill rate (risk-free netting)
+
+**Question that triggered it:** "Are we using cash return as a risk-free
+rate in the alpha calculations?" — No. Alpha was the OLS intercept of
+RAW daily total returns. For a fund whose net sleeve exposure is not
+~100% (under-invested or levered), the raw intercept absorbs the T-bill
+yield on the uninvested/levered part, so "alpha" mixed skill with cash
+drag. Magnitude (5y, R²>0.5 funds): 582 funds >2%/yr apart, 533 in
+1-2%, 345 in 0.5-1%. Sum-of-betas was also polluted by level-matching
+(median 1.15; a plain growth fund like bggsx showed 3.7 with a −10%/yr
+"alpha" that was pure level-splitting).
+
+**Changes:**
+- decompose.py: `rf_series()` = BIL (SPDR 1-3mo T-Bill) daily total
+  return — the local stand-in for the 3-mo T-bill yield; `excess()`
+  nets the fund AND the sleeves against it before every regression.
+  Cash positions now contribute exactly zero (cash IS the rf).
+- CASH_SLEEVES {shv, bil} dropped from the regressors: in excess space
+  they are ~0 columns (ill-conditioning); the cash position is captured
+  by the residual instead. Same for factors.py (shv added to
+  DROP_FROM_REGRESSION).
+- **FULL_WINDOW moved 1990-01-01 → 2007-06-01** (BIL inception):
+  mixing raw pre-2007 with excess post-2007 returns breaks the
+  full-window fit (cosix: R²≈0, zero components selected).
+- **CASH axis redefined** from alpha_ann/cash_yield (the old version
+  was measuring the yield level that no longer lives in the intercept)
+  to **net cash position = 1 − Σβ**: the betas are the net-invested
+  mix, the leftover is cash/T-bills. Pure MMF → 1.0; levered → < 0.
+  Labeled "cash (net posn)". `factors.cash_yield()` removed.
+- Alphas in the app are now true excess-return alphas: intercept of
+  (fund − rf) on (sleeves − rf).
+
+**Verification (probes):** qcmmrx (money market): alpha −0.17%/yr
+(t=−1.3, noise), cash 1.00. vsbsx (short-T index): alpha 0.03%, cash
+0.77 + short duration. bggsx: Σβ 1.25 (was 3.7), alpha −3.6%
+(t=−0.6). IVV: alpha 0.00% (t=+36.7). cosix full-window fixed
+(R² 0.60, vweax 0.45/agg 0.15/ief 0.15).
+
+**Consequence — the CANDIDATE list shrank 250 → 11.** Most of the old
+250 were under-invested funds (bond funds with Σβ<1) whose "alpha" was
+the T-bill yield on their uninvested portion. The 11 that survive are
+funds whose excess alpha is real. Verdict mix now: 1339 sleeve mix,
+1001 weak/unstable alpha, 11 CANDIDATE, 10 correlated, 6 not
+persistent, 17 no 5y window.
+
+**Reference visibility (the "what is alpha computed vs" question):**
+alpha is NOT vs one index — it is vs the fund's own fitted sleeve mix
+(BIC forward selection). fundlab/refback.py recomputes the forward
+selection per fund and stores `ref_5y`/`ref_full` strings in
+search_all.json; the app's alpha-search table now has a "reference
+(5y)" column.
+
+**Backups of the raw-alpha era (NOT deleted):**
+factor_results_rawalpha.json, decompose_results_rawalpha.json,
+search_all_rawalpha.json, cluster_kmeans_rawalpha.json.

fundlab/cluster.py

@@ -30,7 +30,7 @@ HERE = Path(__file__).parent
 RESULTS = HERE / "factor_results.json"
 TREE = HERE / "cluster_tree.json"
 KMEANS = HERE / "cluster_kmeans.json"
-from fundlab.factors import AXES, DRIVERS, cash_yield  # noqa: E402
+from fundlab.factors import AXES, DRIVERS  # noqa: E402
 
 
 def log(msg: str) -> None:
@@ -39,7 +39,6 @@ def log(msg: str) -> None:
 
 def loading_matrix() -> tuple[list[str], np.ndarray]:
     d = json.loads(RESULTS.read_text())
-    yld = cash_yield()
     syms, rows = [], []
     for sym, v in d.items():
         # full window preferred; funds without 250 complete-case rows
@@ -51,13 +50,11 @@ def loading_matrix() -> tuple[list[str], np.ndarray]:
             continue
         row = np.array([f["betas"].get(s, 0.0) or 0.0 for s in DRIVERS],
                        dtype=float)
-        # virtual CASH axis: the fund's intercept (annualized alpha) is
+        # virtual CASH axis = net cash position = 1 - sum of the betas.
-        # its yield capture - for a cash fund that IS the exposure,
+        # Alphas are in excess of the T-bill rate, so the betas are the
-        # because OLS puts a near-constant yield level in the intercept
+        # NET INVESTED mix; whatever is left over sits in cash/T-bills.
-        # rather than in the (small) shv/bil slope. Normalize by the
+        # Pure money-market fund: all betas ~ 0 -> cash ~ 1.
-        # recent cash yield so 1.0 == "earns the cash rate".
+        row = np.append(row, 1.0 - row.sum())
-        alpha = f.get("alpha_ann")
-        row = np.append(row, (alpha / yld if alpha is not None and yld else 0.0))
         # winsorize at +/-4: everything beyond that is an OLS noise fit
         # (R2 ~ 0.01, |t| < 2 - e.g. fyhtx shv +63.8), never a real
         # exposure (no open-end fund runs 400% of a sleeve)
@@ -148,11 +145,11 @@ SLEEVE_NAME = {
     "xlv": "healthcare", "xlp": "staples", "xlu": "utilities",
     "xly": "consumer disc", "xlb": "materials", "dbmf": "CTA/futures",
     "dbb": "commodities",
-    "cash": "cash (yield)",
+    "cash": "cash (net posn)",
 }
 
-# the CASH axis is a yield LEVEL (alpha/cash-yield, 1.0 = earns the cash
+# the CASH axis is a POSITION (net cash, 1.0 = fully in cash), not a
-# rate), not a return SENSITIVITY like the betas. In raw Euclidean space a
+# return SENSITIVITY like the betas. In raw Euclidean space a
 # pure cash fund (0,...,0,~0.5) sits inside the cloud of low-exposure
 # balanced/target-date funds and k-means swallows it. Emphasize the axis
 # for DISTANCE only (labels still use the raw values).

fundlab/decompose.py

@@ -22,6 +22,7 @@ intercept; alphas are annualized (x252); t-stats use the OLS covariance.
 from __future__ import annotations
 
 import json
+from functools import lru_cache
 from pathlib import Path
 
 import numpy as np
@@ -49,6 +50,29 @@ def adj_close(sym: str, root: Path = DATA) -> pd.Series | None:
     return s
 
 
+@lru_cache(maxsize=1)
+def rf_series() -> pd.Series | None:
+    """Daily risk-free rate: BIL (SPDR 1-3 Month T-Bill) total return —
+    the local stand-in for the 3-month T-bill yield. Alphas are computed
+    in EXCESS of this rate (both the fund and the sleeves are netted
+    against it), so the intercept is a true excess-return alpha and a
+    cash position contributes zero (cash IS the risk-free rate). 0 where
+    BIL has no history (pre-2007)."""
+    a = adj_close("bil")
+    if a is None:
+        return None
+    return a.pct_change().fillna(0.0)
+
+
+def excess(panel: pd.DataFrame) -> pd.DataFrame:
+    """Net every column against the daily risk-free rate."""
+    rf = rf_series()
+    if rf is None:
+        return panel
+    rf = rf.reindex(panel.index).fillna(0.0)
+    return panel.sub(rf, axis=0)
+
+
 def returns_panel(symbols: list[str], start: str | None = None) -> pd.DataFrame:
     """Daily simple returns, outer-joined on dates (NaN where a series
     has no data). Callers must handle missing values per regressor —
@@ -207,9 +231,17 @@ CANDIDATES: dict[str, list[str]] = {
 }
 # share classes: same underlying fund
 ALIAS = {"pmfkx": "pmaix", "lcrix": "lcorx", "egrsx": "eagmx"}
-FULL_WINDOW = "1990-01-01"   # effectively all history
+FULL_WINDOW = "2007-06-01"  # from BIL inception: alphas are in excess of
+# the T-bill rate, which needs the rf series (raw pre-2007 returns
+# mixed with excess post-2007 returns breaks the full-window fit)
 RECENT_WINDOW = "2021-01-01"  # last ~5 years
 
+# sleeves that ARE the risk-free rate. Alphas are now computed in excess of
+# the T-bill rate, so these columns are ~0 in excess space and would make
+# the regression ill-conditioned. Cash exposure is captured by the residual
+# (1 - sum of betas) instead, so they are dropped from the regressors.
+CASH_SLEEVES = {"shv", "bil"}
+
 
 def _drift(rb: pd.DataFrame, full_beta: np.ndarray) -> dict[str, float]:
     """Mean |rolling beta - full-sample beta| per component, as a fraction
@@ -225,13 +257,14 @@ def _drift(rb: pd.DataFrame, full_beta: np.ndarray) -> dict[str, float]:
 def decompose(fund: str, start: str = FULL_WINDOW,
               candidates: dict[str, list[str]] | None = None) -> dict:
     cand_syms = (candidates or CANDIDATES).get(fund, [])
-    r = returns_panel([fund] + cand_syms, start=start)
+    r = excess(returns_panel([fund] + cand_syms, start=start))
     if fund not in r.columns:
         return {"fund": fund, "error": "no return history in the data set",
                 "verdict": "no return history", "start": start}
     y = r[fund].to_numpy()
     dates = r.index
-    cand = {s: r[s].to_numpy() for s in r.columns if s != fund}
+    cand = {s: r[s].to_numpy() for s in r.columns
+            if s != fund and s not in CASH_SLEEVES}
     y_ok = ~np.isnan(y)
     if y_ok.sum() < 252:
         return {"fund": fund, "error": "insufficient return history",

fundlab/factors.py

@@ -69,33 +69,20 @@ SLEEVES_V2 = list(dict.fromkeys(BROAD_SLEEVES + EXTRA_SLEEVES))
 # short-rate exposure as huge offsetting shv/bil coefficients (shv +64 /
 # bil -54) - shv (1-3m) alone carries the cash/ultra-short axis, shy
 # keeps the 1-3y axis.
-DROP_FROM_REGRESSION = {"vea", "vug", "finux", "bil"}
+DROP_FROM_REGRESSION = {"vea", "vug", "finux", "bil", "shv"}
+# (bil/shv are the risk-free sleeves - alphas are now computed in
+# excess of the T-bill rate, where they are ~0 columns; the cash
+# position shows up in the residual / the virtual CASH axis instead)
 DRIVERS = [s for s in SLEEVES_V2 if s not in DROP_FROM_REGRESSION]
 
 # clustering/display basis = the regression sleeves + a virtual CASH axis.
-# A cash fund's yield is a near-constant in return space, so OLS puts it in
+# Alphas are computed in EXCESS of the T-bill rate (decompose.excess), so
-# the INTERCEPT (alpha_ann), not in any beta - BIL/SHV betas only measure
+# the beta vector is the fund's NET INVESTED mix and the leftover
-# the fund's response to rate CHANGES. The CASH axis makes the yield
+# (1 - sum of betas) is its cash/T-bill position: a pure money-market
-# capture an explicit driver: alpha_ann normalized by the recent cash
+# fund has all betas ~ 0 -> CASH ~ 1, a levered fund -> CASH < 0.
-# yield (shv trailing 1y total return = the local stand-in for the
-# 13-week T-bill / 0-3m Treasury index, SIXM-style).
 AXES = DRIVERS + ["cash"]
 
 
-@lru_cache(maxsize=1)
-def cash_yield() -> float:
-    """Recent cash level: shv trailing 1y total return (fallback sgov,
-    then bil). ~3.7% in the current regime."""
-    for s in ("shv", "sgov", "bil"):
-        a = decompose.adj_close(s)
-        if a is None:
-            continue
-        year_ago = a[a.index <= a.index[-1] - pd.Timedelta(days=365)]
-        if len(year_ago) >= 1 and year_ago.iloc[-1] > 0:
-            return float(a.iloc[-1] / year_ago.iloc[-1] - 1.0)
-    return 0.03
-
-
 def _resid(y: pd.Series, X: pd.DataFrame) -> pd.Series:
     m = y.notna() & X.notna().all(axis=1)
     Xa = np.column_stack([np.ones(m.sum())] + [X[c][m].to_numpy()
@@ -116,6 +103,7 @@ def _panel() -> pd.DataFrame:
     log(f"building driver panel ({len(DRIVERS)} axes)")
     p = decompose.returns_panel(SLEEVES_V2).sort_index()
     p = p[~p.index.duplicated(keep="last")][DRIVERS]
+    p = decompose.excess(p)   # alphas are in excess of the T-bill rate
     # pure vol axis: vblix residualized on the core equity/duration axes
     core = [c for c in ("ivv", "tlt") if c in p.columns]
     if "vblix" in p.columns and core:
@@ -165,11 +153,16 @@ def _beta_window(y: pd.Series, X: pd.DataFrame, start: str) -> dict | None:
     }
 
 
-def factor_screen(sym: str, start: str = "1990-01-01") -> dict | None:
+def factor_screen(sym: str, start: str = None) -> dict | None:
+    if start is None:
+        start = decompose.FULL_WINDOW
     fund = decompose.adj_close(sym)
     if fund is None or len(fund) < 250:
         return None
     y = fund.pct_change().dropna()
+    rf = decompose.rf_series()
+    if rf is not None:
+        y = y.sub(rf.reindex(y.index).fillna(0.0))
     X = _panel().reindex(y.index)
     # sleeves must have history for most of the fund's window
     avail = [c for c in X.columns if X[c].notna().mean() > 0.3]

fundlab/factors_run.log

@@ -0,0 +1,14 @@
+15:03:47 factor screen: 2384 funds, 2384 to do
+15:03:47 building driver panel (34 axes)
+15:03:55   200/2384
+15:04:02   400/2384
+15:04:10   600/2384
+15:04:18   800/2384
+15:04:30   1000/2384
+15:04:43   1200/2384
+15:04:52   1400/2384
+15:04:59   1600/2384
+15:05:07   1800/2384
+15:05:14   2000/2384
+15:05:21   2200/2384
+15:05:28 factor screen done: 2384 funds -> factor_results.json

fundlab/overnight.pid

@@ -0,0 +1 @@
+5847

fundlab/overnight_rescreen.log

@@ -0,0 +1,36 @@
+15:09:56 === stage screen ===
+15:09:56 screen: 2384 funds, 2384 to do
+15:10:29 screen: 100/2384
+15:11:01 screen: 200/2384
+15:11:33 screen: 300/2384
+15:12:04 screen: 400/2384
+15:12:36 screen: 500/2384
+15:13:09 screen: 600/2384
+15:13:42 screen: 700/2384
+15:14:15 screen: 800/2384
+15:14:45 screen: 900/2384
+15:15:20 screen: 1000/2384
+15:15:56 screen: 1100/2384
+15:16:31 screen: 1200/2384
+15:17:07 screen: 1300/2384
+15:17:38 screen: 1400/2384
+15:18:14 screen: 1500/2384
+15:18:45 screen: 1600/2384
+15:19:16 screen: 1700/2384
+15:19:47 screen: 1800/2384
+15:20:19 screen: 1900/2384
+15:20:51 screen: 2000/2384
+15:21:23 screen: 2100/2384
+15:21:54 screen: 2200/2384
+15:22:26 screen: 2300/2384
+15:22:53 screen done: 2384 funds
+15:22:53 === stage finalize ===
+15:22:53 finalize: 2384 funds screened
+15:22:53    1339  sleeve mix
+15:22:53    1001  weak/unstable alpha
+15:22:53      17  no 5y window
+15:22:53      11  CANDIDATE
+15:22:53      10  alpha, but correlated with current portfolio
+15:22:53       6  alpha in 5y window, but not persistent
+15:22:53   candidates v1 name-filter would have missed: ['aguax', 'etsix', 'femdx', 'hicox', 'prfrx', 'puls', 'rctix', 'rpifx', 'scfzx']
+15:22:53 === overnight run finished in 0.2h ===

fundlab/refback.log

@@ -0,0 +1,98 @@
+[2026-08-30 15:40:42] rows=2384 todo=2384
+[2026-08-30 15:40:48] 25/2384 done (saved)
+[2026-08-30 15:40:53] 50/2384 done (saved)
+[2026-08-30 15:40:59] 75/2384 done (saved)
+[2026-08-30 15:41:04] 100/2384 done (saved)
+[2026-08-30 15:41:10] 125/2384 done (saved)
+[2026-08-30 15:41:16] 150/2384 done (saved)
+[2026-08-30 15:41:22] 175/2384 done (saved)
+[2026-08-30 15:41:28] 200/2384 done (saved)
+[2026-08-30 15:41:33] 225/2384 done (saved)
+[2026-08-30 15:41:39] 250/2384 done (saved)
+[2026-08-30 15:41:45] 275/2384 done (saved)
+[2026-08-30 15:41:51] 300/2384 done (saved)
+[2026-08-30 15:41:57] 325/2384 done (saved)
+[2026-08-30 15:42:03] 350/2384 done (saved)
+[2026-08-30 15:42:08] 375/2384 done (saved)
+[2026-08-30 15:42:14] 400/2384 done (saved)
+[2026-08-30 15:42:20] 425/2384 done (saved)
+[2026-08-30 15:42:26] 450/2384 done (saved)
+[2026-08-30 15:42:32] 475/2384 done (saved)
+[2026-08-30 15:42:37] 500/2384 done (saved)
+[2026-08-30 15:42:42] 525/2384 done (saved)
+[2026-08-30 15:42:48] 550/2384 done (saved)
+[2026-08-30 15:42:54] 575/2384 done (saved)
+[2026-08-30 15:43:00] 600/2384 done (saved)
+[2026-08-30 15:43:06] 625/2384 done (saved)
+[2026-08-30 15:43:11] 650/2384 done (saved)
+[2026-08-30 15:43:18] 675/2384 done (saved)
+[2026-08-30 15:43:24] 700/2384 done (saved)
+[2026-08-30 15:43:29] 725/2384 done (saved)
+[2026-08-30 15:43:36] 750/2384 done (saved)
+[2026-08-30 15:43:42] 775/2384 done (saved)
+[2026-08-30 15:43:48] 800/2384 done (saved)
+[2026-08-30 15:43:53] 825/2384 done (saved)
+[2026-08-30 15:43:59] 850/2384 done (saved)
+[2026-08-30 15:44:05] 875/2384 done (saved)
+[2026-08-30 15:44:10] 900/2384 done (saved)
+[2026-08-30 15:44:16] 925/2384 done (saved)
+[2026-08-30 15:44:22] 950/2384 done (saved)
+[2026-08-30 15:44:28] 975/2384 done (saved)
+[2026-08-30 15:44:34] 1000/2384 done (saved)
+[2026-08-30 15:44:39] 1025/2384 done (saved)
+[2026-08-30 15:44:45] 1050/2384 done (saved)
+[2026-08-30 15:44:51] 1075/2384 done (saved)
+[2026-08-30 15:44:56] 1100/2384 done (saved)
+[2026-08-30 15:45:02] 1125/2384 done (saved)
+[2026-08-30 15:45:08] 1150/2384 done (saved)
+[2026-08-30 15:45:14] 1175/2384 done (saved)
+[2026-08-30 15:45:19] 1200/2384 done (saved)
+[2026-08-30 15:45:25] 1225/2384 done (saved)
+[2026-08-30 15:45:31] 1250/2384 done (saved)
+[2026-08-30 15:45:36] 1275/2384 done (saved)
+[2026-08-30 15:45:42] 1300/2384 done (saved)
+[2026-08-30 15:45:47] 1325/2384 done (saved)
+[2026-08-30 15:45:53] 1350/2384 done (saved)
+[2026-08-30 15:45:59] 1375/2384 done (saved)
+[2026-08-30 15:46:04] 1400/2384 done (saved)
+[2026-08-30 15:46:10] 1425/2384 done (saved)
+[2026-08-30 15:46:16] 1450/2384 done (saved)
+[2026-08-30 15:46:22] 1475/2384 done (saved)
+[2026-08-30 15:46:28] 1500/2384 done (saved)
+[2026-08-30 15:46:33] 1525/2384 done (saved)
+[2026-08-30 15:46:38] 1550/2384 done (saved)
+[2026-08-30 15:46:44] 1575/2384 done (saved)
+[2026-08-30 15:46:49] 1600/2384 done (saved)
+[2026-08-30 15:46:55] 1625/2384 done (saved)
+[2026-08-30 15:47:01] 1650/2384 done (saved)
+[2026-08-30 15:47:06] 1675/2384 done (saved)
+[2026-08-30 15:47:12] 1700/2384 done (saved)
+[2026-08-30 15:47:18] 1725/2384 done (saved)
+[2026-08-30 15:47:24] 1750/2384 done (saved)
+[2026-08-30 15:47:30] 1775/2384 done (saved)
+[2026-08-30 15:47:35] 1800/2384 done (saved)
+[2026-08-30 15:47:41] 1825/2384 done (saved)
+[2026-08-30 15:47:47] 1850/2384 done (saved)
+[2026-08-30 15:47:53] 1875/2384 done (saved)
+[2026-08-30 15:47:59] 1900/2384 done (saved)
+[2026-08-30 15:48:05] 1925/2384 done (saved)
+[2026-08-30 15:48:10] 1950/2384 done (saved)
+[2026-08-30 15:48:16] 1975/2384 done (saved)
+[2026-08-30 15:48:22] 2000/2384 done (saved)
+[2026-08-30 15:48:28] 2025/2384 done (saved)
+[2026-08-30 15:48:33] 2050/2384 done (saved)
+[2026-08-30 15:48:39] 2075/2384 done (saved)
+[2026-08-30 15:48:45] 2100/2384 done (saved)
+[2026-08-30 15:48:51] 2125/2384 done (saved)
+[2026-08-30 15:48:57] 2150/2384 done (saved)
+[2026-08-30 15:49:03] 2175/2384 done (saved)
+[2026-08-30 15:49:09] 2200/2384 done (saved)
+[2026-08-30 15:49:15] 2225/2384 done (saved)
+[2026-08-30 15:49:21] 2250/2384 done (saved)
+[2026-08-30 15:49:27] 2275/2384 done (saved)
+[2026-08-30 15:49:33] 2300/2384 done (saved)
+[2026-08-30 15:49:38] 2325/2384 done (saved)
+[2026-08-30 15:49:44] 2350/2384 done (saved)
+[2026-08-30 15:49:50] 2375/2384 done (saved)
+[2026-08-30 15:49:52] 2384/2384 done (saved)
+[2026-08-30 15:49:52] done: 2384 rows carry a reference

fundlab/refback.py

@@ -0,0 +1,76 @@
+"""Backfill the per-fund alpha REFERENCE into search_all.json.
+
+The alpha search verdict (fundlab/search.py) computes alpha as the OLS
+intercept of the fund's daily total returns against the sleeves BIC
+forward-selection picked from the 21 broad sleeve axes (the same set for
+every fund). The picked sleeves ARE the fund's reference - a fitted
+sleeve mix, not a single index - but the screen never stored them, so
+the app couldn't show them.
+
+This script recomputes the forward selection (full + last-5y) for every
+screened fund and stores a compact, human-readable reference string on
+each row:
+
+  ref_5y   "ivv +0.62, tlt +0.31"   (sleeves picked for the 5y model)
+  ref_full "ivv +0.38, efa +0.15, ..."
+  "pure alpha (no sleeve selected)" when the model has no components
+
+Resumable: rows that already carry ref_5y are skipped.
+
+Run: .venv/bin/python -m fundlab.refback   (logs to fundlab/refback.log)
+"""
+from __future__ import annotations
+
+import json
+import time
+
+from fundlab import decompose
+from fundlab.searchlist import BROAD_SLEEVES
+
+HERE = __import__("pathlib").Path(__file__).parent
+SRC = HERE / "search_all.json"
+LOG = HERE / "refback.log"
+
+
+def log(msg: str) -> None:
+    line = f"[{time.strftime('%Y-%m-%d %H:%M:%S')}] {msg}"
+    print(line, flush=True)
+    with LOG.open("a") as f:
+        f.write(line + "\n")
+
+
+def _fmt(m: dict) -> str:
+    comps = m.get("components") or []
+    if not comps:
+        return "pure alpha (no sleeve selected)"
+    return ", ".join(f"{c['sym']} {c['beta']:+.2f}" for c in comps)
+
+
+def run() -> None:
+    d = json.loads(SRC.read_text())
+    todo = [k for k, v in d.items()
+            if isinstance(v, dict)
+            and isinstance(v.get("r2_5y"), (int, float))
+            and "ref_5y" not in v]
+    log(f"rows={len(d)} todo={len(todo)}")
+    for i, k in enumerate(todo, 1):
+        sym = d[k]["sym"]
+        try:
+            rec = decompose.decompose(sym, start=decompose.RECENT_WINDOW,
+                                      candidates={sym: BROAD_SLEEVES})
+            full = decompose.decompose(sym, candidates={sym: BROAD_SLEEVES})
+            d[k]["ref_5y"] = _fmt(rec)
+            d[k]["ref_full"] = _fmt(full)
+        except Exception as e:  # noqa: BLE001 - batch must not die
+            d[k]["ref_5y"] = f"(backfill failed: {e})"
+        if i % 25 == 0 or i == len(todo):
+            SRC.write_text(json.dumps(d, indent=1))
+            log(f"{i}/{len(todo)} done (saved)")
+    SRC.write_text(json.dumps(d, indent=1))
+    n = sum(1 for v in d.values()
+            if isinstance(v, dict) and v.get("ref_5y"))
+    log(f"done: {n} rows carry a reference")
+
+
+if __name__ == "__main__":
+    run()