Pad/doc/development_plan.md

# Development Plan: reach a lean, usable Android text editor

Status: v2, 2026-08-16. Written against the **live** repo `/home/gmp/pad`
(HEAD 03fb638). v1 (the widget-rebuild plan) is superseded — see §12 for why.

## 1. Decision summary (updated)

- **Primary path: B — finish the live custom path, do NOT rebuild on widgets.**
  The live repo already has the two hardest, most-asset-heavy pieces that a
  rebuild would throw away: a **real filesystem** backend (`internal/io/pool/real/`)
  and a **working Android APK** (JNI + permissions, built + signed). It also has
  a **chunked buffer** (`internal/editor/chunked_buffer.go`) that already implements
  the spec's "file never fully in memory" — the thing v1 said would need fork-level
  work on `widget.Editor`. So the only blocker to the headline feature (Android IME:
  swipe typing, autocorrect) is a **small, precisely-defined wiring gap** (§4), not a
  rewrite.
- **Concurrency:** keep the existing logic-goroutine + channels model for now. It's
  proven to build and (after this plan) pass tests. The v1 "one main goroutine"
  simplification is a *later* cleanup, not a prerequisite for usability.
- **File-size limit:** the chunked buffer targets large files directly; we validate
  it against a real 10 MB file on-device and set the honest limit from that, rather
  than inheriting `widget.Editor`'s ~0.5 GB/MB wall (§2).

## 2. Evidence (verified against gioui.org v0.10.0 sources, the live repo's version)

1. **`widget.Editor` is not virtualized** — it shapes the *entire* document per
   invalidation and keeps a ~200 B/rune in-memory `glyphIndex`. Measured on this
   box: 1 MB ≈ 17 ms/keystroke & ~0.32 GB; 3 MB ≈ 1.7 GB; 4 MB OOM-killed the host.
   Planning figure **~0.5 GB of RAM per 1 MB of text**. A phone cannot edit 10 MB
   in `widget.Editor`. This is *why* the live repo went the chunked-buffer route,
   and it's the reason v1's "just use the widget" was wrong for large files.
2. **The Android IME works through the op layer, not the widget.** The
   `window`'s editor state (`app.Window` → `input.EditorState{Selection, Snippet}`)
   is what `GioInputConnection` (Android's `InputConnection`) reads. It is fed by
   `key.SelectionCmd` and `key.SnippetCmd` ops tagged with a **focusable** handler.
   A handler becomes focusable by emitting a `key.FocusFilter{Target: tag}` (that's
   all `widget.Editor` does — it never touches `io/input` at all). The live code
   already emits `key.FocusCmd{Tag}` + `key.FocusFilter{Target}` and consumes
   `key.EditEvent`/`key.SnippetEvent`, so the *skeleton* is present; the stateful
   ops are missing (§4).
3. **The IME snippet is selection-scoped** — so a chunked/virtualized editor can
   still feed a correct snippet (only the visible/selected window), which the live
   chunked-buffer design is compatible with.
4. **No headless test window** in v0.9 or v0.10 — widget-level Go tests are
   impossible; on-device e2e is required (§9).
5. **The dev agent has no vision** (verified) — the emulator debug loop is
   data-based (state dumps, logcat, gfxinfo, PIL/OCR), not screenshot-judgment
   (§9.3).

## 3. Current state of the live repo (post housekeeping, 2026-08-16)

- `go build ./...` green. **`go test ./...` now green** (was red: `internal/editor`
  and `internal/test/e2e` failed to build after `NewLogic` gained a `path` and
  `openfunc` parameter). Commit `03fb638` fixed the harness + 5 call sites; the e2e
  harness now uses a no-op `openfunc` (matching `impl_other.go`, whose real
  `OpenFile` is a no-op off-Android).
- Uncommitted (yours, left untouched): JNI/Termux bridge, chunk-buffer + logic
  changes. **Recommend committing these** so the working tree is a clean baseline
  before the IME work.

## 4. The IME wiring gap (verified, testable) — THE central work item

Android IME text (swipe, autocorrect) arrives as `key.EditEvent{Range, Text}`
routed to the focused tag. Four things are missing or wrong:

1. **No `key.SelectionCmd` emitted.** Without it the driver's
   `EditorState.Selection.Caret` stays at origin, so the IME doesn't know where the
   caret/selection is → suggestions anchor wrongly and the caret jumps after a
   commit. **Fix:** in the editor's draw/update pass, emit
   `gtx.Execute(key.SelectionCmd{Tag, Range:{cursor,cursor}, Caret:{Pos,Ascent,Descent}})`
   whenever the caret moves. The caret's **pixel** position already exists in
   `internal/ui/render.go:534-554` (the code that draws the caret) — it must be
   plumbed into `app.State` (caret pos in px) and read back here. This is the one
   item coupled to the cursor-positioning work, but it reuses the same geometry.
2. **No `key.SnippetCmd` emitted.** Without it the IME receives an empty snippet →
   no autocorrect context and no in-place swipe replacement. **Fix:** emit
   `gtx.Execute(key.SnippetCmd{Tag, Snippet:{Range: selection, Text: selectedText}})`;
   with no selection, an empty snippet at the caret (exactly what `widget.Editor`
   sends). Feeding it from the chunked buffer is O(selection size), not O(file).
3. **`HandleKeyDown` ignores `EditEvent.Range`.** `state.go:313` does
   `HandleInsert(v.Text)` for any non-backspace text and never deletes
   `v.Range.Start..End`. A *replacement* commit (the normal swipe/autocorrect case)
   therefore inserts the new text **without removing the old** → duplication/corruption.
   **Fix:** `Replace(v.Range.Start, v.Range.End, v.Text)` on the chunked buffer and
   advance the cursor to `Start + len(Text)`. (~10 lines; `ChunkedBuffer` already
   has `Insert`/`Delete`.)
4. **No `key.InputHintOp{Tag, Hint: key.HintText}`.** Cosmetic but correct to add —
   hints the on-screen keyboard into text/autocorrect mode.

Acceptance (all verifiable headlessly, §9): after an IME commit, the buffer equals
the expected post-replacement text, the caret is at the commit end, and the next
frame's `SelectionCmd`/`SnippetCmd` reflect that state. On a real IME, swipe +
autocorrect produce clean, non-duplicated text.

## 5. Phases (Path B)

### Phase 0 — clean baseline (small)
1. Commit the uncommitted JNI/chunk-buffer work.
2. `go test -race ./...` green (add `-race`; the channel model may surface races —
   fix any found).

### Phase 1 — complete the IME (§4)
Implement items 1–4, with **Go unit tests** for item 3 (range-replace on the
chunked buffer is pure and directly testable) and a state-assertion test for
items 1/2 (after N frames, `app.State` holds the expected caret/selection and the
next emitted op set is correct). This is the make-or-break feature.

### Phase 2 — emulator verification (the observation loop, §9.3)
1. Install Android SDK + NDK + platform-tools; create an x86_64 AVD (16 GB VM,
   KVM verified present).
2. Build the APK (`GOOS=android go build ./cmd/pad` with `ANDROID_NDK_HOME`),
   install, launch.
3. Drive + observe: state-dump debug flag (add in Phase 1 as a debug build),
   logcat, `dumpsys gfxinfo`, screenshot→PIL/OCR.
4. **IME experiment with the AOSP keyboard:** `input swipe` over the keyboard area
   and verify behaviorally (did the commit land, non-duplicated, cursor correct?)
   via state dump + logcat — this is what confirms §4 works end to end.

### Phase 3 — large-file validation + honest size limit
1. Open a real **10 MB** file via the chunked buffer; verify smooth scroll + edit
   and measure RAM on-device.
2. Set the documented limit from that measurement; guard larger files with a
   clear "too large to edit" state (browser can still list/preview).

### Phase 4 — the v1 simplifications, now that it's usable (optional, later)
Only after the app is usable: replace globals (`TheState`, `ui.OpenFile`) with
explicit state, prune dead task types, and *then* consider collapsing the
logic-goroutine/channel model to a single owner. These reduce future bug surface
but are not needed for a usable v1.

### Phase 5 — hardening
Rewrite `doc/architecture.md` to match reality; amend `spec.md` per §7; README
(build/install recipe); in-repo device test checklist.

## 6. File-size decision (re-framed)

v1 framed this as "accept a limit vs build a windowed editor." The live repo
already chose the better option — a **chunked buffer** (only viewport ± window
chunks resident, async prefetch, dirty-chunk protection, async line index). So the
decision is now: **validate the existing chunked buffer against a real 10 MB file
(Phase 3) and set the limit from the measurement.** No fork of Gio is on the
critical path. If the chunked buffer proves out (likely), 10 MB+ editing is in
reach without the 0.5 GB/MB wall that kills `widget.Editor`.

## 7. Spec deltas (to write into spec.md)

1. "No practical limits / file never fully in memory" → backed by the chunked
   buffer; add a measured hard limit from Phase 3.
2. Undo across sessions → dropped (in-session only; chunked buffer does not keep
   full-document undo copies).
3. Add: IME/swipe/autocorrect support (now explicit, delivered in Phase 1).
4. Add: external-change detection as the conflict policy (mtime compare on open
   + on resume; changed+clean → reload, changed+dirty → Keep/Reload prompt).

## 8. Non-goals (v1)

- File-system watcher / live browser refresh (re-scan on return to browser).
- Tabs, split view, search-in-file, syntax highlighting, diff/merge.
- Desktop/other platforms (Android-first; window `390×844` dp).
- Rewriting on `widget.Editor` (v1 plan) — kept only as fallback (§12).

## 9. Testing strategy

The **existing** e2e harness is **kept** (it builds and passes now) and extended —
v1's "delete the harness" is wrong for Path B. E2E regression works in three layers:

### Layer 1 — Go unit tests on pure logic (the bulk)
- Port/keep `browser` sort/search/scan tests; keep `editor` chunked-buffer,
  cursor, autosave tests.
- **New:** range-replace on `ChunkedBuffer` (§4.3) — pure, directly testable; the
  exact corruption bug becomes a regression test.
- **New:** IME state transitions — after an `EditEvent{Range,Text}`, assert
  buffer + caret; assert the emitted `SelectionCmd`/`SnippetCmd` reflect state.
- **New:** restore JSON round-trip; offset-vs-mtime validation; external-change
  decision table; file-size guard.
- CI gate: `go build && go vet && go test -race ./...`.

### Layer 2 — UI thin by construction
Draw functions are wirings of the existing element model over `app.State`;
positioning math stays in one place (`render.go`). The cursor/caret geometry that
§4.1 reuses is the same code path as drawing, so IME correctness and visual
correctness share a foundation.

### Layer 3 — scripted on-device e2e (emulator now available)
The in-app selftest drives the **real state machine through the real draw path**
(browser entries → open file → edit → autosave deadline → reload-prompt decision),
logging PASS/FAIL to a file the host `adb pull`s and asserts.

### 9.3 Emulator observation layer (this 16 GB VM, KVM verified; agent has NO vision)
Gio renders into one GL surface, so Android's view hierarchy exposes nothing about
our UI. The debug loop is data-based — *more* precise than pixels for this codebase:
1. **State-dump debug flag** (add in Phase 1): debug builds write `app.State`
   (browser entries, editor text length, cursor px, scroll, dirty, row geometry)
   as JSON to the app data dir on a magic tap or 2 s interval; host `adb pull`s
   and asserts. Primary "eyes."
2. **logcat** — Go panics + our logs.
3. **`dumpsys gfxinfo <pkg>`** — per-frame timing/jank for the 60 fps claims.
4. **Screenshot → PIL color histogram / tesseract OCR** — coarse on-screen checks.
5. **File diffs** via `adb pull` — autosave/restore correctness.
6. **Driving** — `input tap/swipe/keyevent/text`, `am start/force-stop`,
   `ime`/`settings` to pick the AOSP keyboard; swipe-typing exercised via
   `input swipe` over the keyboard area and verified **behaviorally** (did the
   commit land, non-duplicated, cursor correct?), not visually.

## 10. Definition of done (whole effort)

On an emulator (and a real phone for final IME sign-off): launch → previous file
+ caret restored → browse a large directory smoothly → open a real file →
**swipe-type with autocorrect, clean and non-duplicated** → close and reopen, caret
where you left it → external modification → reload prompt → autosave lands on disk
within ~1 s. Repo: `go test -race ./...` green.

## 11. Risks / watch-items
- **§4.1 caret plumbing** is coupled to the cursor-positioning geometry; if that
  geometry is itself buggy, the IME will inherit it. Mitigation: the state-dump
  reports caret px, so we can assert it against expected values in tests.
- **Channel model + `-race`** (Phase 0): the existing goroutine/channel design may
  have latent races; running `-race` before IME work avoids conflating them.
- **Chunked buffer at 10 MB** (Phase 3): unproven on-device; the measurement is
  the gate for the size claim.
- **AOSP keyboard** is a proxy for real IMEs (Gboard, etc.); final swipe/autocorrect
  sign-off needs a real device with a real IME.

## 12. Why v1 (widget rebuild) is now a fallback, not the plan
v1 was written against a stale snapshot (May 31) and concluded "delete
`internal/ui`, build on `widget.Editor`." Two later-verified facts changed that:
(a) the live repo already solved the two hardest parts a rebuild would discard
(real FS, APK, and the chunked buffer that `widget.Editor` cannot do), and (b) the
Android IME is reachable from the op layer with only the §4 gap — so the "free IME"
that motivated the rebuild is actually ~4 small wirings away on the existing code.
**Path A (widget rebuild) remains the fallback if the Phase 1 IME wiring proves
fragile on-device** (e.g., the caret/snippet plumbing drags in a long tail of
cursor-positioning bugs that are cheaper to not own). The emulator (Phase 2) is the
instrument that makes that call.