Greg Pomerantz
16c39f4221
Add the invariant that GlyphLayout offsets are relative to IMEWindowStartByte and that cursor positioning must add/subtract the window base. Record the second tap-to-cursor fix (base offset) alongside the earlier tapLocalY fix in the development plan.
344 lines
19 KiB
Markdown
344 lines
19 KiB
Markdown
# Runtime Architecture
|
||
|
||
This document describes how Pad actually works: the concurrency model, goroutine
|
||
responsibilities, the frame handoff contract, ownership rules, and the internal
|
||
design of the editor and browser. It is written at the level of *invariants and
|
||
contracts*, not line-by-line code, so it stays true as implementation details
|
||
evolve. If a detail below conflicts with the code, the code wins — and this
|
||
document should be fixed.
|
||
|
||
## 1. Concurrency model: single owner, no locks on state
|
||
|
||
The **logic goroutine is the sole owner (reader and writer) of mutable
|
||
`State`**. There is no `sync.Mutex`/`sync.RWMutex` on application state. Every
|
||
other goroutine talks to the owner through channels.
|
||
|
||
```
|
||
┌────────────────────────────────────────────────────────────────────┐
|
||
│ Main goroutine (Gio event loop) │
|
||
│ - w.Event() loop: Config / Frame / Destroy │
|
||
│ - On FrameEvent: lock(handoff) → read Frame snapshot → │
|
||
│ Renderer.Draw → collect gestures + key events → │
|
||
│ e.Frame → unlock(handoff) → send inputs/config/query/layout │
|
||
│ to logic channels (sends happen OUTSIDE the lock) │
|
||
└───────────────▲──────────────────────────────┬─────────────────────┘
|
||
│ handoff lock (frame storage) │ inputChan, configChan,
|
||
│ + w.Invalidate() │ layoutChan, searchQueryChan
|
||
┌───────────────┴───────────┐ ▼
|
||
│ frameReceiver goroutine │ ┌──────────────────────────────────────┐
|
||
│ tight loop: │ │ Logic goroutine (SOLE OWNER of State)│
|
||
│ f := <-frameChan │ │ - select over all input channels │
|
||
│ lock; *frame = f; │ │ - mutates State, builds []Element │
|
||
│ w.Invalidate(); unlock │ │ - sends Frame on frameChan │
|
||
└───────────────────────────┘ │ - dispatches tasks to worker pool │
|
||
└───────────────┬──────────────────────┘
|
||
│ task dispatch / results
|
||
┌───────────────▼──────────────────────┐
|
||
│ Worker pool (8 goroutines) │
|
||
│ - priority: high > medium > low │
|
||
│ - file I/O, directory index, line │
|
||
│ index build, autosave writes │
|
||
│ - results posted on ResultChan │
|
||
└──────────────────────────────────────┘
|
||
```
|
||
|
||
Key invariants:
|
||
|
||
1. **No locks on `State`.** The only mutex is the *handoff* mutex in
|
||
`cmd/pad/main.go`, which guards the one-frame storage slot between
|
||
frameReceiver and the main goroutine. It is a message-passing handoff, not
|
||
a state lock; it is never held while touching a channel.
|
||
2. **The main goroutine never reads logic `State` directly.** It reads only
|
||
the latest `Frame` snapshot (see §4) and writes only via channels.
|
||
3. **Non-owner goroutines never mutate `State`.** They send a request
|
||
(channel) or run a callback on the owner (`Inspect`, test-only).
|
||
4. **The autosave timer goroutine reads nothing.** `time.AfterFunc` only sends
|
||
a token on `autosaveChan`; the owner does all state reads and dispatches
|
||
the write.
|
||
5. **Gio-mutable widget state lives in the main-goroutine-owned `Renderer`**,
|
||
never in logic-owned `State` (see §5).
|
||
6. **Logic work is synchronous and fast.** Processing input, mutating state,
|
||
building the element tree, and sending a frame must stay well under one
|
||
display refresh (~16 ms). File I/O and index building always go to the
|
||
worker pool.
|
||
|
||
## 2. Goroutine responsibilities
|
||
|
||
### 2.1 Main goroutine (`cmd/pad/main.go`)
|
||
|
||
- Runs the Gio event loop.
|
||
- On `app.ConfigEvent`: sends `ConfigEvent{PixelWidth, PixelHeight}` to
|
||
`logic.ConfigChan()`.
|
||
- On `app.FrameEvent`:
|
||
1. Reads `newScale = gtx.Metric.PxPerDp`; if it differs from
|
||
`frame.Scale`, sends `ScaleEvent` (after the draw).
|
||
2. Under the handoff lock: reads the `Frame` snapshot, calls
|
||
`renderer.Draw(gtx, frame.Elems, scale)`, then `renderer.CheckGestures`
|
||
and the focused element's key/edit events (via `key.Filter` +
|
||
`key.FocusFilter`), then `e.Frame(&ops)`.
|
||
3. Outside the lock: sends `[]ui.InputEvent` (if any) to `InputChan`,
|
||
the search text (if it differs from `frame.Query`) to `SearchQueryChan`,
|
||
and `renderer.GlyphLayout()` to `LayoutChan`.
|
||
- Owns `frameReceiver` (started in `run`).
|
||
|
||
### 2.2 Frame receiver
|
||
|
||
Tight loop: read `logic.FrameChan()`, store into the shared `Frame` under the
|
||
handoff lock, call `w.Invalidate()`. The logic goroutine's send on `frameChan`
|
||
(bufsize 1) never blocks for more than one frame cycle.
|
||
|
||
### 2.3 Logic goroutine (`internal/editor/logic.go`)
|
||
|
||
Sole owner of `*State`. `Run()` selects on:
|
||
|
||
| Channel | From | Payload | Action |
|
||
|---|---|---|---|
|
||
| `configChan` | main | `ConfigUpdate` (pixels or scale) | store scale/pixels; recompute layout |
|
||
| `inputChan` | main | `[]ui.InputEvent` | run each `Handler(evt.Data)` on the owner |
|
||
| `searchQueryChan` | main | `string` | store `Browser.Query`, re-filter, new frame |
|
||
| `openFileChan` | main (tap) | `string` path | open file in editor (chunked buffer + async index) |
|
||
| `layoutChan` | main | `ui.GlyphLayout` | store on editor state; derive `LastLineY` for scroll clamping |
|
||
| `retryChan` | logic | `string` filename | autosave retry |
|
||
| `autosaveChan` | timer | `struct{}` token | reconstruct content, dispatch `WriteFile` task |
|
||
| `workerPool.ResultChan()` | pool | task `Result` | apply async results (stat, read, index, write ack) |
|
||
| `inspectChan` | tests | `inspectReq` | run `fn(*State)` **on the owner** and reply (test-only) |
|
||
| `resultChan` | legacy | `ResultEvent` | drained (legacy channel; the live result path is the pool's) |
|
||
| `done` | main | — | drain pool, stop timer, exit |
|
||
|
||
After every state change the logic goroutine rebuilds the element tree and
|
||
sends a `Frame` on `frameChan`.
|
||
|
||
### 2.4 Worker pool (`internal/io/pool`)
|
||
|
||
Fixed 8 workers, two priority lanes (high > low; dispatch routes
|
||
`HighPriority` tasks to the high lane, everything else to low). Tasks are small
|
||
structs with `Execute() Result`; the `FileSystem` interface (`pool.FileSystem`)
|
||
has a real implementation (`io/pool/real`, rooted at `/`) and a mock
|
||
(`io/pool/mock`) for tests.
|
||
|
||
Task types in active use: `StatFile`, `ReadFile`, `BuildLineIndex`,
|
||
`WriteFile` (editor); `BuildIndex`, `LoadPages` (browser). Dormant/dead task
|
||
types — `ReadChunk` (no-op for fully-loaded in-range files), `ReadDir`,
|
||
`StatDir`, `ReadCache`, `WriteCache`, `Invalidate`, `SaveState`, `SaveUndo` —
|
||
are candidates for removal in a cleanup round.
|
||
|
||
## 3. Channel topology summary
|
||
|
||
- **main → logic:** `InputChan`, `ConfigChan`, `LayoutChan`, `SearchQueryChan`,
|
||
`OpenFileChan`.
|
||
- **timer → logic:** `autosaveChan` (token only).
|
||
- **logic → logic:** `retryChan` (self-piped).
|
||
- **logic → frameReceiver:** `FrameChan` (the only outbound state carrier).
|
||
- **pool → logic:** `WorkerPool.ResultChan()`.
|
||
- **tests → logic:** `Inspect` (owner-executed callback; the production
|
||
equivalent is "send a request channel message").
|
||
|
||
## 4. The Frame handoff contract
|
||
|
||
`editor.Frame` is the **only** data that crosses from the logic side to the
|
||
main side:
|
||
|
||
```go
|
||
type Frame struct {
|
||
Elems []ui.Element // element tree for the next draw
|
||
Scale float32 // current px-per-Dp
|
||
FocusedElementID string // which registered element gets key/edit events
|
||
Query string // search text logic is filtering with
|
||
}
|
||
```
|
||
|
||
- `Elems` is the computed tree; the renderer draws it (it is a snapshot, safe
|
||
to read from main).
|
||
- `Scale` lets main detect a density change and pass scale to `Renderer.Draw`.
|
||
- `FocusedElementID` drives which registered `key.Filter`/`key.FocusFilter`
|
||
main uses to harvest keyboard/IME events.
|
||
- `Query` lets main compare against the (main-owned) search `widget.Editor`
|
||
text and forward changes — the browser search box is a Gio widget, so its
|
||
text lives on the main side, not in `State`.
|
||
|
||
## 5. Ownership rules
|
||
|
||
| Owned by | What |
|
||
|---|---|
|
||
| **Logic goroutine** | `State` (browser + editor + chunked buffers + line indexes), worker pool, autosave timer handle |
|
||
| **Main goroutine** | `*app.Window`, `op.Ops`, text shaper, `ui.Renderer` (gesture state, IME dedup state, glyph layout cache), the search-bar `widget.Editor` |
|
||
|
||
Rules:
|
||
|
||
1. **Gio mutates widgets during draw, so anything Gio mutates must be
|
||
main-owned.** The search bar is a `widget.Editor` registered with the
|
||
renderer by ID (`"search_bar"`); its text reaches logic only through
|
||
`SearchQueryChan`.
|
||
2. **The editor content is NOT a `widget.Editor`.** It is the custom
|
||
`ui.TextField` element; all editor state (buffer, cursor, scroll) is
|
||
logic-owned. This is why the editor does not hit `widget.Editor`'s
|
||
O(n²)-ish cost on large files.
|
||
3. **Per-frame element values are rebuilt by logic** (`[]ui.Element` in the
|
||
`Frame`); any cross-frame persistent draw-side state (IME dedup, gesture
|
||
tracking) belongs in the persistent, main-owned `Renderer`.
|
||
|
||
## 6. Editor internals (`internal/editor`)
|
||
|
||
### 6.1 Chunked buffer
|
||
|
||
`ChunkedBuffer` (`chunked_buffer.go`) is the edit buffer for open files.
|
||
|
||
- **Full load on open** for in-range files: the whole file is read into an
|
||
ordered slice of chunks (64 KB) plus **prefix-sum byte offsets** over actual
|
||
chunk lengths. There is no lazy loading and no eviction (both existed as
|
||
plans and were removed).
|
||
- **Byte-indexed** throughout: `CursorPosition`, chunk offsets, and glyph
|
||
`ByteOffsets` are byte offsets.
|
||
- Edits splice the affected chunk(s) only; chunks are not rebalanced.
|
||
- `LineIndex` (per-line byte offsets, `int32`) is built asynchronously by
|
||
`BuildLineIndexTask` and stored on `cb.LineIndex` — the single source of
|
||
truth (the old parallel `EditorState.LineIndex` field was removed).
|
||
- **Size guard:** files larger than `MaxEditableFileSize` (**50 MB**,
|
||
measured on-device) open into a `TooLarge` state: the editor shows a notice
|
||
and edit handlers are no-ops; the browser still lists the file.
|
||
|
||
### 6.2 Virtualized viewport
|
||
|
||
Only the visible byte range is shaped and drawn each frame:
|
||
|
||
- `VisibleByteRange` maps scroll offset + viewport height → `[startLine,
|
||
endLine]` via the `LineIndex`, then to a byte range. The range is always
|
||
bounded by real lines of the document.
|
||
- **Never shape the whole file.** Lesson learned (Phase 3): the shaper's
|
||
internal `document` retains the backing array of its largest layout forever
|
||
(`reset()` keeps the cap), so one whole-file layout permanently inflated
|
||
memory to ~1 GB for a 10 MB file. Shaping ~50 lines keeps it small and flat.
|
||
- The renderer reports `GlyphLayout` back via `LayoutChan`; logic derives
|
||
`LastLineY` for scroll clamping and the max scroll offset.
|
||
- **`GlyphLayout` offsets are window-relative.** The shaper only sees the
|
||
visible window, so `GlyphLayout.ByteOffsets` (and `VisualLineStarts`, `Y`)
|
||
are relative to `IMEWindowStartByte`, not the file start. Any logic-goroutine
|
||
code that maps a glyph offset to the absolute `CursorPosition` (tap-to-position,
|
||
Home/End, vertical cursor move) must add the window base
|
||
(`IMEWindowStartByte`) before storing the cursor and subtract it before
|
||
searching the offsets. For a whole-file window the base is 0 (a no-op).
|
||
Getting this wrong snaps the cursor to the window top on scrolled large files.
|
||
|
||
### 6.3 IME (Android soft keyboard)
|
||
|
||
- The editor exposes to the IME a **windowed snippet**: `IMEWindowText` is the
|
||
visible viewport text, `IMEWindowStartByte` its absolute start. While the
|
||
`TextField` is focused, `drawElement` emits `key.SnippetCmd` (the window)
|
||
and `key.SelectionCmd` (caret as a rune index into the window).
|
||
- **Dedup in the Renderer** (main-owned state): the snippet/selection are
|
||
re-emitted only when they actually change, and a fresh push is forced when
|
||
the field (re)gains focus. This mirrors `widget.Editor`'s behavior and
|
||
prevents per-frame re-push from resetting IME composition on rapid commits.
|
||
- Incoming `key.EditEvent{Range, Text}` has **window-relative rune
|
||
indices**; `HandleReplaceRange` converts them to absolute byte offsets
|
||
(`RuneIndexToByte`) and splices the chunked buffer. Insertion,
|
||
`deleteSurroundingText` (backspace/autocorrect replacement), and composition
|
||
all arrive through this one path.
|
||
|
||
### 6.4 Autosave
|
||
|
||
- Any edit calls `markDirty()`: a 1 s debounce timer; each keystroke restarts
|
||
it. On expiry the timer goroutine sends a token on `autosaveChan`; the owner
|
||
reconstructs the full content from the chunked buffer and dispatches a
|
||
`WriteFile` task.
|
||
- Write failures are tracked per file (`writeFailed`, `retryAttempts`) and
|
||
retried via `retryChan`. There is no save button; autosave is the only
|
||
persistence.
|
||
|
||
### 6.5 Opening a file
|
||
|
||
- A browser tap sends the path on `OpenFileChan`. The logic goroutine creates
|
||
the `ChunkedBuffer`, dispatches `StatFile` (size guard) and the read +
|
||
`BuildLineIndex` tasks, switches `page` to the editor, and sets
|
||
`justOpenedAt`.
|
||
- **Opening-tap swallow:** the tap that opens the file is also delivered as an
|
||
editor tap in the same frame. A short time window (`justOpenedAt`) swallows
|
||
it so the viewport does not jump to the tapped (often EOF) position.
|
||
|
||
## 7. Browser internals (`internal/browser`)
|
||
|
||
- `BrowserState` is **embedded by value** in `State` (single owner; no
|
||
pointer indirection).
|
||
- `BrowserManager` drives async directory loading through the worker pool:
|
||
`ReadDir` + `BuildIndex` on navigation, `LoadPages` for pagination.
|
||
- Features: 4 sort modes (name/date × asc/desc; default newest-first),
|
||
incremental case-insensitive search over names, directory navigation with
|
||
back, single-tap file open.
|
||
- The browser renders via `ListView`/`ListItem` elements; rows carry
|
||
`Interaction` handlers that send channel requests (tap) or mutate state
|
||
directly when the handler runs on the owner (scroll).
|
||
|
||
## 8. Render pipeline (`internal/ui`)
|
||
|
||
- **Logic** builds a `[]ui.Element` tree. Elements are value types; interactive
|
||
ones carry `Interaction{Gesture, Handler}` entries. `Handler` is a static
|
||
function that mutates `TheState` — handlers run on the logic goroutine only.
|
||
- **Main** draws via `Renderer.Draw(gtx, elems, scale)`:
|
||
1. `drawElement` recursively walks the tree, applying transforms/clips.
|
||
2. Interactive elements register hit regions in the *current clip context*
|
||
(`gesture.Click.Add` / `pointer.InputOp`), so hit-testing always matches
|
||
the drawn geometry.
|
||
3. Text elements shape through the shared shaper — **only the visible
|
||
window**, never the whole buffer.
|
||
4. `e.Frame(&ops)` flushes ops.
|
||
- After draw, `CheckGestures` turns raw pointer/gesture state into
|
||
`[]ui.InputEvent` (each carrying its own handler) for the logic channel.
|
||
- **Units:** `ui.Dp`/`ui.Px` convert via the current `PxPerDp` scale
|
||
(`ToDp`/`ToPx`). The window is 390×844 dp; the real pixel size arrives via
|
||
`ConfigEvent`.
|
||
- **Live element catalog:** `Container`, `Label`, `Icon`, `TextField` (editor
|
||
window), `ListView`/`Line`/`ListItem` (browser rows), `GioEditor` (search
|
||
bar), `Cursor`. Types `Button`, `Toast`, `Spacer`, `AlphaIndex`,
|
||
`Selection`, `MergeHunk` exist in `element.go` but are not used by any page
|
||
— future work, not current behavior.
|
||
- **Theme:** `ui.Theme{FontSize: 14}` + palette in `element.go`.
|
||
|
||
## 9. Testing hooks
|
||
|
||
- `Logic.Inspect(fn)` (test-only): runs `fn(*State)` **on the owner** and
|
||
returns its result — the sanctioned way for tests to read state without
|
||
breaking single ownership. It must never be called from production code.
|
||
- The e2e harness (`internal/test/e2e`) drives the **real** `Logic` + worker
|
||
pool + mock filesystem through the real channels, and asserts via
|
||
`Inspect`. It is logic-only: it does not exercise `Renderer.Draw`, so
|
||
draw-path behavior (IME dedup, gesture routing) is validated on-device with
|
||
adb (see `development_plan.md` §Phase 2/3 observation loop).
|
||
- CI gate: `go build ./... && go vet ./... && go test -race ./...`.
|
||
|
||
## 10. Known loose ends (code, not spec)
|
||
|
||
- Globals `TheState` / `TheLogic` / `ui.OpenFile` still exist (handlers are
|
||
static funcs); replacing them with explicit state is a planned cleanup
|
||
(`development_plan.md` Phase 4).
|
||
- Dead task types (`ReadChunk`, `SaveState`, `SaveUndo`, …) and unused element
|
||
types are candidates for removal in the same round.
|
||
|
||
## 11. Performance profiler (default-off, `internal/perf`)
|
||
|
||
Pad ships a built-in profiler that is **off by default** and costs nothing when
|
||
off. It is the in-app frame-timing tool (see `development_plan.md` §9: `gfxinfo`
|
||
cannot measure this app because it renders into a `SurfaceView`).
|
||
|
||
- **Enable** by creating the marker file
|
||
`/storage/emulated/0/PadPerf/enable` before launch. `main.go` then creates a
|
||
`perf.Profiler` that writes `logic_frames.csv` (one row per logic frame: seq,
|
||
ms-since-start, frame delta, page, scroll Dp, max-scroll Dp, total lines,
|
||
visible byte range) and logs a rolling ~1 s `PERF` summary. `DestroyEvent`
|
||
stops it (final flush + `PERF stopped` summary with p50/p90/p99/max).
|
||
- **Hook**: `internal/editor.PerfRecord` is a package-level func, set by
|
||
`main.go` only when enabled. `Logic.emitFrame` calls it on the owner
|
||
goroutine with a `ProbeRecord` **before** sending the frame. When disabled it
|
||
is `nil` and the per-frame cost is a single nil check.
|
||
- **Debug scroll jumps** (for testing, off by default): with the profiler on,
|
||
`main.go` also polls `/storage/emulated/0/PadPerf/cmd` (a one-shot file
|
||
consumed on read). `top`, `bottom`, `frac <0..1>`, and `dp <int>` jump the
|
||
editor's `ScrollOffset` (clamped to `[0, MaxScroll]`) and emit a frame. This
|
||
lets a test drive large-offset scrolls deterministically without pixel taps.
|
||
- The profiler is owned by the goroutine that creates it and is single-goroutine
|
||
(no locks). It does **not** `Sync()` the CSV per flush (only per row batch) to
|
||
avoid periodic fsync hitches in the logic path.
|
||
- **Measured** (emulator, 10 MB file, 2026-08): logic-frame cadence is flat
|
||
across scroll offsets 0.02→1.0 (no large-offset degradation); the visible byte
|
||
range stays ≤ ~4.3 KB (0.04% of the file); PSS plateaus ~250 MB (bounded
|
||
high-water mark, no leak). See `development_plan.md` Phase 6.
|