gmp/Pad
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Doc: reorganize — delete over-detailed docs, rewrite spec + architecture to match code

Browse Source 
- Deleted (drift-prone point-in-time plans / dead specs):
  browser_implementation_plan.md, editor_implementation_plan.md,
  virtual_scroll_render_optimization.md, conflict_resolution.md,
  touch.md, element_model.md, layout_rendering.md, bugs.txt
- Rewrote architecture.md: single-owner/no-lock model, channel topology,
  Frame handoff contract, ownership rules, editor/browser/render internals,
  active vs dead task types, testing hooks.
- Rewrote spec.md: actual behavior (browser, editor, IME, autosave, 50 MB
  limit with measured numbers), actual code layout, invariants to preserve,
  explicit deferred-features table (undo, restoration, sync awareness).
- Added doc/README.md: doc index, documentation policy, build/install
  recipe, on-device observation loop.
- Updated development_plan.md: Phase 5 mostly done; §7 spec deltas written
  with two corrections (no undo at all; no external-change detection).
This commit is contained in:
Greg Pomerantz 2026-08-16 13:22:21 -04:00
parent c2918fa7c1
commit d3d11b5d20
12 changed files with 492 additions and 5450 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

69
doc/README.md Normal file
View File

@ -0,0 +1,69 @@
# Pad documentation
Three documents, kept at the level of *what, why, and invariants* — not
line-by-line code — so they stay true as the implementation evolves.
| Doc | What it is |
|---|---|
| [`spec.md`](./spec.md) | What the app **actually does**, measured performance, the code layout, and an explicit list of deferred features. |
| [`architecture.md`](./architecture.md) | How it works: single-owner concurrency model, channel topology, Frame handoff contract, ownership rules, editor/browser/render internals. |
| [`development_plan.md`](./development_plan.md) | The active plan: completed phases, remaining work, and the on-device observation loop. |
## Documentation policy
1. **Docs describe invariants and contracts, not code lines.** If a document
has to change on every refactor, it is too detailed — delete it or raise
its level of abstraction. Point-in-time implementation plans are deleted
once implemented (the previous `*_implementation_plan.md`,
`touch.md`, `element_model.md`, `layout_rendering.md`,
`virtual_scroll_render_optimization.md`, and `conflict_resolution.md` were
removed in the 2026-08 doc reorganization for this reason).
2. **Unbuilt behavior is not spec'd.** Requirements that are not in the code
live in `spec.md` §7 (deferred), never as if they worked.
3. **Code wins.** When doc and code disagree, fix the doc.
## Build & install (Android, this workstation)
Prereqs on this box (already installed): JDK 17, Go, Android SDK at
`~/android-sdk` (platform-tools, build-tools 35.0.0, emulator, NDK),
`gogio@v0.10.0` in `~/go/bin`, `apktool` at `/tmp/apktool.jar`, debug
keystore at `~/.android/debug.keystore`. Env vars are set in `~/.bashrc`.
The full recipe is `/tmp/build_pad.sh` (machine-local). Steps:
1. `gogio -target android -targetsdk 35 -arch amd64 -o /tmp/pad-raw.apk .`
(from `cmd/pad/`)
2. `apktool d /tmp/pad-raw.apk -o /tmp/pad_decoded -f`
3. Inject `<uses-permission android:name="android.permission.MANAGE_EXTERNAL_STORAGE"/>`
into the decoded `AndroidManifest.xml` (Android 15 needs it for
`/storage/emulated/0`).
4. `apktool b /tmp/pad_decoded -o /tmp/pad-unsigned.apk`
5. `apksigner sign --ks ~/.android/debug.keystore --ks-pass pass:android
--key-pass pass:android --ks-key-alias androiddebugkey --out cmd/pad/pad-emu.apk /tmp/pad-unsigned.apk`
6. `adb install -r cmd/pad/pad-emu.apk`
Run the emulator (headless, AVD `pad_avd`, API 35):
```
nohup emulator -avd pad_avd -no-window -no-audio -no-boot-anim \
-gpu swiftshader_indirect >/tmp/emulator.log 2>&1 &
```
App package/activity: `pad.pad / org.gioui.GioActivity`. Default root
directory: `/storage/emulated/0/Notes` (push test files there with
`adb push file /storage/emulated/0/Notes/`).
## On-device observation loop (quick reference)
Gio renders into one GL surface, so the authoritative debug signals are
data, not pixels:
- `adb logcat -s pad.pad` — app log (IME commits, open-file timing, errors).
- `adb shell input tap|swipe|text` — drive the UI (see `development_plan.md`
for the tap coordinates and IME-tap cadence that works).
- Autosave debounce is 1 s: **wait ~1.6 s before reading a file back from
disk** after typing.
- Memory: `adb shell dumpsys meminfo pad.pad` (watch PSS/RSS; the 3.8 GB
emulator OOMs the app above ~2.5 GB RSS).
- Screenshots (`adb exec-out screencap -p > /tmp/x.png`) are an auxiliary
check only — state, logcat, and file diffs are authoritative.

701
doc/architecture.md
View File

@ -1,505 +1,306 @@
# Runtime Architecture
This document specifies the concurrency model, goroutine responsibilities, inter-goroutine communication, and the synchronous/asynchronous partitioning of logic operations.
This document describes how Pad actually works: the concurrency model, goroutine
responsibilities, the frame handoff contract, ownership rules, and the internal
design of the editor and browser. It is written at the level of *invariants and
contracts*, not line-by-line code, so it stays true as implementation details
evolve. If a detail below conflicts with the code, the code wins — and this
document should be fixed.
## 1. Concurrency Model
## 1. Concurrency model: single owner, no locks on state
Pad runs three concurrent components coordinated by channels and a single mutex:
The **logic goroutine is the sole owner (reader and writer) of mutable
`State`**. There is no `sync.Mutex`/`sync.RWMutex` on application state. Every
other goroutine talks to the owner through channels.
```
┌─────────────────────────────────────────────────────────────┐
│ Main goroutine (Gioui event loop) │
│ - Waits for app events (w.Event()) │
│ - On FrameEvent: Lock → Read Frame → Draw → e.Frame → Unlock│
│ - Batches user input into []InputEvent │
│ - Sends batch to logic goroutine via inputChan │
│ - Sends window changes via configChan │
└─────────────────────────────────────────────────────────────┘
↑ w.Invalidate() │ inputChan / configChan
│ (via Frame Receiver) │ (send)
│ ↓
┌─────────────────────────────────────────────────────────────┐
│ Frame receiver goroutine │
│ - Tight loop: read frameChan │
│ - Lock → Store Frame → w.Invalidate() → Unlock │
└─────────────────────────────────────────────────────────────┘
↑ frameChan (read)
│
┌─────────────────────────────────────────────────────────────┐
│ Logic goroutine │
│ - select on: inputChan, configChan, resultChan │
│ - Processes input batches: loops through []InputEvent │
│ - Dispatches work to priority-aware worker pool │
│ - Sends computed frames on frameChan │
│ - Guarantees: process + layout < 16 ms │
└─────────────────────────────────────────────────────────────┘
↓ highWorkChan / lowWorkChan (dispatch)
│
┌─────────────────────────────────────────────────────────────┐
│ Worker pool (fixed goroutines) │
│ - Priority-aware: highWorkChan > lowWorkChan │
│ - File I/O, diff computation, undo replay on open │
│ - Results posted on resultChan │
└─────────────────────────────────────────────────────────────┘
┌────────────────────────────────────────────────────────────────────┐
│ Main goroutine (Gio event loop) │
│ - w.Event() loop: Config / Frame / Destroy │
│ - On FrameEvent: lock(handoff) → read Frame snapshot → │
│ Renderer.Draw → collect gestures + key events → │
│ e.Frame → unlock(handoff) → send inputs/config/query/layout │
│ to logic channels (sends happen OUTSIDE the lock) │
└───────────────▲──────────────────────────────┬─────────────────────┘
│ handoff lock (frame storage) │ inputChan, configChan,
│ + w.Invalidate() │ layoutChan, searchQueryChan
┌───────────────┴───────────┐ ▼
│ frameReceiver goroutine │ ┌──────────────────────────────────────┐
│ tight loop: │ │ Logic goroutine (SOLE OWNER of State)│
│ f := <-frameChan │ │ - select over all input channels │
│ lock; *frame = f; │ │ - mutates State, builds []Element │
│ w.Invalidate(); unlock │ │ - sends Frame on frameChan │
└───────────────────────────┘ │ - dispatches tasks to worker pool │
└───────────────┬──────────────────────┘
│ task dispatch / results
┌───────────────▼──────────────────────┐
│ Worker pool (8 goroutines) │
│ - priority: high > medium > low │
│ - file I/O, directory index, line │
│ index build, autosave writes │
│ - results posted on ResultChan │
└──────────────────────────────────────┘
```
### Key Invariants
Key invariants:
1. **The mutex is never held while touching a channel.** All channel sends and receives happen outside the critical section.
2. **The frame receiver goroutine runs a tight loop.** It reads from `frameChan`, acquires the mutex, stores the frame, calls `w.Invalidate()`, and releases the mutex. This ensures the logic goroutine's send on `frameChan` never blocks unless the receiver is actively storing a frame.
3. **The main goroutine holds the mutex only during draw + `e.Frame()`.** After releasing the mutex, it sends the batched input to the logic goroutine. This send happens outside the lock.
4. **The logic goroutine guarantees sub-16ms processing.** It processes input, mutates state, computes layout, and sends a frame — all within the display refresh budget.
1. **No locks on `State`.** The only mutex is the *handoff* mutex in
`cmd/pad/main.go`, which guards the one-frame storage slot between
frameReceiver and the main goroutine. It is a message-passing handoff, not
a state lock; it is never held while touching a channel.
2. **The main goroutine never reads logic `State` directly.** It reads only
the latest `Frame` snapshot (see §4) and writes only via channels.
3. **Non-owner goroutines never mutate `State`.** They send a request
(channel) or run a callback on the owner (`Inspect`, test-only).
4. **The autosave timer goroutine reads nothing.** `time.AfterFunc` only sends
a token on `autosaveChan`; the owner does all state reads and dispatches
the write.
5. **Gio-mutable widget state lives in the main-goroutine-owned `Renderer`**,
never in logic-owned `State` (see §5).
6. **Logic work is synchronous and fast.** Processing input, mutating state,
building the element tree, and sending a frame must stay well under one
display refresh (~16 ms). File I/O and index building always go to the
worker pool.
## 2. Goroutine Responsibilities
## 2. Goroutine responsibilities
### 2.1 Main Goroutine (Gioui Event Loop)
### 2.1 Main goroutine (`cmd/pad/main.go`)
Owns the `*app.Window` and runs the event loop. Responsibilities:
- Runs the Gio event loop.
- On `app.ConfigEvent`: sends `ConfigEvent{PixelWidth, PixelHeight}` to
`logic.ConfigChan()`.
- On `app.FrameEvent`:
1. Reads `newScale = gtx.Metric.PxPerDp`; if it differs from
`frame.Scale`, sends `ScaleEvent` (after the draw).
2. Under the handoff lock: reads the `Frame` snapshot, calls
`renderer.Draw(gtx, frame.Elems, scale)`, then `renderer.CheckGestures`
and the focused element's key/edit events (via `key.Filter` +
`key.FocusFilter`), then `e.Frame(&ops)`.
3. Outside the lock: sends `[]ui.InputEvent` (if any) to `InputChan`,
the search text (if it differs from `frame.Query`) to `SearchQueryChan`,
and `renderer.GlyphLayout()` to `LayoutChan`.
- Owns `frameReceiver` (started in `run`).
- **Event dispatch**: calls `w.Event()` to receive `app.FrameEvent`, `app.DestroyEvent`, `app.ConfigEvent`
- **Frame rendering**: on `app.FrameEvent`, acquires the mutex, reads the current frame, draws it via the renderer, calls `e.Frame(&ops)`, releases the mutex
- **Input batching**: collects all user input (taps, key events, scroll) received during the event cycle into a `[]InputEvent`
- **Input delivery**: sends the batch to the logic goroutine on `inputChan`
- **Config delivery**: sends window size or metric changes (`app.ConfigEvent`) to the logic goroutine on `configChan`
### 2.2 Frame receiver
### 2.2 Frame Receiver Goroutine
Tight loop: read `logic.FrameChan()`, store into the shared `Frame` under the
handoff lock, call `w.Invalidate()`. The logic goroutine's send on `frameChan`
(bufsize 1) never blocks for more than one frame cycle.
A dedicated goroutine that bridges the logic goroutine and the main goroutine. Responsibilities:
### 2.3 Logic goroutine (`internal/editor/logic.go`)
- **Frame consumption**: reads from `frameChan` in a tight loop
- **Frame storage**: acquires the mutex, stores the received frame
- **Frame invalidation**: calls `w.Invalidate()` while holding the mutex to request Gio to invoke a `FrameEvent`
- **Lock release**: releases the mutex after invalidation
Sole owner of `*State`. `Run()` selects on:
This goroutine exists so that the logic goroutine can send frames without blocking. The receiver is always ready to consume, and the mutex hold is minimal.
### 2.3 Logic Goroutine
Owns all application state. Runs a `select` loop over multiple channels. Responsibilities:
- **Input processing**: receives a batch of input events from `inputChan`. It loops through the `[]InputEvent` and updates state accordingly.
- **Config processing**: receives window/metric changes from `configChan` and updates layout parameters.
- **Result handling**: receives results from workers on `resultChan`, applies completed async tasks to state.
- **Work dispatch**: identifies long-running tasks and dispatches them to the worker pool via the appropriate priority channel (`highWorkChan` or `lowWorkChan`).
- **Frame production**: computes `[]Element` from current state, sends on `frameChan`.
- **State persistence**: persists state to disk (sync or async, runtime decision — see §5).
The logic goroutine is the sole owner of mutable state. No other goroutine reads or writes state directly.
#### Logic Loop Implementation
The logic goroutine uses a `select` to wait for work:
```go
for {
select {
case inputs := <-inputChan:
for _, in := range inputs {
// handle input (e.g., update buffer, move cursor)
}
sendFrame()
case cfg := <-configChan:
// update window size / metrics
sendFrame()
case res := <-resultChan:
// handle worker result
sendFrame()
}
}
```
### 2.4 Worker Pool
A fixed set of goroutines (e.g., 4) that execute long-running tasks with two-tier priority.
#### Priority Tiers
1. **High Priority (`highWorkChan`)**: UI-critical operations that block user progress.
- Loading a file chunk for the cursor/viewport.
- Loading a directory page in the browser.
- Computing a diff for an active merge session.
2. **Low Priority (`lowWorkChan`)**: Background operations that can be delayed without affecting responsiveness.
- Auto-saving file content.
- Persisting undo stacks.
- Building/updating the line index.
#### Implementation
Workers prioritize high-priority tasks using a nested `select`:
```go
for {
select {
case work := <-highWorkChan:
work.Execute()
default:
select {
case work := <-highWorkChan:
work.Execute()
case work := <-lowWorkChan:
work.Execute()
}
}
}
```
Workers post results on `resultChan`.
## 3. Channel Topology
| Channel | Direction | Purpose | Blocking behavior |
| Channel | From | Payload | Action |
|---|---|---|---|
| `frameChan` | Logic → Frame receiver | Computed frames (`[]Element`) | Unbuffered. Receiver runs tight loop, so logic's send only blocks while receiver holds mutex (fast). |
| `inputChan` | Main → Logic | Batched user input (`[]InputEvent`) | Unbuffered. Logic drains via `select`, main sends after releasing mutex. |
| `configChan` | Main → Logic | Window config changes (`app.ConfigEvent`) | Unbuffered. Logic drains via `select`, main sends after releasing mutex. |
| `resultChan` | Workers → Logic | Completed async task results | Buffered to `workerCount × 4`. Workers post results; logic drains via `select`. Buffer prevents worker blocking when multiple tasks complete simultaneously.
| `highWorkChan` | Logic → Workers | UI-critical async work | Buffered to `workerCount × 2`. |
| `lowWorkChan` | Logic → Workers | Background async work | Buffered to `workerCount × 2`. |
| `configChan` | main | `ConfigUpdate` (pixels or scale) | store scale/pixels; recompute layout |
| `inputChan` | main | `[]ui.InputEvent` | run each `Handler(evt.Data)` on the owner |
| `searchQueryChan` | main | `string` | store `Browser.Query`, re-filter, new frame |
| `openFileChan` | main (tap) | `string` path | open file in editor (chunked buffer + async index) |
| `layoutChan` | main | `ui.GlyphLayout` | store on editor state; derive `LastLineY` for scroll clamping |
| `retryChan` | logic | `string` filename | autosave retry |
| `autosaveChan` | timer | `struct{}` token | reconstruct content, dispatch `WriteFile` task |
| `workerPool.ResultChan()` | pool | task `Result` | apply async results (stat, read, index, write ack) |
| `inspectChan` | tests | `inspectReq` | run `fn(*State)` **on the owner** and reply (test-only) |
| `resultChan` | legacy | `ResultEvent` | drained (legacy channel; the live result path is the pool's) |
| `done` | main | — | drain pool, stop timer, exit |
### Why the Channel Design Works
After every state change the logic goroutine rebuilds the element tree and
sends a `Frame` on `frameChan`.
1. **`frameChan`**: The frame receiver goroutine runs a tight loop. It only holds the mutex during the store + invalidate operation. The logic goroutine's send blocks only during this brief window.
### 2.4 Worker pool (`internal/io/pool`)
2. **`inputChan` / `configChan`**: The main goroutine sends after releasing the mutex. The logic goroutine processes input in < 16ms. The main goroutine produces at most one batch per frame event. The logic's processing pace exceeds production, so the channel never backs up.
Fixed 8 workers, two priority lanes (high > low; dispatch routes
`HighPriority` tasks to the high lane, everything else to low). Tasks are small
structs with `Execute() Result`; the `FileSystem` interface (`pool.FileSystem`)
has a real implementation (`io/pool/real`, rooted at `/`) and a mock
(`io/pool/mock`) for tests.
3. **`resultChan`**: Buffered to `workerCount × 4`. The worker pool is bounded. Results are posted as workers complete. The logic goroutine drains results in its `select` loop. The buffer prevents workers from blocking when multiple tasks complete simultaneously while logic is processing input.
Task types in active use: `StatFile`, `ReadFile`, `BuildLineIndex`,
`WriteFile` (editor); `BuildIndex`, `LoadPages` (browser). Dormant/dead task
types — `ReadChunk` (no-op for fully-loaded in-range files), `ReadDir`,
`StatDir`, `ReadCache`, `WriteCache`, `Invalidate`, `SaveState`, `SaveUndo` —
are candidates for removal in a cleanup round.
4. **Work Channels (buffered to `workerCount × 2`)**: If all workers are busy, logic's dispatch blocks briefly until a worker frees up. Priority ensures critical tasks jump the queue.
## 3. Channel topology summary
## 4. Result Matching
- **main → logic:** `InputChan`, `ConfigChan`, `LayoutChan`, `SearchQueryChan`,
`OpenFileChan`.
- **timer → logic:** `autosaveChan` (token only).
- **logic → logic:** `retryChan` (self-piped).
- **logic → frameReceiver:** `FrameChan` (the only outbound state carrier).
- **pool → logic:** `WorkerPool.ResultChan()`.
- **tests → logic:** `Inspect` (owner-executed callback; the production
equivalent is "send a request channel message").
When multiple tasks are dispatched concurrently, the logic goroutine must match results to their originating tasks. Each `Task` carries identifying context:
## 4. The Frame handoff contract
| Field | Purpose | Example |
|---|---|---|
| `TaskID()` | Unique identifier per dispatch | `"read_dir_/docs_42"` |
| `TaskType()` | Routing key for result handler | `TypeReadDir`, `TypeLoadPages` |
| `DirPath()` | Directory the task operates on | `"/user/documents"` |
The `Result` struct includes `TaskID`, `TaskType`, and `Data`. The logic goroutine routes results by `TaskType` (e.g., `IsBrowserResult()`, `IsFileResult()`) and uses `DirPath()` to apply results to the correct state (e.g., loading pages for the currently browsed directory).
`editor.Frame` is the **only** data that crosses from the logic side to the
main side:
```go
case res := <-wp.ResultChan():
if res.IsBrowserResult() {
switch res.TaskType {
case TypeBuildIndex:
applyDirectoryLoaded(res) // Uses res.Data + DirPath context
case TypeLoadPages:
applyPagesLoaded(res) // Matches pages to current directory
}
type Frame struct {
Elems []ui.Element // element tree for the next draw
Scale float32 // current px-per-Dp
FocusedElementID string // which registered element gets key/edit events
Query string // search text logic is filtering with
}
```
Tasks carry their context (directory path, file path, page indices) in their struct fields. When a task executes, it embeds this context in the `Result.Data` field, so the logic goroutine can apply the result to the correct state.
## 5. Error Handling Policy
Worker tasks may fail due to I/O errors, permission issues, or missing files. The error handling policy ensures the editor remains responsive:
| Task Category | On Error | Recovery |
|---|---|---|
| **Browser tasks** (read dir, build index, load pages) | Log warning; show placeholder in UI | Retry on next browse; user can navigate away and back |
| **File tasks** (read file, write file) | Log warning; show toast notification | Auto-save retries on next debounce cycle |
| **State persistence** (save state, save undo) | Log warning; continue editing | State is re-persisted on next edit |
| **Cache tasks** (read/write cache) | Log warning; fall back to rebuild | Cache is rebuilt on next access |
**Implementation**: Each task returns a `Result` with `Success: false` and `Error` set. The logic goroutine checks `result.IsError()` and handles accordingly. Errors are never re-queued automatically — the caller decides whether to retry based on the task type.
**User notification**: Critical errors (file read/write failures) show a brief toast. Non-critical errors (cache misses, index rebuilds) are logged silently.
## 6. Timeout and Cancellation
Worker tasks operate under time constraints to prevent the pool from stalling:
### Task Timeouts
Tasks have implicit timeouts enforced by the worker pool:
| Task Type | Timeout | Rationale |
|---|---|---|
| High priority (UI-critical) | 5 seconds | User is waiting; must complete or fail fast |
| Low priority (background) | 30 seconds | Can be retried; no user blockage |
If a task exceeds its timeout, the worker cancels the task's `context.Context` and posts a `Result` with `Error: ErrTimeout`.
### Task Cancellation
Tasks carry a `context.Context` that is cancelled when:
1. **User navigates away**: If the user leaves a directory, pending `LoadPages` tasks for that directory are cancelled.
2. **File is closed**: Pending chunk loads for a closed file are cancelled.
3. **Pool shutdown**: `Stop()` closes work channels; workers drain remaining tasks but new dispatches are rejected.
```go
// Task interface includes context for cancellation
func (t *ReadDirTask) Execute() Result {
ctx := t.Context() // Cancelled if user navigates away
entries, err := t.FS.ReadDirContext(ctx, t.Dir)
// ...
}
```
### Timeout Implementation
The `execute()` method wraps task execution with a timeout:
```go
func (wp *WorkerPool) execute(task Task) Result {
ctx, cancel := context.WithTimeout(context.Background(), task.Timeout())
defer cancel()
// Task receives context via Task.Context() or sets it before Execute()
result := task.Execute()
result.Timestamp = time.Now()
return result
}
```
## 7. Synchronous/Asynchronous Partitioning
Every user-facing action has a **synchronous state decision** (what the UI shows) and an optional **asynchronous persistence** (writing to disk). The synchronous path must complete in < 16ms.
### 4.1 Text Editing
| Step | Path | Operation |
|---|---|---|
| User types 'a' | Sync | Insert into in-memory buffer, append to undo chain, advance cursor, compute visible lines, produce frame |
| After frame | Async | Debounce timer (1s) triggers file write via worker |
| After frame | Async | Undo stack persistence (sync or async, runtime decision — §5) |
### 4.2 Backspace
| Step | Path | Operation |
|---|---|---|
| User presses backspace | Sync | Check if last undo entry matches; if so pop it, else start delete chain. Move cursor backward, recompute visible lines, produce frame |
| After frame | Async | Same as text editing (debounced file write, undo persist) |
### 4.3 Undo
| Step | Path | Operation |
|---|---|---|
| User triggers undo | Sync | Pop top chain from in-memory stack, re-anchor using context search in in-memory buffer, replay in reverse, update cursor, produce frame |
| After frame | Async | Persist modified buffer (debounced), persist trimmed undo stack |
### 4.4 Open File
| Step | Path | Operation |
|---|---|---|
| User taps file in browser | Sync | Switch page to editor, set `active_file`, show loading state, produce frame |
| Async | Worker | Read file metadata (size, mtime), check line index cache, compute diff against last known state |
| Async result | Logic | If diff detected → trigger conflict resolution flow. Otherwise → load chunk around cursor |
| Async result | Logic | Place cursor, update scroll, produce frame with editor content |
### 4.5 Conflict Resolution
| Step | Path | Operation |
|---|---|---|
| File watcher detects change | Async | Worker reads file metadata, classifies as external update or sync conflict |
| Classification result | Logic | If sync conflict → set conflict icon in status bar, produce frame |
| User taps conflict icon | Sync | Switch to merge page, set `loading` state, produce frame |
| Async | Worker | Compute line-based diff between ours and theirs |
| Async result | Logic | Hunks computed → display first hunk, produce frame |
| User resolves hunk | Sync | Record choice, advance to next hunk, produce frame |
| User applies merge | Async | Worker writes merged file, deletes conflict file |
| Async result | Logic | Switch to editor page, load merged content, produce frame |
### 4.6 Search
| Step | Path | Operation |
|---|---|---|
| User activates search | Sync | Show search bar element, set focus, produce frame |
| User types in search | Sync | Scan loaded chunks for matches, jump cursor to first match, produce frame |
| User taps up/down | Sync | Jump cursor to previous/next match, produce frame |
### 4.7 Navigation
| Step | Path | Operation |
|---|---|---|
| User scrolls | Sync | Update scroll offset, recompute visible lines, produce frame |
| User taps to place cursor | Sync | Convert tap position → line/col via line index, move cursor, produce frame |
| User jumps to line | Sync | Binary search on line index, compute scroll offset, produce frame |
### 4.8 Directory Browser
| Step | Path | Operation |
|---|---|---|
| User scrolls browser | Sync | Update scroll offset, virtualize visible entries, produce frame |
| User types in browser search | Sync | Filter entries by query, produce frame |
| User taps letter in alpha index | Sync | Jump to corresponding section, produce frame |
| Async | Worker | Load directory entries for newly visible region (lazy loading) |
## 5. State Persistence Strategy
State persistence (writing `state.json`, `cursors.json`, undo stacks) has a **runtime-adaptive strategy**:
### Policy
```
Strategy: Sync | Async
Default: Sync
Monitor: rolling average of persistence duration (exponential moving average)
Threshold: 2 ms average
If average > threshold → switch to Async (log transition)
If average < threshold/2 while Async → switch back to Sync
```
### Rationale
On modern mobile devices with UFS storage, writing small JSON files (< 200 KB for typical undo stacks) completes in single-digit milliseconds. Synchronous persistence is simpler (no crash window between in-memory and on-disk state) and provides stronger crash recovery guarantees.
As state grows (large undo stacks, many cursor entries), persistence may exceed the threshold. The runtime switches to async persistence, accepting the bounded crash window (in-memory state ahead of on-disk state, bounded by the 1s debounce).
### Implementation
The persistence function checks an atomic boolean flag (`sync/atomic.Bool`). If `Sync`, it blocks on the file write. If `Async`, it dispatches the write to a worker and returns immediately. The flag is updated by a background profiler goroutine that monitors the rolling average.
### What is Persisted
| Data | Trigger | Debounce |
|---|---|---|
| `state.json` | Every navigation, edit, cursor movement | 500 ms |
| `cursors.json` | Every cursor movement (in any file) | 500 ms |
| `undo/<hash>.json` | Every edit (insert, delete, undo) | 1000 ms (separate from file save) |
| File content (auto-save) | Every edit | 1000 ms |
| Line index | On file close, or after bulk edit | Immediate |
## 6. Deadlock Analysis
### Scenario: Logic sends frame, main holds mutex
```
Logic: frameChan <- elems // blocks if receiver is not reading
Receiver: lock → store → w.Invalidate() → unlock // fast, sub-ms
```
**Resolution**: The frame receiver runs a tight loop. It only holds the mutex during the store + invalidate. Logic's send blocks for at most the duration of this store — sub-millisecond. No deadlock.
### Scenario: Main sends input, logic is processing previous input
```
Main: inputChan <- inputs // blocks if logic is not reading
Logic: inputs := <-inputChan // select loop, drains before next frame
```
**Resolution**: Logic's 16ms guarantee means it processes input faster than main produces it. Main sends input after releasing the mutex, so it never holds the lock while blocked on a channel send. No deadlock.
### Scenario: Worker posts result, logic is busy
```
Worker: resultChan <- result // blocks if logic is not reading
Logic: select { case r := <-resultChan: ... }
```
**Resolution**: Worker pool is bounded. Workers post results and return to the pool. If logic is briefly busy processing input, the worker's send blocks for at most the duration of logic's current iteration. The worker is not holding any lock during this send. No deadlock.
### Scenario: Logic dispatches work, all workers busy
```
Logic: highWorkChan <- workItem // blocks if highWorkChan is full
Workers: <-highWorkChan // drain at their own pace
```
**Resolution**: Work channels are buffered to pool size. If all workers are busy, logic's dispatch blocks briefly until a worker frees up. This is acceptable because dispatching work is not on the critical input→frame path. No deadlock.
## 7. Go Concurrency Features Used
| Feature | Where | Why |
|---|---|---|
| **Goroutines** | Main, frame receiver, logic, workers | Natural fit for concurrent, channel-coordinated components |
| **Unbuffered channels** | `frameChan`, `inputChan`, `configChan`, `resultChan` | Synchronous handoff ensures no message is lost; timing guarantees prevent blocking |
| **Buffered channels** | `highWorkChan`, `lowWorkChan` (size = pool size) | Allows logic to dispatch without waiting for a free worker |
| **`sync.Mutex`** | Protects current frame | Single critical section (store or read frame), sub-millisecond hold time |
| **`sync/atomic.Bool`** | Persistence strategy flag | Lock-free read in logic goroutine, written by profiler goroutine |
| **`context.Context`** | Worker task cancellation | Cancel in-flight loads if user navigates away or file is closed |
| **`select`** | Logic goroutine's main loop | Multiplex over input, config, and results without polling |
### Features Deliberately Avoided
| Feature | Why avoided |
- `Elems` is the computed tree; the renderer draws it (it is a snapshot, safe
to read from main).
- `Scale` lets main detect a density change and pass scale to `Renderer.Draw`.
- `FocusedElementID` drives which registered `key.Filter`/`key.FocusFilter`
main uses to harvest keyboard/IME events.
- `Query` lets main compare against the (main-owned) search `widget.Editor`
text and forward changes — the browser search box is a Gio widget, so its
text lives on the main side, not in `State`.
## 5. Ownership rules
| Owned by | What |
|---|---|
| **`sync.Mutex` on state** | Single-owner pattern: only logic goroutine mutates state |
| **`sync.RWMutex`** | No readers other than logic; frame access goes through mutex + frame receiver |
| **`sync.Cond`** | Channels provide the same signaling with less boilerplate |
| **`errgroup`** | Workers are long-lived pool goroutines, not task-scoped |
| **`sync.Once`** | No one-time initialization that needs coordination |
| **`atomic.Value`** | Frame is a slice header (two words); mutex is simpler and safer |
| **Logic goroutine** | `State` (browser + editor + chunked buffers + line indexes), worker pool, autosave timer handle |
| **Main goroutine** | `*app.Window`, `op.Ops`, text shaper, `ui.Renderer` (gesture state, IME dedup state, glyph layout cache), the search-bar `widget.Editor` |
## 8. Browser State Management
Rules:
The browser package defines a `BrowserState` struct that holds all browser-specific state. This struct is **embedded** in the editor's `State` struct to maintain the single-owner pattern:
1. **Gio mutates widgets during draw, so anything Gio mutates must be
main-owned.** The search bar is a `widget.Editor` registered with the
renderer by ID (`"search_bar"`); its text reaches logic only through
`SearchQueryChan`.
2. **The editor content is NOT a `widget.Editor`.** It is the custom
`ui.TextField` element; all editor state (buffer, cursor, scroll) is
logic-owned. This is why the editor does not hit `widget.Editor`'s
O(n²)-ish cost on large files.
3. **Per-frame element values are rebuilt by logic** (`[]ui.Element` in the
`Frame`); any cross-frame persistent draw-side state (IME dedup, gesture
tracking) belongs in the persistent, main-owned `Renderer`.
```go
type State struct {
// ... other fields ...
Browser BrowserState // Embedded, not a pointer
}
```
## 6. Editor internals (`internal/editor`)
This ensures:
1. All browser state is owned by the editor package
2. The logic goroutine is the sole owner of browser state
3. No cross-package state access is required
### 6.1 Chunked buffer
The browser package's `BrowserLayout` function is called from the editor package, passing the embedded `BrowserState`:
`ChunkedBuffer` (`chunked_buffer.go`) is the edit buffer for open files.
```go
// In editor/state.go
func (s *State) BrowserLayout(screenW, screenH ui.Dp) []ui.Element {
return browser.BrowserLayout(screenW, screenH, &s.Browser)
}
```
- **Full load on open** for in-range files: the whole file is read into an
ordered slice of chunks (64 KB) plus **prefix-sum byte offsets** over actual
chunk lengths. There is no lazy loading and no eviction (both existed as
plans and were removed).
- **Byte-indexed** throughout: `CursorPosition`, chunk offsets, and glyph
`ByteOffsets` are byte offsets.
- Edits splice the affected chunk(s) only; chunks are not rebalanced.
- `LineIndex` (per-line byte offsets, `int32`) is built asynchronously by
`BuildLineIndexTask` and stored on `cb.LineIndex` — the single source of
truth (the old parallel `EditorState.LineIndex` field was removed).
- **Size guard:** files larger than `MaxEditableFileSize` (**50 MB**,
measured on-device) open into a `TooLarge` state: the editor shows a notice
and edit handlers are no-ops; the browser still lists the file.
This follows the architecture's principle that the editor package owns all application state, while the browser package provides pure functions for layout computation.
### 6.2 Virtualized viewport
## 9. Frame Lifecycle
Only the visible byte range is shaped and drawn each frame:
A frame (computed `[]Element`) follows this lifecycle:
- `VisibleByteRange` maps scroll offset + viewport height → `[startLine,
endLine]` via the `LineIndex`, then to a byte range. The range is always
bounded by real lines of the document.
- **Never shape the whole file.** Lesson learned (Phase 3): the shaper's
internal `document` retains the backing array of its largest layout forever
(`reset()` keeps the cap), so one whole-file layout permanently inflated
memory to ~1 GB for a 10 MB file. Shaping ~50 lines keeps it small and flat.
- The renderer reports `GlyphLayout` back via `LayoutChan`; logic derives
`LastLineY` for scroll clamping and the max scroll offset.
1. **Logic computes**: state mutation + layout → `[]Element`
2. **Logic sends**: `frameChan <- elems` (blocks only while receiver holds mutex)
3. **Receiver stores**: `lock → currentElems = elems → w.Invalidate() → unlock`
4. **Main receives `FrameEvent`**: `lock → renderer.Draw(gtx, currentElems) → e.Frame(&ops) → unlock`
5. **Frame is on screen**: Gioui has submitted the frame to the display
6. **Next frame**: logic computes a new `[]Element`, repeats from step 2
### 6.3 IME (Android soft keyboard)
At no point are two frames rendered simultaneously. The mutex ensures the main goroutine never reads a frame while the receiver is writing it. The frame receiver calls `w.Invalidate()` only after storing, so Gioui never requests a draw for a frame that hasn't been stored yet.
- The editor exposes to the IME a **windowed snippet**: `IMEWindowText` is the
visible viewport text, `IMEWindowStartByte` its absolute start. While the
`TextField` is focused, `drawElement` emits `key.SnippetCmd` (the window)
and `key.SelectionCmd` (caret as a rune index into the window).
- **Dedup in the Renderer** (main-owned state): the snippet/selection are
re-emitted only when they actually change, and a fresh push is forced when
the field (re)gains focus. This mirrors `widget.Editor`'s behavior and
prevents per-frame re-push from resetting IME composition on rapid commits.
- Incoming `key.EditEvent{Range, Text}` has **window-relative rune
indices**; `HandleReplaceRange` converts them to absolute byte offsets
(`RuneIndexToByte`) and splices the chunked buffer. Insertion,
`deleteSurroundingText` (backspace/autocorrect replacement), and composition
all arrive through this one path.
## 9. Process Death and Recovery
### 6.4 Autosave
The concurrency model has no shared mutable state between goroutines (state is owned by the logic goroutine, frame is protected by mutex). On process death:
- Any edit calls `markDirty()`: a 1 s debounce timer; each keystroke restarts
it. On expiry the timer goroutine sends a token on `autosaveChan`; the owner
reconstructs the full content from the chunked buffer and dispatches a
`WriteFile` task.
- Write failures are tracked per file (`writeFailed`, `retryAttempts`) and
retried via `retryChan`. There is no save button; autosave is the only
persistence.
- **In-memory state is lost**: restored from `.pad/state.json`, `.pad/cursors.json`, `.pad/undo/`
- **In-flight worker tasks are lost**: workers are short-lived; tasks are idempotent (file reads, diff computation)
- **In-flight persistence is lost**: bounded by the debounce window; last persisted state is the recovery point
- **Merge state does not survive**: if the app dies during conflict resolution, the merge session is re-computed on relaunch (diff is idempotent)
### 6.5 Opening a file
The recovery flow (SPEC §8) reads persisted state, restores the active file, cursor, and undo stack. The concurrency model restarts fresh: new goroutines, new channels, same invariants.
- A browser tap sends the path on `OpenFileChan`. The logic goroutine creates
the `ChunkedBuffer`, dispatches `StatFile` (size guard) and the read +
`BuildLineIndex` tasks, switches `page` to the editor, and sets
`justOpenedAt`.
- **Opening-tap swallow:** the tap that opens the file is also delivered as an
editor tap in the same frame. A short time window (`justOpenedAt`) swallows
it so the viewport does not jump to the tapped (often EOF) position.
## 10. Memory Monitoring and Management
## 7. Browser internals (`internal/browser`)
Pad implements a monitoring system for heap usage and latency, with hooks for future global memory management.
- `BrowserState` is **embedded by value** in `State` (single owner; no
pointer indirection).
- `BrowserManager` drives async directory loading through the worker pool:
`ReadDir` + `BuildIndex` on navigation, `LoadPages` for pagination.
- Features: 4 sort modes (name/date × asc/desc; default newest-first),
incremental case-insensitive search over names, directory navigation with
back, single-tap file open.
- The browser renders via `ListView`/`ListItem` elements; rows carry
`Interaction` handlers that send channel requests (tap) or mutate state
directly when the handler runs on the owner (scroll).
### 10.1 Monitoring
## 8. Render pipeline (`internal/ui`)
A background profiler goroutine periodically collects system metrics:
- **Logic** builds a `[]ui.Element` tree. Elements are value types; interactive
ones carry `Interaction{Gesture, Handler}` entries. `Handler` is a static
function that mutates `TheState` — handlers run on the logic goroutine only.
- **Main** draws via `Renderer.Draw(gtx, elems, scale)`:
1. `drawElement` recursively walks the tree, applying transforms/clips.
2. Interactive elements register hit regions in the *current clip context*
(`gesture.Click.Add` / `pointer.InputOp`), so hit-testing always matches
the drawn geometry.
3. Text elements shape through the shared shaper — **only the visible
window**, never the whole buffer.
4. `e.Frame(&ops)` flushes ops.
- After draw, `CheckGestures` turns raw pointer/gesture state into
`[]ui.InputEvent` (each carrying its own handler) for the logic channel.
- **Units:** `ui.Dp`/`ui.Px` convert via the current `PxPerDp` scale
(`ToDp`/`ToPx`). The window is 390×844 dp; the real pixel size arrives via
`ConfigEvent`.
- **Live element catalog:** `Container`, `Label`, `Icon`, `TextField` (editor
window), `ListView`/`Line`/`ListItem` (browser rows), `GioEditor` (search
bar), `Cursor`. Types `Button`, `Toast`, `Spacer`, `AlphaIndex`,
`Selection`, `MergeHunk` exist in `element.go` but are not used by any page
— future work, not current behavior.
- **Theme:** `ui.Theme{FontSize: 14}` + palette in `element.go`.
- **Heap Usage**: Collected via `runtime.ReadMemStats`. Tracks total allocated memory and heap size.
- **Persistence Latency**: Tracks the EMA of state persistence duration (see §5).
- **Logic Latency**: Tracks the time taken for the Logic goroutine to process input and compute layout.
## 9. Testing hooks
These metrics are reported on a **Debug Page** within the application, allowing for real-time performance analysis during development and by power users.
- `Logic.Inspect(fn)` (test-only): runs `fn(*State)` **on the owner** and
returns its result — the sanctioned way for tests to read state without
breaking single ownership. It must never be called from production code.
- The e2e harness (`internal/test/e2e`) drives the **real** `Logic` + worker
pool + mock filesystem through the real channels, and asserts via
`Inspect`. It is logic-only: it does not exercise `Renderer.Draw`, so
draw-path behavior (IME dedup, gesture routing) is validated on-device with
adb (see `development_plan.md` §Phase 2/3 observation loop).
- CI gate: `go build ./... && go vet ./... && go test -race ./...`.
### 10.2 Memory Pressure (Future Enhancement)
## 10. Known loose ends (code, not spec)
While initial versions use per-component eviction (e.g., LRU for chunk cache and line indices), a future **Global Memory Governor** is planned:
- **Centralized Eviction**: Instead of components deciding when to evict independently, the Governor monitors total heap pressure.
- **Pressure Tiers**:
- **Normal**: No proactive eviction.
- **Warning**: Proactively evict oldest line indices and non-visible file chunks.
- **Critical**: Evict all non-visible data; force garbage collection.
- **Implementation**: Components register "evictable" resources with the Governor, which calls back to release memory when thresholds are exceeded.
This ensures that the "No Practical Limits" philosophy does not lead to Out-Of-Memory (OOM) crashes on devices with limited RAM.
- Globals `TheState` / `TheLogic` / `ui.OpenFile` still exist (handlers are
static funcs); replacing them with explicit state is a planned cleanup
(`development_plan.md` Phase 4).
- Dead task types (`ReadChunk`, `SaveState`, `SaveUndo`, …) and unused element
types are candidates for removal in the same round.

1133
doc/browser_implementation_plan.md
View File

File diff suppressed because it is too large Load Diff

2
doc/bugs.txt
View File

@ -1,2 +0,0 @@
This file lists descriptions of outstanding bugs. Bugs are separated by empty lines.

233
doc/conflict_resolution.md
View File

@ -1,233 +0,0 @@
# Conflict Resolution Specification
## 1. Overview
Pad operates on a directory synced by Syncthing. External modifications to files originate from edits on other devices, delivered via Syncthing. This document specifies how Pad detects, classifies, and resolves such changes.
## 2. Syncthing Conflict File Format
When Syncthing detects that two devices have modified the same file, it preserves both versions:
- **Original file** (`notes.txt`) — the local device's version (what Pad edited)
- **Conflict file** (`notes.sync-conflict-2024-05-15-1430-ABCDEF1.txt`) — the remote device's version
**Naming pattern:** `<base>.sync-conflict-<date>-<time>-<modifiedBy>.<ext>`
| Component | Format | Example |
|---|---|---|
| `base` | Original filename without extension | `notes` |
| `date` | ISO date | `2024-05-15` |
| `time` | 24-hour time, no separator | `1430` |
| `modifiedBy` | Remote device short ID | `ABCDEF1` |
To resolve the original file from a conflict file, strip the `.sync-conflict-<date>-<time>-<modifiedBy>` suffix and reattach the extension.
## 3. Change Classification
When the file system watcher detects a change, Pad classifies it into one of two categories:
| Category | Trigger | Meaning |
|---|---|---|
| **External update** | File content changed on disk, no conflict file exists | Another device edited and synced; our device received the update cleanly |
| **Sync conflict** | A `.sync-conflict-*` file appeared for a file we have open or know about | Both devices edited the same file; Syncthing could not merge automatically |
### Detection Logic
```
ON file_modified(event):
original = event.path
conflict = find_conflict_file_for(original)
IF conflict exists:
→ SYNC CONFLICT
ELSE:
→ EXTERNAL UPDATE
ON new_file(event):
IF event.path matches conflict pattern:
original = resolve_original(event.path)
→ SYNC CONFLICT for original
```
## 4. External Update (No Conflict)
The file was modified on another device and synced cleanly. Our local edits (if any) were already synced before the remote edit, so there is no divergence.
### Procedure
1. **Detect** — watcher fires `file_modified` for the active file
2. **Read** — load the new content from disk
3. **Diff** — compute diff between the editor's buffer (old) and disk content (new) to understand what changed
4. **Re-anchor undo stack** — for each operation in the undo stack, attempt to locate its context in the new file content (see §6)
5. **Replace buffer** — swap the edit buffer with the new content
6. **Preserve cursor** — if the cursor position falls within a changed region, move it to the start of the changed region; otherwise keep it at the same byte offset
7. **Continue** — user can edit immediately, undo works with re-anchored operations
### Edge Cases
| Scenario | Behavior |
|---|---|
| User has un-saved edits (within debounce window) | Flush auto-save immediately, then proceed |
| User is actively typing | Complete the current keystroke, then apply update on next detection |
| File was truncated externally | Load empty/new content, reset cursor to 0 |
| Undo operations cannot be re-anchored | Drop unanchorable operations silently (context no longer exists in new content) |
## 5. Sync Conflict
Both devices edited the same file. Syncthing preserved both versions. Pad must help the user integrate the changes.
### Procedure
1. **Detect** — watcher fires `new_file` for a conflict file, or `file_modified` with a corresponding conflict file present
2. **Identify** — resolve the original file path from the conflict file name
3. **Load both versions:**
- **Ours** (`notes.txt`) — the local file, may contain the user's current edits
- **Theirs** (`notes.sync-conflict-*.txt`) — the remote device's version
4. **Diff** — compute a line-based diff between ours and theirs, producing a list of conflicting hunks
5. **Present merge UI** — show the user each hunk with resolution options (see §7)
6. **Apply resolution** — produce a merged file from the user's choices
7. **Persist** — write merged content to the original file (`notes.txt`), delete the conflict file
8. **Re-anchor undo stack** — same as external update (§4, step 4)
9. **Replace buffer** — swap edit buffer with merged content
10. **Continue** — user can edit immediately
### Edge Cases
| Scenario | Behavior |
|---|---|
| Conflict file for a file the user never opened | Leave it alone (out of scope; Syncthing will handle on next sync cycle) |
| Multiple conflict files for the same original (rare, from rapid edits) | Resolve the most recent one first; delete all conflict files after resolution |
| User closes the app while merge UI is shown | Persist the unresolved state; reopen merge UI on relaunch |
| Conflict file is deleted by user externally | Treat as abandoned; no action needed |
## 6. Undo Stack Re-anchoring
After any external change or conflict resolution, the undo stack must be reconciled with the new file content.
### Algorithm
For each chain in the undo stack (most recent first):
1. **Locate the head's context** in the new file content:
- For **insert chains**: search for `before` + `after` pattern (text surrounding the insertion point)
- For **delete chains**: search for `after` pattern (text after the deletion point)
2. **If context is found:**
- Update the head's `pos` to the new location
- All tail entries inherit from the re-anchored head
- Chain is preserved
3. **If context is not found:**
- Remove the entire chain from the undo stack
- The text surrounding the operation has changed too much to reliably replay
4. **If context is found multiple times:**
- Pick the occurrence closest to the original `pos`
- If ambiguity remains, pick the first occurrence
### Limitations
- Re-anchoring is best-effort. If the external change modified text within a chain's context window, that chain is lost.
- This is acceptable — the alternative (full diff-based rebasing) is significantly more complex and error-prone.
## 7. Merge UI
The merge UI presents conflicting hunks to the user for per-hunk resolution.
### Hunk Structure
Each hunk represents a region where ours and theirs diverge:
```
─── Hunk 3 of 7 ─────────────────────────
Line 142 — Line 148
Context (unchanged):
The quick brown fox
Ours:
jumps over the lazy dog
The end.
Theirs:
jumps over the sleepy cat
A new paragraph begins.
The end.
```
### Resolution Options (per hunk)
| Choice | Result |
|---|---|
| **Keep ours** | Use our version of this hunk |
| **Keep theirs** | Use their version of this hunk |
| **Merge both** | Concatenate ours + theirs (ours first, then a blank line, then theirs) |
### UI Behavior
- **One hunk at a time** — shown full screen, swipe or tap to navigate between hunks
- **Tap a version** to preview it (highlight in the full file context)
- **Tap a resolution button** to accept and move to next hunk
- **Apply all** button appears after the last hunk is resolved
- **Cancel** — abort merge, keep our version, leave conflict file untouched (will reappear on next sync)
### State Persistence
The merge session (which conflict file, which hunks, which resolutions) is persisted to the state file. If the app is killed during a merge:
1. On relaunch, detect the unresolved conflict file
2. Re-compute the diff (idempotent)
3. Restore any resolutions the user already made (from state file)
4. Present the merge UI from the first unresolved hunk
## 8. Diff Engine
Pad needs a line-based diff engine that produces hunks with context.
### Requirements
- **Line-based** — hunks are groups of lines, not character offsets
- **Streaming** — must not load entire files into memory; process line by line
- **Context-aware** — each hunk includes N context lines before and after the change (configurable, default 3)
- **Positioned** — each hunk reports the starting line number in both versions
- **Efficient** — must handle files with millions of lines without O(n²) behavior
### Output Format
```go
type Hunk struct {
OurStartLine int // Starting line in our version (1-indexed)
OurLines []string // Lines from our version (replaced by TheirLines)
TheirStartLine int // Starting line in their version (1-indexed)
TheirLines []string // Lines from their version (replaces OurLines)
ContextBefore []string // Unchanged lines before the hunk
ContextAfter []string // Unchanged lines after the hunk
}
```
### Implementation Options
| Approach | Pros | Cons |
|---|---|---|
| **Go `textdiff` / `diffmatchpatch`** | Available, battle-tested | Character-based, not line-based; may need wrapping |
| **Shell `diff` invocation** | Standard, line-based, efficient | Requires subprocess, harder to stream on Android |
| **Custom Myers diff** | Full control, line-based, streaming | More code to write and test |
| **Import `github.com/sergi/go-diff`** | Line-based, Go-native | External dependency, may not stream |
**Recommendation:** Import a Go-native line-based diff library (e.g., `go-diff` or `diffmatchpatch` with line-level wrapping). Evaluate during implementation.
## 9. Interaction with Auto-save
| Event | Auto-save behavior |
|---|---|
| External update applied | Auto-save the new buffer content (so our disk copy matches the synced version) |
| Merge in progress | Pause auto-save (buffer is in an inconsistent state) |
| Merge applied | Auto-save the merged content immediately |
| Conflict file detected while auto-save debounce is active | Flush auto-save first, then proceed with conflict detection |
## 10. Out of Scope
- Three-way merge (we don't have access to the common ancestor; Syncthing doesn't preserve it)
- Automatic merge (all changes require user confirmation)
- Binary file conflicts (Pad only handles text files)
- Folder-level conflicts (only file-level)

41
doc/development_plan.md
View File

@ -1,7 +1,13 @@
# Development Plan: reach a lean, usable Android text editor
Status: v3, 2026-08-16 (Phases 0–3 complete). Written against the **live** repo
`/home/gmp/pad`. v1 (the widget-rebuild plan) is superseded — see §12 for why.
Status: v4, 2026-08-16 (Phases 0–3 complete; doc reorganization done). Written
against the **live** repo `/home/gmp/pad`. v1 (the widget-rebuild plan) is
superseded — see §12 for why. Doc reorganization (2026-08-16): the over-detailed
docs (`*_implementation_plan.md`, `touch.md`, `element_model.md`,
`layout_rendering.md`, `virtual_scroll_render_optimization.md`,
`conflict_resolution.md`, `bugs.txt`) were deleted; `spec.md` and
`architecture.md` were rewritten to describe the actual app; `doc/README.md`
adds the doc policy + build/install recipe. See `doc/README.md`.
Phases 0–3 done: single-owner no-lock architecture, Android IME wiring, on-device
IME validation (passing), viewport-on-open fix, chunked-buffer drift fix, the
whole-file shaper memory-leak fix, a measured 50 MB size limit, and the IME
@ -223,9 +229,16 @@ explicit state, prune dead task types, and *then* consider collapsing the
logic-goroutine/channel model to a single owner. These reduce future bug surface
but are not needed for a usable v1.
### Phase 5 — hardening
Rewrite `doc/architecture.md` to match reality; amend `spec.md` per §7; README
(build/install recipe); in-repo device test checklist.
### Phase 5 — hardening (mostly done)
- ✓ Rewrite `doc/architecture.md` to match reality (2026-08-16: now describes
the single-owner/no-lock model, channel topology, Frame contract, ownership
rules; over-detailed companion docs deleted — see `doc/README.md` policy).
- ✓ Amend `spec.md` per §7 (2026-08-16: rewritten; unbuilt features moved to an
explicit “deferred” table; see the corrections noted under §7 below).
- ✓ Build/install recipe (2026-08-16: in `doc/README.md`; machine-local script
`/tmp/build_pad.sh`).
- ☐ In-repo device test checklist (the adb tap/swipe/IME sequences used in
Phases 2–3 are in this plan's phase notes but not a standalone checklist).
## 6. File-size decision (re-framed)
@ -238,15 +251,17 @@ critical path. The chunked buffer proved out on a real 10 MB file (Phase 3, §5)
10 MB+ editing works with ~linear, bounded memory, without the 0.5 GB/MB wall
that kills `widget.Editor`.
## 7. Spec deltas (to write into spec.md)
## 7. Spec deltas (WRITTEN into spec.md, 2026-08-16 rewrite)
1. "No practical limits / file never fully in memory" → backed by the chunked
buffer; add a measured hard limit from Phase 3.
2. Undo across sessions → dropped (in-session only; chunked buffer does not keep
full-document undo copies).
3. Add: IME/swipe/autocorrect support (now explicit, delivered in Phase 1).
4. Add: external-change detection as the conflict policy (mtime compare on open
+ on resume; changed+clean → reload, changed+dirty → Keep/Reload prompt).
1. ✓ "No practical limits / file never fully in memory" → replaced by the
measured 50 MB hard limit + `TooLarge` state (spec.md §2.3).
2. ✓ Correction: undo is **not implemented at all** (not "in-session only") —
there is no undo stack in the code; listed as deferred (spec.md §7).
3. ✓ IME/swipe/autocorrect support is explicit (delivered Phase 1; spec.md
§2.2). Real-device swipe sign-off remains the one open validation item.
4. ✓ Correction: external-change detection is **not implemented** (no mtime
compare on open/resume, no watcher) — listed as deferred (spec.md §7), not
as a current requirement.
## 8. Non-goals (v1)

503
doc/editor_implementation_plan.md
View File

@ -1,503 +0,0 @@
# Editor Implementation Plan
## 1. Design Principles
- **Single-Owner Pattern**: The logic goroutine is the exclusive authority for modifying the editor buffer, cursor, and selection state.
- **Asynchronous I/O**: File saving and loading operations are delegated to the worker pool. The UI never blocks on disk operations.
- **Buffer-Based Editing**: For performance, the editor will eventually utilize a gap buffer or rope data structure. For MVP, string manipulation is acceptable if performance targets are met.
- **Event-Driven**: Keyboard input, mouse interactions, and menu actions are passed through the existing `inputChan` to the logic goroutine.
## 2. Core State Structure
The `State` struct in `internal/editor/state.go` is extended to track active editing session data:
```go
type EditorState struct {
Buffer string
CursorPosition int
GlyphLayout GlyphLayout
SelectionStart int // -1 if no selection (unused)
SelectionEnd int // -1 if no selection (unused)
CursorVisible bool // Blink state (no timer yet)
Filename string // Current file path being edited
fileVersion map[string]int // per-file: version of the loaded buffer
lastWriteVersion map[string]int // per-file: version of last successful write
saveTimer *time.Timer // nil = no pending save
}
```
**Dirty** is computed, not stored:
```go
func (e *EditorState) IsDirty() bool {
bufVer := e.fileVersion[e.Filename]
writeVer := e.lastWriteVersion[e.Filename]
return bufVer > writeVer
}
```
**Current state** (as of June 2026): `EditorState` is defined with `Dirty bool` (single boolean, cleared on file switch). `UndoStack` is deferred. `SelectionStart`/`SelectionEnd` are present but unused. **Save file feature not yet implemented.**
## 3. Auto-Save Design
Auto-save is transparent, asynchronous, and fire-and-forget. No save button, no UI indicators, no "do you want to save?" dialog.
### 3.1 Per-File Version Tracking
The `Dirty` boolean is replaced by per-file version maps:
| Field | Meaning |
|---|---|
| `fileVersion[filename]` | Version of the buffer currently loaded for this file (monotonically increasing) |
| `lastWriteVersion[filename]` | Version of the last **successful** write for this file |
| **Dirty** | Computed: `fileVersion > lastWriteVersion` |
**Dirty lifecycle:**
| Event | Effect |
|---|---|
| Edit | `fileVersion[filename]++` → Dirty becomes true |
| Write succeeds | `lastWriteVersion[filename] = fileVersion[filename]` → Dirty becomes false |
| Write fails | `lastWriteVersion` unchanged → Dirty stays true |
| Switch files | Old file's versions preserved in map → Dirty preserved |
| Re-open file | `fileVersion` preserved (not reset) → Dirty preserved |
| Next edit on dirty file | New save attempt (retry) |
This means:
- Dirty state survives file switches
- Dirty state survives file re-opens
- Failed writes leave the file dirty
- No automatic retry — the next edit triggers a new save attempt
### 3.2 Debounce Timer
A 1-second debounce timer fires after the last edit. The timer is reset on each new edit, so only one write is dispatched per debounce window.
```go
// In inputChan handler, after processing events:
if l.state.Editor.IsDirty() && l.state.Editor.Filename != "" {
if l.saveTimer != nil {
l.saveTimer.Stop()
}
gen := l.saveGeneration
l.saveGeneration++
buffer := l.state.Editor.Buffer // capture in logic goroutine
l.saveTimer = time.AfterFunc(1*time.Second, func() {
if l.saveGeneration == gen {
l.workerPool.DispatchNonBlocking(
pool.NewWriteFileTask(l.state.Editor.Filename, []byte(buffer), l.mockFS),
)
}
})
}
```
### 3.3 Generation Counter
The `saveGeneration` field prevents stale timer callbacks from dispatching writes:
```go
type Logic struct {
saveTimer *time.Timer
saveGeneration int // incremented each time a new timer starts
}
```
Each timer callback captures its own `gen` value. When the callback fires, it checks `l.saveGeneration == gen`. If they differ, the timer was replaced and the callback drops silently.
**Why `Stop()` alone isn't enough:** `time.AfterFunc` creates a goroutine that waits on a timer channel. `Stop()` closes the channel, preventing future firings. But if the callback goroutine has already woken up (the timer fired), `Stop()` doesn't interrupt it. The generation check handles this in-flight case.
**Scenario: Two rapid edits**
```
Time 0s: Edit → gen=0, saveGen=1, Timer A starts
Time 0.5s: Edit → gen=1, saveGen=2, Timer A stopped, Timer B starts
Time 1.0s: Timer A fires → gen(0) != saveGen(2) → drop
Time 1.5s: Timer B fires → gen(1) == saveGen(2) → dispatch
```
Only the latest timer dispatches. All others are filtered by the generation check.
### 3.4 Failure Mode Handling
Three likely failure modes:
| Failure | Recoverable? | Response |
|---|---|---|
| Device I/O error | No | Log error, leave dirty, no retry |
| Network filesystem unavailable | Maybe | Log error, leave dirty, no retry |
| Removable media not found | No (not relevant on mobile) | Log error, leave dirty, no retry |
**No automatic retry.** The dirty state persists until the next edit triggers a new save attempt. If the filesystem recovers (e.g., network comes back), the next save will succeed. If not, the file stays dirty and the user will eventually notice.
### 3.5 Stale Result Filtering
Write results are filtered by file path:
```go
case res.TaskType == pool.TypeWriteFile:
if res.FilePath != l.state.Editor.Filename {
return // File switch — stale result for a different file
}
if res.IsSuccess() {
l.state.Editor.markSaved()
} else {
log.Printf("Auto-save failed for %s: %v", res.FilePath, res.Error)
// Dirty stays true — next edit triggers retry
}
```
When a write fails, `lastWriteVersion` is not updated. The file stays dirty. The next edit on that file triggers a new save attempt.
### 3.6 File Switch While Save In Flight
```
Edit A → Dirty=true → timer starts
Timer fires → write dispatched
Open file B → timer cancelled, dirty cleared for B
Write A result arrives → path check: A != B → ignored
```
The file switch cancels the timer and clears dirty for the new file. An in-flight save for the old file is harmless — its result is ignored by the path check.
### 3.7 Atomic Write (Temp + Rename)
The mock filesystem implements atomic writes via temp file + rename:
```go
func (fs *FileSystem) WriteFileAtomic(path string, content []byte) error {
// 1. Write to .tmp/ subdirectory (top-level, outside user directories)
tempPath := ".tmp/" + filepath.Base(path)
fs.files[tempPath] = &File{...}
// 2. Atomic rename — insert + delete in one locked operation
fs.files[path] = &File{...}
delete(fs.files, tempPath)
// 3. Send change notifications (outside lock)
fs.sendNotifications(...)
}
```
Key design points:
- Temp files live in top-level `.tmp/` — never in any user directory
- The rename is atomic: both insert and delete happen while holding the mutex
- No partial writes are ever visible
- Change notifications fire after the lock is released
### 3.8 Concurrency Correctness
| Concern | How it's handled |
|---|---|
| Multiple writes in flight | Generation counter prevents old timer callbacks from dispatching |
| Out-of-order completion | Not an issue — only one write dispatched at a time (guaranteed by generation counter) |
| Write fails | `lastWriteVersion` unchanged → dirty stays true → next edit retries |
| Write dropped (channel full) | No result → dirty stays true → next edit retries |
| File switch while save in flight | Timer cancelled; stale result ignored by path check |
| Re-open dirty file | `fileVersion` preserved → dirty stays true → next edit retries |
| Buffer race with timer callback | Buffer captured in logic goroutine before timer starts |
| No blocking on critical path | `DispatchNonBlocking` never blocks; timer callback is separate goroutine |
## 4. Implementation Phases
### Phase 1: Basic Buffer Management & Cursor
- [x] Define `EditorState` (Cursor, Selection, Dirty flag). — **DONE**
- [x] Implement `GlyphLayout` capture in `drawWrappedText` and feedback via `layoutChan`. — **DONE**
- [x] Implement cursor navigation (Arrow keys: Left, Right, Up, Down). — **DONE** (Left/Right: byte-based ±1; Up/Down: GlyphLayout-based)
- [x] Implement basic buffer updates (Insert character, Delete/Backspace). — **DONE**
- [x] Implement cursor display. — **DONE** (Rendered inline in `Renderer.drawWrappedText` using `GlyphLayout`)
- [ ] Implement cursor blink animation. — **NOT STARTED** (`CursorVisible` field exists but no timer drives it)
- [x] Tap-to-position cursor from GlyphLayout. — **DONE** (`SetCursorFromPoint` groups glyphs by Y, finds closest X)
### Phase 2: File IO Integration
- [x] Implement `SaveFile` handler: Debounced auto-save via `WriteFileTask` to worker pool. — **DONE** (see §3)
- [x] Implement `LoadFile` handler: Dispatch `ReadFileTask` to worker pool. — **DONE**
- [x] Status bar integration: Show "Saving..." indicator, "Modified" status. — **DONE**
### Phase 3: Text Editing Operations
- [ ] Implement Cut/Copy/Paste interactions. — **NOT STARTED** (icon elements exist but handlers are `nil`)
- [x] Implement multi-line text navigation (Home/End, PageUp/PageDown). — **DONE**
- [ ] Implement text selection display and mouse interaction. — **NOT STARTED** (`SelectionStart`/`SelectionEnd` fields exist in `EditorState` but are unused)
- [x] Implement soft-wrap toggle. — **DONE** (`ToggleWordWrap` in `state.go`, wired to bottom bar "Wrap" label tap)
### Phase 4: Polish & Advanced Features
- [ ] Undo/Redo stack implementation. — **NOT STARTED** (`UndoStack` field is commented out in `EditorState`)
- [ ] Performance testing with large files (>1MB). — **NOT STARTED**
### 4.5 Real Filesystem Integration
To support real filesystem operations with minimal changes, we are abstracting the filesystem behind an interface.
### 4.5.1 Design
- **Interface Definition**: Define a `FileSystem` interface in `internal/io/pool/` that captures the required operations (`ReadDir`, `ReadFile`, `WriteFile`, `DirExists`, etc.).
- **Mock Update**: Update the existing `internal/io/pool/mock` to satisfy this interface.
- **Real Implementation**: Create `internal/io/pool/real` that implements the `FileSystem` interface using standard `os` and `path/filepath` packages.
- **Dependency Injection**: Update worker pool tasks to accept the `FileSystem` interface instead of the concrete `*mock.FileSystem` type.
- **Atomic Writes**: The `RealFileSystem` must also implement atomic writes via temporary files and `os.Rename` to maintain the consistency guarantees of the mock.
### 4.5.2 Implementation Roadmap
1. Define `pool.FileSystem` interface.
2. Update `mock.FileSystem` to satisfy the interface.
3. Update `pool` tasks to accept the interface.
4. Implement `real.FileSystem`.
5. Update `main.go` to inject the correct implementation based on build/runtime flags.
---
## 5. Interaction Flow (Example: Keyboard/Mouse Input)
1. **User** performs an action (types a character, taps an icon).
2. **UI (Renderer)** registers the element's interaction (`event.Op` for keys, `gesture.Add` for mouse) within its clip context.
3. **Main Loop** captures the event (using `key.Filter` for keys, `gesture.Update` for mouse).
4. **Main Loop** generates an `InputEvent` and sends it to `inputChan`.
5. **Logic Goroutine** receives the event, triggers the corresponding `Handler` in `TheState`.
6. **Logic Goroutine** modifies state, and sends updated elements back through `frameChan`.
7. **Renderer** paints the new frame.
---
## 6. Performance Targets
| Operation | Target | Mechanism |
|---|---|---|
| Typing latency | < 16ms | Single-owner logic update, immediate frame redraw |
| File Load (1MB) | < 100ms | Async worker pool read, background loading |
| File Save | < 100ms | Async worker pool write |
| Undo/Redo | < 10ms | O(1) or O(N) memory-based buffer update |
---
## 7. Input Handling Mechanism
### 7.1 Mouse/Touch Input
- Uses `gesture.Click` and `gesture.Scroll`.
- Registered via `event.Op` and gesture `Add()` within the clip context of the element.
- Processed by `Renderer.CheckGestures` and dispatched via `InputEvent` to the logic goroutine.
### 7.2 Keyboard Input
- **Registration**: Elements register as input handlers using `event.Op(gtx.Ops, elementID)`. This tags the current clip context for input routing.
- **Focus**: Focus is managed by the logic goroutine. When an element is focused, `key.FocusCmd{Tag: elementID}` is submitted to the operation stack.
- **Filtering**: Keyboard events are processed in the main loop using `key.Filter{Focus: focusedElementID}` to ensure events are only routed to the active element.
- **Routing**: `key.Event` (for key presses) and `key.EditEvent` (for text input) are converted into `InputEvent` structs and sent to the logic goroutine via `inputChan` for state updates.
---
## 8. Glyph Layout Architecture
### 8.1 Problem
The editor must know the exact screen position of every character after shaping and word wrapping. This is required for:
- Accurate cursor rendering (no fixed-width approximation)
- Correct cursor navigation (arrow keys respect visual line boundaries)
- Scroll-aware positioning (cursor follows text when viewport changes)
- Screen resize resilience (cursor tracks text as wrap points shift)
- Future features: mouse click-to-place, text selection, search highlight positioning
### 8.2 Core Principle
**The renderer is the single source of truth for where characters appear on screen.** Logic never guesses positions — it reads the layout the renderer already computed during the draw pass.
### 8.3 Data Structure
```go
type GlyphLayout struct {
ByteOffsets []int // byte offset of each glyph in the buffer
X []Dp // screen X (Dp) of each glyph, relative to text region origin
Y []Dp // screen Y (Dp) of each glyph, relative to text region origin
Advance []Dp // advance width (Dp) of each glyph
}
```
Each index `i` represents one glyph. `ByteOffsets[i]` is the byte position in the buffer, `(X[i], Y[i])` is its screen location, and `Advance[i]` is its width. The slice length equals the total number of glyphs (one per rune).
**Derived values:**
- `LastLineY` = `Y[len(Y)-1]` (last glyph's baseline Y) — replaces the separate `lastLineY` feedback
- Visual line breaks = any index `i` where `Y[i] > Y[i-1]`
- Cursor at byte offset `b` → binary search `ByteOffsets` for exact match, then read `(X[idx], Y[idx])`
Storing all X advances is not expensive — a single page of text is a tiny amount of memory.
### 8.4 Capture Point
**Implemented.** `drawWrappedText` in `render.go` captures full per-glyph layout data during the shaping loop:
```go
func (r *Renderer) drawWrappedText(gtx layout.Context, str string, reg Region, wrapWidth Dp, scrollOffset Dp, cursorPos int) {
// ... shaping setup with WrapHeuristically ...
r.shp.LayoutString(params, str)
var layout GlyphLayout
byteOffset := 0
for g, ok := r.shp.NextGlyph(); ok; g, ok = r.shp.NextGlyph() {
layout.ByteOffsets = append(layout.ByteOffsets, byteOffset)
layout.X = append(layout.X, Dp(float32(g.X>>6)/r.scale.Scale()))
layout.Y = append(layout.Y, Dp(float32(g.Y)/r.scale.Scale()))
layout.Advance = append(layout.Advance, Dp(float32(g.Advance>>6)/r.scale.Scale()))
for i := uint16(0); i < g.Runes; i++ {
_, sz := utf8.DecodeRuneInString(str[byteOffset:])
byteOffset += sz
}
// ... draw logic ...
}
// Store captured layout; derive lastLineY from it.
r.glyphLayout = layout
if len(layout.Y) > 0 {
r.lastLineY = layout.Y[len(layout.Y)-1]
}
}
```
The layout is stored on the renderer as `r.glyphLayout`. `lastLineY` is derived from `layout.Y[len-1]`. Exposed via `Renderer.GlyphLayout()`.
### 8.5 Feedback Path
**Implemented.** A `layoutChan` carries the full `GlyphLayout` from renderer to logic. `LastLineY` is derived from `GlyphLayout.Y[len-1]`.
```go
// main.go — after each FrameEvent
glyphLayout := renderer.GlyphLayout()
logic.LayoutChan() <- glyphLayout
// logic.go
case layout := <-l.layoutChan:
l.state.Editor.GlyphLayout = layout
var derivedLastLineY ui.Dp
if len(layout.Y) > 0 {
derivedLastLineY = layout.Y[len(layout.Y)-1]
}
if derivedLastLineY != l.state.LastLineY {
l.state.LastLineY = derivedLastLineY
l.frameChan <- l.state.layout(l.browserManager)
}
```
### 8.6 Editor State Consumption
**Implemented.** `EditorState` contains a `GlyphLayout` field. All cursor operations are GlyphLayout-based.
```go
type EditorState struct {
Buffer string
CursorPosition int
GlyphLayout GlyphLayout
// ...
}
```
**Cursor rendering** — handled inline in `drawWrappedText` (`render.go`). After shaping, the renderer binary-searches `ByteOffsets` for `cursorPos` and draws a 2dp vertical bar at the computed position. No separate cursor element is needed in the element tree.
**Cursor movement** (`HandleCursorMove`, `state.go`) — byte-based ±1 with bounds clamping. For proportional fonts this is a reasonable approximation; a glyph-index-based version could be added later.
**Vertical cursor movement** (`HandleVerticalCursorMove`, `state.go`) — fully GlyphLayout-based. Finds the current glyph's Y via binary search, scans for the target line's Y, then picks the glyph with the closest X on that line.
**Tap-to-position** (`SetCursorFromPoint`, `state.go`) — GlyphLayout-based. Groups glyphs by Y-baseline, identifies the target visual line from the tap Y, then finds the closest glyph by X distance on that line. Handles edge case of tapping past the rightmost glyph.
### 8.7 Correctness Guarantees
| Scenario | How it works |
|---|---|
| **Word wrap** | Shaper's `WrapHeuristically` produces different Y values at wrap points. Layout captures them exactly. |
| **Screen resize** | Next frame → new `wrapWidth` → shaper produces new layout → main loop sends new layout → cursor follows text. |
| **Editing** | Buffer changes → layout re-computed on next frame → `ByteOffsets` reflect new positions. |
| **Scroll** | Y values are computed minus `scrollOffset` → cursor Y tracks naturally. |
| **LastLineY** | Derived from `layout.Y[len-1]` — no separate tracking needed. |
| **Non-fixed-width fonts** | `Advance` comes from actual glyph metrics, not approximation. |
| **Empty buffer** | Layout is empty (`len == 0`). Cursor rendering handles this as the "after last glyph" case. |
### 8.8 Summary of Changes
| File | Status | Notes |
|---|---|---|
| `internal/ui/unit.go` | **Done** | `GlyphLayout` struct defined with `ByteOffsets`, `X`, `Y`, `Advance` |
| `internal/ui/render.go` | **Done** | `drawWrappedText` captures full `GlyphLayout`; derives `lastLineY` from it; renders cursor inline |
| `internal/ui/element.go` | **Done** | `TextField.Draw` passes `CursorPosition` to `drawWrappedText` |
| `internal/editor/logic.go` | **Done** | Uses `layoutChan` (full `GlyphLayout`) instead of `lastLineYChan` |
| `internal/editor/state.go` | **Done** | `GlyphLayout` stored on `EditorState`; cursor rendering handled by `Renderer` |
| `cmd/pad/main.go` | **Done** | Sends `renderer.GlyphLayout()` on `LayoutChan()` after each frame |
---
## 9. Current Implementation Status (as of June 2026)
This section documents what has been implemented beyond the original plan, and what gaps remain.
### 9.1 Implemented Components
| Component | Status | Notes |
|---|---|---|
| `EditorState` struct | **Done** | `Buffer`, `CursorPosition`, `GlyphLayout`, `fileVersion`, `lastWriteVersion`, `Filename`, `saveTimer` |
| `GlyphLayout` capture | **Done** | Full per-glyph capture in `drawWrappedText` (ByteOffsets, X, Y, Advance) |
| `layoutChan` feedback | **Done** | Main loop sends `GlyphLayout` after each frame; logic derives `LastLineY` from it |
| `HandleInsert` | **Done** | String concatenation at cursor position, advances cursor, calls `markDirty()` |
| `HandleBackspace` | **Done** | Removes byte before cursor, decrements cursor, calls `markDirty()` |
| `HandleDelete` | **Done** | Removes byte after cursor, calls `markDirty()` |
| `HandleCursorMove` | **Done** | Byte-based ±1 movement, bounded by buffer length |
| `HandleVerticalCursorMove` | **Done** | GlyphLayout-based: binary search for current Y, scan for target Y, find closest X |
| `SetCursorFromPoint` | **Done** | GlyphLayout-based tap-to-position: groups glyphs by Y, finds closest X on target line |
| `HandleKeyDown` | **Done** | Dispatches `key.Name` (arrows, delete, return) and `key.EditEvent` (text input) |
| `EditorLayout` | **Done** | Full layout: status bar, text field, bottom bar |
| Cursor rendering | **Done** | Inline in `Renderer.drawWrappedText` — 2dp vertical bar positioned from `GlyphLayout` |
| File open flow | **Done** | `OpenFile` → `ReadFileTask` → worker pool → `handleWorkerResult` → `Editor.Buffer` |
| Soft-wrap toggle | **Done** | `ToggleWordWrap` wired to bottom bar label |
| Page navigation | **Done** | Browser ↔ Editor page switching via `GoToBrowser`/`GoToEditor` |
| Keyboard focus | **Done** | `FocusedElementID` + `key.FocusCmd` + `key.Filter`/`key.FocusFilter` routing |
| Scroll handling | **Done** | `HandleScroll` clamped to `[0, MaxScroll]` derived from `LastLineY` |
| Mock filesystem | **Done** | `populateMockFileSystem` with realistic directory structure |
| Worker pool | **Done** | 4-goroutine pool with `ReadFileTask`, `BuildIndexTask`, `WriteFileTask`, etc. |
| Per-file dirty tracking | **Done** | `fileVersion` + `lastWriteVersion` maps, computed `IsDirty()` |
| Auto-save debounce | **Done** | 1s debounce with generation counter, `DispatchNonBlocking` |
| Atomic write (mock) | **Done** | `WriteFileAtomic` — temp file + mutex-protected rename |
| Stale result filtering | **Done** | Path check + version tracking in `handleWorkerResult` |
| `WriteFileTask` atomic | **Done** | `Execute()` calls `WriteFileAtomic`, `Result` carries `FilePath` |
| Generation counter | **Done** | `saveGeneration` prevents stale timer callbacks from dispatching |
| `OpenFile` filename | **Done** | Tracks `Filename`, cancels pending timer on file switch |
| Dynamic filename in UI | **Done** | `EditorLayout` uses `TheState.Editor.Filename` |
| `markDirty()` / `markSaved()` | **Done** | Helper methods on `EditorState` |
| `IsDirty()` computed | **Done** | `fileVersion > lastWriteVersion` |
| `Result.FilePath` field | **Done** | Added to `Result` struct for stale result filtering |
| `WriteFileAtomic` in mock | **Done** | Top-level `.tmp/` directory, atomic rename, no partial writes |
### 9.2 Test Coverage
| Test File | Coverage |
|---|---|
| `buffer_test.go` | `TestInsertChar` — verifies insert at end of buffer |
| `buffer_test.go` | `TestBackspace` — verifies backspace at end of buffer |
| `cursor_test.go` | `TestCursorPositioning_Basic` — verifies GlyphLayout-based tap-to-position |
| `cursor_test.go` | `TestCursorPositioning_EmptyDocument` — verifies empty document handling |
| `cursor_test.go` | `TestCursorPositioning_IncompleteLayout` — verifies graceful handling of partial layout |
| `cursor_test.go` | `TestHandleCursorMove_Bounds` — verifies cursor clamping at buffer start/end |
| `integration_test.go` | `TestOpenFileIntegration` — verifies full file open flow through worker pool |
| `auto_save_test.go` | `TestAutoSave_Fires` — edit → debounce → WriteFileTask dispatched |
| `auto_save_test.go` | `TestAutoSave_DebounceCoalesces` — rapid edits → single WriteFileTask |
| `auto_save_test.go` | `TestAutoSave_GenerationFiltering` — old timer callbacks drop |
| `auto_save_test.go` | `TestAutoSave_FileSwitchCancelsTimer` — switch file → old not saved |
| `auto_save_test.go` | `TestAutoSave_StaleResultIgnored` — result for different file ignored |
| `auto_save_test.go` | `TestAutoSave_FailureLeavesDirty` — write fails → dirty stays true |
| `auto_save_test.go` | `TestAutoSave_SuccessClearsDirty` — write succeeds → dirty becomes false |
| `auto_save_test.go` | `TestAutoSave_PersistsAcrossSwitch` — edit A → switch B → switch A → A still dirty |
| `auto_save_test.go` | `TestAutoSave_AtomicWrite` — temp file + rename, no partial writes visible |
### 9.3 Known Gaps
| Gap | Description |
|---|---|
| Cursor blink | `CursorVisible` field exists but no blink timer is driven |
| Home/End/PageUp/PageDown | No handlers for these keys |
| Cut/Copy/Paste | Icon elements exist with `nil` handlers |
| Text selection | `SelectionStart`/`SelectionEnd` fields present but unused |
| Undo/Redo | Commented out in `EditorState`, no implementation |
| Status bar dynamics | Filename, line/col, byte count are static strings |
| `Dirty` flag UI | `Dirty` is computed but not displayed in status bar |
| Real filesystem backend | Mock `WriteFileAtomic` uses in-memory `.tmp/`; production needs `.pad/tmp/` temp+rename |
| File close save | No explicit save-on-file-close; last edits rely on debounce timer |
### 9.4 File-by-File Change Summary for Auto-Save
| File | Changes |
|---|---|
| `internal/editor/state.go` | Added `Filename`, `fileVersion`, `lastWriteVersion`, `saveTimer`; removed `Dirty` boolean; added `IsDirty()`, `markDirty()`, `markSaved()`; updated `OpenFile` to cancel timer and track filename |
| `internal/editor/logic.go` | Added `saveGeneration`; added debounce logic in `inputChan` handler; added `TypeWriteFile` handling in `handleWorkerResult` |
| `internal/io/pool/task.go` | `WriteFileTask.Execute()` calls `WriteFileAtomic`; `Result` carries `FilePath` |
| `internal/io/pool/result.go` | Added `FilePath` field to `Result` |
| `internal/io/pool/mock/filesystem.go` | Added `WriteFileAtomic` — temp file + mutex-protected rename |
| `internal/editor/auto_save_test.go` | New test file: 9 tests covering debounce, generation filtering, stale results, atomic write, failure retry |

602
doc/element_model.md
View File

@ -1,602 +0,0 @@
# Element Model Specification
## 1. Overview
Pad's logic layer produces a slice of positioned elements. Each element knows how to draw itself: the `Element` interface includes a `Draw` method that delegates low-level rendering to a `*Renderer`. Elements are Go structs with both structural data (text, alignment, etc.) and behavioral code (their `Draw` implementation).
```
Logic: (State, Event) → []Element
Draw: Element.Draw(gtx, r) → r.drawText(), r.drawBg(), ...
Test: assert on []Element directly (no renderer needed)
```
The Renderer provides low-level primitives (`drawText`, `drawBg`, `drawPng`) that elements call during their `Draw` method. It handles Dp→Px conversion and Gio shaper interop. The Renderer's `Draw(gtx, elems)` iterates the slice and calls each element's `Draw` method.
## 2. Core Types
### 2.1 Coordinate System
Consistent with Gioui and standard 2D graphics:
- **Origin (0, 0)**: top-left corner of the application surface
- **Y-axis**: increases downward
- **X-axis**: increases rightward
- **Screen orientation**: the application surface rotates with the device (handled by the OS / Gioui surface). The logic layer receives the current surface dimensions and recomputes layouts accordingly.
### 2.2 Units
The project uses distinct Go types to prevent accidental mixing of coordinate units at compile time:
| Unit | Go Type | Use |
|---|---|---|
| **Dp** (device-independent pixels) | `ui.Dp` (alias of `unit.Dp`) | Element positions, sizes, spacing in the logic/layout layer |
| **Px** (physical pixels) | `ui.Px` (alias of `int`) | Gio interop only (`gtx.Constraints`, `gtx.Dp()`, etc.) |
| **Sp** (scaled pixels) | `unit.Sp` (`float32`) | Font sizes (respects user text scaling preference; Gio's shaper requires `unit.Sp`) |
### Type Safety
`ui.Dp` and `ui.Px` are distinct named types. The compiler prevents accidental mixing:
```go
var pos ui.Dp = ui.Dp(100)
var winW ui.Px = ui.Px(780)
// Compile error: invalid operation: pos + winW (mismatched types ui.Dp and ui.Px)
// Must use explicit conversion:
converted := ui.ToPx(pos, scale) // Dp → Px
```
### Conversion Functions
```go
// Convert Dp to Px using the scale factor (pixels per DP)
func ToPx(dp Dp, scale float32) Px
// Convert Px to Dp using the scale factor
func ToDp(px Px, scale float32) float32
```
### Architecture
- **Logic/Layout layer**: Works exclusively in `Dp`. Element regions, positions, and sizes are all in `Dp`.
- **Renderer**: Converts `Dp` → `Px` at Gio interop boundaries using `gtx.Metric.PxPerDp`.
- **main.go**: Converts `app.ConfigEvent` pixel dimensions to `Dp` before passing to the logic layer.
The Renderer holds a `ScaleProvider` interface (backed by `*editor.State`) that provides the current scale factor. It converts Dp↔Px on demand via `r.toPx()` / `r.toDp()`. The main window clip is applied at the top of `Renderer.Draw()` to bound all drawing to the window area.
### 2.3 Element Interface
All UI elements implement the `Element` interface. Elements know their region and visibility, and **each element draws itself** by delegating to the `*Renderer`:
```go
type Element interface {
Region() Region
Visible() bool
Draw(gtx layout.Context, r *Renderer)
}
```
Concrete elements are Go structs with unexported fields for `region`, `visible`, and `id`. Constructors (`NewLabel`, `NewContainer`, `NewIcon`, etc.) set these fields, keeping the API clean and preventing external mutation.
```go
type Region struct {
X, Y Dp // ui.Dp, not unit.Dp
W, H Dp
}
type Label struct { /* unexported region, visible, id + exported Text, Align, ... */ }
func NewLabel(text string, fontSize unit.Sp, region Region, align TextAlign) Label
func (l Label) Region() Region
func (l Label) Visible() bool
func (l Label) Draw(gtx layout.Context, r *Renderer) { r.drawText(...) }
func (l Label) ID() string
```
The Renderer's `Draw(gtx, elems)` iterates the element slice, skips invisible elements, and calls each element's `Draw` method. `Container` elements additionally clip to their bounds and offset their children. This keeps the logic layer testable — tests assert on concrete element values without any framework or interface indirection in the test code.
## 3. Element Catalog
### 3.0 Container
Composite element that holds child elements within a clipped region with an optional background. Children's regions are relative to the Container's origin.
```go
type Container struct {
/* region, visible, id — unexported */
background Color
children []Element
}
func NewContainer(region Region, bg Color, children []Element) Container
func (c Container) Draw(gtx layout.Context, r *Renderer)
```
`Container.Draw()` draws its background (if any). The Renderer's `drawElement` handles clipping to the container's bounds and offsetting children before recursing into them. Children are drawn in slice order.
Used for: StatusBar, BottomBar, and any grouped UI region with a shared background and containment clipping.
### 3.1 Static Text
```go
type Label struct {
/* region, visible, id — unexported */
Text string
Align TextAlign // start, center, end
FontSize unit.Sp // 0 = theme default
Color Color // 0 = theme default
Bold bool
}
type TextAlign int // Start, Center, End
```
Used for: headers, status bar text, file sizes, line numbers.
### 3.1b Icon
Displays a named icon image (loaded from embedded PNGs). The icon **automatically scales to fill its region** when `Size` is 0, using the region's `W` and `H` as the target dimensions.
```go
type Icon struct {
/* region, visible, id — unexported */
Name string // icon name, e.g. "cut", "copy", "paste"
Size Dp // 0 = auto-scale to region W×H
}
func NewIcon(name string, region Region, size Dp) Icon
func (i Icon) Draw(gtx layout.Context, r *Renderer)
```
Icons are loaded from the embedded filesystem (`icons/*.png`). The `Renderer` caches them in a map and renders via `r.drawPng()`. When `Size` is 0 (the default), `Icon.Draw` passes `region.W` and `region.H` as the target dimensions, and `r.drawPng()` applies an affine scale transform so the source image fills that space. The `Size` field can be set to a non-zero value to override the region-based sizing.
Used for: action icons in the StatusBar (cut, copy, paste).
### 3.2 Interactive Text
```go
type TextField struct {
/* region, visible, id — unexported */
Value string
Placeholder string
Focused bool // true = show cursor, accept keyboard input
Multiline bool // true = full-height text area (editor)
ScrollOffset unit.Dp // vertical scroll position (Dp)
VisibleLines []Line // for multiline: the lines to render
WordWrap bool // true = wrap long lines visually (default)
WrapWidth unit.Dp // width at which wrapping occurs (auto if 0)
}
type Line struct {
Text string
LineNumber int // 1-indexed, for display
}
```
Used for: search bar (`Multiline: false`), editor buffer (`Multiline: true`).
The editor's `TextField` only contains lines visible in the current viewport. The logic layer determines which lines to include based on scroll offset and viewport height.
**Word wrap**: when `WordWrap` is true, long lines are broken visually at word boundaries within the element's region width. This is a display-only feature — no newlines are inserted into the underlying text. The logic layer computes wrapped display lines from the raw text, region width, and estimated character advance. Users can toggle word wrap off (horizontal scroll instead).
### 3.3 Search Bar
```go
type SearchBar struct {
/* region, visible, id — unexported */
Query string // current search text
Match int // current match index (0-based)
Total int // total number of matches
Forward bool // true = last search was forward, false = backward
}
```
Used for: in-editor text search. Appears as a narrow bar below the header. Contains a text field and up/down arrows. The logic layer:
1. Finds all occurrences of `Query` in the visible buffer (or full file for small files)
2. Sets `Match` to the current position in the match list
3. Moves the editor cursor to the matched text on each keystroke or arrow tap
4. Up arrow = previous match, down arrow = next match
### 3.4 Cursor
```go
type Cursor struct {
/* region, visible, id — unexported */
Line int // line number (0-indexed into VisibleLines)
Column int // character offset within the line
Blinking bool // current blink state
Selection *Selection
}
type Selection struct {
StartLine, StartCol int
EndLine, EndCol int
}
```
Rendered as an overlay inside a `TextField`. The logic layer computes cursor position from byte offset.
### 3.5 Lists
```go
type ListView struct {
/* region, visible, id — unexported */
Items []ListItem
ScrollOffset int // index of the first visible item
Selected int // index of selected item (-1 = none)
}
type ListItem struct {
Text string
Subtext string // optional secondary text (e.g., file size, date)
Selected bool // highlighted
}
```
Used for: directory browser. Only contains items for the current viewport.
### 3.6 Alphabet Index
```go
type AlphaIndex struct {
/* region, visible, id — unexported */
Letters []string // visible letters (e.g., ["A", "B", "C", ...])
ActiveLetter string // currently pressed letter (for highlighting)
}
```
Used for: quick navigation in the directory browser.
### 3.7 Buttons
```go
type Button struct {
/* region, visible, id — unexported */
Text string
Enabled bool
Primary bool // true = emphasized style (e.g., filled background)
}
func NewButton(text string, enabled bool, primary bool, region Region) Button
func (b Button) Draw(gtx layout.Context, r *Renderer)
```
Used for: merge resolution (ours/theirs/both), dismiss, apply.
### 3.8 Merge Hunk
```go
type MergeHunk struct {
/* region, visible, id — unexported */
HunkNumber int // N of M
TotalHunks int
LineRange string // display text, e.g., "Lines 142–148"
ContextLines []string // unchanged lines (shared)
OurLines []string // our version of the changed region
TheirLines []string // their version of the changed region
Resolution HunkResolution
}
type HunkResolution int // Unresolved, KeepOurs, KeepTheirs, MergeBoth
```
Used for: conflict resolution UI. The logic layer produces one `MergeHunk` element plus navigation buttons.
### 3.9 Status Bar (Top Bar) — Composition
The status bar is **not a standalone element type**. It is composed from `Container`, `Label`, and `Icon` elements:
```go
// EditorLayout builds the status bar as:
Container{
region: Region{X: margin, Y: margin, W: screenWidth - margin*2, H: 52},
background: Color{R: 230, G: 230, B: 230, A: 255},
children: []Element{
// Row 1: filename
NewLabel("filename.txt", 14, Region{X: 0, Y: 2, W: statusBarW, H: 20}, AlignStart),
// Row 2: cut, copy, paste icons (Size=0 → auto-scale to region W×H)
NewIcon("cut", Region{X: 0, Y: 28, W: 24, H: 24}, 0),
NewIcon("copy", Region{X: 48, Y: 28, W: 24, H: 24}, 0),
NewIcon("paste", Region{X: 96, Y: 28, W: 24, H: 24}, 0),
},
}
```
The status bar appears at the top of the editor page. It has a **two-line layout**:
- **Line 1**: Filename (truncated with ellipsis if too long). Tapping the ellipsis toggles between truncated and full multi-line view.
- **Line 2**: Action icons (Cut, Copy, Paste) on the left. Icons are `24×24` DP each with `Size=0`, so they auto-scale to fill their regions.
The Container clips its children to its bounds and provides a shared background. Children use regions relative to the Container's origin — the Renderer applies `op.Offset` automatically when drawing Container children.
When a sync conflict is detected for the active file, a conflict `Icon` appears. Tapping it navigates to the merge resolution page.
### 3.10 Bottom Bar — Composition
The bottom bar is **not a standalone element type**. It is composed from `Container` and `Label` elements:
```go
// EditorLayout builds the bottom bar as:
Container{
region: Region{X: margin, Y: screenHeight - margin - 24, W: screenWidth - margin*2, H: 24},
background: Color{R: 230, G: 230, B: 230, A: 255},
children: []Element{
NewLabel("Ln 47, Col 12", 12, Region{X: 0, Y: 2, W: barW, H: 20}, AlignStart),
NewLabel("1024 / 50000", 12, Region{X: 0, Y: 2, W: barW, H: 20}, AlignCenter),
NewLabel("Wrap: On", 12, Region{X: 0, Y: 2, W: barW, H: 20}, AlignEnd),
},
}
```
The bottom bar appears at the bottom of the editor page. It displays cursor position, byte position, and word wrap status. It is always visible with a fixed height of 24 DP.
**Layout**:
```
┌──────────────────────────────────────────────────────────────┐
│ Ln 10, Col 5 1024 / 50000 Wrap: On │
└──────────────────────────────────────────────────────────────┘
```
- **Left**: Cursor position (line number, column number)
- **Center**: Byte position (current byte / total file bytes)
- **Right**: Word wrap status (tap to toggle On/Off)
### 3.11 Toast / Notification
```go
type Toast struct {
/* region, visible, id — unexported */
Text string
Timeout int // auto-dismiss after this many milliseconds
}
func NewToast(region Region, text string, timeout int) Toast
```
Used for: "undo skipped (text changed)", "file saved", "conflict detected".
### 3.12 Spacer
```go
type Spacer struct {
/* region, visible, id — unexported */
// Region.H defines the spacer height
}
```
## 4. Layout Model
The logic layer performs a layout pass before emitting elements. Each element's `Region` is already computed. Elements draw themselves: the Renderer iterates the slice and calls `Element.Draw(gtx, r)` for each visible element. `Container` elements additionally clip to their bounds and offset their children.
### Layout Pass
```
1. Given: screen width, screen height, state
2. Compute: region for each element
3. Emit: []Element with filled Region fields
```
The layout pass is pure: `(ScreenSize) → []Element`. It is testable.
Currently implemented: `EditorLayout(screenWidth, screenHeight ui.Dp) []ui.Element` in `internal/editor/state.go`. It produces:
```
Screen: 390 × 844 Dp
Container (StatusBar) → Region{X:10, Y:10, W:370, H:52}
Container (BottomBar) → Region{X:10, Y:810, W:370, H:24}
```
The logic layer knows the screen dimensions (converted from raw pixels to Dp in `main.go`) and computes regions accordingly.
### Scrolling
Scrollable elements (ListView, TextField multiline) contain only the visible items. The logic layer:
1. Tracks scroll offset (pixels or item index)
2. Computes which items are visible given the offset and viewport height
3. Emits only those items
4. Updates `ScrollOffset` field for the renderer to maintain scroll position
Scroll events from the renderer are routed back to the logic layer, which updates the offset and re-renders.
## 5. Interaction Model
Elements declare their input behaviors at construction time via the `Interactive` interface. Each element can register one or more `Interaction`s, pairing a gesture type with a handler function.
### 5.1 Core Types
```go
type InputType int
const (
Tap InputType = iota
DoubleTap
)
// Interaction pairs a gesture type with a handler function.
type Interaction struct {
Gesture InputType
Handler func(any) // called with raw Gio gesture event data
}
// Interactive is implemented by elements that respond to input events.
type Interactive interface {
ID() string
Interactions() []Interaction
}
// InputEvent carries a handler and its raw event data.
type InputEvent struct {
Handler func(any)
Data any
}
```
### 5.2 Declaring Interactions
Elements declare interactions when constructed:
```go
// Wrap label: tap toggles word wrap
wrapLabel := NewLabel("Wrap: On", 12, region, AlignEnd, "wrap",
[]Interaction{{Gesture: Tap, Handler: ToggleWordWrap}})
```
Every interactive element type (`Label`, `Icon`, `Button`, `TextField`, `ListView`, `AlphaIndex`, `MergeHunk`, `SearchBar`, `Cursor`, `Toast`, `Spacer`) has an `interactions` field and implements `Interactions() []Interaction`.
### 5.3 Click Area Clipping
The click area is **not** the element's full region. It is clipped to the actual painted content:
- **Text elements** (`Label`, `Button`): The click area is the bounding box of the shaped text glyphs. For `AlignEnd` labels (e.g., "Wrap: On"), only the text area at the right edge is clickable, not the full container width.
- **Icon elements**: The click area is the icon image's bounding box.
- **Text fields and lists**: The click area covers the element's full region (the entire text area or list viewport).
This clipping is implemented in `Renderer.drawText()`: after shaping the text, a `clip.Rect` is pushed around the text's bounding box, then `gesture.Click.Add()` is called inside that clip. Only paint ops within the clip contribute to the click area.
```
┌──────────────────────────────────────────────────────────────┐
│ Ln 10, Col 5 1024 / 50000 [Wrap: On] │
│ ──── click area ──── ──── click area ──── click area │
│ (full width) (full width) (text only) │
└──────────────────────────────────────────────────────────────┘
```
### 5.4 Gesture Registration and Event Routing
Click gesture registration and event routing follow this flow:
1. **Registration**: In `drawElement`, the renderer checks if the element implements `Interactive`. For each `Interaction`, it stores the `*gesture.Click` and handler in `clicks map[string]clickReg`.
2. **Click area**: During `drawText` (or `drawPng` for icons), `click.Add(gtx.Ops)` is called inside the content's clip, associating the gesture with the painted area.
3. **Polling**: After `renderer.Draw()`, `renderer.CheckGestures()` polls all registered `gesture.Click` instances via `click.Update(q)`.
4. **Routing**: Click events are returned as `InputEvent{Handler, Data}`. The main loop sends them to the logic goroutine via `inputChan`.
5. **Execution**: The logic goroutine calls `evt.Handler(evt.Data)`. Handlers are static functions (e.g., `ToggleWordWrap`) that read/write global `TheState` directly, avoiding closures and circular imports.
### 5.5 Handler Design
Handlers are static functions defined in the `editor` package:
```go
func ToggleWordWrap(data any) {
state := editor.TheState.Load()
state.Lock()
state.WordWrap = !state.WordWrap
state.Unlock()
}
```
They receive the raw Gio gesture event (`gesture.ClickEvent`) as `any`. They access global `TheState` directly, avoiding the need for closures or element-ID dispatch tables.
## 6. Rendering Order
Elements are rendered in slice order. Later elements draw on top of earlier ones. Typical order:
```
1. Background (full screen)
2. Status bar (top)
3. Search bar or main content
4. List or text area
5. Overlay elements (cursor, selection highlight)
6. Bottom bar
```
## 7. Theme
Minimal theming via a `Theme` struct passed to the renderer:
```go
type Theme struct {
FontSize unit.Sp
}
```
Elements that don't specify explicit colors/sizes use theme defaults. The theme is part of the app state, not a global. Screen dimensions, header/status bar heights, and padding are layout constants known to the logic layer's layout functions, not theme fields.
## 8. Page Compositions
### 8.1 Browser Page
```
[]Element{
Label{Text: "My Documents"}, // header
TextField{Placeholder: "Search..."}, // search bar
ListView{Items: [...], ScrollOffset: 0}, // file list
AlphaIndex{Letters: ["A"..."Z"]}, // right sidebar
}
```
### 8.2 Editor Page
```
[]Element{
// StatusBar: Container with Labels and Icons
Container{
background: gray,
children: []Element{
NewLabel("notes.txt", 14, ..., AlignStart), // filename
NewIcon("cut", Region{X:0, Y:28, W:24, H:24}, 0), // cut (auto-scale)
NewIcon("copy", Region{X:48, Y:28, W:24, H:24}, 0), // copy (auto-scale)
NewIcon("paste", Region{X:96, Y:28, W:24, H:24}, 0), // paste (auto-scale)
},
},
// (future) TextField{Multiline: true, ...},
// (future) Cursor{Line: 5, Column: 12, Selection: &Selection{...}},
// BottomBar: Container with Labels
Container{
background: gray,
children: []Element{
NewLabel("Ln 47, Col 12", 12, ..., AlignStart),
NewLabel("1024 / 50000", 12, ..., AlignCenter),
NewLabel("Wrap: On", 12, ..., AlignEnd),
},
},
}
```
StatusBar and BottomBar are compositions of `Container`, `Label`, and `Icon` — not standalone element types. The `EditorLayout` function in `internal/editor/state.go` builds these compositions. Icon elements use `Size=0` so they auto-scale to fill their `24×24` DP regions.
Future pages will add `TextField` (editor content), `SearchBar`, `Cursor`, and `Button` elements.
### 8.3 Merge Page
```
[]Element{
Label{Text: "notes.txt — Conflict"},
Label{Text: "Hunk 3 of 7 (Lines 142–148)"},
MergeHunk{ContextLines: [...], OurLines: [...], TheirLines: [...]},
Button{Text: "Ours", OnPress: "resolve_ours"},
Button{Text: "Theirs", OnPress: "resolve_theirs"},
Button{Text: "Both", OnPress: "resolve_both"},
Button{Text: "Next", Primary: true, OnPress: "next_hunk"},
}
```
## 9. Testing
Tests import the logic layer, call layout functions, and assert on the resulting `[]Element`. Since `Element` is an interface, tests type-assert to concrete types:
```go
func TestEditorLayout(t *testing.T) {
elems := EditorLayout(ui.Dp(390), ui.Dp(844))
// StatusBar is a Container
status, ok := elems[0].(Container)
assert.True(t, ok)
assert.Equal(t, ui.Dp(52), status.Region().H)
assert.Equal(t, 3, len(status.children))
// BottomBar is a Container
bottom, ok := elems[1].(Container)
assert.True(t, ok)
assert.Equal(t, ui.Dp(24), bottom.Region().H)
}
```
No Android SDK, no Gioui, no display server. Pure Go tests.
## 10. Out of Scope
- Element animations (may be added later)
- Element transitions between pages
- Right-to-left text
- Dynamic font sizing / accessibility scaling
- Custom element types beyond the catalog above

689
doc/layout_rendering.md
View File

@ -1,689 +0,0 @@
# Text Shaper Usage Guide
This document describes how to use the Gio text shaper correctly for text layout, measurement, and rendering. It serves as a reference to avoid common mistakes and ensure consistency across the codebase.
## 1. Creating a Shaper
```go
import (
"gioui.org/text"
"gioui.org/x/gofont"
)
shp := text.NewShaper(text.WithCollection(gofont.Collection()))
```
**Important**: Always provide a font collection. Without it, the shaper may not load fonts correctly and will return zero-width glyphs.
## 2. Layout Parameters — CRITICAL
The `text.Parameters` struct controls how text is shaped. **Three fields are mandatory** for correct single-line text layout:
```go
params := text.Parameters{
PxPerEm: fixed.I(gtx.Sp(size)), // Font size in device pixels (REQUIRED)
MinWidth: 0, // Minimum width (REQUIRED for single-line)
MaxWidth: availableWidth, // Maximum width (REQUIRED for single-line)
MaxLines: 1, // Limit to one line (REQUIRED for single-line)
}
```
### 2.1 PxPerEm Calculation
**CORRECT**: Use `gtx.Sp(size)` to convert SP to device pixels:
```go
params := text.Parameters{
PxPerEm: fixed.I(gtx.Sp(size)), // size is unit.Sp
}
```
**WRONG**: Don't use arbitrary multipliers like `1024 * pixelsPerDp`:
```go
// WRONG - this produces incorrect widths
params := text.Parameters{
PxPerEm: fixed.Int26_6(1024 * pixelsPerDp),
}
```
**Why**: `gtx.Sp(size)` correctly converts SP to device pixels accounting for DPI scaling. The `fixed.I()` function converts the result to `fixed.Int26_6` format.
### 2.2 MinWidth/MaxWidth/MaxLines — REQUIRED
**Without `MinWidth`, `MaxWidth`, and `MaxLines`, the shaper has no horizontal space constraint and will wrap every character into its own line.** This is the most common mistake when using the shaper.
```go
// WRONG - every character wraps to its own line
shp.LayoutString(text.Parameters{PxPerEm: fixed.I(gtx.Sp(size))}, str)
// CORRECT - text flows horizontally within constraints
shp.LayoutString(text.Parameters{
PxPerEm: fixed.I(gtx.Sp(size)),
MinWidth: 0,
MaxWidth: availableWidth,
MaxLines: 1,
}, str)
```
**Why**: Gio's `widget.Label` sets these from `gtx.Constraints`:
- `MinWidth: cs.Min.X`
- `MaxWidth: cs.Max.X`
- `MaxLines: l.MaxLines`
Without these, the shaper treats each character as a separate line with `FlagLineBreak` set.
### 2.3 Word Wrap
Set `MaxWidth` to enable word wrap:
```go
params := text.Parameters{
PxPerEm: fixed.I(gtx.Sp(size)),
MinWidth: 0,
MaxWidth: int(widthInPixels),
MaxLines: 0, // 0 = no limit, allows multi-line
WrapPolicy: text.WrapHeuristically,
}
```
The shaper automatically breaks text into lines at word boundaries when `MaxWidth` is set and `MaxLines` is 0 or unset.
## 3. Layout Functions
### 3.1 LayoutString (for single strings)
```go
shp.LayoutString(params, "Hello, World!")
```
### 3.2 Layout (for readers)
```go
shp.Layout(params, strings.NewReader("Hello, World!"))
```
## 4. Iterating Through Glyphs
After layout, iterate through glyphs using `NextGlyph()`:
```go
for {
g, ok := shp.NextGlyph()
if !ok {
break
}
// Process glyph g
}
```
### 4.1 Glyph Fields
```go
type Glyph struct {
ID GlyphID // Glyph ID
X fixed.Int26_6 // Dot position in document coordinates
Y int32 // Baseline position (same for all glyphs on a line)
Ascent fixed.Int26_6 // Line ascent
Descent fixed.Int26_6 // Line descent
Advance fixed.Int26_6 // Logical width (horizontal advance)
Runes uint16 // Number of runes this glyph represents
Offset fixed.Point26_6 // Glyph offset from (X, Y)
Bounds fixed.Rectangle26_6 // Glyph bounds relative to dot
Flags Flags // FlagLineBreak, FlagRunBreak, FlagClusterBreak, etc.
}
```
### 4.2 Accumulating Widths
To get per-character cumulative widths:
```go
func charWidths(gtx layout.Context, shp *text.Shaper, str string, size unit.Sp) []int {
shp.LayoutString(text.Parameters{
PxPerEm: fixed.I(gtx.Sp(size)),
MinWidth: 0,
MaxWidth: 1000,
MaxLines: 1,
}, str)
var widths []int
var cumWidth fixed.Int26_6 = 0
for {
g, ok := shp.NextGlyph()
if !ok {
break
}
cumWidth += g.Advance
widths = append(widths, int(cumWidth>>6)) // Convert from Int26_6 to int
}
return widths
}
```
**Key points**:
- `g.Advance` is in `fixed.Int26_6` format (16.6 fixed-point)
- Convert to int by shifting right by 6 bits: `int(cumWidth>>6)`
- The cumulative width is in **device pixels** (not Dp!)
- **Always** set `MinWidth`, `MaxWidth`, `MaxLines` or widths will be wrong
### 4.3 Measuring Text Width for Alignment
When you need the total text width for centering or right-alignment, sum the advances and convert to Dp:
```go
var totalAdvance fixed.Int26_6
for g, ok := shp.NextGlyph(); ok; g, ok = shp.NextGlyph() {
totalAdvance += g.Advance
}
// totalAdvance>>6 is in DEVICE PIXELS — must divide by scale to get Dp
textW_Dp := ui.Dp(float32(totalAdvance>>6) / scale.Scale())
```
**Critical**: `totalAdvance>>6` gives device pixels, but UI regions (`reg.W`, `reg.X`) are in Dp. If you use the raw value as Dp, centering will appear roughly correct (off by factor of 2, halved by division in center formula) but right-alignment will have a visible gap. Always divide by the scale factor to convert device pixels → Dp before using with region coordinates.
**Wrong** (works for AlignStart, breaks AlignCenter/AlignEnd):
```go
textW := ui.Dp(totalAdvance>>6) // treats device pixels as Dp!
```
**Correct**:
```go
textW := ui.Dp(float32(totalAdvance>>6) / r.scale.Scale()) // convert to Dp
```
### 4.3 Getting Per-Character Widths
For hit testing (mapping UI coordinates → byte offset), we need per-character cumulative widths:
```go
func charWidths(gtx layout.Context, shp *text.Shaper, str string, size unit.Sp) []int {
shp.LayoutString(text.Parameters{
PxPerEm: fixed.I(gtx.Sp(size)),
MinWidth: 0,
MaxWidth: 1000,
MaxLines: 1,
}, str)
var widths []int
var cumWidth fixed.Int26_6 = 0
for {
g, ok := shp.NextGlyph()
if !ok {
break
}
cumWidth += g.Advance
widths = append(widths, int(cumWidth>>6))
}
return widths
}
```
## 5. Rendering Glyphs Directly
Gio's `paintGlyph` (`widget/label.go`) draws glyphs using `shaper.Shape()` (for vector glyphs) and `shaper.Bitmaps()` (for bitmap glyphs like emoji). This is the correct approach to avoid double-shaping.
### 5.1 Drawing Text — Match Gio's paintGlyph Exactly
```go
func drawText(gtx layout.Context, shp *text.Shaper, str string, size unit.Sp, x, y unit.Dp, col color.NRGBA) {
// Layout text with width constraints
shp.LayoutString(text.Parameters{
PxPerEm: fixed.I(gtx.Sp(size)),
MinWidth: 0,
MaxWidth: 1000,
MaxLines: 1,
}, str)
drawLineText(gtx, shp, x, y, col)
}
func drawLineText(gtx layout.Context, shp *text.Shaper, x, y unit.Dp, col color.NRGBA) {
// Match Gio's textView: record into a macro for clipping
m := op.Record(gtx.Ops)
var glyphs [32]text.Glyph
line := glyphs[:0]
for g, ok := shp.NextGlyph(); ok; g, ok = shp.NextGlyph() {
line = append(line, g)
if g.Flags&text.FlagLineBreak != 0 || cap(line)-len(line) == 0 {
drawLine(gtx, shp, line, x, y, col)
line = line[:0]
}
}
if len(line) > 0 {
drawLine(gtx, shp, line, x, y, col)
}
call := m.Stop()
call.Add(gtx.Ops)
}
func drawLine(gtx layout.Context, shp *text.Shaper, line []text.Glyph, x, y unit.Dp, col color.NRGBA) {
if len(line) == 0 {
return
}
first := line[0]
// shaper.Shape(line) returns a path where glyph positions are relative to
// the first glyph. Offset by (x + first.X, y + first.Y) to place the line
// at the desired document position. Matches Gio's paintGlyph:
// lineOff = (glyph.X, glyph.Y) - viewport.Min
// op.Affine(f32.Affine2D{}.Offset(lineOff))
offX := float32(gtx.Dp(x)) + float32(first.X)/64.0
offY := float32(gtx.Dp(y)) + float32(first.Y)
t := op.Affine(f32.Affine2D{}.Offset(f32.Pt(offX, offY))).Push(gtx.Ops)
// Draw vector glyphs
path := shp.Shape(line)
outline := clip.Outline{Path: path}.Op().Push(gtx.Ops)
paint.ColorOp{Color: col}.Add(gtx.Ops)
paint.PaintOp{}.Add(gtx.Ops)
outline.Pop()
// Draw bitmap glyphs (emoji, etc.)
if call := shp.Bitmaps(line); call != (op.CallOp{}) {
call.Add(gtx.Ops)
}
t.Pop()
}
```
### 5.2 Offset Calculation Explained
The offset calculation is critical:
```go
offX := float32(gtx.Dp(x)) + float32(first.X)/64.0
offY := float32(gtx.Dp(y)) + float32(first.Y)
```
- `x, y` are the desired document position (in DP)
- `first.X` is the first glyph's X position in `fixed.Int26_6` (divide by 64 to get pixels)
- `first.Y` is the first glyph's Y position (baseline) in pixels
- `shp.Shape(line)` returns a path where glyph positions are **relative to the first glyph**
- So we offset by `(x + first.X, y + first.Y)` to place the line at `(x, y)`
**This matches Gio's `paintGlyph` exactly**:
```go
// Gio's paintGlyph:
if len(line) == 0 {
it.lineOff = f32.Point{X: fixedToFloat(glyph.X), Y: float32(glyph.Y)}
.Sub(layout.FPt(it.viewport.Min))
}
// ...
t := op.Affine(f32.Affine2D{}.Offset(it.lineOff)).Push(gtx.Ops)
```
Since `viewport.Min = (0,0)` in our case, `lineOff = (glyph.X, glyph.Y)`.
### 5.3 Avoiding Double-Shaping
**WRONG** (shapes text twice):
```go
// First pass: measure
widths := charWidths(gtx, shp, text, size)
// Second pass: render (material.Label calls shaper again)
material.Label(th, size, text).Layout(gtx)
```
**CORRECT** (shapes text once for rendering):
```go
// Single pass: layout and draw using shaper.Shape()
drawText(gtx, shp, text, size, x, y, col)
```
**NOTE**: For truncation, we may need to layout twice:
1. First layout to measure widths and find truncation point
2. Second layout to draw the truncated text
This is acceptable because truncation is only needed when text doesn't fit, and the string is short (filename in StatusBar).
## 6. Comparison with Gio's paintGlyph
Gio's `paintGlyph` (`widget/label.go`) demonstrates the correct approach:
```go
func (it *textIterator) paintGlyph(gtx layout.Context, shaper *text.Shaper, glyph text.Glyph, line []text.Glyph) ([]text.Glyph, bool) {
visibleOrBefore := it.processGlyph(glyph, true)
if it.visible {
if len(line) == 0 {
it.lineOff = f32.Point{X: fixedToFloat(glyph.X), Y: float32(glyph.Y)}
.Sub(layout.FPt(it.viewport.Min))
}
line = append(line, glyph)
}
if glyph.Flags&text.FlagLineBreak != 0 || cap(line)-len(line) == 0 || !visibleOrBefore {
t := op.Affine(f32.Affine2D{}.Offset(it.lineOff)).Push(gtx.Ops)
path := shaper.Shape(line)
outline := clip.Outline{Path: path}.Op().Push(gtx.Ops)
it.material.Add(gtx.Ops) // sets color
paint.PaintOp{}.Add(gtx.Ops)
outline.Pop()
if call := shaper.Bitmaps(line); call != (op.CallOp{}) {
call.Add(gtx.Ops)
}
t.Pop()
line = line[:0]
}
return line, visibleOrBefore
}
```
**Key insights**:
1. `lineOff` is set on the first glyph: `(glyph.X, glyph.Y) - viewport.Min`
2. `shaper.Shape(line)` returns a path relative to the first glyph
3. The offset places the first glyph at `lineOff`, and subsequent glyphs follow
4. `it.material` is a pre-recorded color call; we use `paint.ColorOp` + `paint.PaintOp` instead
5. Everything is wrapped in `op.Record`/`m.Stop()` for clipping
## 7. Common Pitfalls
### 7.1 Missing MinWidth/MaxWidth/MaxLines
**Mistake**: Calling `LayoutString` without width constraints:
```go
shp.LayoutString(text.Parameters{PxPerEm: fixed.I(gtx.Sp(size))}, str)
```
**Result**: Every character wraps to its own line with `FlagLineBreak` set and `X=0`.
**Fix**: Always set `MinWidth`, `MaxWidth`, and `MaxLines`:
```go
shp.LayoutString(text.Parameters{
PxPerEm: fixed.I(gtx.Sp(size)),
MinWidth: 0,
MaxWidth: availableWidth,
MaxLines: 1,
}, str)
```
### 7.2 Wrong PxPerEm Calculation
**Mistake**: Using arbitrary multipliers like `1024 * pixelsPerDp`
**Fix**: Use `fixed.I(gtx.Sp(size))` to get the font size in device pixels
### 7.3 Double-Shaping
**Mistake**: Calling `charWidths()` to measure, then `material.Label()` to render
**Fix**: Use `shp.Shape()` to render the already-laid-out glyphs
### 7.4 Wrong Offset Calculation
**Mistake**: Using just `(x, y)` without adding `first.X`/`first.Y`:
```go
// WRONG - glyphs will be at wrong position
offX := float32(gtx.Dp(x))
offY := float32(gtx.Dp(y))
```
**Mistake**: Subtracting `first.Y`:
```go
// WRONG - pushes text off-screen
offY := float32(gtx.Dp(y)) - float32(first.Y)
```
**Fix**: Add `first.X` and `first.Y` to the offset:
```go
offX := float32(gtx.Dp(x)) + float32(first.X)/64.0
offY := float32(gtx.Dp(y)) + float32(first.Y)
```
### 7.5 Not Converting Int26_6 to int
**Mistake**: Using `int(g.Advance)` directly without shifting
**Fix**: Use `int(g.Advance >> 6)` to convert from fixed.Int26_6 to int
## 8. Usage in Pad
### 8.1 Filename Truncation (StatusBar)
```go
// Measure widths
shp.LayoutString(text.Parameters{
PxPerEm: fixed.I(gtx.Sp(size)),
MinWidth: 0,
MaxWidth: availableWidth,
MaxLines: 1,
}, filename)
// Truncate if needed
widths := measureWidths(shp)
if widths[len(widths)-1] > availableWidth {
// Find truncation point
truncated = filename[:i] + "..."
}
// Re-layout and render
shp.LayoutString(text.Parameters{...}, truncated)
drawLineText(gtx, shp, x, y, col)
```
### 8.2 TextField (Editor Content)
For the TextField element, we'll use the same approach:
1. Layout text with `MaxWidth` for word wrap
2. Iterate through glyphs to get per-character positions
3. Use glyph positions for:
- Hit testing (UI coordinates → byte offset)
- Cursor positioning
- Selection rendering
- IME bridge sync
### 8.3 StatusBar / BottomBar
The StatusBar and BottomBar are `Container` compositions of `Label` and `Icon` elements. Their text is rendered via the Renderer's `drawText` → `drawLineText` → `drawLine` pipeline (same as all other text). Truncation may require a double-layout (measure then draw truncated), which is acceptable because the strings are short.
## 9. Debugging Tips
When text doesn't render correctly, add debug logging to trace:
```go
fmt.Printf("[DEBUG] glyph X=%d Y=%d advance=%d flags=%b\n",
g.X, g.Y, g.Advance, g.Flags)
fmt.Printf("[DEBUG] offX=%f offY=%f\n", offX, offY)
fmt.Printf("[DEBUG] clip=(%d, %d, %d, %d)\n", clipMin.X, clipMin.Y, clipMax.X, clipMax.Y)
```
Look for:
- **All `X=0`**: Missing `MaxWidth`/`MinWidth`/`MaxLines`
- **All `FlagLineBreak` set**: Same as above
- **Text off-screen**: Wrong offset calculation
- **Text clipped**: Clip region doesn't include text bounds
## 9. Multi-line Text Spacing
When laying out multiple lines of text at different positions, you need to understand how Gio calculates baseline spacing.
### 9.1 Gio's Line Height Calculation
In `gio/text/gotext.go` (`calculateYOffsets` and `LayoutRunes`):
```go
// First line baseline starts at the ascent height
currentY := lines[0].ascent.Ceil()
// Subsequent baselines are spaced by lineHeight
for i := range lines {
if i > 0 {
currentY += lines[i].lineHeight.Round()
}
lines[i].yOffset = currentY
}
// lineHeight = max(ascent + descent) * LineHeightScale
// Default LineHeightScale = 1.2
if params.LineHeight != 0 {
maxHeight = params.LineHeight
}
if params.LineHeightScale == 0 {
params.LineHeightScale = 1.2
}
maxHeight = floatToFixed(fixedToFloat(maxHeight) * params.LineHeightScale)
```
**Key points**:
- **First line baseline**: `ascent.Ceil()` (not 0)
- **Baseline-to-baseline spacing**: `lineHeight * LineHeightScale`
- **Default LineHeightScale**: 1.2 (adds 20% extra padding)
- **LineHeightScale 1.0**: tightest spacing, no extra padding
### 9.2 Positioning Multiple Lines
When drawing multiple lines at different positions (e.g., filename + icons in StatusBar), use the baseline spacing as your guide:
```go
// Line 1: Filename at origin
filenameLineY := reg.Y
// Line 2: Icons — baseline spacing ≈ font size with LineHeightScale=1.0
// For 16SP font: baseline-to-baseline ≈ 20DP
iconsLineY := filenameLineY + unit.Dp(20)
```
**Why 20DP?** For a 16SP font on a typical display:
- `PxPerEm = 16 * 1.25 = 20` device pixels (2x display)
- `ascent + descent ≈ PxPerEm = 20`
- With `LineHeightScale = 1.0`: baseline spacing = 20DP
### 9.3 Controlling Spacing
You have two options to control line spacing:
**Option A: Set `LineHeightScale` in LayoutString**
```go
shp.LayoutString(text.Parameters{
PxPerEm: fixed.I(gtx.Sp(size)),
MinWidth: 0,
MaxWidth: availableWidth,
MaxLines: 1,
LineHeightScale: 1.0, // Tight spacing, no extra padding
}, str)
```
**Option B: Set `LineHeight` to a specific value**
```go
shp.LayoutString(text.Parameters{
PxPerEm: fixed.I(gtx.Sp(size)),
MinWidth: 0,
MaxWidth: availableWidth,
MaxLines: 1,
LineHeight: fixed.I(gtx.Sp(18)), // Fixed 18SP baseline spacing
}, str)
```
### 9.4 Common Mistakes
**Mistake**: Adding arbitrary spacing between lines without considering baseline positioning.
**Fix**: Remember that the shaper's Y value is the **baseline**, not the top of the text. The visual top of the text is at `baseline - ascent`. So the visual gap between two lines is:
```
visualGap = (line2Baseline - line1Baseline) - (ascent1 + ascent2)
= lineHeight - (ascent1 + ascent2)
```
With `LineHeightScale = 1.0`: `visualGap ≈ 0` (lines touch)
With `LineHeightScale = 1.2`: `visualGap ≈ 0.2 * lineHeight` (20% padding)
## 9.5 Coordinate System and Type Safety
The project uses distinct Go types (`ui.Dp` and `ui.Px`) to prevent accidental mixing of coordinate units at compile time.
### Architecture
```
main.go State/Logic layer Renderer
───────── ───────────────── ────────
app.ConfigEvent EditorLayout(ui.Dp) r.toPx(dp)
PixelWidth/Px → regions in Dp r.toDp(px)
→ stored in State
ScaleEvent
→ stored in State
```
### Key Rules
1. **Logic layer works exclusively in Dp**: `EditorLayout` accepts `ui.Dp` dimensions and returns regions in `ui.Dp`. No pixel conversions in the layout layer.
2. **main.go converts pixels to Dp**: Raw pixel dimensions from `app.ConfigEvent` are stored in `State.PixelWidth`/`State.PixelHeight`. The `State.layout()` method converts them to Dp using the current scale before calling `EditorLayout`.
3. **Renderer converts on demand**: The `Renderer` holds a `ScaleProvider` interface (backed by `*editor.State`) and converts Dp↔Px via `r.toPx(dp)` / `r.toDp(px)` at Gio interop boundaries.
4. **Explicit conversions only**: Use `ui.ToPx(dp, scale)` and `ui.ToDp(px, scale)` for conversions. The `Dp` and `Px` types are distinct named types — the compiler prevents mixing them directly.
### StatusBar Composition Example
The StatusBar is built as a `Container` with child `Label` and `Icon` elements:
```go
// In EditorLayout (state.go):
statusBar := ui.NewContainer(
ui.Region{X: margin, Y: margin, W: screenWidth - margin*2, H: ui.Dp(52)},
ui.Color{R: 230, G: 230, B: 230, A: 255},
[]ui.Element{
ui.NewLabel("filename.txt", 14,
ui.Region{X: 0, Y: ui.Dp(2), W: statusBarW, H: ui.Dp(20)},
ui.AlignStart),
// Icons: Size=0 → auto-scale to fill region W×H
ui.NewIcon("cut", ui.Region{X: 0, Y: ui.Dp(28), W: ui.Dp(24), H: ui.Dp(24)}, 0),
ui.NewIcon("copy", ui.Region{X: ui.Dp(48), Y: ui.Dp(28), W: ui.Dp(24), H: ui.Dp(24)}, 0),
ui.NewIcon("paste", ui.Region{X: ui.Dp(96), Y: ui.Dp(28), W: ui.Dp(24), H: ui.Dp(24)}, 0),
},
)
```
The Container's region is in screen-space Dp. Child regions are relative to the Container's origin (the Renderer applies `op.Offset` for children). The Renderer converts all Dp values to pixels via `r.toPx()` when submitting Gio ops. Icon elements use `Size=0` so the renderer auto-scales them to fill their `24×24` DP regions via an affine transform.
### Common Pitfalls
- **Don't mix Dp and Px**: The compiler will catch it. Use explicit `ui.ToPx` / `ui.ToDp` conversions.
- **Don't use `gtx.Constraints` in element Draw methods**: They're modified by clips. The Renderer's `toPx`/`toDp` methods use the scale from `State`, not constraints.
- **Container children use relative regions**: Child element regions are relative to the Container's origin, not screen space. The Renderer applies `op.Offset` automatically when drawing Container children.
## 9.6 Background Drawing with Clips
When drawing backgrounds (rectangles with solid colors), use `clip.Rect{...}.Push(gtx.Ops)` followed immediately by `clip.Pop()`. **Never use `op.Record`/`m.Stop()` macros for simple background clips** — the macro leaves the clip on the stack, preventing subsequent elements from drawing.
**Correct pattern**:
```go
bgClip := clip.Rect{
Min: image.Point{X: pxMinX, Y: pxMinY},
Max: image.Point{X: pxMaxX, Y: pxMaxY},
}.Push(gtx.Ops)
paint.ColorOp{Color: bgColor}.Add(gtx.Ops)
paint.PaintOp{}.Add(gtx.Ops)
bgClip.Pop() // Immediately pop — clip stack must be clean
```
**Wrong pattern** (leaves clip on stack):
```go
m := op.Record(gtx.Ops)
clip.Rect{...}.Op().Push(gtx.Ops) // Push inside macro
paint.ColorOp{...}.Add(gtx.Ops)
paint.PaintOp{}.Add(gtx.Ops)
call := m.Stop()
call.Add(gtx.Ops) // When replayed, clip is pushed but never popped!
```
**Why this matters**: When the StatusBar draws its background with a macro, the clip inside is pushed during macro replay but never popped. The BottomBar then draws while that stale clip is still active, causing it to be clipped out of visibility.
## 10. Summary
- **Always** use `fixed.I(gtx.Sp(size))` for `PxPerEm`
- **Always** set `MinWidth`, `MaxWidth`, `MaxLines` in `LayoutString` parameters
- **Always** offset by `(x + first.X, y + first.Y)` in `drawLine`
- **Always** wrap glyph drawing in `op.Record`/`m.Stop()` for clipping safety
- **Baseline spacing** = `lineHeight * LineHeightScale` (default 1.2)
- **For tight multi-line layouts**, use `LineHeightScale: 1.0` or set explicit `LineHeight`
- **Visual gap** between lines = `baselineSpacing - (ascent1 + ascent2)`
- **Logic layer works exclusively in Dp**. `State.layout()` converts pixels→Dp. Renderer converts Dp→Px via `r.toPx()`.
- **Explicit conversions** (`ui.ToPx`, `ui.ToDp`) — `Dp` and `Px` are distinct named types.
- **Follow** Gio's `paintGlyph` as the reference implementation for text rendering

600
doc/spec.md
View File

@ -1,465 +1,157 @@
# Pad — Text Editor Specification
This is the specification of what Pad **actually is** (v1, current code).
Behavior that was once spec'd but never built is listed as deferred in §7, not
described as a requirement. If code and this document disagree, the code wins
— and this document should be fixed.
## 1. Overview
Pad is a minimal, high-performance plain text editor for Android. It is configured with a single directory of text files and provides instant file access, automatic persistence, and seamless recovery from process death. The editor is designed for a directory synced across devices, with reactive awareness of external file modifications.
Pad is a minimal plain-text editor for Android, written in Go with the Gio UI
framework. It presents a single root directory of text files (a browser) and a
full-screen editor with the Android soft keyboard. The design goal is a
fast, honest editor for a directory that is synced across devices — instant
open, autosave on every edit, and large files that stay smooth.
**Design philosophy:**
- **No practical limits** — files and directories of arbitrary size must work without degradation
- **Minimal surface area** — plain text only, no formatting, no file management UI
- **Automatic everything** — save, state persistence, sync awareness happen without user intervention
- **Survive anything** — process kill, battery death, device restart: the editor restores exactly where the user left off
- **Plain text only** — no formatting, no syntax highlighting.
- **Automatic everything** — autosave with a 1 s debounce; there is no save
button.
- **Minimal surface area** — a browser and an editor, nothing else.
- **Bounded memory** — a measured file-size limit with an explicit
"too large" state, instead of silently degrading or OOMing.
## 2. Core Requirements
**Platform:** Android-first. Window `390×844` dp. Default root directory is
`/storage/emulated/0/Notes` on Android (overridable with `-root`); `.`
elsewhere.
## 2. Implemented behavior
### 2.1 File browser
- Lists files and directories under the current directory.
- **Sort:** four modes — name asc/desc, date asc/desc — cycled by the sort
button. Default: date descending (newest first).
- **Search:** incremental, case-insensitive filter over entry names via the
search bar (a Gio `widget.Editor`).
- **Pagination:** directory entries are loaded asynchronously in pages through
the worker pool; the first paint does not wait for a full directory read.
- **Navigation:** tap a directory to enter it, back to return. Single tap
opens a file — no long-press menus, no file management actions.
### 2.2 Editor
- **Input via the Android IME:** soft-keyboard typing, composition,
backspace, and autocorrect replacements all arrive through one IME
replace-range path and are converted from rune indices to byte offsets.
(Swipe-typing support follows from the same IME path; final sign-off on a
physical device is the one open validation item.)
- **Word wrap:** on by default; wrapped lines are virtual — the buffer stores
only real newlines.
- **Virtualized viewport:** only the visible byte range is shaped and drawn
each frame (typically ~4 KB of a large file), keeping frame cost and shaper
memory constant regardless of file size.
- **Cursor + scroll:** tap to place the cursor, drag/scroll to pan, Home/End
and Page Up/Down on hardware keyboards, arrow keys.
- **Autosave:** every edit restarts a 1 s debounce; on expiry the full content
is written to disk by a worker. Failed writes are retried. This is the only
persistence mechanism.
### 2.3 Large files (measured)
- Editable limit: **50 MB** (`MaxEditableFileSize`), measured on-device:
a 10 MB file (130,954 lines) opens in ~120 ms (stat ~76 ms, read ~27 ms,
line-index ~18 ms) and idles at **~150 MB PSS / ~230 MB RSS, flat under
scroll**. 50 MB extrapolates to a few hundred MB — acceptable on a modern
phone.
- Files above the limit open into a **"too large to edit"** state: a notice is
shown, edit operations are no-ops, and the browser still lists the file.
- The buffer is chunked (64 KB chunks, prefix-sum offsets, full load for
in-range files); edits splice only affected chunks. Details:
`architecture.md` §6.
## 3. Code organization (actual)
```
cmd/pad/ # Gio entry point, main loop, frameReceiver,
# Android defaults (impl_android.go)
internal/
browser/ # BrowserState, BrowserManager, sort, search,
# pagination, layout, handlers
editor/ # Logic goroutine, State, ChunkedBuffer, LineIndex,
# IME handling, autosave, Frame handoff (frame.go)
io/pool/ # Worker pool (priority lanes), task types,
# real/ — real filesystem (rooted at /)
# mock/ — in-memory FS for tests
# types/ — shared types (LineIndex, ...)
ui/ # Element model, Renderer, units (Dp/Px), theme
ui/icons/ # Vector icons
test/e2e/ # Logic-level e2e harness + tests
doc/ # spec.md (this file), architecture.md,
# development_plan.md
```
## 4. Architecture (summary)
The logic goroutine is the **sole owner** of mutable state; the main (Gio)
goroutine reads only a `Frame` snapshot and writes only via channels; a
frame-receiver goroutine stores frames and invalidates the window; a
priority-aware worker pool does all file I/O. No mutex guards application
state. The IME snippet, search box, and gesture state live on the
main-goroutine side because Gio mutates them during draw.
Full details, channel topology, and ownership rules: [`architecture.md`](./architecture.md).
## 5. Invariants to preserve (future development must not break these)
1. **Single owner, no locks on state** (architecture.md §1). Any new feature
adds a channel or an owner-executed callback — never a direct state touch
from another goroutine.
2. **Main never reads logic `State`** — only the `Frame` snapshot.
3. **Never shape the whole file.** The text shaper's internal line storage
retains the largest layout ever performed; one whole-file layout permanently
inflates memory (this caused a 1 GB leak, fixed in Phase 3).
4. **The IME snippet is the visible window**, and rune↔byte conversion happens
in one place (`HandleReplaceRange` / `RuneIndexToByte`).
5. **Autosave is the only persistence.** If state persistence (last file,
cursor, scroll) is added later, it must go through the same
owner-dispatches-a-task pattern.
6. **Logic work stays < 16 ms.** Anything that can block or scan more than the
viewport goes to the worker pool.
## 6. Performance expectations (validated on-device)
| Operation | Target | Measured (10 MB file, Android 35 emulator) |
|---|---|---|
| Open file | instant feel | ~120 ms (stat + read + line index) |
| Scroll | 60 fps | flat ~240 MB RSS, no growth, no OOM |
| Type | responsive | single IME path; rapid commits land cleanly |
| Memory | bounded | ~150 MB PSS / ~230 MB RSS at 10 MB file, flat |
## 7. Deferred / not implemented (explicit non-goals for v1)
These were in the original v1 spec but are **not in the code**. They are
recorded here so future rounds don't mistake doc text for behavior:
| Feature | Status | Notes |
|---|---|---|
| Undo (any) | **not implemented** | No undo stack exists; the old `SaveUndoTask` is dead code. |
| State restoration on relaunch | **not implemented** | Last file / cursor / scroll are not persisted. |
| External change detection | **not implemented** | No mtime compare on open/resume, no watcher, no Keep/Reload prompt. |
| Syncthing conflict handling | **not implemented** | No `.sync-conflict-*` file detection or merging. |
| File-system watcher | **not implemented** | Browser does not live-refresh; it re-scans on navigation. |
| Alphabetical index sidebar | **not implemented** | `AlphaIndex` element exists but is unused. |
| In-file search, tabs, split view | **not implemented** | — |
| Files > 50 MB | **not supported** | `TooLarge` state instead. |
| Desktop / other platforms | **not supported** | Android-first. |
## 8. Product requirements (standing)
| Requirement | Detail |
|---|---|
| Instant open | Files open immediately regardless of size; content loads on demand |
| Auto-save | Every edit is persisted to disk with a short debounce; no save button exists |
| Unlimited scale | No caps on file size, line count, or directory entry count |
| External change detection | The editor monitors the configured directory for modifications from sync tools |
| Cross-session undo | Full undo history survives process death, file close, and device restart |
| State restoration | On relaunch, the editor restores the exact file, cursor position, and scroll state |
| Per-file cursor memory | Reopening any previously visited file restores the last cursor position |
| Plain text only | No formatting, no syntax highlighting, no markdown rendering |
## 3. Architecture
### 3.1 Code Organization
```
cmd/
pad/ # Mobile entry point (Gioui window + frame loop)
internal/
browser/ # Directory listing, pagination, search, alphabetical index
editor/ # Virtual scroll renderer, chunked edit buffer, input handling
filesystem/ # Chunked file I/O, debounced writes, atomic operations
watcher/ # File system monitoring, change detection, notification
conflict/ # External change reconciliation, merge strategies
state/ # App state persistence, per-file cursor map, restore logic
undo/ # Undo operation log, context anchoring, rebasing
ui/ # Element types, renderer, Gioui drawing
```
### 3.2 Runtime Architecture
The runtime uses four concurrent components coordinated by channels and a single mutex:
- **Main goroutine** — Gioui event loop; renders frames under mutex, collects user input
- **Frame receiver goroutine** — tight loop that stores frames from logic and triggers redraws
- **Logic goroutine** — sole owner of application state; processes input, computes layout, dispatches async work
- **Worker pool** — fixed goroutines for file I/O, diff computation, undo replay
Details: [`architecture.md`](./architecture.md) — concurrency model, channel topology, deadlock analysis, sync/async partitioning, persistence strategy.
## 4. File Browser
The browser presents the configured directory's contents. It must handle directories with hundreds of thousands of entries.
### Behavior
- **Lazy loading** — directory entries are loaded in pages, not all at once
- **Alphabetical index** — a tap-on-letter sidebar jumps to the corresponding section
- **Search** — incremental text filter over entry names
- **Single tap to open** — no long-press menus, no file management actions
### Performance
- First paint: directory root visible before all entries are read
- Scroll: smooth at 60 fps with virtualized list (only visible rows rendered)
- Search: results filter in real time, no debounce on keystrokes
## 5. Text Editor
The editor renders plain text with virtual scrolling and chunked loading. The entire file is never loaded into memory.
### Virtual Scrolling
Only lines visible on screen (plus a small prefetch buffer) are rendered. Scrolling triggers on-demand loading of adjacent chunks.
### Chunked Buffer
- Files are divided into fixed-size chunks (configurable, e.g., 64 KB)
- Only chunks near the cursor or visible region reside in memory
- Chunks are evicted under memory pressure, reloaded from disk on demand
- Edits at chunk boundaries are handled by splitting/merging chunks
### Line Index
The editor maintains a line-offset index (line number → byte offset in file) for every file it has opened. This index is:
- **Built** in a single streaming pass over the file (O(n) time, O(lines) space for the index array)
- **Cached** on disk inside `.pad/indices/` (one file per indexed file, named by the hash of the original path)
- **Invalidated** when the file's modification time or size changes (detected on open)
- **Updated incrementally** during editing sessions (insert/delete shifts subsequent offsets in the in-memory copy)
- **Evicted** under disk pressure (LRU by last access time)
The index enables O(log n) jumps to any line number via binary search. Without it, finding line N in a large file requires scanning from the start — O(N) in bytes. For a 1 GB file with average 50-byte lines, that's ~20M lines and 20 MB of index data (4 bytes per line as uint32). This is acceptable on modern devices.
During an editing session, the in-memory index is updated with each edit. On file close, the updated index is written back. On next open, if the disk file matches the index's mtime/size stamp, the cached index is reused.
### Input
- Standard text input: insert characters, backspace, delete
- Selection: tap-and-drag or tap-hold to select a range
- Paste: clipboard content inserted at cursor or replacing selection
- No keyboard shortcuts beyond what the OS input method provides
### Word Wrap
Long lines are wrapped visually at word boundaries within the viewport width. This is a display-only feature — no newlines are inserted into the underlying text or written to disk. The logic layer computes wrapped display lines from the raw text, region width, and estimated character advance.
Word wrap is **on by default**. Users can toggle it off for horizontal scrolling instead.
### Search
The editor has an in-editor search bar (appears below the header when activated). It contains a text field and up/down arrow buttons. Behavior:
1. As the user types, the logic layer finds all occurrences of the query in the loaded buffer
2. The cursor jumps to the first match immediately
3. Up/down arrows cycle through matches (up = previous, down = next)
4. Each keystroke that changes the query re-runs the search and jumps to the first match
5. The search bar displays `Match N of M` so the user knows their position
For large files, search runs over the currently loaded chunks. If a match exists in an unloaded region, the editor loads that chunk and scrolls to it.
## 6. Undo System
The undo system records reversible edit operations with context anchoring, persisted to disk for cross-session survival.
### 6.1 Operation Model
Edits are recorded as a **stack of chained operations**. A chain is a contiguous sequence of operations sharing a common anchor (position and context). A new chain begins whenever the cursor moves or a new selection is made.
### 6.2 Entry Formats
**Insert chain** (typing at a fixed cursor position):
```
Head: {pos: 1024, before: "<N bytes before>", after: "<M bytes after>", inserted: "h"}
Tail: {inserted: "e"}
Tail: {inserted: "l"}
Tail: {inserted: "l"}
Tail: {inserted: "o"}
```
Entries without `pos`, `before`, or `after` inherit from the head. Position advances by the length of the previous entry's `inserted` text.
**Delete chain** (backspacing without hitting original text):
```
Head: {pos: 1023, after: "<M bytes after>", deleted: "o"}
Tail: {deleted: "l"}
Tail: {deleted: "l"}
```
Entries without `pos` or `after` inherit from the head. Position for entry `i` is `head.pos + i`. The `before` field is omitted because it shifts with each deletion; `after` is the stable anchor.
**Selection replacement** (select text, type to replace):
```
Head: {pos: 5000, before: "<N>", after: "<M>", deleted: "<selected text>", inserted: "x"}
Tail: {inserted: "y"}
Tail: {inserted: "z"}
```
The head records both the deleted selection and the first inserted character. Continuation typing appends to the chain.
**Field semantics:**
| Field | Present on | Meaning |
|---|---|---|
| `pos` | Head only | Byte offset of the first operation in the chain |
| `before` | Head of insert/replace chains | N bytes immediately before `pos`, used for re-anchoring |
| `after` | Head of all chains | M bytes immediately after the affected range, used for re-anchoring |
| `deleted` | Head of delete chains, selection replace | Text that was removed |
| `inserted` | Every entry | Text that was added (empty string for pure deletion entries) |
Fields absent from an entry are inherited from the chain head.
### 6.3 Backspace Semantics
Backspace interacts with the undo stack:
1. **If the last stack entry is an `inserted` entry** and the character immediately before the cursor matches that entry's `inserted` text: **pop** that entry from the stack. No new operation is recorded. The character is removed from the file.
2. **Otherwise** (stack empty, or last entry is a delete): record a **new chain head** with `deleted` set to the backspaced character, `pos` set to the byte offset of that character, and `after` set to the context after the cursor.
3. **Subsequent backspaces** (after entering delete mode): each appends a `{deleted: "<char>"}` entry to the current chain, inheriting `pos` and `after` from the head.
This means backspacing over text the user just typed is "free" — it removes operations from the stack rather than adding new ones. Undo after backspace naturally restores the popped character.
### 6.4 Undo Replay
To undo, pop the top chain from the stack and replay in reverse:
1. **Re-anchor**: search the file for the head's context (`before` + `after` for insert chains, `after` for delete chains). If found, update `pos` to the new location. If not found, skip the entire chain.
2. **Apply in reverse**: for each entry from last to first:
- **Insert entry**: delete `inserted` text at the current position
- **Delete entry**: insert `deleted` text at the current position
- Advance/retreat position by the length of the text affected
3. For delete chains, replay right-to-left (last entry first) so that each insert shifts subsequent text right without affecting earlier positions.
### 6.5 Stack Limits
| Parameter | Value | Rationale |
|---|---|---|
| Max operations | 1000 | Sufficient for ~1000 typing sessions or individual keystrokes |
| Context window | 50 bytes each side | Unique enough for prose/code, small enough for persistence |
| `deleted` field size | **1 MB inline** | Below: stored directly in the operation. Above: stored in a separate backup file (see below) |
| Total undo disk budget | 10 MB | Per `.pad/undo/` directory; oldest files evicted on excess |
| Redo | Not supported | Can be added later; keeps initial implementation minimal |
**Large deletion handling**: When a single deletion exceeds 1 MB, the deleted text is written to `.pad/undo/<hash>_backup_<seq>.bin` and the operation records a `deleted_file` path instead of inline text. The backup file is cleaned up when the operation is popped (by undo or eviction). This preserves the "no practical limits" philosophy — even a 4 GB file deletion is undoable — while keeping the serialized undo stack small enough for fast persistence.
```json
{
"type": "delete_chain",
"pos": 0,
"after": "...",
"deleted_file": ".pad/undo/abc123_backup_001.bin",
"deleted_size": 4294967296,
"entries": []
}
```
The `deleted_size` field is used for display (e.g., toast: "Undo: 4.0 GB restored") and for eviction decisions.
### 6.6 Persistence
The undo stack for the active file is serialized to `.pad/undo/<hash>.json` (see §8) and persisted on every edit, debounced separately from auto-save. On restore, each operation's context is verified; operations that cannot be anchored are silently dropped from the stack.
The stack survives:
- Process kill by the OS
- Device restart
- File close and reopen
- App restart
The stack does **not** survive:
- Explicit user action to clear it (if such a feature is added)
- External file changes that make anchoring impossible (text context no longer exists)
- Eviction due to disk budget exceeded (oldest undo files pruned first)
## 7. Auto-save
Every edit to a file is written to disk automatically.
### Behavior
- **Debounce**: 500–1000 ms after the last keystroke (configurable)
- **Atomic write**: write to a temporary file inside `.pad/tmp/`, then rename over the target. The `.pad/` directory is excluded from Syncthing sync (via `.stignore`), so temp files are invisible to the sync daemon and do not generate spurious sync events
- **No UI**: there is no save button, no unsaved indicator, no "do you want to save?" dialog
### Interaction with Undo
Auto-save writes the current file state. The undo stack is persisted separately. Undo works transparently across saves — saving does not "commit" or "clear" undo history.
### Syncthing Compatibility
The `.pad/` directory contains editor-internal data (state, indices, temp files) and is excluded from sync. Syncthing ignores it via a `.stignore` file placed in the configured directory:
```
.pad/
```
This ensures:
- Temp files used for atomic writes are never synced
- State and index files remain local to the device
- Only the user's text files are synchronized across devices
## 8. State Management
The editor persists its state across multiple files inside `.pad/`. This multi-file approach avoids the overhead of a database dependency and keeps each file small enough for fast atomic writes.
### Directory Layout
```
.pad/
state.json # App state (active file, cursor, scroll, browser)
cursors.json # Per-file cursor map
undo/
<file_hash>.json # Undo stack for a specific file (one per actively edited file)
indices/
<file_hash>.bin # Line-offset index (see §5)
tmp/
# Temp files for atomic writes
```
All `.pad/` contents are excluded from Syncthing sync (see §7).
### App State (`state.json`)
```json
{
"active_file": "path/to/open/file.txt",
"cursor_pos": 1024,
"scroll_offset": 500,
"browser_scroll": 1200,
"browser_path": "/user/documents",
"browser_query": ""
}
```
| Field | Description |
|---|---|
| `active_file` | Path of the currently open file (relative to configured directory) |
| `cursor_pos` | Byte offset of the cursor in the active file |
| `scroll_offset` | Vertical scroll position (line or pixel offset) |
| `browser_scroll` | Scroll position in the directory browser |
| `browser_path` | Currently browsed directory path (relative to configured directory) |
| `browser_query` | Current search query in the browser |
### Cursor Map (`cursors.json`)
```json
{
"path/to/file1.txt": 2048,
"path/to/file2.txt": 512
}
```
Map of file path → last cursor position. Evicts oldest entries to stay under 1000 entries.
### Undo Stacks (`undo/<hash>.json`)
Each actively edited file gets its own undo stack file, named by the SHA-256 hash of the file path. This keeps undo data isolated per-file and avoids serializing all undo stacks into one large blob.
```json
{
"file": "path/to/open/file.txt",
"file_mtime": 1715000000,
"file_size": 50000,
"operations": [
{
"type": "insert_chain",
"pos": 1024,
"before": "...",
"after": "...",
"entries": [
{"inserted": "h"},
{"inserted": "e"},
{"inserted": "l"},
{"inserted": "l"},
{"inserted": "o"}
]
}
]
}
```
The `file_mtime` and `file_size` fields are used to detect if the on-disk file has changed since the undo stack was last used (external update or sync conflict). If they differ, the stack is discarded.
Old undo stack files are evicted when disk usage exceeds a budget (e.g., 10 MB total), using LRU by last access time.
### Why Not SQLite?
SQLite (via `modernc.org/sqlite`) was considered but rejected for the initial implementation:
- **Simplicity**: JSON files are human-readable, trivially debuggable, and require no schema migrations
- **Workload**: The editor performs sequential appends to undo stacks and point reads for state — JSON files handle this well
- **Atomicity**: Temp+rename gives us crash-safe writes without a transaction log
- **Dependency**: One fewer dependency in the Go module
If profiling reveals that JSON serialization or file I/O is a bottleneck (unlikely for the expected workload), SQLite can be adopted later with minimal API changes.
### Write Strategy
- **Continuous**: state is written on every navigation, edit, or cursor movement
- **Debounced**: writes are batched with a short debounce (separate from auto-save debounce)
- **Atomic**: write to temp file in `.pad/tmp/`, then rename (same as auto-save)
- **Bounded**: `cursors.json` evicts oldest entries (max 1000). Undo stacks are capped at 1000 operations per file and evicted by total disk size.
### Restore Flow
1. App launches → read `state.json`
2. If `active_file` exists on disk → open it, load chunk around `cursor_pos`, place cursor
3. If `active_file` was deleted → show browser at `browser_scroll` position
4. If no state file exists → show browser at root
5. Load undo stack from `undo/<hash>.json`; verify file mtime/size match; drop stack if mismatched
## 9. File System Monitor
The editor watches the configured directory for external modifications (e.g., from a sync tool on another device).
### Detection
- Uses the platform's file system watch API (inotify on Android/Linux equivalent)
- Detects: file content changes, file creation, file deletion, file rename
- Watches the entire directory tree recursively
### Response
| Event | Action |
|---|---|
| File modified externally (editor not open) | Update browser listing (timestamp, size) |
| File modified externally (editor is open) | Trigger conflict resolution (§10) |
| File created | Appear in browser on next scroll/search |
| File deleted | Remove from browser; if open, close and show browser |
| File renamed | Update browser listing |
## 10. Conflict Resolution
When the file system monitor detects an external modification to the currently open file:
### Detection
Compare the file's last-modified timestamp and size against the editor's known state. If either differs, the file has changed externally.
### Strategies
| Strategy | Behavior |
|---|---|
| **Discard local** | Replace buffer with disk content, preserve cursor position if possible |
| **Keep local** | Ignore external change, continue editing (next sync may overwrite) |
| **Manual** | Show a brief prompt asking the user to choose |
The default strategy is configurable. The prompt (if shown) is the only modal UI in the application.
### Interaction with Undo
When an external change is applied to the buffer:
- The undo stack is **preserved** (not flushed)
- Operations are re-anchored at undo time using context search (§6.4)
- Operations that cannot be anchored are silently skipped
- New operations recorded after the external change use positions in the new coordinate space
## 11. Performance Guarantees
| Operation | Target | Mechanism |
|---|---|---|
| Open any file | < 100 ms | Chunked load: only the chunk around the cursor is read |
| Scroll | 60 fps | Virtual rendering: only visible lines + prefetch buffer |
| Type | < 16 ms per keystroke | In-memory chunk edit, async chunk flush |
| Search in directory | < 100 ms for 100k files | Lazy index built on first browse, cached on disk |
| Search in file | < 50 ms for 1M lines | In-memory scan of loaded chunks; line index for navigation |
| Jump to line | < 10 ms | Binary search on cached line-offset index |
| Undo | < 100 ms | Context search is O(n) in file size but bounded by context window |
| State restore | < 500 ms | Read state file, open file, load chunk, place cursor |
| Auto-save | < 100 ms | Debounced atomic write |
### Memory
| Component | Budget |
|---|---|
| Chunk cache | Configurable, e.g., 16–64 MB |
| Undo stack (in memory) | ~200 KB for 1000 ops with 50B context (excluding large `deleted` fields) |
| Undo stack (on disk) | 10 MB total budget across all files |
| State files | < 1 KB each (`state.json`, `cursors.json`) |
| Line index | ~4 bytes per line (stored on disk, loaded fully for active file) |
## 12. Out of Scope
- Syntax highlighting, themes, or formatting
- Multiple tabs or split views
- Find and replace (search is in scope; replace may be added later)
- File management (create, delete, rename, move) — handled by external tools
- Cloud sync — handled by external tools (Syncthing, Dropbox, etc.)
- Collaborative editing
- Redo (may be added later)
- Keyboard shortcut configuration
- Plugins or extensions
| Instant open | Files open without visible lag; content and line index load async. |
| Auto-save | Every edit persisted with a 1 s debounce; no save button. |
| External change awareness | Deferred (§7) — the sync-awareness story is explicitly out of v1. |
| Plain text only | No formatting, no highlighting, no file management UI. |
| Bounded memory | 50 MB editable limit with an explicit "too large" state. |

556
doc/touch.md
View File

@ -1,556 +0,0 @@
# Touch and Input Handling
This document specifies how Pad handles low-level pointer and keyboard events using Gioui primitives, and how these are translated into editor actions (cursor movement, selection, scrolling).
## 1. Gioui Primitives
Pad avoids high-level widgets and uses the following low-level Gioui ops and events. Ops are submitted in the paint function; Events are returned by `w.Event()`.
| Op (submitted in paint) | Event / Gesture | Use Case |
|---|---|---|
| `gesture.Click.Add` | `gesture.ClickEvent` | Tap detection for UI elements (buttons, status bar labels, icons). |
| `pointer.InputOp` | `pointer.Event` | Editor hit-region. Captures `Press`, `Release`, `Move`, `Drag`, `Scroll` for cursor movement, selection, scrolling. |
| `key.InputOp` | `key.Event` | Enables keyboard focus. Captures `Edit` (text insertion), `Press` (Backspace, Enter, Arrows). |
| `gtx.Execute(key.FocusCmd` | — | Requests keyboard focus for the active element. |
| `gtx.Execute(key.SoftKeyboardCmd)` | — | Explicitly shows/hides the Android soft keyboard. |
### 1.1 Op/Event Flow
Because Logic does not have access to `*op.Ops`, the Op/Event flow is split:
1. **Logic → Elements**: Logic computes `[]Element`. Interactive elements declare `Interaction{Gesture, Handler}` entries.
2. **Renderer → Ops**: During `drawElement`, the renderer registers interactions (creates `gesture.Click` instances) and submits ops into `*op.Ops`. Click registration happens within the element's clip context.
3. **Gioui → Events**: Gioui returns events via `w.Event()`. Gesture events are polled from `input.Source` via `click.Update(q)`.
4. **Main → Logic**: The Main goroutine batches gesture events into `[]InputEvent` (each carrying its own handler) and sends them to Logic via `inputChan`.
5. **Logic → State**: Logic calls `evt.Handler(evt.Data)`. Handlers are static functions that access global `TheState` directly.
This ensures that hit-regions are always defined in the same frame where the elements are drawn, and Logic remains testable without the SDK.
## 2. Coordinate Mapping
The editor operates in three coordinate spaces:
1. **UI Space (DP)**: The raw (X, Y) coordinates from Gioui events, relative to the window origin.
2. **Text Space (Lines/Cols)**: The logical position within the text file, accounting for scroll offset and word wrap.
3. **Byte Space (Offsets)**: The raw byte offset in the UTF-8 file buffer.
### 2.1 Hit Regions
Each interactive element registers its hit region via `gesture.Click.Add(gtx.Ops)`. The click area is determined by the **current clip context** on the operation stack at the time `Add()` is called, not by explicit screen coordinates.
**Gio's Clip-Based Model**: Gio doesn't need explicit screen coordinates for click registration. Instead, it uses the clip stack to determine the click area. When you call `clip.Rect{...}.Push(gtx.Ops)`, you add a clipping region. When you call `.Pop()`, you remove it. Everything drawn or registered between Push and Pop is constrained to that region.
**The Correct Pattern**:
```go
// 1. Set the clip to the element's bounds
clipRect := clip.Rect{
Min: image.Point{X: pxMinX, Y: pxMinY},
Max: image.Point{X: pxMaxX, Y: pxMaxY},
}.Push(gtx.Ops)
// 2. Register the click area (uses current clip)
click.Add(gtx.Ops)
// 3. Draw the element (also clipped to same region)
drawElement(gtx)
// 4. Pop the clip
clipRect.Pop()
```
**Why This Works**: The clip stack ensures that:
1. Click registration happens within the same coordinate system as drawing
2. The click area is automatically constrained to the element's visible bounds
3. Nested elements inherit their parent's clip context
4. No manual coordinate conversion is needed
**Critical Detail**: The click area matches the **drawn content position**, not the entire clip region. If text is positioned on the right side of a parent region, the click area will be at the text's actual position, not the entire parent clip.
When Logic receives a click event, the `InputEvent` carries its own handler function. No tag-based dispatch is needed — the handler knows exactly what to do.
### 2.2 Mapping UI → Byte Offset (Editor/TextField)
When a `pointer.Event` occurs at `(Ex, Ey)` on the editor hit-region:
1. **Adjust for Scroll**: `Y = Ey + scroll_offset`.
2. **Identify Visual Line**: `visual_line = floor(Y / line_height)`.
3. **Resolve to Logical Line**: If word wrap is enabled, use the cached wrap positions to map `visual_line` → `logical_line`. Otherwise, they are the same.
4. **Identify Column**:
- Retrieve text for `logical_line` from the chunked buffer.
- Use the font shaper to measure glyph widths until the cumulative width exceeds `Ex`.
- Clamp to the line length.
5. **Result**: Convert `(logical_line, col_index)` to a byte offset using the **Line Index**.
### 2.3 Mapping UI → Element (Non-Text Elements)
For buttons, search bar, alpha index, merge hunks:
- Each element has a `Region()` (rectangle). The Main goroutine checks if `(Ex, Ey)` falls within the region.
- For the **Alpha Index**, the Y-coordinate maps to a letter (e.g., `letter = alphabet[floor((Y - alpha_top) / (alpha_height / 26))]`).
- For **Merge Hunks**, the tag identifies which hunk button (accept-ours / accept-theirs) was tapped.
## 3. Gestures and State Machine
The Logic goroutine maintains a small state machine for the active gesture. The state is transient (does not survive process death) and is reset on gesture completion.
### 3.1 Gesture States
| State | Entry Condition | Exit Condition | Action |
|---|---|---|---|
| `Idle` | Default state | `Press` received | — |
| `Pressed` | `Press` at (X, Y) | `Release` or `Drag` | Record press time and position |
| `Tapping` | `Release` received (short duration, no move) | Timeout or next `Press` | Move cursor, clear selection |
| `DoubleTap` | Second `Tapping` within 300ms | Gesture complete | Select word at (X, Y) |
| `LongPress` | `Press` held > 500ms | `Release` | Select word (future: magnifier) |
| `Selecting` | `Drag` received | `Release` | Update `selection_end` to current (X, Y) |
| `Scrolling` | `Drag` on non-focused area or two-finger drag | `Release` | Update `scroll_offset` |
### 3.2 Gesture Disambiguation
The Logic goroutine distinguishes gestures using timing and movement thresholds:
- **Tap vs Long Press**: If `Press` → `Release` occurs within 500ms, it's a tap. Otherwise, it's a long press.
- **Tap vs Drag**: If the finger moves more than 16 DP between `Press` and `Release`, it's a drag. Otherwise, it's a tap.
- **Single vs Double Tap**: If two taps occur within 300ms, the second tap triggers a word selection.
- **Select vs Scroll**: If the keyboard is visible (focused `TextField`), a single-finger drag is a selection. If the keyboard is hidden, a single-finger drag is a scroll. Two-finger drags are always scrolls.
### 3.3 Selection Logic
Selection is defined by two byte offsets: `selection_start` and `selection_end`.
- If `selection_start == selection_end`, there is no selection (just a cursor).
- During a **Drag** gesture:
- On `Press`: Set `selection_start = selection_end = offset_at(X, Y)`.
- On `Drag`: Update `selection_end = offset_at(X, Y)`.
- The UI renders a `TextField` with highlighted regions between the two offsets.
### 3.4 Double Click (Word Selection)
When a double-click is detected:
1. Identify the character at the click offset.
2. Expand left and right until a non-word character (space, punctuation, newline) is hit.
3. Set `selection_start` and `selection_end` to these boundaries.
## 4. Input Batching and Latency
As specified in `architecture.md`, the Main goroutine batches events. For touch, this is critical:
### 4.1 Event Compression in the Main Loop
The Main goroutine collects events during its event cycle. For a drag gesture, multiple `pointer.Event` (Type: `Move`) may fire before the next `FrameEvent`. The Main goroutine:
1. Accumulates all events into a `[]InputEvent` slice.
2. On the next `FrameEvent`, it sends the entire batch to Logic via `inputChan`.
3. Logic processes the batch sequentially, updating the gesture state and cursor/selection.
4. Logic produces a single frame reflecting the final state after all events.
This means that if 3 `Drag` events fire in one frame, Logic sees all 3 and produces one frame with the cursor/selection at the final position. There is no "churn" of intermediate frames.
### 4.2 Latency Budget
- **Event Capture**: Gioui captures the touch event immediately (sub-ms).
- **Batching**: Main goroutine holds events until the next `FrameEvent` (up to 16ms, but typically < 8ms).
- **Processing**: Logic processes the batch and computes layout (< 16ms).
- **Rendering**: Frame receiver stores and invalidates (sub-ms).
- **Total**: < 32ms from touch to visual feedback (typically < 16ms).
### 4.3 Gesture Continuity
Because Logic maintains the gesture state machine across frames, a multi-frame drag gesture is continuous:
- Frame 1: `Press` at A → Logic enters `Pressed` state.
- Frame 2: `Drag` to B → Logic enters `Selecting` state, updates `selection_end`.
- Frame 3: `Drag` to C → Logic updates `selection_end` again.
- Frame 4: `Release` → Logic finalizes selection, enters `Idle` state.
Each frame produces a new `[]Element` with the updated selection highlight.
## 5. Keyboard Interaction
The `TextField` element must be "focused" to receive keyboard events. Focus is a transient state managed by Logic.
### 5.1 Focus Acquisition
1. **Tap on Editor**: User taps the `TextField` region. Main goroutine sends `pointer.Event` (Type: `Press`) to Logic.
2. **Focus Request**: Logic sets `focused = true` and computes a frame with `gtx.Execute(key.FocusCmd` and `gtx.Execute(key.SoftKeyboardCmd{Show: true}`.
3. **Keyboard Appears**: Gioui shows the Android soft keyboard. Subsequent `w.Event()` calls return `key.Event`.
### 5.2 Focus Loss
1. **Tap Outside Editor**: User taps a non-editor region (e.g., status bar, directory browser). Logic sets `focused = false` and computes a frame with `gtx.Execute(key.SoftKeyboardOp{Show: false})`.
2. **Keyboard Disappears**: Gioui hides the soft keyboard.
### 5.3 Key Events
| Event | Type | Gio Constant | Action |
|---|---|---|---|
| Text insertion | `key.EditEvent` | — | Insert `event.Text` at cursor, append to undo chain |
| Backspace | `key.Event` (Press) | `key.NameDeleteBackward` | Delete character before cursor, append to delete chain |
| Delete | `key.Event` (Press) | `key.NameDeleteForward` | Delete character after cursor |
| Enter | `key.Event` (Press) | `key.NameReturn` | Insert newline at cursor |
| Tab | `Press` | `key.NameTab` | Insert spaces (configurable, e.g., 4 spaces) |
| Arrow Up | `key.Event` (Press) | `key.NameUpArrow` | Move cursor up one visual line (GlyphLayout-based) |
| Arrow Down | `key.Event` (Press) | `key.NameDownArrow` | Move cursor down one visual line (GlyphLayout-based) |
| Arrow Left | `key.Event` (Press) | `key.NameLeftArrow` | Move cursor left one byte |
| Arrow Right | `key.Event` (Press) | `key.NameRightArrow` | Move cursor right one byte |
| Ctrl+A | `Press` | `A` (with Ctrl) | Select all text (future) |
| Ctrl+Z | `Press` | `Z` (with Ctrl) | Undo (future) |
| Ctrl+Y | `Press` | `Y` (with Ctrl) | Redo (future) |
| Ctrl+F | `Press` | `F` (with Ctrl) | Show search bar (future) |
**Current implementation** (as of June 2026): `HandleKeyDown` in `state.go` dispatches `key.Name` events via a `switch` on `key.Name` constants. `key.EditEvent` is handled separately for text input. Up/Down arrow navigation uses `HandleVerticalCursorMove` which is GlyphLayout-based. Left/Right uses `HandleCursorMove` which is byte-based ±1.
### 5.4 IME and Composition (Pure Gioui)
Android soft keyboards use IME composition for languages like Chinese, Japanese, or Korean. Gioui's `key.Event` (Type: `Edit`) handles the final committed text. Logic treats the committed text as a single insertion at the cursor.
**Limitation**: Gioui's `key.InputOp` on Android does not provide a full `InputConnection`. The IME has no context — it cannot read surrounding text for autocorrect, cannot anchor swipe gestures to our cursor, and cannot provide predictive suggestions. Only raw character commits are received.
## 5.5 Android IME Bridge via Fragment
To access the full suite of Android IME features (autocorrect, swipe typing, voice input, predictions), Pad uses a **hidden `EditText`** hosted in a native `Fragment`. This is the same pattern used by Passgo (see `cmd/passgo-gui/impl_android.go`, `PgpConnect.java`).
### 5.5.1 Fragment Registration
The Fragment is registered on app startup using `app.ViewEvent`:
1. **Go side**: `handleEvent` receives `app.ViewEvent` on app start. `e.View` is a `uintptr` pointing to the `GioView` (`android.view.View`).
2. **C side (JNI)**: `registerFragment()` receives the `jobject view`:
- Gets the `Context` from the View via `getContext()`
- Gets the `ClassLoader` from the Context via `getClassLoader()`
- Loads the Fragment class via `findClass("pad/ime/ImeFragment")`
- Creates an instance by calling the constructor with the View
3. **Java side (`ImeFragment.java`)**:
- `ImeFragment` extends `Fragment`
- Constructor receives the `View` (GioView), extracts `Context`
- In `onAttach()`, casts `Context` → `Activity`
- Uses `act.getFragmentManager().beginTransaction().add(inst, "ImeFragment").commitNow()`
- Inflates a layout containing a transparent, zero-size `EditText`
### 5.5.2 IME Fragment Architecture
```
┌─────────────────────────────────────────────────────────────┐
│ Android Activity │
│ ┌───────────────────────────────────────────────────────┐ │
│ │ GioView (OpenGL rendering) │ │
│ │ - pointer.InputOp → pointer.Event │ │
│ │ - key.InputOp → key.Event │ │
│ └───────────────────────────────────────────────────────┘ │
│ ┌───────────────────────────────────────────────────────┐ │
│ │ ImeFragment (transparent overlay) │ │
│ │ ┌─────────────────────────────────────────────────┐ │ │
│ │ │ EditText (hidden, zero-size) │ │ │
│ │ │ - Holds text window around cursor │ │ │
│ │ │ - Selection synced to cursor position │ │ │
│ │ │ - IME receives all keyboard events │ │ │
│ │ └─────────────────────────────────────────────────┘ │ │
│ └───────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
```
### 5.5.3 Two-Way Sync: Gio ↔ EditText
The IME bridge maintains a tight sync loop between the Gio editor state and the hidden `EditText`:
**Gio → EditText (Cursor Sync)**:
1. User taps to move cursor in Gio editor.
2. Logic updates `cursor_pos` in state.
3. Main goroutine calls a JNI function: `SetEditTextSelection(byteOffset)`.
4. C side calls `editText.setSelection(byteOffset)` on the `EditText`.
5. IME now knows the cursor position and can provide context-aware suggestions.
**EditText → Gio (IME Events)**:
1. User types/swipes/uses voice input on the soft keyboard.
2. IME commits text to the `EditText` via `EditText.onTextChanged()`.
3. A `TextWatcher` on the `EditText` captures the change.
4. The change is sent via JNI callback to Go: `ImeTextChanged(replacementText, start, before, count)`.
5. C side allocates memory for the string and calls a Go export function.
6. Go side creates a synthetic `key.Event` (Type: `Edit`) and sends it to the Main goroutine.
7. Main goroutine batches it into `[]InputEvent` alongside any pointer/keyboard events.
8. Logic processes the synthetic event as a text insertion at the cursor.
### 5.5.4 Text Window Management
The `EditText` does not hold the entire file — only a **text window** around the cursor:
- **Window size**: ±1KB of text around the cursor (configurable).
- **On cursor move**: The Main goroutine sends a JNI call to update the `EditText`'s text and selection.
- **On text insertion/deletion**: The Logic goroutine updates the in-memory buffer, then sends a JNI call to update the `EditText`'s text and selection.
This ensures the IME always has context for suggestions, even for multi-gigabyte files.
### 5.5.5 IME Feature Support
| Feature | Supported | How |
|---|---|---|
| Autocorrect | Yes | IME reads text window from `EditText`, suggests corrections |
| Swipe typing | Yes | IME anchors swipe to `EditText` selection (cursor position) |
| Voice input | Yes | IME commits voice text to `EditText` |
| Predictive bar | Yes | IME reads text window for context |
| Emoji keyboard | Yes | IME commits emoji to `EditText` |
| CJK composition | Yes | IME composition buffer → `EditText` → Gio |
### 5.5.6 Event Merging in the Main Loop
IME events from the Fragment are merged with Gioui events in the Main goroutine:
```
Main goroutine event cycle:
1. w.Event() → returns next event (FrameEvent, pointer.Event, key.Event)
2. Check for pending IME events (non-blocking channel read)
3. If IME event exists → create InputEvent, add to batch
4. If Gioui event exists → create InputEvent, add to batch
5. Send entire batch to Logic via inputChan
```
IME events are **not** returned by `w.Event()` — they arrive asynchronously via a JNI callback channel. The Main goroutine drains this channel in every cycle, ensuring IME events are batched alongside Gioui events and delivered to Logic atomically.
### 5.5.7 Implementation Notes
- The `ImeFragment` lives in `cmd/pad/impl_android.go` (Go), `jni_android.c` (C), and `ime/ImeFragment.java` (Java).
- The JNI callback uses a `BlockingQueue` or unbuffered channel to pass events from the Java main looper to the Main goroutine.
- The `EditText` is transparent (`android:background="@android:color/transparent"`) and zero-size (`android:layout_width="0dp" android:layout_height="0dp"`), so it does not affect the UI.
- The `EditText` is **not** focusable by touch — it only receives focus programmatically when the Gio editor needs the keyboard.
## 5.6 Cut, Copy, and Paste
Pad implements its own clipboard (not Android's `ClipboardManager`) with **persistent storage** — the clipboard survives process death and app restart.
### 5.6.1 Clipboard Storage
The clipboard is stored as a string field in the Logic goroutine's state and persisted to disk:
- **File**: `.pad/clipboard.json`
- **Format**: `{"text": "..."}` (UTF-8 string)
- **Write trigger**: Every cut or copy operation triggers an async write to disk (debounced, 100ms).
- **Load trigger**: On app start, Logic reads `.pad/clipboard.json` and restores the clipboard.
- **Atomic write**: Written via temp file + rename in `.pad/tmp/` (same pattern as auto-save).
This ensures the clipboard is available even if the system kills the app while it is in the background.
### 5.6.2 StatusBar Layout with Cut/Copy/Paste Icons
The StatusBar (top bar) has **fixed icon slots** for cut, copy, and paste. Icons appear/disappear without reflowing other elements.
**Layout** (vertical stack, top to bottom):
```
┌──────────────────────────────────────────────────────────────┐
│ filename.txt │ ← Line 1: filename (truncated with ellipsis)
│ [Cut] [Copy] [Paste] [Conf] [Search] │ ← Line 2: action icons (left/right)
└──────────────────────────────────────────────────────────────┘
StatusBar (top bar)
┌──────────────────────────────────────────────────────────────┐
│ TextField (editor content) │
└──────────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────────┐
│ Ln 10, Col 5 1024 / 50000 [Word Wrap: On] │ ← BottomBar
└──────────────────────────────────────────────────────────────┘
```
**Fixed slot positions** (in Dp, from the edges):
- **Cut icon**: leftmost action icon (24×24 DP, X: 0)
- **Copy icon**: second from left (24×24 DP, X: 48)
- **Paste icon**: third from left (24×24 DP, X: 96)
- **Conflict icon**: right side (24×24 DP, future)
- **Search icon**: rightmost (24×24 DP, future)
- **Filename**: top line, truncated with ellipsis if too long
**Visibility rules**:
| Icon | Visible When |
|---|---|
| **Cut** | `selection_start != selection_end` |
| **Copy** | `selection_start != selection_end` |
| **Paste** | `clipboard != ""` |
| **Conflict** | sync conflict detected for active file |
| **Search** | Always visible (toggle search bar) |
Icons are rendered as `Icon` elements with `Size=0`, so they auto-scale to fill their `24×24` DP regions via an affine transform in `r.drawPng()`. The StatusBar's `Region` height is computed dynamically based on whether the filename line is visible (2 lines when filename is shown, 1 line when in merge/search mode).
### 5.6.3 Cut Operation
1. User taps the Cut icon.
2. Logic extracts text: `clipboard = buffer[selection_start:selection_end]`.
3. Logic writes clipboard to `.pad/clipboard.json` (async).
4. Logic deletes selected text from buffer (append to delete chain).
5. Logic sets `selection_start = selection_end = cursor_pos` (clears selection).
6. Logic produces a new frame (Cut/Copy icons hidden, Paste icon may appear).
### 5.6.4 Copy Operation
1. User taps the Copy icon.
2. Logic extracts text: `clipboard = buffer[selection_start:selection_end]`.
3. Logic writes clipboard to `.pad/clipboard.json` (async).
4. Selection is **not** cleared (text remains in buffer).
5. Logic produces a new frame (Cut/Copy icons still visible, Paste icon appears).
### 5.6.4a Search Button
The Search button toggles the search bar visible/invisible.
1. User taps the Search icon.
2. Logic toggles `search_active` state field.
3. If `search_active` is true:
- Show `SearchBar` element below the StatusBar.
- Set focus to the search bar's text field.
- Show the soft keyboard (`gtx.Execute(key.SoftKeyboardCmd{Show: true}`).
4. If `search_active` is false:
- Hide `SearchBar` element.
- Clear search query.
- Remove focus from search bar.
- Hide soft keyboard (`gtx.Execute(key.SoftKeyboardCmd{Show: false}`).
5. Logic produces a new frame.
The Search button is always visible in the StatusBar (unlike Cut/Copy/Paste which are conditional). It serves as the primary way to activate search on Android (where Ctrl+F may not be available).
### 5.6.5 Paste Operation
1. User taps the Paste icon.
2. Logic inserts `clipboard` text at cursor position.
3. Logic appends an insert operation to the undo chain.
4. Logic advances cursor past the inserted text.
5. Logic produces a new frame (no change to Cut/Copy/Paste visibility).
### 5.6.6 Input Routing
Interactive elements declare their behavior at construction time via the `Interactive` interface. Each element registers `Interaction` entries pairing a gesture type (`Tap`, `DoubleTap`) with a handler function.
**Flow**:
1. **Registration**: `drawElement` registers interactions via `registerInteraction`. For `Tap` gestures, a `*gesture.Click` is created and stored in the renderer's `clicks` map, keyed by element ID. The handler is stored alongside it.
2. **Click area**: `click.Add(gtx.Ops)` is called within the element's clip context. The clip defines both the coordinate system and the clipping bounds for the click area. The click area matches the drawn content position within that clip, not the entire clip region.
3. **Polling**: After `renderer.Draw()`, `CheckGestures()` polls all registered `gesture.Click` instances via `click.Update(q)`.
4. **Routing**: Click events are returned as `InputEvent{Handler, Data}`. The main loop sends them to the logic goroutine via `inputChan`.
5. **Execution**: The logic goroutine calls `evt.Handler(evt.Data)`. Handlers are static functions (e.g., `ToggleWordWrap`, `DoCut`) that access global `TheState` directly.
This design eliminates element-ID dispatch tables — each click event carries its own handler, and the handler knows exactly what to do.
### 5.6.7 Clipboard State Machine
The clipboard is a simple string field. It has no state machine — it is either empty or contains text. On cut/copy, the text is replaced. On paste, the text is unchanged.
**Edge cases**:
- **Paste into empty clipboard**: No-op (Paste icon is hidden when clipboard is empty).
- **Cut with no selection**: No-op (Cut icon is hidden when there is no selection).
- **Copy with no selection**: No-op (Copy icon is hidden when there is no selection).
- **Multiple cuts/copies**: Each cut/copy replaces the previous clipboard content.
- **Clipboard persistence**: If the app is killed, the clipboard is restored from `.pad/clipboard.json` on restart.
## 5.7 Filename Display with Ellipsis Toggle
The filename is displayed on a separate line at the top of the StatusBar. When the filename is too long to fit the screen width, it is truncated with an ellipsis (`...`). Tapping the ellipsis toggles between the truncated single-line view and a full multi-line view.
### 5.7.1 Truncation Behavior
- **Truncated view**: Filename is truncated to fit the screen width, with `...` at the end. Example: `very_long_filename...`
- **Full view**: Filename is displayed in full, potentially spanning multiple lines. Example: `very_long_filename_that_does_not_fit_on_a_single_line.txt`
### 5.7.2 Toggle Interaction
1. **Tap on ellipsis**: Toggling between truncated and full views.
2. **Tap elsewhere on filename**: No action (filename text is not interactive).
3. **Ellipsis region**: The ellipsis (`...`) is a separate `Button` element with a `Tap` interaction. Tapping it triggers the toggle.
### 5.7.3 Input Routing
- The ellipsis button declares a `Tap` interaction with handler `ToggleFilename`.
- When the user taps the ellipsis, the handler is called via `evt.Handler(evt.Data)`.
- The handler toggles the `filename_expanded` state field.
- Logic produces a new frame with the updated filename display.
### 5.7.4 State Field
The Logic goroutine maintains a boolean field `filename_expanded`:
- `false` (default): truncated view with ellipsis.
- `true`: full multi-line view.
This field is **not** persisted to disk — it is reset on app restart (same as gesture state).
### 5.7.5 Layout
In the truncated view, the StatusBar has 2 lines:
```
Line 1: filename.txt...
Line 2: [Cut] [Copy] [Paste] [Conf] [Search]
```
In the full view, the StatusBar has 3+ lines (depending on filename length):
```
Line 1: very_long_filename_that_does_not_fit_on_a_single_line.txt
Line 2: [Cut] [Copy] [Paste] [Conf] [Search]
```
The `Region.H` of the StatusBar is computed dynamically based on the number of lines. The BottomBar is always at the bottom of the screen (24 DP fixed height).
## 5.8 Bottom Bar
The Bottom Bar appears at the bottom of the editor page. It displays cursor position, byte position, and word wrap status. It is always visible.
### 5.8.1 Layout
```
┌──────────────────────────────────────────────────────────────┐
│ Ln 10, Col 5 1024 / 50000 [Word Wrap: On] │
└──────────────────────────────────────────────────────────────┘
```
- **Left**: Cursor position (e.g., "Ln 10, Col 5")
- **Center**: Byte position (e.g., "1024 / 50000")
- **Right**: Word wrap button (tap to toggle On/Off)
The BottomBar's `Region.H` is fixed at 24 DP. It sits below the editor content.
### 5.8.2 Cursor Position
The cursor position is computed from the current byte offset:
1. Logic converts byte offset to (line, column) using the **Line Index**.
2. The `CursorPos` field is set to `"Ln {line}, Col {column}"`.
3. The field is updated on every cursor movement (tap, arrow keys, backspace, etc.).
### 5.8.3 Byte Position
The byte position shows the current byte offset and total file size:
- **Format**: `"current / total"` (e.g., "1024 / 50000")
- **Current**: byte offset of the cursor in the file
- **Total**: total file size in bytes
- The field is updated on every cursor movement.
### 5.8.4 Word Wrap Button
The word wrap button toggles word wrap on/off:
- `true` → "Word Wrap: On"
- `false` → "Word Wrap: Off"
**Interaction**:
1. User taps the word wrap button in the BottomBar.
2. Main goroutine receives a click event from `CheckGestures()`.
3. Main goroutine sends `InputEvent{Handler: ToggleWordWrap, Data: clickEvent}` to Logic.
4. The handler toggles the `WordWrap` field in `TheState`.
5. Logic produces a new frame (BottomBar shows updated status).
The word wrap status is persisted to `.pad/state.json` and restored on app start.
## 6. Scroll Momentum
Pad implements a momentum model in the Logic goroutine to provide smooth scrolling.
### 6.1 Scroll Sources
| Source | Event | Behavior |
|---|---|---|
| **Mouse Wheel** | `pointer.Event` (Source: `MouseWheel`) | Direct scroll by `event.Scroll.Y` DP |
| **Finger Drag (No Focus)** | `pointer.Event` (Type: `Move`, no keyboard) | Direct 1:1 mapping of finger movement to `scroll_offset` |
| **Fling** | `pointer.Event` (Type: `Release`, high velocity) | Momentum-based scroll with exponential decay |
### 6.2 Fling Logic
When a `Release` event occurs with a high vertical velocity:
1. **Velocity Calculation**: Logic tracks the position and timestamp of the last few `Move` events. On `Release`, it calculates the velocity (DP/ms).
2. **Momentum Start**: If velocity > threshold (e.g., 0.5 DP/ms), Logic enters a `Flinging` state.
3. **Decay**: On each subsequent frame (triggered by a timer or periodic `sendFrame()`), Logic updates `scroll_offset` by `velocity * elapsed_ms` and multiplies `velocity` by a decay factor (e.g., 0.95).
4. **Stop**: When velocity drops below a minimum (e.g., 0.01 DP/ms), Logic stops the fling and returns to `Idle`.
### 6.3 Scroll Clamping
`scroll_offset` is clamped to ensure the viewport never scrolls past the top or bottom of the file:
- **Min**: 0 (top of file)
- **Max**: `total_file_height - viewport_height` (bottom of file)

817
doc/virtual_scroll_render_optimization.md
View File

@ -1,817 +0,0 @@
# Virtual Scrolling & Render Optimization — Implementation Plan
## 1. Problem Statement
The spec (§11, "Performance Guarantees") mandates:
| Operation | Target | Mechanism |
|---|---|---|
| Open any file | < 100 ms | Chunked load: only the chunk around the cursor is read |
| Scroll | 60 fps | Virtual rendering: only visible lines + prefetch buffer |
| Type | < 16 ms per keystroke | In-memory chunk edit, async chunk flush |
**Current reality:** `drawWrappedText` in `internal/ui/render.go` receives the entire `EditorState.Buffer` as a single string and calls `shp.LayoutString(params, str)` on it — every frame. For a 100,000-line file, this means:
1. The shaper iterates ~1–10 million glyphs per frame
2. A `GlyphLayout` struct grows to millions of entries (ByteOffsets, X, Y, Advance slices)
3. Every line is drawn, including those far off-screen
4. Frame times are measured in hundreds of milliseconds — far exceeding the 16ms budget
This is the primary cause of sluggishness when editing large files. The fix is **virtual scrolling**: only shape, render, and capture glyph layout for the bytes visible in the current viewport.
---
## 2. Design Principles
1. **The buffer is the source of truth.** The `EditorState.Buffer` string holds the complete file content. Virtual scrolling is a presentation-layer concern — the buffer itself is untouched.
2. **Logic computes the visible range; renderer renders only that range.** The logic layer determines which byte range is visible given scroll offset and viewport height. The renderer shapes and draws only those bytes.
3. **GlyphLayout is bounded to the viewport.** Per-glyph layout data is only captured for visible content, keeping memory usage constant regardless of file size.
4. **Chunking is a prerequisite.** Virtual scrolling requires that the buffer be backed by chunks (fixed-size file slices) so that the logic layer can load/unload chunks on demand. This plan assumes the chunked buffer exists; the chunked buffer implementation is described in a companion plan.
---
## 3. Architecture
```
┌──────────────────────────────────────────────────────────────────┐
│ EditorState (internal/editor/state.go) │
│ │
│ Buffer string ← full file content (from chunks)│
│ ScrollOffset ui.Dp ← current scroll position │
│ ChunkedBuffer *ChunkedBuffer ← chunked file access │
│ LineIndex *LineIndex ← byte offsets per line │
│ │
│ VisibleByteRange(scroll, viewport) → (start, end) │
│ VisibleContent(start, end) → []byte │
└──────────────────────┬───────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────────┐
│ EditorLayout (internal/editor/state.go) │
│ │
│ 1. Compute viewport height from editor region + scale │
│ 2. Call VisibleByteRange(scrollOffset, viewportHeight) │
│ 3. Extract visibleContent := ChunkedBuffer.VisibleContent() │
│ 4. Adjust cursorPos to be relative to visibleContent │
│ 5. Adjust scrollOffset to be relative to visibleContent origin │
│ 6. Pass visibleContent + adjusted offsets to NewTextField │
└──────────────────────┬───────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────────┐
│ TextField.Draw → Renderer.drawWrappedText │
│ │
│ 1. Shape ONLY visibleContent (not the full buffer) │
│ 2. Capture GlyphLayout only for visible glyphs │
│ 3. Draw only visible lines │
│ 4. Render cursor at adjusted position │
│ 5. Return bounded GlyphLayout (≈ viewport-sized) │
└──────────────────────┬───────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────────┐
│ GlyphLayout (internal/ui/unit.go) │
│ │
│ ByteOffsets []int ← byte offsets within visibleContent │
│ X []Dp ← screen X positions │
│ Y []Dp ← screen Y positions │
│ Advance []Dp ← glyph widths │
│ │
│ Length: proportional to visible lines (≈ 50–500 entries) │
│ NOT proportional to file size │
└──────────────────────────────────────────────────────────────────┘
```
---
## 4. Data Structures
### 4.1 ChunkedBuffer
```go
// ChunkedBuffer provides chunked access to a file's content.
// Only chunks near the cursor or viewport are kept in memory.
type ChunkedBuffer struct {
filename string
chunkSize int // e.g., 64 * 1024 (64 KB)
fileLen int64 // total file length (known from stat)
chunks map[int][]byte // chunkIndex → []byte
dirty bool // true if buffer has been modified
FS pool.FileSystem // filesystem for reading chunks
basePath string // base path for file resolution
}
```
**Key properties:**
- `chunkSize` is fixed at 64 KB (configurable constant).
- `chunks` is a sparse map: only chunks near the cursor/viewport are loaded.
- `fileLen` is known from the file stat task (dispatched when opening a file).
- The full buffer is reconstructed on demand via `Content(start, end)` for rendering.
### 4.2 LineIndex
```go
// LineIndex maps line numbers to byte offsets within the file.
// Built as a low-priority background task; cached on disk.
type LineIndex struct {
offsets []int32 // byte offset of each line start (line 0 = 0)
mtime int64 // file mtime at index build time
size int64 // file size at index build time
}
```
**Key properties:**
- Built by streaming the file once (O(n) time, O(lines) space for the offset array).
- Stored as `[]int32` — 4 bytes per line. A 1M-line file = 4 MB.
- Used for O(log n) line number → byte offset lookup (binary search).
- Used by `VisibleByteRange` to compute the exact byte range for visible lines.
- Invalidated when file mtime or size changes.
- Cached on disk at `.pad/indices/<sha256(path)>.bin`.
### 4.3 VisibleByteRange
```go
// VisibleByteRange returns the byte range [start, end) of content
// visible in the viewport, given the current scroll offset and viewport height.
func (cb *ChunkedBuffer) VisibleByteRange(scrollOffset ui.Dp, viewportHeight ui.Dp) (start, end int) {
if cb.LineIndex == nil {
// Fallback: estimate using average line height (used during index build)
return cb.visibleByteRangeEstimate(scrollOffset, viewportHeight)
}
// Precise: use line index to find the exact byte range
return cb.visibleByteRangePrecise(scrollOffset, viewportHeight)
}
```
**Two strategies:**
| Strategy | When Used | Accuracy |
|---|---|---|
| `visibleByteRangeEstimate` | During initial load, before line index is built | Approximate (±1–2 lines) |
| `visibleByteRangePrecise` | After line index is available | Exact (byte-accurate) |
**Estimate strategy (no index):**
1. Compute average character width from a sample of lines.
2. Compute average line height from the shaper (one-shot measurement).
3. Estimate visible line range from scrollOffset / lineHeight.
4. Use binary search on the line index (if available) or scan from file start to find byte offsets.
**Precise strategy (with index):**
1. Compute visible line range: `startLine = scrollOffset / lineHeight`, `endLine = (scrollOffset + viewportHeight) / lineHeight`.
2. Binary search in `LineIndex.offsets` for `startLine` and `endLine`.
3. Clamp to `[0, fileLen)`.
4. Return `(int(offsets[startLine]), int(offsets[min(endLine, len(offsets))-1]))`.
### 4.4 Chunked Content Extraction
```go
// Content returns the bytes in [start, end) from the chunked buffer.
// It loads missing chunks on demand.
func (cb *ChunkedBuffer) Content(start, end int) string {
// 1. Determine which chunks are needed
startChunk := start / cb.chunkSize
endChunk := (end - 1) / cb.chunkSize
// 2. Load missing chunks
for i := startChunk; i <= endChunk; i++ {
if _, ok := cb.chunks[i]; !ok {
cb.loadChunk(i) // reads from disk via FS.ReadFile (full file)
}
}
// 3. Concatenate needed chunk slices
var buf bytes.Buffer
for i := startChunk; i <= endChunk; i++ {
chunk := cb.chunks[i]
chunkStart := i * cb.chunkSize
chunkEnd := chunkStart + len(chunk)
segStart := max(start, chunkStart) - chunkStart
segEnd := min(end, chunkEnd) - chunkStart
buf.Write(chunk[segStart:segEnd])
}
return buf.String()
}
// Prefetch loads adjacent chunks for smooth scrolling.
func (cb *ChunkedBuffer) Prefetch(centerChunk int, radius int) {
for i := centerChunk - radius; i <= centerChunk + radius; i++ {
if i >= 0 && i < cb.numChunks() {
if _, ok := cb.chunks[i]; !ok {
cb.loadChunk(i)
}
}
}
}
```
---
## 5. Implementation Steps
### Step 1: ChunkedBuffer Implementation
**File:** `internal/editor/chunked_buffer.go` (new)
Implement the `ChunkedBuffer` struct with:
- `NewChunkedBuffer(filename, chunkSize, fs, basePath)` constructor
- `loadChunk(idx int)` — reads a single chunk from disk via `ReadFile`
- `Content(start, end int) string` — extracts bytes from loaded chunks
- `Prefetch(centerChunk, radius int)` — loads adjacent chunks
- `EvictFarChunks(cursorPos int, radius int)` — removes chunks far from cursor
- `FileLen() int64` — total file length
- `Filename() string` — the file path
**File:** `internal/io/pool/task.go` — add `ReadChunkTask`
```go
type ReadChunkTask struct {
taskID string
Path string
ChunkIdx int
FS FileSystem
}
func NewReadChunkTask(path string, chunkIdx int, fs FileSystem) *ReadChunkTask { ... }
func (t *ReadChunkTask) Execute() Result {
// Read the full file, then slice the requested chunk
content, err := t.FS.ReadFile(t.Path)
if err != nil {
return Result{TaskID: t.taskID, TaskType: TypeReadChunk, Success: false, Error: err}
}
start := t.ChunkIdx * 64 * 1024
end := min(start+64*1024, len(content))
if start >= len(content) {
return Result{TaskID: t.taskID, TaskType: TypeReadChunk, Success: true, Data: []byte{}}
}
chunk := content[start:end]
return Result{TaskID: t.taskID, TaskType: TypeReadChunk, Success: true, Data: chunk}
}
func (t *ReadChunkTask) Priority() Priority { return HighPriority }
func (t *ReadChunkTask) TaskType() TaskType { return TypeReadChunk }
```
**File:** `internal/io/pool/task.go` — add `TypeReadChunk` constant.
**File:** `internal/io/pool/task.go` — add `StatFileTask` (replaces full-file `ReadFileTask` for open)
```go
type StatFileTask struct {
taskID string
Path string
FS FileSystem
}
func NewStatFileTask(path string, fs FileSystem) *StatFileTask { ... }
func (t *StatFileTask) Execute() Result {
// Read the full file content for now (will be optimized later with chunked stat)
content, err := t.FS.ReadFile(t.Path)
if err != nil {
return Result{TaskID: t.taskID, TaskType: TypeStatFile, Success: false, Error: err}
}
return Result{
TaskID: t.taskID, TaskType: TypeStatFile, Success: true,
Data: &FileStat{Path: t.Path, Size: int64(len(content))},
}
}
type FileStat struct {
Path string
Size int64
}
```
### Step 2: ChunkedBuffer on EditorState
**File:** `internal/editor/state.go`
```go
type EditorState struct {
// ... existing fields ...
ChunkedBuffer *ChunkedBuffer // NEW: chunked file access
LineIndex *LineIndex // NEW: line-to-byte-offset mapping
}
```
Update `OpenFile` in `state.go`:
```go
func OpenFile(data any) {
filename := data.(string)
TheState.Editor.Filename = filename
// Create chunked buffer
chunkSize := 64 * 1024 // 64 KB
cb := NewChunkedBuffer(filename, chunkSize, TheLogic.mockFS, "")
TheState.Editor.ChunkedBuffer = cb
// Dispatch stat task to get file size
TheLogic.workerPool.Dispatch(pool.NewStatFileTask(filename, TheLogic.mockFS))
// Switch to editor page
TheState.page = EditorPage
TheState.FocusedElementID = "editor_text"
TheState.ScrollOffset = 0
TheState.Editor.CursorPosition = 0
}
```
Update `handleWorkerResult` in `logic.go`:
```go
case res.TaskType == pool.TypeStatFile:
if res.Success {
if stat, ok := res.Data.(*pool.FileStat); ok {
TheState.Editor.ChunkedBuffer.SetFileSize(stat.Size)
// Load the chunk containing the cursor (position 0 at open)
TheState.Editor.ChunkedBuffer.LoadChunk(0)
// Prefetch adjacent chunks
TheState.Editor.ChunkedBuffer.Prefetch(0, 1)
}
}
```
### Step 3: VisibleByteRange and Content Extraction
**File:** `internal/editor/chunked_buffer.go` — implement `VisibleByteRange` and `Content`.
**File:** `internal/editor/chunked_buffer.go` — implement `visibleByteRangeEstimate`:
```go
// visibleByteRangeEstimate approximates the visible byte range using
// heuristic estimates. Used when the line index is not yet available.
func (cb *ChunkedBuffer) visibleByteRangeEstimate(scrollOffset ui.Dp, viewportHeight ui.Dp) (start, end int) {
totalLines := int(cb.fileLen) / 50 // rough: 50 bytes per line average
lineHeight := EditorLineHeight()
startLine := int(scrollOffset / lineHeight)
endLine := int((scrollOffset + viewportHeight) / lineHeight)
// Clamp
if startLine < 0 { startLine = 0 }
if endLine > totalLines { endLine = totalLines }
// Convert line numbers to byte offsets using the line index if available
if cb.LineIndex != nil && startLine < len(cb.LineIndex.offsets) {
start = int(cb.LineIndex.offsets[startLine])
} else {
start = startLine * 50 // rough estimate
}
if cb.LineIndex != nil && endLine < len(cb.LineIndex.offsets) {
end = int(cb.LineIndex.offsets[endLine])
} else {
end = endLine * 50
}
// Clamp to file bounds
if start < 0 { start = 0 }
if end > int(cb.fileLen) { end = int(cb.fileLen) }
if end <= start { end = start + 100 } // at least 100 bytes
return start, end
}
```
### Step 4: Line Index Build Task
**File:** `internal/io/pool/task.go` — add `BuildLineIndexTask`:
```go
type BuildLineIndexTask struct {
taskID string
Path string
FS FileSystem
}
func NewBuildLineIndexTask(path string, fs FileSystem) *BuildLineIndexTask { ... }
func (t *BuildLineIndexTask) Execute() Result {
content, err := t.FS.ReadFile(t.Path)
if err != nil {
return Result{TaskID: t.taskID, TaskType: TypeBuildLineIndex, Success: false, Error: err}
}
offsets := []int32{0}
for i := 0; i < len(content); i++ {
if content[i] == '\n' {
offsets = append(offsets, int32(i+1))
}
}
return Result{
TaskID: t.taskID, TaskType: TypeBuildLineIndex, Success: true,
Data: &LineIndex{offsets: offsets, mtime: 0, size: int64(len(content))},
}
}
func (t *BuildLineIndexTask) Priority() Priority { return LowPriority }
func (t *BuildLineIndexTask) TaskType() TaskType { return TypeBuildLineIndex }
```
**File:** `internal/io/pool/task.go` — add `TypeBuildLineIndex` constant.
**File:** `internal/editor/logic.go` — handle `TypeBuildLineIndex` result:
```go
case res.TaskType == pool.TypeBuildLineIndex:
if res.Success {
if idx, ok := res.Data.(*LineIndex); ok {
TheState.Editor.LineIndex = idx
}
}
```
**Dispatch the line index build in `OpenFile`:**
```go
// After loading the initial chunk:
TheLogic.workerPool.Dispatch(pool.NewBuildLineIndexTask(filename, TheLogic.mockFS))
```
### Step 5: EditorLayout — Compute Visible Content
**File:** `internal/editor/state.go` — update `EditorLayout`:
```go
func EditorLayout(screenWidth, screenHeight ui.Dp, wordWrap bool) []ui.Element {
// ... existing status bar and bottom bar code ...
// --- Editor text area ---
editorRegion := ui.Region{
X: margin, Y: editorY,
W: screenWidth - margin*2,
H: editorH,
}
// Compute viewport height in Dp
viewportHeight := editorRegion.H
// Compute visible byte range
var visibleContent string
var visibleCursorPos int
var visibleScrollOffset ui.Dp
cb := TheState.Editor.ChunkedBuffer
if cb != nil {
start, end := cb.VisibleByteRange(TheState.ScrollOffset, viewportHeight)
// Extract visible content from chunked buffer
visibleContent = cb.Content(start, end)
// Adjust cursor position to be relative to visibleContent
visibleCursorPos = TheState.Editor.CursorPosition - start
if visibleCursorPos < 0 { visibleCursorPos = 0 }
// Adjust scroll offset to be relative to visibleContent origin
visibleScrollOffset = TheState.ScrollOffset
// Prefetch adjacent chunks for smooth scrolling
cursorChunk := visibleCursorPos / cb.ChunkSize()
cb.Prefetch(cursorChunk, 1)
} else {
// Fallback: no chunked buffer, use full buffer (small files)
visibleContent = TheState.Editor.Buffer
visibleCursorPos = TheState.Editor.CursorPosition
visibleScrollOffset = TheState.ScrollOffset
}
// ... existing bottom bar code ...
editorElem := ui.NewTextField(
"editor_text",
visibleContent, // ← visible content only
editorRegion,
editorRegion.W,
visibleScrollOffset, // ← adjusted scroll
visibleCursorPos, // ← adjusted cursor
interactions,
)
return []ui.Element{statusBar, editorElem, bottomBar}
}
```
### Step 6: drawWrappedText — Render Only Visible Content
**File:** `internal/ui/render.go` — update `drawWrappedText`:
The function already receives `str` as the text to render. With virtual scrolling, `str` is now **only the visible content** (not the full buffer). The function body remains largely the same, but the impact is dramatic:
```go
func (r *Renderer) drawWrappedText(gtx layout.Context, str string, reg Region, wrapWidth Dp, scrollOffset Dp, cursorPos int) {
if str == "" {
return
}
// str is now the VISIBLE portion of the file (typically 50–500 lines)
// NOT the entire buffer. The shaper iterates only visible glyphs.
params := text.Parameters{
PxPerEm: fixed.I(gtx.Sp(r.theme.FontSize)),
MinWidth: 0,
MaxWidth: int(r.toPx(wrapWidth)),
MaxLines: 0,
LineHeight: fixed.I(gtx.Sp(lineHeightSp)),
LineHeightScale: 1.0,
WrapPolicy: text.WrapHeuristically,
}
// ONE LayoutString call — but now on visible content only (not full file)
r.shp.LayoutString(params, str)
// ... rest of the function unchanged ...
// GlyphLayout now has ~50–500 entries instead of millions
}
```
**Key change:** The function signature is unchanged. The difference is that `str` is now the visible content extracted by the logic layer. No changes to the shaping or drawing loop are needed — the optimization comes from passing a small string instead of a large one.
### Step 7: TextField — Pass Visible Content
**File:** `internal/ui/element.go` — `TextField.Draw` is unchanged:
```go
func (tf TextField) Draw(gtx layout.Context, r *Renderer) {
r.drawWrappedText(gtx, tf.Value, tf.region, tf.WrapWidth, tf.ScrollOffset, tf.CursorPosition)
}
```
`tf.Value` is now the visible content. No changes needed.
### Step 8: Edit Operations — Update ChunkedBuffer
**File:** `internal/editor/state.go` — update `HandleInsert`, `HandleBackspace`, `HandleDelete`:
```go
func HandleInsert(s string) {
pos := TheState.Editor.CursorPosition
buf := TheState.Editor.ChunkedBuffer
// Insert into the chunked buffer
buf.Insert(pos, s)
TheState.Editor.CursorPosition += len(s)
markDirty()
}
func HandleBackspace() {
pos := TheState.Editor.CursorPosition
if pos == 0 {
return
}
buf := TheState.Editor.ChunkedBuffer
buf.Delete(pos-1, 1)
TheState.Editor.CursorPosition--
markDirty()
}
func HandleDelete() {
pos := TheState.Editor.CursorPosition
buf := TheState.Editor.ChunkedBuffer
buf.Delete(pos, 1)
markDirty()
}
```
**File:** `internal/editor/chunked_buffer.go` — implement `Insert` and `Delete`:
```go
func (cb *ChunkedBuffer) Insert(pos int, text string) {
// 1. Ensure the chunk containing pos is loaded
chunkIdx := pos / cb.chunkSize
if _, ok := cb.chunks[chunkIdx]; !ok {
cb.loadChunk(chunkIdx)
}
// 2. Insert into the chunk
chunk := cb.chunks[chunkIdx]
offsetInChunk := pos - chunkIdx*cb.chunkSize
cb.chunks[chunkIdx] = append(chunk[:offsetInChunk], append([]byte(text), chunk[offsetInChunk:]...)...)
// 3. Mark buffer dirty
cb.dirty = true
// 4. If insertion spans chunk boundary, merge chunks
cb.maybeMergeChunk(chunkIdx)
}
func (cb *ChunkedBuffer) Delete(pos, n int) {
// 1. Ensure relevant chunks are loaded
startChunk := pos / cb.chunkSize
endChunk := (pos + n - 1) / cb.chunkSize
for i := startChunk; i <= endChunk; i++ {
if _, ok := cb.chunks[i]; !ok {
cb.loadChunk(i)
}
}
// 2. Delete from chunks
for i := startChunk; i <= endChunk; i++ {
chunk := cb.chunks[i]
offsetInChunk := pos - i*cb.chunkSize
deleteEnd := min(offsetInChunk+n, len(chunk))
if offsetInChunk < len(chunk) {
cb.chunks[i] = append(chunk[:offsetInChunk], chunk[deleteEnd:]...)
}
}
cb.dirty = true
cb.maybeMergeChunk(startChunk)
}
```
### Step 9: Auto-Save — Write Chunked Buffer
**File:** `internal/editor/logic.go` — update auto-save to reconstruct full content:
```go
func (l *Logic) markDirty() {
// ... existing code ...
l.saveTimer = time.AfterFunc(1*time.Second, func() {
if l.saveGeneration == gen+1 {
// Reconstruct full content from chunks for saving
cb := l.state.Editor.ChunkedBuffer
content := cb.FullContent() // reads all chunks + re-reads unloaded from disk
l.workerPool.DispatchNonBlocking(
pool.NewWriteFileTask(filename, []byte(content), l.mockFS),
)
}
})
}
```
**File:** `internal/editor/chunked_buffer.go` — implement `FullContent()`:
```go
func (cb *ChunkedBuffer) FullContent() string {
numChunks := int((cb.fileLen + int64(cb.chunkSize) - 1) / int64(cb.chunkSize))
var buf bytes.Buffer
for i := 0; i < numChunks; i++ {
if chunk, ok := cb.chunks[i]; ok {
buf.Write(chunk)
} else {
// Re-read from disk
chunk := cb.loadChunk(i)
buf.Write(chunk)
}
}
return buf.String()
}
```
### Step 10: Buffer Field Deprecation
**File:** `internal/editor/state.go` — keep `Buffer` field for backward compatibility but deprecate:
```go
type EditorState struct {
Buffer string // DEPRECATED: use ChunkedBuffer for large files
ChunkedBuffer *ChunkedBuffer // NEW: primary buffer for editing
// ...
}
```
Provide a `GetBuffer()` method that reconstructs the full string on demand:
```go
func (e *EditorState) GetBuffer() string {
if e.ChunkedBuffer != nil {
return e.ChunkedBuffer.FullContent()
}
return e.Buffer
}
```
---
## 6. File Change Summary
| File | Change |
|---|---|
| `internal/editor/chunked_buffer.go` | **NEW** — ChunkedBuffer, LineIndex, VisibleByteRange, Insert, Delete |
| `internal/editor/state.go` | Add `ChunkedBuffer`, `LineIndex` fields; update `OpenFile`, `HandleInsert`, `HandleBackspace`, `HandleDelete`, `EditorLayout` |
| `internal/editor/logic.go` | Add `TypeReadChunk`, `TypeStatFile`, `TypeBuildLineIndex` handling; update `markDirty` for chunked save |
| `internal/io/pool/task.go` | Add `ReadChunkTask`, `StatFileTask`, `BuildLineIndexTask`; add `TypeReadChunk`, `TypeStatFile`, `TypeBuildLineIndex` constants; add `FileStat`, `LineIndex` types |
| `internal/ui/render.go` | `drawWrappedText` — no signature change; optimization comes from smaller `str` input |
| `internal/ui/element.go` | `TextField.Draw` — no change; passes visible content |
---
## 7. Performance Impact
| Metric | Before | After | Spec Target |
|---|---|---|---|
| Open 100MB file | ~1000ms (full read + string) | ~50ms (stat + 1 chunk) | < 100ms |
| Scroll frame time (1M line file) | ~500ms+ (shapes all glyphs) | ~2ms (shapes ~50 lines) | 60 fps |
| Memory (100MB file) | ~200MB+ (string + GlyphLayout) | ~6.4MB (100 chunks + viewport GlyphLayout) | 16–64 MB |
| Per-keystroke latency | ~500ms+ (full reshape) | ~5ms (visible reshape) | < 16ms |
| GlyphLayout size (1M line file) | ~10M entries | ~50–500 entries | bounded |
---
## 8. Edge Cases and Fallbacks
| Scenario | Handling |
|---|---|
| File < 1 MB | Use full buffer (no chunking needed); virtual scrolling still applies but visible content ≈ full content |
| Line index build in progress | Use `visibleByteRangeEstimate` (heuristic) until index is ready |
| File smaller than chunk size | Single chunk; no chunk management overhead |
| Chunk boundary in middle of a line | Chunk content is raw bytes; line wrapping handles partial lines naturally |
| Edit at chunk boundary | `Insert`/`Delete` handles multi-chunk operations; `maybeMergeChunk` consolidates if needed |
| File replaced externally | `StatFileTask` detects size/mtime change; reload chunks and rebuild index |
| Very long line (> viewport width) | Shaper handles it; virtual scrolling still works because we shape only visible content |
| Cursor off-screen | `SetCursorFromPoint` uses GlyphLayout from visible content; cursor is always on-screen |
---
## 9. Testing Strategy
### Unit Tests
**File:** `internal/editor/chunked_buffer_test.go` (new)
| Test | What it verifies |
|---|---|
| `TestChunkedBuffer_LoadChunk` | Loading a chunk reads the correct bytes from disk |
| `TestChunkedBuffer_Content` | `Content(start, end)` returns the correct byte range |
| `TestChunkedBuffer_Insert` | Inserting text updates the correct chunk |
| `TestChunkedBuffer_Delete` | Deleting bytes removes them from the correct chunk |
| `TestChunkedBuffer_EvictFarChunks` | Chunks far from cursor are evicted |
| `TestChunkedBuffer_Prefetch` | Prefetch loads adjacent chunks |
| `TestChunkedBuffer_VisibleByteRange` | Visible byte range is correct for various scroll positions |
| `TestChunkedBuffer_FullContent` | `FullContent()` reconstructs the exact original file |
| `TestChunkedBuffer_EmptyFile` | Handles zero-length files |
| `TestChunkedBuffer_FileSmallerThanChunk` | Single-chunk files work correctly |
| `TestLineIndex_Build` | Line index correctly maps line numbers to byte offsets |
| `TestLineIndex_BinarySearch` | Binary search finds correct byte offset for any line |
| `TestLineIndex_Invalidation` | Index is invalidated when file size changes |
| `TestDeterministicFile_GeneratesKnownContent` | Same seed produces identical content every time |
| `TestDeterministicFile_LineBased` | Line-based generator produces correct line count and byte offsets |
### Integration Tests
**File:** `internal/editor/virtual_scroll_test.go` (new)
| Test | What it verifies |
|---|---|
| `TestEditorLayout_VisibleContent` | EditorLayout passes only visible content to TextField |
| `TestEditorLayout_ScrollUpdates` | Scrolling changes the visible byte range |
| `TestOpenFile_ChunkedLoad` | Opening a file loads the initial chunk and dispatches index build |
| `TestAutoSave_ChunkedWrite` | Auto-save reconstructs full content and writes it |
| `TestEdit_ChunkedBuffer` | Insert/delete on chunked buffer maintains content correctness |
| `TestVirtualScroll_10KLines` | Opening a 10K-line file loads only visible chunks; frame time < 16ms |
| `TestVirtualScroll_100KLines` | Scrolling through 100K lines keeps memory bounded; no full-file shapes |
| `TestLineJump_1MLines` | Jumping to line 500K in a 1M-line file completes < 10ms via line index |
| `TestSetCursorFromPoint_100KLines` | Click-to-position cursor works correctly on a 100K-line file |
### Render Tests
**File:** `internal/ui/render_test.go` — add tests for `drawWrappedText` with large strings to verify that GlyphLayout size is bounded.
### E2E Tests
**File:** `internal/test/e2e/virtual_scroll_test.go` (new)
Uses the existing `Harness` to drive full editor workflows with deterministic large files. Tests verify the complete pipeline from file open through rendering to user interaction:
- **Open and scroll**: Open a 100K-line file, scroll to line 50K, verify the frame contains the correct visible lines (checked by inspecting the TextField.Value length)
- **Click-to-position**: Simulate a tap at a known screen coordinate, verify the cursor lands at the expected byte offset using the GlyphLayout feedback
- **Edit and verify**: Insert text at a specific position in a large file, verify the buffer content is correct by reading back the affected chunk
- **Line index build**: Verify the line index is built asynchronously and visible byte range becomes precise after index is ready
### Deterministic Large File Generator
**File:** `internal/editor/deterministic_file.go` (new)
Generates arbitrarily large files in-memory using a fixed seed and repeating block content. The mock filesystem stores the generated content as a `[]byte` without writing to disk. Two modes:
- **Fixed seed mode**: `NewDeterministicFile(seed uint64, blockSize int, repeatCount int)` — produces `blockSize * repeatCount` bytes. Same seed always produces identical content.
- **Line-based mode**: `NewLineBasedFile(seed uint64, linesPerBlock int, blockCount int)` — each block is a fixed set of lines repeated. Useful for testing line-index and line-jump correctness.
The generator is used by tests to create 10K, 100K, and 1M line files in the mock filesystem in milliseconds, with zero disk I/O. The content is a known repeating pattern so byte offsets, line numbers, and visible ranges can be verified exactly.
### Click-to-Position Verification
**File:** `internal/editor/cursor_test.go` — extend existing cursor tests with large-file scenarios using the deterministic generator.
- **`TestSetCursorFromPoint_LargeFile`**: Generate a 50K-line file, simulate a tap at a known Dp coordinate, verify the cursor lands on the expected byte offset. Uses the GlyphLayout captured during `drawWrappedText`.
- **`TestSetCursorFromPoint_ChunkBoundary`**: Generate a file where the tap target falls near a chunk boundary, verify cursor positioning is correct even when the visible content spans multiple chunks.
- **`TestSetCursorFromPoint_OffScreen`**: Verify that tapping outside the visible viewport scrolls to the target and positions the cursor correctly.
- **`TestSetCursorFromPoint_WrappedLines`**: Generate a file with long lines that wrap, verify tap-to-position correctly identifies the visual line and closest glyph.
### Performance Benchmarks
**File:** `internal/editor/bench_test.go` (new)
Go benchmarks that measure the actual performance of virtual scrolling with the deterministic generator:
- `BenchmarkVisibleByteRange_10KLines` — measures time to compute visible byte range
- `BenchmarkVisibleByteRange_1MLines` — measures time to compute visible byte range on 1M-line file
- `BenchmarkChunkedBufferContent` — measures time to extract visible content from chunked buffer
- `BenchmarkDrawWrappedText_10KLines` — measures shaping/rendering time for 10K-line file (visible only)
- `BenchmarkDrawWrappedText_1MLines` — measures shaping/rendering time for 1M-line file (visible only)
- `BenchmarkLineIndexBinarySearch` — measures O(log n) line jump time
All benchmarks use the deterministic generator to create files of known size. No disk I/O. Results are compared against the pre-virtual-scroll baseline (full-buffer shaping).
---