Greg Pomerantz
180fa966c8
Two feature bodies accumulated in the working tree:
1. Pinch to change the app font size, continuously (no snapping):
- internal/ui/pinch_tracker.go: logic-free touch state machine.
Two-mover formation (the resting palm can land first or last;
movement is the only signal valid for both), pair = the mover
pair whose distance changed most, baseline = press distance
(formDist), lazy pending releases, survivor-scroll forwarding
after a pair break. Robust to ~1 fps frames: a whole pinch can
land in one drain (formDist/brokeFactor/lazy releases).
- render.go: pinch probe (raw pointer events) + grab lifecycle so
the pair is exclusive (scroll sees nothing of the pair) and the
survivor's finger keeps working as a scroll after the pinch.
- state.go/logic.go/session.go/frame.go: app-local float font
scale, content-point pin (buffer byte + offset from baseline,
not a layout point, so rewrap keeps the same character under
the center), restore/font pins, session persistence.
- pinch_test.go, pinch_font_test.go, tag_identity_test.go,
real_draw_probe_test.go: unit + real-Renderer/real-Router tests.
2. Soft keyboard must not shift content:
- Root cause: gioui.org/app calls Router.RevealFocus on any frame
the viewport shrinks (IME open under adjustResize) and
synthesizes a pointer.Scroll nudge aimed at the focused field's
stale pre-resize bounds; gesture.Scroll consumed it -> a 32 dp
content jump.
- Fix: main.go flags the shrink frame; render.go drains that one
synthetic scroll for the gesture's tag before Update (scroll-
range clamping cannot work: the router UNIONs ranges across
frames). Finger scroll (pointer.Drag) and the flinger are
untouched. reveal_focus_drain_test.go reproduces RevealFocus at
the router level and verifies the drain + zero delta.
Also: tools/touchinject (platform-signed emulator multi-touch
injection harness + e2e script, adb has no two-finger input),
docs (spec 2.2 + development_plan 18-20), .gitignore, gofmt.
233 lines
13 KiB
Markdown
233 lines
13 KiB
Markdown
# Pad documentation
|
||
|
||
Four documents, kept at the level of *what, why, and invariants* — not
|
||
line-by-line code — so they stay true as the implementation evolves.
|
||
|
||
| Doc | What it is |
|
||
|---|---|
|
||
| [`spec.md`](./spec.md) | What the app **actually does**, measured performance, the code layout, and an explicit list of deferred features. |
|
||
| [`architecture.md`](./architecture.md) | How it works: single-owner concurrency model, channel topology, Frame handoff contract, ownership rules, editor/browser/render internals. |
|
||
| [`development_plan.md`](./development_plan.md) | The active plan: completed phases, remaining work, and the on-device observation loop. |
|
||
| [`release.md`](./release.md) | The release process (`scripts/release.sh`): the gates, the install-to-all-devices policy, and the rule that on-device profiling/testing is diagnostic and needs explicit approval. |
|
||
|
||
## Package inventory
|
||
|
||
| Package | Role |
|
||
|---------|------|
|
||
| `internal/editor` | Chunked buffer, line index, virtualized viewport, IME ops, autosave, logic goroutine, state, Frame |
|
||
| `internal/browser` | Directory browsing, search, sort, pagination, browser state machine |
|
||
| `internal/ui` | Element tree, frame layout, renderer (op-based), IME wiring, gestures |
|
||
| `internal/perf` | Default-off performance profiler (per-logic-frame cadence + scroll state → CSV) |
|
||
| `internal/io/pool` | Worker pool (8 workers, 2 priority lanes) + file/dir tasks |
|
||
| `internal/test/e2e` | Harness driving the real `Logic` + `Inspect` |
|
||
| `cmd/pad` | `main.go` (entry), `impl_android.go` (base path) |
|
||
|
||
## Documentation policy
|
||
|
||
1. **Docs describe invariants and contracts, not code lines.** If a document
|
||
has to change on every refactor, it is too detailed — delete it or raise
|
||
its level of abstraction. Point-in-time implementation plans are deleted
|
||
once implemented (the previous `*_implementation_plan.md`,
|
||
`touch.md`, `element_model.md`, `layout_rendering.md`,
|
||
`virtual_scroll_render_optimization.md`, and `conflict_resolution.md` were
|
||
removed in the 2026-08 doc reorganization for this reason).
|
||
2. **Unbuilt behavior is not spec'd.** Requirements that are not in the code
|
||
live in `spec.md` §7 (deferred), never as if they worked.
|
||
3. **Code wins.** When doc and code disagree, fix the doc.
|
||
|
||
## Build & install (Android, this workstation)
|
||
|
||
Prereqs on this box (already installed): JDK 17, Go, Android SDK at
|
||
`~/android-sdk` (platform-tools, build-tools 35.0.0, emulator, NDK),
|
||
`gogio@v0.10.0` in `~/go/bin`, debug keystore at
|
||
`~/.android/debug.keystore`. Env vars are set in `~/.bashrc`.
|
||
|
||
**Build + install with `./scripts/build_emu.sh`** (add `--no-install` to
|
||
build only). It is self-contained: checks the prereqs, auto-downloads
|
||
apktool v3.0.3 to `~/android-sdk/tools/apktool.jar` if missing, and uses a
|
||
temporary work dir. Output: `cmd/pad/pad-emu.apk`.
|
||
|
||
What it does, and why (gogio cannot inject manifest permissions):
|
||
|
||
1. `gogio -target android -targetsdk 35 -arch amd64` → raw APK (from `cmd/pad/`)
|
||
2. `apktool d` → decode
|
||
3. Inject `<uses-permission android:name="android.permission.MANAGE_EXTERNAL_STORAGE"/>`
|
||
(without it, Android 11+ blocks listing `/storage/emulated/0`)
|
||
4. `apktool b` → rebuild
|
||
5. `apksigner sign` with the debug key → `cmd/pad/pad-emu.apk`
|
||
6. `adb install -r` (skipped with `--no-install`)
|
||
|
||
## Emulator + on-device debug: `scripts/emu.sh`
|
||
|
||
All emulator lifecycle and on-device debug operations go through
|
||
`scripts/emu.sh` (headless AVD `pad_avd`, API 35): `up` / `down` /
|
||
`status`, `app start|stop|restart`, `shot`, `log`, `tap X Y`, `type TEXT`,
|
||
`cmd <top|bottom|frac F|dp N>` (one-shot editor debug command),
|
||
`perf on|off|pull` (in-app profiler), `push`/`pull` (Notes files). See the
|
||
script header for the full list and the device paths it encodes.
|
||
|
||
Snapshot behavior: the AVD auto-saves an instant-boot snapshot on clean
|
||
`down` (restore ~10 s vs ~20 s cold boot). A corrupted snapshot makes the
|
||
emulator segfault during restore; `up` detects that and falls back to a
|
||
cold boot, and the next `down` re-saves a good snapshot (self-healing).
|
||
Always stop with `down` — SIGKILLing the emulator corrupts the snapshot.
|
||
|
||
App package/activity: `pad.pad / org.gioui.GioActivity`. Default root
|
||
directory: `/storage/emulated/0/Notes`.
|
||
|
||
## Screen coordinates
|
||
|
||
The app has several coordinate spaces (screen/display/app-local px, app dp,
|
||
text-local, window-relative glyph bytes, absolute file bytes); mixing them
|
||
up is the #1 cause of tap-test and cursor bugs. (Measured 2026-08-16 on
|
||
`pad_avd`; the 128 px offset and the 2.625 scale are device-config facts —
|
||
re-derive with `adb shell wm size`, `adb shell wm density`, `dumpsys
|
||
window`, and a fresh screenshot with the keyboard open if the device config
|
||
ever changes.)
|
||
|
||
| space | size | used by |
|
||
|--------------|-----------|------------------------------------------|
|
||
| screen px | 1080×2400 | `adb input tap`, `screencap` output |
|
||
| display px | 900×2000 | what you see when reading a PNG (vision) |
|
||
| app-local px | 1080×2272 | what Gio receives (`evtPos`) |
|
||
| pt | — | app layout units |
|
||
|
||
Conversions:
|
||
|
||
- display → screen: **× 1.2** (both axes)
|
||
- screen → app-local: **Y − 128** (surface starts below the status bar; X unchanged)
|
||
- app-local px → pt: **÷ 2.625** (density 420)
|
||
|
||
There are TWO independent scale factors, not one:
|
||
|
||
- **Density** (`PxPerDp`, 2.625 here): the px↔dp conversion above. All
|
||
geometry bookkeeping is in density-dp, so the pipeline is scale-free and
|
||
holds at any display density (verified algebraically; the tap/scroll proof
|
||
in Phase 10 uses no device numbers).
|
||
- **User font scale** (`PxPerSp/PxPerDp`, e.g. `settings put system
|
||
font_scale 1.3`): scales the *text* only (the shaper draws in sp), so the
|
||
rendered line pitch is `16.8 × fontScale` dp. The logic side tracks this
|
||
via `EffectiveLineHeight()` and every line-height consumer uses it
|
||
(window start, sub-line remainder, tap mapping, scroll clamp, caret,
|
||
handles, highlight). Change it and the line pitch on screen changes
|
||
(57 px/line at 1.3× vs 44 px/line at 1.0× on this AVD).
|
||
- **App-local pinch font scale** (a third factor, in-app only): a two-finger
|
||
pinch in the editor multiplies a continuous float32 scale (1.0 default,
|
||
clamped 0.5–3.0, never rounded) on top of the two factors above; the
|
||
rendered line pitch is `16.8 × fontScale × appFontScale` dp. The logic
|
||
side folds it into `EffectiveLineHeight()` (system × app) and the
|
||
renderer multiplies the editor's sp size by it. The pinch CENTER is the
|
||
anchor: the logic captures the **content point** there (the glyph byte +
|
||
offset from its baseline, with a line/fragment/sub-line fallback) and
|
||
re-anchors it under every newly shaped layout — including the re-wrap a
|
||
few frames after the font change — so the character under the fingers
|
||
holds still, not merely its (rewrap-moved) visual line. It is persisted in
|
||
the relaunch session (`AppFontScale`), and
|
||
the session's `ScrollSub` is stored as a *fraction* of the line height so
|
||
scroll restore is font-independent. Test hook: `scripts/emu.sh cmd pinch
|
||
<F>` (relative, anchored at the editor region center) / `fontsize <F>`
|
||
(absolute, top-anchored) drive the same `HandleFontPinch` path a real
|
||
pinch delivers.
|
||
|
||
On-device verification notes:
|
||
|
||
- The profiler CSV (`/storage/emulated/0/PadPerf/logic_frames.csv`) flushes
|
||
to disk at most every 2 s, so `tail` can be up to 2 s stale — a scroll
|
||
fling still settling reads as "settled" if two reads fall in the same
|
||
flush window. Compare rows ≥ 2.5 s apart, taken after the last input.
|
||
- The tap test is the ground truth and does not depend on the profiler:
|
||
screenshot → visually identify a line → tap it → type a marker → read the
|
||
file off the device → the marker must be on exactly the tapped line.
|
||
|
||
Full pipeline, screen → file byte (each hop has exactly one place it
|
||
happens; the app is internally consistent, so errors only appear at the
|
||
hops, not inside a space):
|
||
|
||
1. **screen px → window px → app dp** — OS/Gio. Taps you inject land here;
|
||
this is the only space `adb`/screenshots touch.
|
||
2. **app dp → text-local dp** — the tap/drag handlers
|
||
(`localX = x − EditorRegion.X`, `localY = tapLocalY(...)`).
|
||
**Y adds only the sub-line scroll remainder**, never the full scroll:
|
||
the visible glyph layout is window-relative, so adding the full scroll
|
||
maps a tap to a line far below the window (the Phase 7 tap-to-position
|
||
bug). The remainder and the window's start line come from ONE shared
|
||
float64 floor decomposition of `ScrollOffset` (line k, remainder r, with
|
||
k·lineHeight ≤ ScrollOffset < (k+1)·lineHeight): the window starts at
|
||
content line k, the renderer shifts the windowed layout up by r, and a
|
||
tap a dp below the region top maps to content line k +
|
||
⌊(a+r)/lineHeight⌋ = ⌊(a+ScrollOffset)/lineHeight⌋ — the line actually
|
||
under the finger, for every scroll offset (architecture.md §6.2).
|
||
3. **text-local dp → window-relative glyph byte** —
|
||
`visualLine = y / lineHeight`, then glyph x-search within that line
|
||
group. `GlyphLayout.ByteOffsets` are relative to the **top of the
|
||
visible window** (`IMEWindowStartByte`), not the file start.
|
||
4. **window-relative byte → absolute file byte** — add/subtract the window
|
||
base (`IMEWindowStartByte`, the `glyphBase()` helper) at each cursor
|
||
boundary (tap, Home, End, vertical move, selection handles). Forgetting
|
||
this hop is what snapped the cursor to the window top on scrolled files.
|
||
|
||
Selection / menu / IME spaces (all app dp unless noted):
|
||
|
||
- `EditorState.SelectionStart/End`, `SelectionAnchor`, `CursorPosition`:
|
||
**absolute file bytes** (architecture.md §6.3).
|
||
- `ui.TextField.SelectionStart/End`: **window-relative** bytes into the
|
||
visible `Value` (−1 = none) — rendering + IME use only.
|
||
- `SelectionDragEvent.X/Y`: finger position in **app dp** (renderer reports
|
||
the finger; logic converts, exactly like a tap — §6.3a).
|
||
- `MenuRect`: app dp; `MenuItem.X/Y`: **menu-local** (relative to the menu
|
||
origin — `Menu.Draw` adds the origin back when drawing; a tap is inside
|
||
the menu iff it is inside `MenuRect`, item index =
|
||
`(x − MenuRect.X) / itemW`).
|
||
- IME `key.EditEvent.Range`: **window-relative rune indices** (§6.4).
|
||
|
||
Invariants: architecture.md §6.2 (window base), §6.3 (selection/caret),
|
||
§6.3a (touch selection), §6.4 (IME).
|
||
|
||
**The tap rule (covers ~95% of needs):** to tap something visible, read its
|
||
(x, y) off the PNG you are looking at, multiply both by **1.2**, and
|
||
`input tap x y`. Screenshot px *are* screen px — no 128 offset involved.
|
||
The 128 offset only matters when converting screen ↔ app-local (e.g.
|
||
predicting which text line a tap will hit, or reasoning about `evtPos`
|
||
values). The app itself is internally consistent — rendering and input
|
||
share the same origin — so the offset only bites at the adb/screenshot
|
||
boundary.
|
||
|
||
Fixed geometry (screen px unless noted):
|
||
|
||
- status bar: top 128 px; nav bar: bottom 63 px
|
||
- Gboard (no suggestion bar): keyboard top ≈ 1518; rows QWERTY ≈ 1716,
|
||
ASDF ≈ 1880, ZXCV ≈ 2020 (backspace key ≈ (990, 2020)); space/enter
|
||
row ≈ 2168; icon toolbar ≈ 1548. Re-measure from a screenshot if Gboard
|
||
changes.
|
||
- App space (pt): editor region starts at (10, 62); line height 16.8 pt.
|
||
|
||
Tap-test pitfalls:
|
||
|
||
- The first tap right after launch, file open, or logcat clear is often
|
||
swallowed — re-tap before concluding anything is broken.
|
||
- The first `input text` after a cursor move can silently fail — retype.
|
||
- `input text` goes to whatever has IME focus; if typed text lands in the
|
||
wrong place, suspect cursor/selection state (see the GlyphLayout
|
||
window-relative invariant in `architecture.md`), not tap math.
|
||
- Don't memorize content positions (file rows, buttons) — they drift with
|
||
content. Measure from the current screenshot with the ×1.2 rule.
|
||
|
||
## On-device observation loop (quick reference)
|
||
|
||
Gio renders into one GL surface, so the authoritative debug signals are
|
||
data, not pixels (use `scripts/emu.sh log|shot|perf|cmd` for the first
|
||
four):
|
||
|
||
- `logcat -s pad.pad` — app log (errors, limits, recovery; the normal path
|
||
is quiet by design).
|
||
- `input tap|swipe|text` — drive the UI. **All tap coordinates are screen
|
||
px: read the position off the screenshot, ×1.2** (full reference:
|
||
§Screen coordinates). The first tap right after launch/open is sometimes
|
||
swallowed — re-tap.
|
||
- Autosave debounce is 1 s: **wait ~1.6 s before reading a file back from
|
||
disk** after typing.
|
||
- Memory: `adb shell dumpsys meminfo pad.pad` (watch PSS/RSS; the 3.8 GB
|
||
emulator OOMs the app above ~2.5 GB RSS).
|
||
- Screenshots are an auxiliary check only — state, logcat, and file diffs
|
||
are authoritative.
|