Pad/doc
Greg Pomerantz 6c6a0c1a27 Fix write-concurrency race: per-file write protocol + unique temps
The review-identified race: saves are async (owner snapshots content,
worker pool writes), nothing serialized per file, and every write of a
file used the SAME deterministic temp ('.<name>.tmp'). Two overlapping
writes (autosave x autosave, retry x autosave, or the synchronous
FlushAll on GoToBrowser/Shutdown x a worker write) interleaved on the
shared temp and could rename a byte-mixture into place; even without
interleaving, last-rename-wins could promote a STALE snapshot.

Owner-side protocol (logic.go, requestSave + result handler):
- at most one write in flight per file (writeInFlight maps filename ->
  the file version whose content the in-flight write carries);
- a save requested while one is in flight is deferred (savePending) and
  re-issued by the write's result handler with a FRESH snapshot, so
  'last rename wins' coincides with 'newest snapshot wins';
- on success the SNAPSHOT version (not the current one) is recorded as
  written, so an edit that arrived during the write leaves the file
  dirty and triggers the re-issue;
- FlushAll (GoToBrowser, Shutdown) defers via the same protocol instead
  of writing concurrently on the shared temp;
- shutdown drain: on done the owner waits (bounded 5 s) for in-flight
  writes and armed retries to settle before exiting, so the post-exit
  synchronous FlushAll and workerPool.Stop cannot race a straggling
  worker write;
- retry timer now sends a non-blocking token (no timer-goroutine stall
  on a full channel); emitFrame no longer blocks on a slow/gone main
  (frames are snapshots; the next emission wins) - also required so the
  drain can never deadlock on frame delivery.

Mechanism (real/filesystem.go):
- WriteFileAtomic uses a unique per-call temp ('.<name>.tmp.<pid>.<seq>'),
  making same-file staging-file interleaving structurally impossible even
  if the serialization regressed (defense in depth);
- each successful write best-effort removes stale temps of the same file
  (crash leftovers, plus the legacy deterministic name for upgraded
  installs); a failed write removes its own temp.

Tests:
- write_serialization_test.go (e2e): a counting FS wrapper proves the
  peak concurrent same-file saves is 1 across two deliberately
  overlapping autosaves (2 s saves; the second edit lands inside the
  first save's window and its token is deferred, then re-issued with the
  newer content), and that a Flush during an in-flight save adds no
  concurrent writer and the newest snapshot still wins. Mutation-verified:
  disabling the deferral fails it with peak = 2. (The pool's WriteFileTask
  calls FS.WriteFile, not WriteFileAtomic - the real FS is atomic only
  because WriteFile delegates to WriteFileAtomic; the wrapper mirrors that
  delegation or the overlap window does not exist.)
- filesystem_test.go: stale-temp test updated to the new pattern, also
  covering the legacy name and asserting a different file's temp is
  untouched.
- real_file_fuzz_test.go: stray-temp check matches both patterns.

Docs: architecture.md 6.5 rewritten (protocol invariants), spec.md
autosave line, development_plan.md v11 + Phase 12.

On-device smoke: open, type, autosave lands exact content on disk, no
temp files left, clean relaunch. Full suite green under -race.

Residuals (documented): no fsync before rename (power-loss window only);
external-change detection absent; a drain-deadline exit with a straggling
write can only lose freshness (unique temps keep every rename a complete
snapshot).
2026-08-17 16:29:47 -04:00
..
architecture.md Fix write-concurrency race: per-file write protocol + unique temps 2026-08-17 16:29:47 -04:00
development_plan.md Fix write-concurrency race: per-file write protocol + unique temps 2026-08-17 16:29:47 -04:00
README.md Track Android user font scale in all line-height geometry (Phase 11) 2026-08-17 10:59:34 -04:00
spec.md Fix write-concurrency race: per-file write protocol + unique temps 2026-08-17 16:29:47 -04:00

Pad documentation

Three documents, kept at the level of what, why, and invariants — not line-by-line code — so they stay true as the implementation evolves.

Doc What it is
spec.md What the app actually does, measured performance, the code layout, and an explicit list of deferred features.
architecture.md How it works: single-owner concurrency model, channel topology, Frame handoff contract, ownership rules, editor/browser/render internals.
development_plan.md The active plan: completed phases, remaining work, and the on-device observation loop.

Package inventory

Package Role
internal/editor Chunked buffer, line index, virtualized viewport, IME ops, autosave, logic goroutine, state, Frame
internal/browser Directory browsing, search, sort, pagination, browser state machine
internal/ui Element tree, frame layout, renderer (op-based), IME wiring, gestures
internal/perf Default-off performance profiler (per-logic-frame cadence + scroll state → CSV)
internal/io/pool Worker pool (8 workers, 2 priority lanes) + file/dir tasks
internal/test/e2e Harness driving the real Logic + Inspect
cmd/pad main.go (entry), impl_android.go (base path)

Documentation policy

  1. Docs describe invariants and contracts, not code lines. If a document has to change on every refactor, it is too detailed — delete it or raise its level of abstraction. Point-in-time implementation plans are deleted once implemented (the previous *_implementation_plan.md, touch.md, element_model.md, layout_rendering.md, virtual_scroll_render_optimization.md, and conflict_resolution.md were removed in the 2026-08 doc reorganization for this reason).
  2. Unbuilt behavior is not spec'd. Requirements that are not in the code live in spec.md §7 (deferred), never as if they worked.
  3. Code wins. When doc and code disagree, fix the doc.

Build & install (Android, this workstation)

Prereqs on this box (already installed): JDK 17, Go, Android SDK at ~/android-sdk (platform-tools, build-tools 35.0.0, emulator, NDK), gogio@v0.10.0 in ~/go/bin, debug keystore at ~/.android/debug.keystore. Env vars are set in ~/.bashrc.

Build + install with ./scripts/build_emu.sh (add --no-install to build only). It is self-contained: checks the prereqs, auto-downloads apktool v3.0.3 to ~/android-sdk/tools/apktool.jar if missing, and uses a temporary work dir. Output: cmd/pad/pad-emu.apk.

What it does, and why (gogio cannot inject manifest permissions):

  1. gogio -target android -targetsdk 35 -arch amd64 → raw APK (from cmd/pad/)
  2. apktool d → decode
  3. Inject <uses-permission android:name="android.permission.MANAGE_EXTERNAL_STORAGE"/> (without it, Android 11+ blocks listing /storage/emulated/0)
  4. apktool b → rebuild
  5. apksigner sign with the debug key → cmd/pad/pad-emu.apk
  6. adb install -r (skipped with --no-install)

Emulator + on-device debug: scripts/emu.sh

All emulator lifecycle and on-device debug operations go through scripts/emu.sh (headless AVD pad_avd, API 35): up / down / status, app start|stop|restart, shot, log, tap X Y, type TEXT, cmd <top|bottom|frac F|dp N> (one-shot editor debug command), perf on|off|pull (in-app profiler), push/pull (Notes files). See the script header for the full list and the device paths it encodes.

Snapshot behavior: the AVD auto-saves an instant-boot snapshot on clean down (restore ~10 s vs ~20 s cold boot). A corrupted snapshot makes the emulator segfault during restore; up detects that and falls back to a cold boot, and the next down re-saves a good snapshot (self-healing). Always stop with down — SIGKILLing the emulator corrupts the snapshot.

App package/activity: pad.pad / org.gioui.GioActivity. Default root directory: /storage/emulated/0/Notes.

Screen coordinates

The app has several coordinate spaces (screen/display/app-local px, app dp, text-local, window-relative glyph bytes, absolute file bytes); mixing them up is the #1 cause of tap-test and cursor bugs. (Measured 2026-08-16 on pad_avd; the 128 px offset and the 2.625 scale are device-config facts — re-derive with adb shell wm size, adb shell wm density, dumpsys window, and a fresh screenshot with the keyboard open if the device config ever changes.)

space size used by
screen px 1080×2400 adb input tap, screencap output
display px 900×2000 what you see when reading a PNG (vision)
app-local px 1080×2272 what Gio receives (evtPos)
pt app layout units

Conversions:

  • display → screen: × 1.2 (both axes)
  • screen → app-local: Y 128 (surface starts below the status bar; X unchanged)
  • app-local px → pt: ÷ 2.625 (density 420)

There are TWO independent scale factors, not one:

  • Density (PxPerDp, 2.625 here): the px↔dp conversion above. All geometry bookkeeping is in density-dp, so the pipeline is scale-free and holds at any display density (verified algebraically; the tap/scroll proof in Phase 10 uses no device numbers).
  • User font scale (PxPerSp/PxPerDp, e.g. settings put system font_scale 1.3): scales the text only (the shaper draws in sp), so the rendered line pitch is 16.8 × fontScale dp. The logic side tracks this via EffectiveLineHeight() and every line-height consumer uses it (window start, sub-line remainder, tap mapping, scroll clamp, caret, handles, highlight). Change it and the line pitch on screen changes (57 px/line at 1.3× vs 44 px/line at 1.0× on this AVD).

On-device verification notes:

  • The profiler CSV (/storage/emulated/0/PadPerf/logic_frames.csv) flushes to disk at most every 2 s, so tail can be up to 2 s stale — a scroll fling still settling reads as "settled" if two reads fall in the same flush window. Compare rows ≥ 2.5 s apart, taken after the last input.
  • The tap test is the ground truth and does not depend on the profiler: screenshot → visually identify a line → tap it → type a marker → read the file off the device → the marker must be on exactly the tapped line.

Full pipeline, screen → file byte (each hop has exactly one place it happens; the app is internally consistent, so errors only appear at the hops, not inside a space):

  1. screen px → window px → app dp — OS/Gio. Taps you inject land here; this is the only space adb/screenshots touch.
  2. app dp → text-local dp — the tap/drag handlers (localX = x EditorRegion.X, localY = tapLocalY(...)). Y adds only the sub-line scroll remainder, never the full scroll: the visible glyph layout is window-relative, so adding the full scroll maps a tap to a line far below the window (the Phase 7 tap-to-position bug). The remainder and the window's start line come from ONE shared float64 floor decomposition of ScrollOffset (line k, remainder r, with k·lineHeight ≤ ScrollOffset < (k+1)·lineHeight): the window starts at content line k, the renderer shifts the windowed layout up by r, and a tap a dp below the region top maps to content line k + ⌊(a+r)/lineHeight⌋ = ⌊(a+ScrollOffset)/lineHeight⌋ — the line actually under the finger, for every scroll offset (architecture.md §6.2).
  3. text-local dp → window-relative glyph bytevisualLine = y / lineHeight, then glyph x-search within that line group. GlyphLayout.ByteOffsets are relative to the top of the visible window (IMEWindowStartByte), not the file start.
  4. window-relative byte → absolute file byte — add/subtract the window base (IMEWindowStartByte, the glyphBase() helper) at each cursor boundary (tap, Home, End, vertical move, selection handles). Forgetting this hop is what snapped the cursor to the window top on scrolled files.

Selection / menu / IME spaces (all app dp unless noted):

  • EditorState.SelectionStart/End, SelectionAnchor, CursorPosition: absolute file bytes (architecture.md §6.3).
  • ui.TextField.SelectionStart/End: window-relative bytes into the visible Value (1 = none) — rendering + IME use only.
  • SelectionDragEvent.X/Y: finger position in app dp (renderer reports the finger; logic converts, exactly like a tap — §6.3a).
  • MenuRect: app dp; MenuItem.X/Y: menu-local (relative to the menu origin — Menu.Draw adds the origin back when drawing; a tap is inside the menu iff it is inside MenuRect, item index = (x MenuRect.X) / itemW).
  • IME key.EditEvent.Range: window-relative rune indices (§6.4).

Invariants: architecture.md §6.2 (window base), §6.3 (selection/caret), §6.3a (touch selection), §6.4 (IME).

The tap rule (covers ~95% of needs): to tap something visible, read its (x, y) off the PNG you are looking at, multiply both by 1.2, and input tap x y. Screenshot px are screen px — no 128 offset involved. The 128 offset only matters when converting screen ↔ app-local (e.g. predicting which text line a tap will hit, or reasoning about evtPos values). The app itself is internally consistent — rendering and input share the same origin — so the offset only bites at the adb/screenshot boundary.

Fixed geometry (screen px unless noted):

  • status bar: top 128 px; nav bar: bottom 63 px
  • Gboard (no suggestion bar): keyboard top ≈ 1518; rows QWERTY ≈ 1716, ASDF ≈ 1880, ZXCV ≈ 2020 (backspace key ≈ (990, 2020)); space/enter row ≈ 2168; icon toolbar ≈ 1548. Re-measure from a screenshot if Gboard changes.
  • App space (pt): editor region starts at (10, 62); line height 16.8 pt.

Tap-test pitfalls:

  • The first tap right after launch, file open, or logcat clear is often swallowed — re-tap before concluding anything is broken.
  • The first input text after a cursor move can silently fail — retype.
  • input text goes to whatever has IME focus; if typed text lands in the wrong place, suspect cursor/selection state (see the GlyphLayout window-relative invariant in architecture.md), not tap math.
  • Don't memorize content positions (file rows, buttons) — they drift with content. Measure from the current screenshot with the ×1.2 rule.

On-device observation loop (quick reference)

Gio renders into one GL surface, so the authoritative debug signals are data, not pixels (use scripts/emu.sh log|shot|perf|cmd for the first four):

  • logcat -s pad.pad — app log (errors, limits, recovery; the normal path is quiet by design).
  • input tap|swipe|text — drive the UI. All tap coordinates are screen px: read the position off the screenshot, ×1.2 (full reference: §Screen coordinates). The first tap right after launch/open is sometimes swallowed — re-tap.
  • Autosave debounce is 1 s: wait ~1.6 s before reading a file back from disk after typing.
  • Memory: adb shell dumpsys meminfo pad.pad (watch PSS/RSS; the 3.8 GB emulator OOMs the app above ~2.5 GB RSS).
  • Screenshots are an auxiliary check only — state, logcat, and file diffs are authoritative.