doc: screen coordinate reference (screen vs display vs app-local px)

The coordinate-space confusion (screen px 1080x2400 for input tap/
screencap, display px 900x2000 for reading PNGs, app-local px offset by
the 128px status bar, pt at density 2.625) cost real debugging time.
Wrote it down in doc/README.md as the permanent reference:

- the four spaces + conversions (display x1.2 = screen; screen Y-128 =
  app-local; /2.625 = pt)
- the practical tap rule: read (x,y) off the PNG, x1.2, input tap --
  screenshot px ARE screen px, no 128 offset for tap-what-you-see
- fixed geometry (status/nav bars, Gboard rows, editor region in pt),
  with re-measure guidance
- tap-test pitfalls (first-tap-swallowed, first input text can fail,
  wrong-place text = cursor/selection suspect not tap math, don't
  memorize drifting content positions)

All keyboard row values re-measured from a fresh screenshot and verified
live: tapping the Q key at (62, 1712) and backspace at (990, 2020) both
landed first try (file content confirmed). This also exposed that the
old notes had mixed display-px row values with screen-px key values in
the same list -- exactly the confusion this section eliminates.

Also fixed the stale Phase 6 'letterboxed window' note in
development_plan.md (window is full-screen; only the 128px status-bar
offset is real; content positions drift; the OpenFileChan logcat line
was removed) and pointed emu.sh's tap help at the x1.2 rule.
This commit is contained in:
Greg Pomerantz 2026-08-16 22:19:40 -04:00
parent b1a1719f17
commit ab60bb7f16
3 changed files with 66 additions and 13 deletions

View File

@ -74,6 +74,55 @@ Always stop with `down` — SIGKILLing the emulator corrupts the snapshot.
App package/activity: `pad.pad / org.gioui.GioActivity`. Default root App package/activity: `pad.pad / org.gioui.GioActivity`. Default root
directory: `/storage/emulated/0/Notes`. directory: `/storage/emulated/0/Notes`.
## Screen coordinates
Three coordinate spaces are in play; mixing them up is the #1 cause of
tap-test errors. (Measured 2026-08-16 on `pad_avd`; re-derive with
`adb shell wm size`, `adb shell wm density`, `dumpsys window`, and a fresh
screenshot with the keyboard open if the device config ever changes.)
| space | size | used by |
|--------------|-----------|------------------------------------------|
| screen px | 1080×2400 | `adb input tap`, `screencap` output |
| display px | 900×2000 | what you see when reading a PNG (vision) |
| app-local px | 1080×2272 | what Gio receives (`evtPos`) |
| pt | — | app layout units |
Conversions:
- display → screen: **× 1.2** (both axes)
- screen → app-local: **Y 128** (surface starts below the status bar; X unchanged)
- app-local px → pt: **÷ 2.625** (density 420)
**The tap rule (covers ~95% of needs):** to tap something visible, read its
(x, y) off the PNG you are looking at, multiply both by **1.2**, and
`input tap x y`. Screenshot px *are* screen px — no 128 offset involved.
The 128 offset only matters when converting screen ↔ app-local (e.g.
predicting which text line a tap will hit, or reasoning about `evtPos`
values). The app itself is internally consistent — rendering and input
share the same origin — so the offset only bites at the adb/screenshot
boundary.
Fixed geometry (screen px unless noted):
- status bar: top 128 px; nav bar: bottom 63 px
- Gboard (no suggestion bar): keyboard top ≈ 1518; rows QWERTY ≈ 1716,
ASDF ≈ 1880, ZXCV ≈ 2020 (backspace key ≈ (990, 2020)); space/enter
row ≈ 2168; icon toolbar ≈ 1548. Re-measure from a screenshot if Gboard
changes.
- App space (pt): editor region starts at (10, 62); line height 16.8 pt.
Tap-test pitfalls:
- The first tap right after launch, file open, or logcat clear is often
swallowed — re-tap before concluding anything is broken.
- The first `input text` after a cursor move can silently fail — retype.
- `input text` goes to whatever has IME focus; if typed text lands in the
wrong place, suspect cursor/selection state (see the GlyphLayout
window-relative invariant in `architecture.md`), not tap math.
- Don't memorize content positions (file rows, buttons) — they drift with
content. Measure from the current screenshot with the ×1.2 rule.
## On-device observation loop (quick reference) ## On-device observation loop (quick reference)
Gio renders into one GL surface, so the authoritative debug signals are Gio renders into one GL surface, so the authoritative debug signals are
@ -82,9 +131,10 @@ four):
- `logcat -s pad.pad` — app log (errors, limits, recovery; the normal path - `logcat -s pad.pad` — app log (errors, limits, recovery; the normal path
is quiet by design). is quiet by design).
- `input tap|swipe|text` — drive the UI (see `development_plan.md` for the - `input tap|swipe|text` — drive the UI. **All tap coordinates are screen
tap coordinates and IME-tap cadence that works). The first tap right px: read the position off the screenshot, ×1.2** (full reference:
after launch/open is sometimes swallowed — re-tap. §Screen coordinates). The first tap right after launch/open is sometimes
swallowed — re-tap.
- Autosave debounce is 1 s: **wait ~1.6 s before reading a file back from - Autosave debounce is 1 s: **wait ~1.6 s before reading a file back from
disk** after typing. disk** after typing.
- Memory: `adb shell dumpsys meminfo pad.pad` (watch PSS/RSS; the 3.8 GB - Memory: `adb shell dumpsys meminfo pad.pad` (watch PSS/RSS; the 3.8 GB

View File

@ -281,13 +281,12 @@ scroll (bounded high-water mark from the shaper glyph cache + Go heap; grows
~13 MB over the first ~30 scrolls then flat) — **no leak**, well under the 2.5 GB ~13 MB over the first ~30 scrolls then flat) — **no leak**, well under the 2.5 GB
OOM line. The profiler's own overhead is negligible (same plateau with it off). OOM line. The profiler's own overhead is negligible (same plateau with it off).
**Test-harness gotcha:** the 390×844 Dp window is **letterboxed** on the **Test-harness gotcha:** on-screen tap coordinates are not the naive
1080×2400 screen (~28 px left / ~133 px top offset), so on-screen tap Dp→px map — see `README.md` §Screen coordinates (screen px vs display px
coordinates are offset from the naive 1:1 Dp→px map. The editor back-arrow hits vs app-local; the tap rule is "read off the screenshot, ×1.2"). Don't
at ~`(86, 264)` px, not the glyph's apparent 1:1 position. Browser file rows memorize content positions (file rows etc.); measure them from the current
(newest-first) start ~y=520 px, ~134 px apart. Verify every open via the exact screenshot. Verify file opens from the title bar / file content, not
logcat line `Logic: OpenFileChan /storage/emulated/0/Notes/<file>`, not a loose logcat (the open log line was later removed as noise).
`OpenFileChan` match.
### Phase 7 — tap-to-position-cursor verification + fix — DONE (2026-08-16) ### Phase 7 — tap-to-position-cursor verification + fix — DONE (2026-08-16)

View File

@ -8,9 +8,13 @@
# scripts/emu.sh app start|stop|restart # scripts/emu.sh app start|stop|restart
# scripts/emu.sh shot [FILE] # screenshot (default /tmp/pad_shot_<ts>.png) # scripts/emu.sh shot [FILE] # screenshot (default /tmp/pad_shot_<ts>.png)
# scripts/emu.sh log [LINES] # last N lines of app logcat (default 20) # scripts/emu.sh log [LINES] # last N lines of app logcat (default 20)
# scripts/emu.sh tap X Y # tap (note: the first tap right after # scripts/emu.sh tap X Y # tap at SCREEN px (1080x2400 space).
# # launch/open is sometimes swallowed — # # Tap-what-you-see rule: read (x,y) off
# # re-tap if nothing happens) # # the PNG you are viewing and multiply
# # both by 1.2 (vision displays
# # screenshots at 900x2000). Note: the
# # first tap right after launch/open is
# # sometimes swallowed — re-tap.
# scripts/emu.sh type TEXT # type into the focused field (spaces ok) # scripts/emu.sh type TEXT # type into the focused field (spaces ok)
# scripts/emu.sh cmd <top|bottom|frac F|dp N> # one-shot editor debug command # scripts/emu.sh cmd <top|bottom|frac F|dp N> # one-shot editor debug command
# scripts/emu.sh perf on|off # enable/disable the in-app profiler # scripts/emu.sh perf on|off # enable/disable the in-app profiler