doc: screen coordinate reference (screen vs display vs app-local px)
The coordinate-space confusion (screen px 1080x2400 for input tap/ screencap, display px 900x2000 for reading PNGs, app-local px offset by the 128px status bar, pt at density 2.625) cost real debugging time. Wrote it down in doc/README.md as the permanent reference: - the four spaces + conversions (display x1.2 = screen; screen Y-128 = app-local; /2.625 = pt) - the practical tap rule: read (x,y) off the PNG, x1.2, input tap -- screenshot px ARE screen px, no 128 offset for tap-what-you-see - fixed geometry (status/nav bars, Gboard rows, editor region in pt), with re-measure guidance - tap-test pitfalls (first-tap-swallowed, first input text can fail, wrong-place text = cursor/selection suspect not tap math, don't memorize drifting content positions) All keyboard row values re-measured from a fresh screenshot and verified live: tapping the Q key at (62, 1712) and backspace at (990, 2020) both landed first try (file content confirmed). This also exposed that the old notes had mixed display-px row values with screen-px key values in the same list -- exactly the confusion this section eliminates. Also fixed the stale Phase 6 'letterboxed window' note in development_plan.md (window is full-screen; only the 128px status-bar offset is real; content positions drift; the OpenFileChan logcat line was removed) and pointed emu.sh's tap help at the x1.2 rule.
This commit is contained in:
parent
b1a1719f17
commit
ab60bb7f16
|
|
@ -74,6 +74,55 @@ Always stop with `down` — SIGKILLing the emulator corrupts the snapshot.
|
|||
App package/activity: `pad.pad / org.gioui.GioActivity`. Default root
|
||||
directory: `/storage/emulated/0/Notes`.
|
||||
|
||||
## Screen coordinates
|
||||
|
||||
Three coordinate spaces are in play; mixing them up is the #1 cause of
|
||||
tap-test errors. (Measured 2026-08-16 on `pad_avd`; re-derive with
|
||||
`adb shell wm size`, `adb shell wm density`, `dumpsys window`, and a fresh
|
||||
screenshot with the keyboard open if the device config ever changes.)
|
||||
|
||||
| space | size | used by |
|
||||
|--------------|-----------|------------------------------------------|
|
||||
| screen px | 1080×2400 | `adb input tap`, `screencap` output |
|
||||
| display px | 900×2000 | what you see when reading a PNG (vision) |
|
||||
| app-local px | 1080×2272 | what Gio receives (`evtPos`) |
|
||||
| pt | — | app layout units |
|
||||
|
||||
Conversions:
|
||||
|
||||
- display → screen: **× 1.2** (both axes)
|
||||
- screen → app-local: **Y − 128** (surface starts below the status bar; X unchanged)
|
||||
- app-local px → pt: **÷ 2.625** (density 420)
|
||||
|
||||
**The tap rule (covers ~95% of needs):** to tap something visible, read its
|
||||
(x, y) off the PNG you are looking at, multiply both by **1.2**, and
|
||||
`input tap x y`. Screenshot px *are* screen px — no 128 offset involved.
|
||||
The 128 offset only matters when converting screen ↔ app-local (e.g.
|
||||
predicting which text line a tap will hit, or reasoning about `evtPos`
|
||||
values). The app itself is internally consistent — rendering and input
|
||||
share the same origin — so the offset only bites at the adb/screenshot
|
||||
boundary.
|
||||
|
||||
Fixed geometry (screen px unless noted):
|
||||
|
||||
- status bar: top 128 px; nav bar: bottom 63 px
|
||||
- Gboard (no suggestion bar): keyboard top ≈ 1518; rows QWERTY ≈ 1716,
|
||||
ASDF ≈ 1880, ZXCV ≈ 2020 (backspace key ≈ (990, 2020)); space/enter
|
||||
row ≈ 2168; icon toolbar ≈ 1548. Re-measure from a screenshot if Gboard
|
||||
changes.
|
||||
- App space (pt): editor region starts at (10, 62); line height 16.8 pt.
|
||||
|
||||
Tap-test pitfalls:
|
||||
|
||||
- The first tap right after launch, file open, or logcat clear is often
|
||||
swallowed — re-tap before concluding anything is broken.
|
||||
- The first `input text` after a cursor move can silently fail — retype.
|
||||
- `input text` goes to whatever has IME focus; if typed text lands in the
|
||||
wrong place, suspect cursor/selection state (see the GlyphLayout
|
||||
window-relative invariant in `architecture.md`), not tap math.
|
||||
- Don't memorize content positions (file rows, buttons) — they drift with
|
||||
content. Measure from the current screenshot with the ×1.2 rule.
|
||||
|
||||
## On-device observation loop (quick reference)
|
||||
|
||||
Gio renders into one GL surface, so the authoritative debug signals are
|
||||
|
|
@ -82,9 +131,10 @@ four):
|
|||
|
||||
- `logcat -s pad.pad` — app log (errors, limits, recovery; the normal path
|
||||
is quiet by design).
|
||||
- `input tap|swipe|text` — drive the UI (see `development_plan.md` for the
|
||||
tap coordinates and IME-tap cadence that works). The first tap right
|
||||
after launch/open is sometimes swallowed — re-tap.
|
||||
- `input tap|swipe|text` — drive the UI. **All tap coordinates are screen
|
||||
px: read the position off the screenshot, ×1.2** (full reference:
|
||||
§Screen coordinates). The first tap right after launch/open is sometimes
|
||||
swallowed — re-tap.
|
||||
- Autosave debounce is 1 s: **wait ~1.6 s before reading a file back from
|
||||
disk** after typing.
|
||||
- Memory: `adb shell dumpsys meminfo pad.pad` (watch PSS/RSS; the 3.8 GB
|
||||
|
|
|
|||
|
|
@ -281,13 +281,12 @@ scroll (bounded high-water mark from the shaper glyph cache + Go heap; grows
|
|||
~13 MB over the first ~30 scrolls then flat) — **no leak**, well under the 2.5 GB
|
||||
OOM line. The profiler's own overhead is negligible (same plateau with it off).
|
||||
|
||||
**Test-harness gotcha:** the 390×844 Dp window is **letterboxed** on the
|
||||
1080×2400 screen (~28 px left / ~133 px top offset), so on-screen tap
|
||||
coordinates are offset from the naive 1:1 Dp→px map. The editor back-arrow hits
|
||||
at ~`(86, 264)` px, not the glyph's apparent 1:1 position. Browser file rows
|
||||
(newest-first) start ~y=520 px, ~134 px apart. Verify every open via the exact
|
||||
logcat line `Logic: OpenFileChan /storage/emulated/0/Notes/<file>`, not a loose
|
||||
`OpenFileChan` match.
|
||||
**Test-harness gotcha:** on-screen tap coordinates are not the naive
|
||||
Dp→px map — see `README.md` §Screen coordinates (screen px vs display px
|
||||
vs app-local; the tap rule is "read off the screenshot, ×1.2"). Don't
|
||||
memorize content positions (file rows etc.); measure them from the current
|
||||
screenshot. Verify file opens from the title bar / file content, not
|
||||
logcat (the open log line was later removed as noise).
|
||||
|
||||
### Phase 7 — tap-to-position-cursor verification + fix — DONE (2026-08-16)
|
||||
|
||||
|
|
|
|||
|
|
@ -8,9 +8,13 @@
|
|||
# scripts/emu.sh app start|stop|restart
|
||||
# scripts/emu.sh shot [FILE] # screenshot (default /tmp/pad_shot_<ts>.png)
|
||||
# scripts/emu.sh log [LINES] # last N lines of app logcat (default 20)
|
||||
# scripts/emu.sh tap X Y # tap (note: the first tap right after
|
||||
# # launch/open is sometimes swallowed —
|
||||
# # re-tap if nothing happens)
|
||||
# scripts/emu.sh tap X Y # tap at SCREEN px (1080x2400 space).
|
||||
# # Tap-what-you-see rule: read (x,y) off
|
||||
# # the PNG you are viewing and multiply
|
||||
# # both by 1.2 (vision displays
|
||||
# # screenshots at 900x2000). Note: the
|
||||
# # first tap right after launch/open is
|
||||
# # sometimes swallowed — re-tap.
|
||||
# scripts/emu.sh type TEXT # type into the focused field (spaces ok)
|
||||
# scripts/emu.sh cmd <top|bottom|frac F|dp N> # one-shot editor debug command
|
||||
# scripts/emu.sh perf on|off # enable/disable the in-app profiler
|
||||
|
|
|
|||
Loading…
Reference in New Issue
Block a user