gmp/Pad
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Pinch-to-font-size (continuous, content-point pinned) + IME-open scroll fix

Browse Source 
Two feature bodies accumulated in the working tree:

1. Pinch to change the app font size, continuously (no snapping):
   - internal/ui/pinch_tracker.go: logic-free touch state machine.
     Two-mover formation (the resting palm can land first or last;
     movement is the only signal valid for both), pair = the mover
     pair whose distance changed most, baseline = press distance
     (formDist), lazy pending releases, survivor-scroll forwarding
     after a pair break. Robust to ~1 fps frames: a whole pinch can
     land in one drain (formDist/brokeFactor/lazy releases).
   - render.go: pinch probe (raw pointer events) + grab lifecycle so
     the pair is exclusive (scroll sees nothing of the pair) and the
     survivor's finger keeps working as a scroll after the pinch.
   - state.go/logic.go/session.go/frame.go: app-local float font
     scale, content-point pin (buffer byte + offset from baseline,
     not a layout point, so rewrap keeps the same character under
     the center), restore/font pins, session persistence.
   - pinch_test.go, pinch_font_test.go, tag_identity_test.go,
     real_draw_probe_test.go: unit + real-Renderer/real-Router tests.

2. Soft keyboard must not shift content:
   - Root cause: gioui.org/app calls Router.RevealFocus on any frame
     the viewport shrinks (IME open under adjustResize) and
     synthesizes a pointer.Scroll nudge aimed at the focused field's
     stale pre-resize bounds; gesture.Scroll consumed it -> a 32 dp
     content jump.
   - Fix: main.go flags the shrink frame; render.go drains that one
     synthetic scroll for the gesture's tag before Update (scroll-
     range clamping cannot work: the router UNIONs ranges across
     frames). Finger scroll (pointer.Drag) and the flinger are
     untouched. reveal_focus_drain_test.go reproduces RevealFocus at
     the router level and verifies the drain + zero delta.

Also: tools/touchinject (platform-signed emulator multi-touch
injection harness + e2e script, adb has no two-finger input),
docs (spec 2.2 + development_plan 18-20), .gitignore, gofmt.
This commit is contained in:
Greg Pomerantz 2026-08-23 09:00:51 -04:00
parent 0f1b6e6290
commit 180fa966c8
23 changed files with 3302 additions and 45 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
.gitignore vendored
View File

@ -13,6 +13,10 @@
cmd/pad/pad
cmd/pad/classes
# touchinject harness build artifacts (regenerated by build.sh)
tools/touchinject/build/
tools/touchinject/dex/
# IDE
.idea/
.vscode/

21
cmd/pad/main.go
View File

@ -157,9 +157,13 @@ func run(w *app.Window) error {
// applied to the main-owned find input (edge-triggered, see
// Frame.FindClearSeq).
var lastFindClearSeq int
// lastFrameW/H hold the previous FrameEvent's window size (px); 0 = no
// frame yet. Used to spot shrink frames (see ZeroWheelScroll below).
var lastFrameW, lastFrameH int
for {
switch e := w.Event().(type) {
e := w.Event()
switch e := e.(type) {
case app.DestroyEvent:
if prof != nil {
prof.Stop()
@ -237,6 +241,8 @@ func run(w *app.Window) error {
curScale = 1 // no frame yet
}
curFontScale := frame.FontScale // 0 until the logic has the value
// App-local pinch font scale for the editor text (1.0 = default).
renderer.SetAppFontScale(frame.AppFontScale)
renderer.Draw(gtx, frame.Elems, curScale)
glyphLayout := renderer.GlyphLayout()
// Send search query update to the logic goroutine when it changes.
@ -255,7 +261,19 @@ func run(w *app.Window) error {
}
newFind := findEditor.Text()
sendFind := newFind != frame.FindQuery
// On a frame that shrinks the window (the IME opening under
// adjustResize), Gio's window synthesizes a scroll-to-focus
// pointer.Scroll via RevealFocus — it reads the focused field's
// stale pre-resize bounds and nudges the editor content. Flag the
// frame so CheckGestures drains that one synthetic event before the
// scroll gesture consumes it (Renderer.ZeroWheelScroll); finger
// scroll and the flinger are unaffected, and normal frames are
// untouched.
renderer.ZeroWheelScroll = lastFrameH > 0 &&
(e.Size.Y < lastFrameH || e.Size.X < lastFrameW)
lastFrameW, lastFrameH = e.Size.X, e.Size.Y
events := renderer.CheckGestures(e.Source, gtx.Metric)
renderer.ZeroWheelScroll = false
// Keep frames flowing while a long press is pending: a stationary
// finger generates no pointer events, so without this the window
// would sleep and the long-press threshold would never be reached.
@ -384,6 +402,7 @@ func run(w *app.Window) error {
WindowStartByte: frame.WindowStartByte,
WindowStartLine: frame.WindowStartLine,
EditSeq: frame.EditSeq,
ScrollOffset: frame.ScrollOffset,
}
}
default:

17
doc/README.md
View File

@ -111,6 +111,23 @@ There are TWO independent scale factors, not one:
(window start, sub-line remainder, tap mapping, scroll clamp, caret,
handles, highlight). Change it and the line pitch on screen changes
(57 px/line at 1.3× vs 44 px/line at 1.0× on this AVD).
- **App-local pinch font scale** (a third factor, in-app only): a two-finger
pinch in the editor multiplies a continuous float32 scale (1.0 default,
clamped 0.5–3.0, never rounded) on top of the two factors above; the
rendered line pitch is `16.8 × fontScale × appFontScale` dp. The logic
side folds it into `EffectiveLineHeight()` (system × app) and the
renderer multiplies the editor's sp size by it. The pinch CENTER is the
anchor: the logic captures the **content point** there (the glyph byte +
offset from its baseline, with a line/fragment/sub-line fallback) and
re-anchors it under every newly shaped layout — including the re-wrap a
few frames after the font change — so the character under the fingers
holds still, not merely its (rewrap-moved) visual line. It is persisted in
the relaunch session (`AppFontScale`), and
the session's `ScrollSub` is stored as a *fraction* of the line height so
scroll restore is font-independent. Test hook: `scripts/emu.sh cmd pinch
<F>` (relative, anchored at the editor region center) / `fontsize <F>`
(absolute, top-anchored) drive the same `HandleFontPinch` path a real
pinch delivers.
On-device verification notes:

295
doc/development_plan.md
View File

@ -1048,3 +1048,298 @@ Verified on device: with the exclusion active, horizontal drags from the
left-edge start handle no longer trigger `startBackNavigation`
(`dumpsys window` shows the exclusion region; logcat shows zero
back-gesture previews).
## 18. Pinch-to-change-font-size, continuous (2026-08-22)
A two-finger pinch in the editor now changes the app's font size smoothly,
without snapping to whole points. The app-local scale is a float32
(default 1.0, clamped 0.5–3.0) layered on top of the system user font
setting; it is never rounded anywhere in the pipeline.
**Renderer (`internal/ui`).** Gio v0.10 has no two-finger pinch primitive,
so the Renderer owns a probe event tag clipped to the editor text region
(next to the long-press probe). It tracks the active pointers across frames
(window-px positions keyed by pointer ID) and emits one relative factor per
frame — `pinchDist(cur)/pinchDist(prev)` — as `ui.FontPinchEvent` to the
editor's new `ui.Pinch` interaction handler. `pinchDist` (two lowest-ID
pointers) is a pure function, unit-tested. While a pinch is active, scroll
emission is suppressed so the first finger does not drag the text.
`drawWrappedText` multiplies the editor's sp size by the frame's
`AppFontScale` (main goroutine feeds it via `SetAppFontScale` before
Draw); ascent/line-height/highlight/caret/handles all follow because they
derive from the same size.
**Logic (`internal/editor`).** `State.appFontScale` + `HandleFontPinch`
(multiply by the per-frame factor and clamp). The anchor is the pinch
CENTER, not the viewport top — and it is a **content point**, not a layout
point: `State.captureContentPin` names the glyph under the midpoint (the
ABSOLUTE buffer byte — the layout's `ByteOffsets` are window-relative, so
the capture adds `IMEWindowStartByte` — plus the point's offset from that
glyph's baseline), captured under the pre-change layout. A rewrap moves
the *text* of a visual line (the same fragment index holds different bytes
after the rewrap), so pinning (line, fragment) would leave a different
character at the center; naming the byte does not. The offset is applied
in two phases: (1) immediately, `rescaleScrollAnchored` rescales `S + m`
about the center (continuous, valid until rewrap lands); (2) on **every
newly shaped layout at the current scale** — the re-shape after the font
change and the rewrap corrections that follow it — `refineContentPin`
recomputes the offset from the pinned byte's fresh baseline:
`S' = vk·lh + Y + Dy − m`, where `vk = VisualsBefore(WindowStartLine)` is
the window's FIRST visual line (the window top sits at content `vk·lh`,
NOT `floor(S/lh)·lh` — a different line whenever the viewport top lands
mid-way through a wrapped logical line) and `Y` is the byte's baseline in
the fresh layout (located by its absolute byte, window start from
`LayoutFeedback.WindowStartByte`). That lands the byte exactly on the
center and is a fixed point when the layout already agrees (no drift, no
oscillation). Two stale-data traps had to be closed: the scale change
**invalidates the last shaped layout** (`invalidateShapedLayout`) —
otherwise the next frame computes its window start with the OLD line
height and the NEW rescaled offset, a window ~10k lines off — and
`refreshFontPin` skips feedback shaped at a different scale (during a
pinch every frame changes the scale, so all but the latest feedback are
stale). While armed (2 s, refreshed by feedback) the pin rides every
frame; a (line, fragment, sub-line) anchor (`captureFontPin`/`applyFontPin`)
stands in for points off any glyph; an edit (EditSeq mismatch), a scroll,
or the timeout disarms it. `SetAppFontScale` (the `fontsize` debug
command) keeps the top-anchored behavior (no fingers to center on) and
invalidates the stale layout too. `EffectiveLineHeight()` is now system ×
app; every geometry consumer (window start, tap mapping, scroll clamp,
restore) was already routed through it. `Frame.AppFontScale` carries the
value to the renderer; `LayoutFeedback.ScrollOffset` carries the shaped
scroll back (used by the fallback path and diagnostics).
**Persistence.** The session snapshot gains `AppFontScale`, and
`ScrollSub` is now stored as a *fraction* of the line height (font-
independent; legacy Dp values > 1 are converted on restore).
**Testing.** `pinch_font_test.go` (continuous product of small factors,
clamp at both ends, center-anchor invariance, glyph hit-testing, content-
pin capture, the refine keeping the pinned BYTE on the center across a
rewrap that moves it to another fragment, the fixed-point property, the
(line, fragment) fallback, fragment clamp on pinch-out, bad-data no-op),
`pinch_test.go` (pinchDist/pinchMid geometry,
factor-series telescoping). On the emulator
(`scripts/emu.sh cmd pinch <F>` / `fontsize <F>` — one-shot commands that
drive the same `HandleFontPinch` path a real pinch delivers, since adb has
no two-finger input; `pinch` anchors at the editor-region center): five
×1.05 steps produced line pitches 44→46→49→51→54→56 px (autocorrelation-
measured) — continuous, no whole-point snapping; on a 40k-line wrapped
file scrolled to the middle, a pinch in/out cycle (×1.5 → ×0.75 →
×1.125) exercised the rewrap in BOTH directions (1→2 and 2→1 fragments
per line) and the pin converged to a fixed point within 2–3 layout-
feedback frames at every step, keeping the captured BYTE's line on the
region center (verified against the app's own geometry, not the pixels);
the ground-truth tap test (tap a line, type a marker, read the file)
passed at a 1.125× scale after two rewrapping pinches; `fontsize` keeps
the top anchor across a 1→2 rewrap; font scale, file, cursor and scroll
all survive a full restart; 0.5/3.0 clamps hold; one-finger scroll is
unaffected (and takes over the viewport, disarming the pin).
**Bugs found by the emulator round (both would have passed the unit
suite).** (1) `GlyphLayout.ByteOffsets` are window-relative, not absolute:
capturing the pin's byte without adding the window start made the pin
chase a moving offset and never converge. (2) The scale change left the
old `GlyphLayout` in place; the next frame computed its window start with
the OLD line height and the NEW rescaled offset — a window ~10k lines
from the viewport (visible as a ~1000-line jump). Fixed by
`invalidateShapedLayout()` on every scale change.
**Bug found by the on-device round (the emulator round could never have
found it — adb has no two-finger input, and the debug `pinch` command
bypasses the probe entirely).** On the phone a real two-finger pinch did
nothing. On-device logcat (probe event logs + per-frame event counts)
showed the scroll gesture receiving every finger move while both probe
tags received zero events. Root cause: the probes were declared as
`struct{}` fields of the Renderer. The unnamed fieldless `struct{}` is a
SINGLE canonical Go type, so `pressProbe`, `pinchProbe` (and a
diagnostic third) were the SAME tag value. Gio's router keys handlers by
tag value, so all three `event.Op` registrations collapsed into one
handler; the press probe's drain (which runs first in `CheckGestures`)
consumed every event for that tag and the pinch probe was structurally
starved. Fixed by giving each probe its own named type
(`pressProbeTag`, `pinchProbeTag`), with a regression test
(`TestProbeTagIdentity`) asserting the tags remain distinct map keys, and
`TestRealDrawOpsProbeHit`, which runs the real `Renderer.Draw` op stream
through a real `input.Router` and asserts the probe tags receive the
pointer press. Verified on the Pixel 9 Pro: 927 probe events across a
multi-pinch session, 137 per-frame factors emitted and applied (net
scale 1.70×, font visibly enlarged), scroll suppressed mid-pinch,
anchor held.
## 19. Pinch tracker: explicit pair, two-mover formation, slow frames (2026-08-23)
The §18 probe design ("two lowest-ID pointers") was replaced by an explicit
pair state machine (`internal/ui/pinch_tracker.go`, pure and unit-tested;
the renderer's `consumePinchProbe` is now a thin adapter that feeds events,
executes the tracker's grabs, and emits its factor). Four on-device failure
modes drove the rewrite, plus a whole class of slow-frame bugs only visible
on the ~1 fps emulator:
1. **Single-finger scroll changed the font** — the pair was re-derived from
whatever pointers happened to be present, so a scroll finger got paired
with a stale pointer and its drags became "pinch".
2. **Scroll-down enlarged the font** — same root: the scroll finger's
distance to a stale second pointer grows as it moves.
3. **Two fingers produced a sudden zoom before the pinch** — the baseline
(`prevDist`) survived from the previous pinch, so a new pinch 2.5× wider
emitted 2.5× on its first frame.
4. **Pinch-out stopped and became a scroll** — the pair was not explicit;
once a finger moved past the scroll slop the router handed the pointer
to the scroll gesture and the "pair" silently switched composition.
**Explicit, stable pair + grabs.** When the pair forms, the adapter issues
`pointer.GrabCmd` for BOTH fingers (exclusive delivery to the probe:
releases arrive even off-clip, and scroll/click are dropped with a Cancel —
so the pair can never be stolen mid-gesture, failure mode 4). The pair's
composition and its baseline are never re-derived from ambient pointers.
`factor()` = current pair distance / previous frame's distance, one factor
per frame (the Android driver replays historical samples — several drags per
frame — so the font sees one factor per frame, not per sample). A sanity
clamp drops factors outside 0.1–10 and advances the baseline, so a
teleporting pointer (ID-reuse noise) cannot jump the font. When a pair
finger lifts, the other becomes the **survivor**: it stays grabbed (Gio
v0.10 has no release-grab) and its drags are forwarded as a plain scroll
delta (`survivorScroll`), so the finger is not dead. A second finger that
returns re-forms the pair with a fresh baseline.
**Formation requires TWO MOVING fingers — and nothing else.** The dominant
real-world case is a palm edge already down when the two pinch fingers
land; a static rule about "which finger is the palm" (the oldest? the
newest? the still one?) cannot survive both palm-first and palm-last hand
lands. Movement is the only signal that works for both: while 2–3 fresh
fingers are down the tracker is *pending*; the pair forms — at `factor()`
time, after the whole frame's events, never per-event (per-event locking
in the first mover pair seen would pair a finger with a drifting palm) —
when two pending fingers have each moved more than `pinchMoveEps` (10 px)
from where they pressed, and it is the mover pair whose **distance**
changed most (a drifting palm's distance to a finger changes little; a
pinch's does). A lone mover is a scroll, never a pair; a unison movement
(a two-finger slide) leaves the distance unchanged and forms nothing.
Consequences verified: a resting (pruned, >300 ms) or still palm can never
enter the distance; the three fresh-fingers case pairs the pinch fingers;
the re-form candidate (a finger landing on a survivor) must also move
before it becomes the pair.
**Baseline = the PRESS distance.** Any spread that happened before the pair
starts is owed, not lost: the formation frame emits `d_current /
d_press`, and a pinch that breaks before its first `factor()` settles the
same owed factor at the break.
**Slow frames (the ~1 fps emulator batches a whole gesture into one
drain).** (a) *Born-and-dead in one frame*: presses, drags and BOTH
releases in one drain — pending releases are held **lazy** (`released`
map) until `factor()`: the pair forms at the fingers' final positions,
then breaks there (no survivor when both released), settling the owed
factor. (b) *Pair broke mid-frame*: `brokeFactor`/`brokeMid` are settled
at the break (against `prevDist`, or the press distance when fresh) and
emitted by the subsequent `factor()` call, which would otherwise see
`on==false` and drop the frame's movement.
**Emulator multi-touch injection.** adb has no two-finger input, so the
failure modes could not be tested end-to-end until `tools/touchinject`: a
platform-signed (AOSP test key, `INJECT_EVENTS` granted) toy app whose
broadcast receiver injects a scripted `MotionEvent` stream
(`down/move/up/wait`, display px) through
`InputManager.injectInputEvent` — `/dev/uhid` is a dead end (no kernel
module in the image). Two known flakes, both documented in the harness:
the receiver process is "cached" and the 1.5 GB emulator OOM-kills it
mid-script occasionally (the harness verifies `=== done` in logcat and
re-runs; service routing is blocked by Android 12+ background-start
restrictions, and the AVD's locked bootloader blocks the system-app
escalation); and burst drags (all moves within one frame's drain) do not
scroll — the app is on-demand-rendering at ~1 fps, so scroll tests space
the moves ~80 ms apart, which also matches what a real finger produces
over several frames.
**End-to-end results (real injected MotionEvents, big wrapped file).**
Single-finger drag: zero factors, scale unchanged (font), content scrolls.
Two-finger pinch-out 300→596 px: exactly one factor 596/300 = 1.98667,
font ~2×. Palm-first three fingers (palm resting and still, pinch fingers
landing 80/120 ms later): pair is the two pinch fingers — the factor
tracks the pinch, the palm never enters the distance. Lift one finger
mid-pinch: the factor stream stops at the lift (font frozen), the
survivor's 600 px drag scrolls the content 600/3.5 = 171.4 dp. The
one-frame leak at formation (the pair's own drags of the formation frame
still reach scroll, since the grabs commit next frame) is bounded by the
scroll slop — the deliberate cost of not grabbing on press, which would
kill single-finger scrolls.
**Testing.** `pinch_test.go` now covers: factor-series telescoping
(baseline = press distance), single-finger never scales, resting/stale
palm excluded, extra finger during an active pinch ignored, fresh baseline
per pinch, sanity clamp, the three slow-frame shapes (full pinch in one
frame, stationary born-dead, born-and-dead), palm-first three fingers,
and survivor scroll + re-form (candidate must move).
`real_draw_probe_test.go` runs the real `Renderer.Draw` op stream through
a real `input.Router`: press frame (pending, nothing), formation frame
(grabs + owed factor, one-frame scroll leak), post-formation frames (scroll
sees nothing of the pair), off-clip survival, release via the grab,
survivor scroll forwarding, re-form.
## 20. IME open: content must not shift (2026-08-23)
**Bug.** With the soft keyboard open (adjustResize), the editor content
jumped up by exactly 32 dp (112 px) every time the keyboard appeared.
Top-anchored layout keeps the window start line put when only the
viewport height changes, so the shift was not our layout: KBW
instrumentation of every `ScrollOffset` writer showed
`HandleScroll` receiving a single +112 px delta at the resize frame.
**Root cause (Gio, not the app).** `gioui.org/app` `window.go`, on every
frame whose viewport *shrank*, calls `Router.RevealFocus(viewport)` —
"scroll the focused widget into view". For a text editor the focused
field's registered bounds (stale — from the pre-resize, taller frame)
extend below the new viewport, so RevealFocus synthesizes a
`pointer.Scroll` event (`Source: Touch`, position (0,0), Y = the nudge)
delivered to the focused field's scroll handler. `gesture.Scroll`
consumes it like any wheel scroll → `HandleScroll` → the 32 dp jump.
The event is invisible to the app: it never enters the pointer queue
(no `MotionEvent` on the Android side), it is manufactured by the router
during `processEvent(frameEvent)`, before the app's frame handler runs.
Reproduced at the router level: `RevealFocus` on a shrunken viewport
queues exactly one scroll event for the gesture's tag.
**Why not the obvious fixes.**
- Zeroing the scroll *range* on the shrink frame does nothing: the router
UNIONs scroll ranges into the handler's filter across frames
(`pointerFilter.Add`/`Merge`), so the historical max can never shrink
back to zero — the clamp stays at ±∞ forever.
- Patching `app/window.go` to drop the shrink→RevealFocus call would mean
shipping a forked gioui (the build constraint is clean v0.10.0).
- `adjustNothing` removes the resize but hides the cursor line under the
keyboard.
**Fix (app-side, two files).**
- `cmd/pad/main.go`: on each `FrameEvent`, detect a shrink
(`e.Size` smaller than the previous frame's) and set
`renderer.ZeroWheelScroll` for that one frame.
- `internal/ui/render.go` (`CheckGestures`): when flagged, drain
`pointer.Scroll` events for the editor scroll gesture's tag
(`q.Event(pointer.Filter{Target: reg.scroll, Kinds: pointer.Scroll})`)
before `gesture.Scroll.Update` consumes anything. Only the synthesized
nudge matches: finger scroll is `pointer.Drag`, inertia is the
flinger, and on a phone there is no trackpad wheel. Normal frames are
untouched.
**Verification.**
- `reveal_focus_drain_test.go`: real `input.Router` + real
`Renderer.Draw` ops; the focused field (KeyDown interaction required —
it records the `event.Op` tag reference, and a per-frame
`key.FocusFilter` consumer marks the handler focusable, else the key
queue clears the focus each frame), shrunken viewport, `RevealFocus` →
exactly one synthetic scroll queued for the gesture tag; the drain
terminates and `gesture.Scroll.Update` then returns 0.
- Emulator E2E (real injected taps, keyboard really opens, window
2560→1527 px): pre-fix the scroll gesture emitted delta=112 on the
shrink frame and a screenshot cross-correlation showed a 112 px content
shift; post-fix the drain consumes the event, the gesture delta is 0,
and the cross-correlation shift is 0 (corr 0.987). The perf-CSV
(ScrollDP per logic frame) shows no 32 dp step when the keyboard opens
on the final build.
**Notes.** The `aosp_atd` emulator later started ANR-ing Pad on first
frame — the ANR trace shows the main thread in
`GioView.onFrameCallback` → `glDeleteBuffers` → gfxstream guest →
`madvise` (91 s system time): the emulated GPU's buffer-free path,
unrelated to input handling. Final verification therefore used a small
file (fast first frame) plus the router-level test.

28
doc/spec.md
View File

@ -60,6 +60,34 @@ elsewhere.
hardware keyboard, arrow keys, Home/End, and Page Up/Down move the cursor
(verified on the Android emulator; Gio's mobile focus-navigation default
for arrow keys is overridden — see architecture.md §2.1).
- **Soft keyboard does not shift content:** opening or closing the IME
resizes the window (adjustResize); the editor is top-anchored, so the
visible text stays put. Gio's window otherwise synthesizes a scroll-to-
focus nudge on any frame the viewport shrinks (RevealFocus, aimed at the
focused field's stale pre-resize bounds), which would shift the content
up; the app drains that one synthetic scroll on the shrink frame
(development_plan.md §20).
- **Pinch to change font size:** a two-finger pinch inside the editor text
changes the app's font size continuously — the rendered size tracks the
inter-finger distance with no snapping to whole points (the scale is a
float32 multiplied by the per-frame distance ratio, clamped to 0.5×–3.0×
of the 14sp base). It is an *app-local* scale layered on top of the
system user font setting. The CONTENT UNDER THE PINCH CENTER STAYS
FIXED: the logic captures the **content point** under the finger
midpoint — the glyph byte and the point's offset from that glyph's
baseline — and re-derives the scroll offset to keep that point on the
center. A content point, not a layout point: when the font change
re-wraps a line, a *visual* line's text moves (fragment 2 of the new
wrap is different text), so pinning (line, fragment) would leave a
different character at the center. The pin therefore names the
character itself (byte + baseline offset) and re-anchors it under every
newly shaped layout — the re-wrap included — which is what keeps the
character under the fingers through the rewrap. (A (line, fragment,
sub-line) anchor stands in as fallback for points off any glyph.)
Single-finger scroll is suppressed mid-pinch (the pinch owns the two
fingers) and takes over the viewport on the first scroll after the
pinch. The scale is part of the relaunch session (§2.4) and survives
restarts.
- **Text selection:** two input paths. **Touch** (the Android-native model):
long-press selects the word under the finger (on a blank spot it places the
caret and offers a paste-only menu); double-tap selects the word; the

10
internal/editor/frame.go
View File

@ -26,6 +26,10 @@ type Frame struct {
Elems []ui.Element
Scale float32
FontScale float32 // user font-size setting the logic bookkeeping used
// AppFontScale is the app-local pinch font scale (1.0 = default, 0 =
// not set yet). The renderer multiplies the editor font size by it;
// FontScale is already folded into gtx.Metric (PxPerSp).
AppFontScale float32
FocusedElementID string
Query string
// FindQuery: the in-file search query the logic goroutine has processed
@ -36,6 +40,10 @@ type Frame struct {
// FindClearSeq mirrors EditorState.Find.ClearSeq: main wipes the widget
// input once per NEW value (the X button cleared the logic-side query).
FindClearSeq int
// ScrollOffset is the editor scroll offset this frame's elements laid
// out at, shipped with the shaped glyph layout (LayoutFeedback) so the
// logic can express layout positions in content coordinates.
ScrollOffset ui.Dp
// WindowStartByte / WindowStartLine / EditSeq: the editor window this
// frame's elements describe. The main goroutine forwards them with the
// shaped glyph layout (LayoutFeedback) so the logic goroutine can apply
@ -61,10 +69,12 @@ func (l *Logic) frameOf(elems []ui.Element) Frame {
Elems: elems,
Scale: l.state.scale,
FontScale: l.state.fontScale,
AppFontScale: l.state.appFontScale,
FocusedElementID: l.state.FocusedElementID,
Query: l.state.Browser.Query,
FindQuery: l.state.Editor.Find.Query,
FindClearSeq: l.state.Editor.Find.ClearSeq,
ScrollOffset: l.state.ScrollOffset,
WindowStartByte: l.state.Editor.IMEWindowStartByte,
WindowStartLine: l.state.WindowStartLine,
WindowText: l.state.Editor.IMEWindowText,

67
internal/editor/logic.go
View File

@ -126,6 +126,17 @@ type Logic struct {
restoreScrollLine int
restoreScrollSub float64
restorePinDeadline time.Time
// fontPin/fontPinM implement the pinch font-pin (see
// setFontPin/refreshFontPin in session.go): the CONTENT point under the
// pinch center (glyph byte + offset from its baseline, with a
// line/fragment/sub-line fallback) and the center's region-relative Y
// (dp). While armed, every shaped layout re-derives the scroll offset to
// keep that content point under the center as the font scale — and, a
// few frames later, the rewrap — changes.
fontPin contentPin
fontPinM float64
fontPinArmed bool
fontPinDeadline time.Time
scaleSeen bool
restoreContentLanded bool
sessionSaver func(SessionState)
@ -307,6 +318,12 @@ func (l *Logic) Run() {
l.refreshRestorePin()
}
}
// Pinch font-pin (see refreshFontPin): the fresh layout may
// have rewrapped the pinned line; re-derive so the pinned
// content point stays under the pinch center.
if l.fontPinArmed {
l.refreshFontPin(fb)
}
// Search settle (see EditorState.findSettle): the shaping above
// may have corrected the wrap counts around a find-jumped
// viewport; re-scroll while the correction still matters.
@ -372,6 +389,7 @@ func (l *Logic) openFile(path string) {
if l.restoreFile != "" && l.restoreFile != path {
l.abortRestore() // a different open cancels the in-flight one
}
l.releaseFontPin() // the pin names lines of the previous file
// Discard find results for the previous file; the query is kept
// (see EditorState.findReset) and re-scanned against the new file.
TheState.Editor.findReset()
@ -492,10 +510,11 @@ func (l *Logic) EnableDebugCmdPoll(dir string) {
}()
}
// applyDebugCmd applies a one-shot debug scroll command from the cmd-file
// poller. Commands: "open <path>" (any page), and, on the editor page,
// "top", "bottom", "frac <0..1>", "dp <int>". Must be called on the logic
// goroutine.
// applyDebugCmd applies a one-shot debug command from the cmd-file poller.
// Commands: "open <path>" (any page), and, on the editor page, "top",
// "bottom", "frac <0..1>", "dp <int>", "pinch <factor>" (relative app font
// scale, as the renderer's pinch probe would deliver) and "fontsize <v>"
// (absolute app font scale). Must be called on the logic goroutine.
func (l *Logic) applyDebugCmd(cmd string) {
s := l.state
fields := strings.Fields(cmd)
@ -515,6 +534,45 @@ func (l *Logic) applyDebugCmd(cmd string) {
l.emitFrame() // OpenFile only mutates state (the tap path emits via its handler)
return
}
// App-local font scale (pinch zoom). Drives the full logic->frame->render
// path the same way a real pinch does (HandleFontPinch is the handler a
// FontPinchEvent carries); adb has no two-finger input, so these
// commands are the on-emulator test hook. `pinch` anchors at the CENTER
// of the editor region (where a real pinch usually starts); `fontsize`
// is an absolute top-anchored set.
if fields[0] == "pinch" || fields[0] == "fontsize" {
if s.page != EditorPage {
log.Printf("DebugCmd: %q ignored (not on editor page)", cmd)
return
}
if len(fields) < 2 {
log.Printf("DebugCmd: %s needs a value", fields[0])
return
}
f, err := strconv.ParseFloat(fields[1], 32)
if err != nil || f <= 0 {
log.Printf("DebugCmd: bad %s value %q", fields[0], fields[1])
return
}
if fields[0] == "pinch" {
if f > 100 { // a single frame's pinch never spans this much
log.Printf("DebugCmd: pinch factor out of range: %v", f)
return
}
HandleFontPinch(ui.FontPinchEvent{
Scale: float32(f),
Center: ui.Point{
X: s.EditorRegion.X + s.EditorRegion.W/2,
Y: s.EditorRegion.Y + s.EditorRegion.H/2,
},
})
} else {
SetAppFontScale(float32(f))
}
log.Printf("DebugCmd: %q -> appFontScale=%.4f scroll=%d", cmd, s.appFontScale, int(s.ScrollOffset))
l.emitFrame()
return
}
if s.page != EditorPage {
log.Printf("DebugCmd: %q ignored (not on editor page)", cmd)
return
@ -555,6 +613,7 @@ func (l *Logic) applyDebugCmd(cmd string) {
if target > s.MaxScroll {
target = s.MaxScroll
}
l.releaseFontPin() // a debug scroll takes over the viewport
s.ScrollOffset = target
log.Printf("DebugCmd: %q -> scroll=%d maxScroll=%d", cmd, int(s.ScrollOffset), int(s.MaxScroll))
l.emitFrame()

386
internal/editor/pinch_font_test.go Normal file
View File

@ -0,0 +1,386 @@
package editor
import (
"math"
"testing"
"pad/internal/ui"
)
// These tests cover the app-local pinch font scale: EffectiveLineHeight must
// follow the product of the system font scale and the app scale, and the
// pinch handler must apply the relative factor continuously (no rounding to
// whole points) while keeping the viewport top anchored (scroll offset
// rescaled in lockstep with the line height).
func TestEffectiveLineHeight_FollowsAppFontScale(t *testing.T) {
TheLogic = nil
TheState = NewState()
defer func() { TheState.appFontScale = 1.0 }()
cases := []struct {
sys, app float32
want float64
}{
{0, 1.0, float64(EditorLineHeight())}, // both unknown/default
{1.3, 1.0, float64(EditorLineHeight()) * 1.3},
{1.0, 1.7, float64(EditorLineHeight()) * 1.7},
{1.3, 1.7, float64(EditorLineHeight()) * 1.3 * 1.7},
{1.0, 0.5, float64(EditorLineHeight()) * 0.5},
}
for _, c := range cases {
TheState.fontScale = c.sys
TheState.appFontScale = c.app
got := float64(EffectiveLineHeight())
if math.Abs(got-c.want) > 1e-5 {
t.Errorf("sys=%v app=%v: EffectiveLineHeight=%v want %v", c.sys, c.app, got, c.want)
}
}
}
func TestHandleFontPinch_ContinuousAndAnchored(t *testing.T) {
TheLogic = nil
TheState = NewState()
defer func() { TheState.appFontScale = 1.0; TheState.ScrollOffset = 0 }()
// Start scrolled to a line boundary: 100 lines at the default pitch.
lh := float64(EffectiveLineHeight())
TheState.ScrollOffset = ui.Dp(100 * lh)
// A sequence of small relative factors: the product is 1.1^5 ~ 1.61051,
// a non-representable float32 — rounding to whole points (or to any
// fixed step) would not land here.
for i := 0; i < 5; i++ {
HandleFontPinch(ui.FontPinchEvent{Scale: 1.1})
}
want := float64(1.1) * 1.1 * 1.1 * 1.1 * 1.1
if math.Abs(float64(TheState.appFontScale)-want) > 1e-6 {
t.Errorf("appFontScale=%v want %v (continuous, no snapping)", TheState.appFontScale, want)
}
// The scroll offset rescaled by the same ratio: the same line (100) at
// the same sub-line fraction (0) stays on top.
wantOff := 100 * lh * want
if got := float64(TheState.ScrollOffset); math.Abs(got-wantOff) > 1e-3 {
t.Errorf("ScrollOffset=%v want %v (anchor preserved)", got, wantOff)
}
// EffectiveLineHeight follows the new scale.
wantLH := lh * want
if got := float64(EffectiveLineHeight()); math.Abs(got-wantLH) > 1e-5 {
t.Errorf("EffectiveLineHeight=%v want %v", got, wantLH)
}
}
func TestHandleFontPinch_Clamps(t *testing.T) {
TheLogic = nil
TheState = NewState()
defer func() { TheState.appFontScale = 1.0; TheState.ScrollOffset = 0 }()
// Zoom out past the minimum.
HandleFontPinch(ui.FontPinchEvent{Scale: 0.1})
if TheState.appFontScale != MinAppFontScale {
t.Errorf("appFontScale=%v want MinAppFontScale %v", TheState.appFontScale, MinAppFontScale)
}
// Zoom in past the maximum.
HandleFontPinch(ui.FontPinchEvent{Scale: 100})
if TheState.appFontScale != MaxAppFontScale {
t.Errorf("appFontScale=%v want MaxAppFontScale %v", TheState.appFontScale, MaxAppFontScale)
}
// At the maximum, further zoom-in is a no-op (including the scroll).
TheState.ScrollOffset = ui.Dp(123)
HandleFontPinch(ui.FontPinchEvent{Scale: 1.5})
if TheState.appFontScale != MaxAppFontScale {
t.Errorf("appFontScale=%v want MaxAppFontScale %v", TheState.appFontScale, MaxAppFontScale)
}
if float64(TheState.ScrollOffset) != 123 {
t.Errorf("ScrollOffset=%v want 123 (unchanged at the clamp)", TheState.ScrollOffset)
}
}
func TestSetAppFontScale_Absolute(t *testing.T) {
TheLogic = nil
TheState = NewState()
defer func() { TheState.appFontScale = 1.0; TheState.ScrollOffset = 0 }()
lh := float64(EffectiveLineHeight())
TheState.ScrollOffset = ui.Dp(50 * lh)
SetAppFontScale(2.0)
if TheState.appFontScale != 2.0 {
t.Fatalf("appFontScale=%v want 2.0", TheState.appFontScale)
}
if got, want := float64(TheState.ScrollOffset), 50*lh*2.0; math.Abs(got-want) > 1e-3 {
t.Errorf("ScrollOffset=%v want %v", got, want)
}
SetAppFontScale(0) // clamps to the minimum
if TheState.appFontScale != MinAppFontScale {
t.Errorf("appFontScale=%v want MinAppFontScale", TheState.appFontScale)
}
}
func TestHandleFontPinch_IgnoresBadData(t *testing.T) {
TheLogic = nil
TheState = NewState()
defer func() { TheState.appFontScale = 1.0 }()
HandleFontPinch("not an event")
HandleFontPinch(ui.FontPinchEvent{Scale: 0})
HandleFontPinch(ui.FontPinchEvent{Scale: -2})
if TheState.appFontScale != 1.0 {
t.Errorf("appFontScale=%v want unchanged 1.0", TheState.appFontScale)
}
}
// The pinch CENTER stays fixed: the continuous content coordinate under the
// center (display-line units) is invariant through the scale change. A
// top-anchored zoom would move that coordinate by m*(1-ratio) display
// lines — for a center 400dp below the region top at ×1.5 that is ~59
// display lines off, which is exactly what this test rejects.
func TestHandleFontPinch_CenterAnchor(t *testing.T) {
TheLogic = nil
TheState = NewState()
defer func() {
TheState.appFontScale = 1.0
TheState.ScrollOffset = 0
TheState.EditorRegion = ui.Region{}
}()
TheState.EditorRegion = ui.Region{Y: 100, H: 800}
lh := float64(EffectiveLineHeight())
TheState.ScrollOffset = ui.Dp(100 * lh)
// Center 400dp below the region top.
const m = 400.0
wantU := (float64(TheState.ScrollOffset) + m) / lh // content coord under the center
HandleFontPinch(ui.FontPinchEvent{Scale: 1.5, Center: ui.Point{Y: 100 + m}})
lhNew := float64(EffectiveLineHeight())
gotU := (float64(TheState.ScrollOffset) + m) / lhNew
// Tolerance is float32-scale + ui.Dp rounding accumulated over two
// conversions — still 4 orders of magnitude smaller than the ~59-line
// drift a top-anchored zoom would produce here.
if math.Abs(gotU-wantU) > 1e-4 {
t.Errorf("content under center: u=%v want %v (center must stay fixed)", gotU, wantU)
}
// Top-anchor sanity: the TOP moved (that is the point of center anchoring).
if float64(TheState.ScrollOffset) == 100*lh*1.5 {
t.Errorf("scroll=%v equals the top-anchored value; the center was not the anchor", TheState.ScrollOffset)
}
}
// With word wrap, the anchor must be the LOGICAL line under the center, not
// the display line: when the font change re-wraps the lines above the
// anchor (their fragment counts grow), the display-line index under the
// center changes, but the same logical line / fragment / sub-line must stay
// under it. applyFontPin is what the logic re-runs as rewrap corrections
// land (the font-pin).
func TestFontPin_SurvivesRewrap(t *testing.T) {
TheLogic = nil
TheState = NewState()
defer func() {
TheState.appFontScale = 1.0
TheState.ScrollOffset = 0
TheState.Editor.ChunkedBuffer = nil
}()
// 1000 logical lines, each wrapped into 2 fragments at the current size.
w := NewWrapIndex(1000)
for i := 0; i < 1000; i++ {
w.Set(i, 2)
}
TheState.Editor.ChunkedBuffer = &ChunkedBuffer{WrapIndex: w}
lh := float64(EffectiveLineHeight())
const m = 200.0 // region top (EditorRegion zero) + 200dp
// Anchor: logical line 60, its 1st fragment, 0.5 down it.
TheState.ScrollOffset = ui.Dp(float64(w.VisualsBefore(60))*lh + 0.5*lh - m)
line, frag, sub, ok := TheState.captureFontPin(m)
if !ok || line != 60 || frag != 0 || math.Abs(sub-0.5) > 1e-5 {
t.Fatalf("capture: line=%d frag=%d sub=%v ok=%v, want line 60 frag 0 sub 0.5", line, frag, sub, ok)
}
// The font grows ×1.5 (the line under the fingers keeps its 2 fragments
// for now): the same point stays under m.
TheState.appFontScale = 1.5
TheState.applyFontPin(line, frag, sub, m)
lhNew := float64(EffectiveLineHeight())
u := (float64(TheState.ScrollOffset) + m) / lhNew
if got := int(w.LineForVisual(int32(u))); got != 60 {
t.Errorf("after scale: line under center=%d want 60", got)
}
// Rewrap lands: every line now wraps into 3 fragments. The display-line
// index under the center moves from 120.5 to 180.5, but re-applying the
// pin must keep logical line 60 (fragment 0, sub 0.5) under the center.
for i := 0; i < 1000; i++ {
w.Set(i, 3)
}
TheState.applyFontPin(line, frag, sub, m)
u = (float64(TheState.ScrollOffset) + m) / lhNew
if got := int(w.LineForVisual(int32(u))); got != 60 {
t.Errorf("after rewrap: line under center=%d want 60 (display-line anchoring would fail here)", got)
}
// And a display-line anchor (what NOT to do) would be off: at the new
// counts, display line 120.5 is logical line 40, not 60.
if got := int(w.LineForVisual(120)); got == 60 {
t.Errorf("test is stale: display line 120 unexpectedly maps to line 60")
}
}
// fabLayout builds a GlyphLayout the way the shaper produces one: glyphsPerFrag
// glyphs per fragment, baseline of fragment f at (f+0.8)*lh (ascent < lh, the
// shaper's convention), 20dp advance each.
func fabLayout(lh float64, fragments, glyphsPerFrag int, byte0 int) ui.GlyphLayout {
gl := ui.GlyphLayout{LineHeight: ui.Dp(lh)}
for f := 0; f < fragments; f++ {
for g := 0; g < glyphsPerFrag; g++ {
gl.ByteOffsets = append(gl.ByteOffsets, byte0+f*glyphsPerFrag+g)
gl.X = append(gl.X, ui.Dp(10+20*float64(g)))
gl.Y = append(gl.Y, ui.Dp((float64(f)+0.8)*lh))
gl.Advance = append(gl.Advance, ui.Dp(20))
}
}
return gl
}
func TestGlyphAtLocalPoint(t *testing.T) {
gl := fabLayout(16.8, 2, 4, 0) // bytes 0-3 on fragment 0, 4-7 on fragment 1
if i, ok := glyphAtLocalPoint(gl, 50, 25); !ok || i != 6 {
t.Errorf("(50,25): i=%d ok=%v, want glyph 6 (byte 6, X=50 on fragment 1)", i, ok)
}
if _, ok := glyphAtLocalPoint(gl, 5, 25); ok {
t.Error("(5,25): left margin has no glyph, want not-ok")
}
if i, ok := glyphAtLocalPoint(gl, 1000, 25); !ok || i != 7 {
t.Errorf("(1000,25): past the line end pins its last glyph, got i=%d ok=%v want 7", i, ok)
}
if _, ok := glyphAtLocalPoint(ui.GlyphLayout{}, 50, 25); ok {
t.Error("empty layout: want not-ok")
}
}
func TestCaptureContentPin(t *testing.T) {
TheLogic = nil
TheState = NewState()
defer func() {
TheState.appFontScale = 1.0
TheState.ScrollOffset = 0
TheState.Editor.GlyphLayout = ui.GlyphLayout{}
}()
TheState.Editor.GlyphLayout = fabLayout(16.8, 2, 4, 0)
TheState.ScrollOffset = 0
// Point at (x=50, y=25): fragment 1 (baseline 30.24), the glyph at X=50.
pin := TheState.captureContentPin(50, 25)
if !pin.HaveGlyph || pin.Byte != 6 {
t.Fatalf("pin: byte=%d haveGlyph=%v, want byte 6", pin.Byte, pin.HaveGlyph)
}
if math.Abs(pin.Dy-(25-30.24)) > 1e-5 { // ui.Dp stores float32-precision values
t.Errorf("dy=%v want %v (offset from the glyph baseline)", pin.Dy, 25-30.24)
}
// Fallback anchor (no WrapIndex): line 1, sub = 25/16.8 - 1 ~ 0.488.
if pin.Line != 1 || pin.Frag != 0 || math.Abs(pin.Sub-(25/16.8-1)) > 1e-6 {
t.Errorf("fallback: line=%d frag=%d sub=%v, want line 1 frag 0 sub %v", pin.Line, pin.Frag, pin.Sub, 25/16.8-1)
}
}
// The REFINEMENT is what makes the content point stay fixed across a rewrap:
// the pinned byte moves to a different fragment in the new layout, and the
// refined offset places that byte's baseline (plus the captured dy) exactly
// at the pinch center — where the (line, fragment) anchor would leave a
// different character there.
func TestRefineContentPin_RewrapKeepsContentPoint(t *testing.T) {
const m = 25.0 // region-relative pinch center
const dy = 25.0 - 30.24 // from TestCaptureContentPin's point (above baseline)
// OLD layout (font 1.0x): byte 6 on fragment 1. (Capture would have
// returned Byte=6, Dy=dy.)
_ = fabLayout(16.8, 2, 4, 0)
// Font grows to 1.5x: the line re-wraps to 3 fragments x 3 glyphs; byte 6
// lands on fragment 2 (a DIFFERENT fragment than the old fragment 1 it
// was captured on... old frag index was 1, new is 2).
glNew := fabLayout(25.2, 3, 3, 0)
// The new layout is shaped for the window starting at document byte 0,
// whose first visual line is visual line 3 of the document (vk=3):
// the window top sits at content 3*25.2.
vk := 3
off, ok := refineContentPin(glNew, vk, 0, dy, m, 6)
if !ok {
t.Fatal("refine: want ok")
}
// The pinned point (byte 6's baseline + dy) must sit exactly at m.
contentY := float64(off) + m // content coordinate of the region-relative m
// byte 6's baseline: window y = (2+0.8)*25.2, window top at content vk*lh.
baselineContent := float64(vk)*float64(glNew.LineHeight) + float64(glNew.Y[6])
if math.Abs(baselineContent+dy-contentY) > 1e-5 { // float32-precision layout Y
t.Errorf("pinned point at content %v, want %v (baseline %v + dy)", contentY, baselineContent+dy, baselineContent)
}
// (A (line, fragment) anchor would have failed here: the byte captured on
// the old fragment 1 sits on the NEW fragment 2, a full line further down.
// Pinning the old fragment index would leave a different character at the
// center — exactly the drift the content pin exists to remove.)
}
// Steady state: a layout shaped at offset S with the pinned point at the
// center must refine back to exactly S (the fixed point — no drift, no
// oscillation while the pin is armed).
func TestRefineContentPin_FixedPoint(t *testing.T) {
const m = 25.0
lh := ui.Dp(25.2)
S := ui.Dp(100)
vk, r := scrollDecompose(S, lh) // window's first visual line = v0 (no wrap)
const dy = 3.0
// Place byte 4 at window y = m + r - dy so the point sits at m when the
// window (top at content vk*lh) is shaped at S.
wantY := m + r - dy
gl := ui.GlyphLayout{LineHeight: lh}
for b := 0; b < 6; b++ {
gl.ByteOffsets = append(gl.ByteOffsets, b)
gl.X = append(gl.X, ui.Dp(10+20*float64(b%3)))
gl.Y = append(gl.Y, ui.Dp(wantY)) // one line's worth for this test
gl.Advance = append(gl.Advance, ui.Dp(20))
}
off, ok := refineContentPin(gl, int(vk), 0, dy, m, 4)
if !ok {
t.Fatal("refine: want ok")
}
if math.Abs(float64(off)-float64(S)) > 1e-9 {
t.Errorf("fixed point: refine(S)=%v want S=%v (no drift)", off, S)
}
}
// Pinch-out clamp of the pinned fragment: a line that re-wraps to FEWER
// fragments must not pin a fragment that no longer exists.
func TestFontPin_FragmentClampedOnPinchOut(t *testing.T) {
TheLogic = nil
TheState = NewState()
defer func() {
TheState.appFontScale = 1.0
TheState.ScrollOffset = 0
TheState.Editor.ChunkedBuffer = nil
}()
w := NewWrapIndex(100)
for i := 0; i < 100; i++ {
w.Set(i, 3)
}
TheState.Editor.ChunkedBuffer = &ChunkedBuffer{WrapIndex: w}
// Pin line 5, fragment 2 (the third fragment).
TheState.appFontScale = 1.0
const m = 100.0
TheState.applyFontPin(5, 2, 0.25, m)
// Pinch out: the line now wraps into 2 fragments.
for i := 0; i < 100; i++ {
w.Set(i, 2)
}
TheState.appFontScale = 0.8
TheState.applyFontPin(5, 2, 0.25, m) // frag 2 must clamp to 1
lh := float64(EffectiveLineHeight())
u := (float64(TheState.ScrollOffset) + m) / lh
// The anchor is now line 5, its LAST fragment (index 1), 0.25 down it.
wantU := float64(w.VisualsBefore(5)+1) + 0.25
if math.Abs(u-wantU) > 1e-6 {
t.Errorf("anchor u=%v want %v (last remaining fragment of line 5)", u, wantU)
}
}

118
internal/editor/session.go
View File

@ -24,6 +24,7 @@ package editor
import (
"log"
"math"
"time"
"pad/internal/ui"
@ -56,12 +57,14 @@ type SessionState struct {
Cursor int // cursor byte offset
Scroll float64 // editor scroll offset, Dp
ScrollLine int // logical line at the viewport top (-1 = unknown)
ScrollSub float64 // Scroll's sub-line remainder, Dp (0 <= r < lineHeight)
ScrollSub float64 // Scroll's sub-line remainder as a FRACTION of the line height (0 <= f < 1); font-independent. Snapshots written before the pinch feature stored Dp instead; restore converts values > 1.
SelStart int // selection start byte (-1 = no selection)
SelEnd int // selection end byte, exclusive
FindQuery string // find bar query ("" = none)
FindVisible bool // find bar was open
FindCurByte int // start byte of the current find match (-1 = none)
// AppFontScale is the app-local pinch font scale (0 = default 1.0).
AppFontScale float64
}
// SetSessionSaver registers the callback that persists snapshots (the cmd
@ -106,7 +109,11 @@ func (l *Logic) SnapshotSession() SessionState {
if w := cb.WrapIndex; w != nil {
scrollLine = w.LineForVisual(int32(v0))
}
scrollSub = r0
// Store the sub-line remainder as a fraction of the line height so
// it stays valid if the font scale changes between save and restore
// (r0 < lh always, so the fraction is < 1 — that is also the legacy
// Dp discriminator used on restore).
scrollSub = r0 / float64(lh)
}
return SessionState{
File: e.Filename,
@ -119,6 +126,7 @@ func (l *Logic) SnapshotSession() SessionState {
FindQuery: f.Query,
FindVisible: f.Visible,
FindCurByte: curByte,
AppFontScale: float64(s.appFontScale),
}
}
@ -229,6 +237,17 @@ func (l *Logic) BeginRestore(s SessionState) {
// struct).
l.restoreScroll = ui.Dp(s.Scroll)
l.restoreScrollArmed = s.Scroll > 0
// The app-local font scale lands immediately, BEFORE any layout: the
// restore's line-based offset math reads it through EffectiveLineHeight.
if as := s.AppFontScale; as > 0 {
if as < MinAppFontScale {
as = MinAppFontScale
}
if as > MaxAppFontScale {
as = MaxAppFontScale
}
l.state.appFontScale = float32(as)
}
l.state.page = EditorPage
l.state.FocusedElementID = "editor_text"
l.state.justOpenedAt = time.Now()
@ -297,12 +316,19 @@ func (l *Logic) maybeApplyRestoreScroll() bool {
if cb := l.state.Editor.ChunkedBuffer; cb != nil && cb.WrapIndex != nil && s.ScrollLine < cb.WrapIndex.Len() {
base = float64(cb.WrapIndex.VisualsBefore(s.ScrollLine))
}
l.restoreScroll = ui.Dp(base*float64(lh) + s.ScrollSub)
sub := s.ScrollSub
if sub > 1 {
// Legacy snapshot: the sub-line remainder was stored in Dp, not
// as a fraction. Convert with the current line height (the
// error is a fraction of one sub-line line, at most).
sub = sub / float64(lh)
}
l.restoreScroll = ui.Dp(base*float64(lh) + sub*float64(lh))
// Arm the line-pin (see refreshRestorePin): wrap-count corrections
// landing below this line would shift the mapping and drag the
// viewport off the restored line while the index settles.
l.restoreScrollLine = s.ScrollLine
l.restoreScrollSub = float64(s.ScrollSub)
l.restoreScrollSub = sub // fraction of the line height
l.restorePinDeadline = time.Now().Add(restorePinTimeout)
} else {
l.restoreScrollLine = -1
@ -358,13 +384,95 @@ func (l *Logic) refreshRestorePin() {
// stale and would under-clamp the re-derived offset; the layout pass of
// the emitted frame clamps to the fresh value. An edit shrinking the
// file mid-pin is covered the same way.
off := ui.Dp(float64(w.VisualsBefore(l.restoreScrollLine))*float64(lh) + l.restoreScrollSub)
off := ui.Dp(float64(w.VisualsBefore(l.restoreScrollLine))*float64(lh) + l.restoreScrollSub*float64(lh))
if off != l.state.ScrollOffset {
l.state.ScrollOffset = off
l.emitFrame()
}
}
// fontPinTimeout bounds the pinch font-pin: rewrap corrections keep landing
// for a while after the last pinch frame (shaping lags the font change);
// after this long the user has moved on and the pin stands down.
const fontPinTimeout = 2 * time.Second
// setFontPin arms/updates the pinch font-pin from a pre-change capture
// (see captureContentPin: the glyph under the pinch center + offset from its
// baseline) and applies the IMMEDIATE anchor: the continuous content
// coordinate under the center scaled by the font ratio. Exact until the
// rewrap lands; the shaped-layout feedback (refreshFontPin) then snaps the
// pinned character exactly onto the center. Must be called on the logic
// goroutine.
func (l *Logic) setFontPin(pin contentPin, m float64, ratio float64) {
l.fontPin = pin
l.fontPinM = m
l.fontPinArmed = true
l.fontPinDeadline = time.Now().Add(fontPinTimeout)
s := l.state
s.rescaleScrollAnchored(ratio, float64(s.EditorRegion.Y)+m)
// No emitFrame here: the caller (input path or debug command) emits the
// frame carrying the new scale and the rescaled offset.
}
// refreshFontPin re-derives the scroll offset from a freshly shaped layout so
// the pinned content point stays under the pinch center (see
// refineContentPin). Runs on every layout feedback while armed: the font
// change's own re-shaping is the first, and rewrap corrections follow it. If
// the pinned byte is not in the shaped window (no layout yet, or the point
// is in a blank margin), the (line, fragment, sub-line) fallback anchor
// stands in. An edit landing since the capture (EditSeq mismatch) or leaving
// the editor page disarms the pin: its byte no longer names the same content.
// No MaxScroll clamp: the layout pass of the emitted frame clamps to the
// fresh value. Must be called on the logic goroutine.
func (l *Logic) refreshFontPin(fb ui.LayoutFeedback) {
if !l.fontPinArmed {
return
}
if time.Now().After(l.fontPinDeadline) {
l.fontPinArmed = false
return
}
s := l.state
if s.page != EditorPage || fb.EditSeq != l.fontPin.EditSeq {
l.fontPinArmed = false
return
}
l.fontPinDeadline = time.Now().Add(fontPinTimeout)
// Skip feedback shaped at a DIFFERENT scale: during a pinch every frame
// changes the scale, so all but the latest feedback carry layouts whose
// LineHeight no longer matches the geometry the pin computes in. A
// stale-scale layout would place the point by the old geometry for one
// frame (a visible jump) before the next corrects it.
if fb.GlyphLayout.LineHeight > 0 &&
math.Abs(float64(fb.GlyphLayout.LineHeight)-float64(EffectiveLineHeight())) > 0.5 {
return
}
old := s.ScrollOffset
vk := -1
if cb := s.Editor.ChunkedBuffer; cb != nil && cb.WrapIndex != nil && fb.WindowStartLine >= 0 {
vk = int(cb.WrapIndex.VisualsBefore(fb.WindowStartLine))
}
if l.fontPin.HaveGlyph && vk >= 0 {
if off, ok := refineContentPin(fb.GlyphLayout, vk, fb.WindowStartByte, l.fontPin.Dy, l.fontPinM, l.fontPin.Byte); ok {
s.ScrollOffset = off
} else {
s.applyFontPin(l.fontPin.Line, l.fontPin.Frag, l.fontPin.Sub, l.fontPinM)
}
} else {
s.applyFontPin(l.fontPin.Line, l.fontPin.Frag, l.fontPin.Sub, l.fontPinM)
}
if s.ScrollOffset != old {
l.emitFrame()
}
}
// releaseFontPin disarms the pinch font-pin: the user (a scroll, a debug
// command, a file switch) has taken over the viewport. Must be called on
// the logic goroutine.
func (l *Logic) releaseFontPin() {
l.fontPinArmed = false
}
// applyRestorePositions clamps the restored snapshot's cursor and selection
// to a file of n bytes and applies them. Must be called on the logic
// goroutine.

318
internal/editor/state.go
View File

@ -17,6 +17,15 @@ import (
// EditorFontSize is the font size used for editor text.
const EditorFontSize = 14 // unit.Sp
// App-local font-size scale bounds (pinch zoom), applied on top of the
// system user font scale. The scale itself is a continuous float32 — it is
// never rounded to a whole point value; the bounds only stop the pinch from
// leaving the usable range.
const (
MinAppFontScale = 0.5
MaxAppFontScale = 3.0
)
// EditorLineHeightScale is the baseline-to-baseline spacing multiplier.
const EditorLineHeightScale = 1.2
@ -190,6 +199,7 @@ type State struct {
PixelHeight int // raw pixel height from Gio ConfigEvent
scale float32
fontScale float32 // user font-size setting (PxPerSp/PxPerDp); 0 = unknown -> 1.0
appFontScale float32 // app-local pinch font scale (1.0 = default); 0 = unknown -> 1.0
page Page // current page (Browser or Editor)
WordWrap bool
ScrollOffset ui.Dp // vertical scroll position in Dp
@ -232,6 +242,7 @@ type State struct {
func NewState() *State {
return &State{
scale: 1.0,
appFontScale: 1.0,
page: BrowserPage, // Reverted to BrowserPage
WordWrap: true, // Enable word wrap by default
lastEvictionTime: time.Now(),
@ -278,17 +289,306 @@ func stateFontScale() float32 {
}
// EffectiveLineHeight is the editor line height in density-dp WITH the user
// font-size setting applied. The shaper draws baselines at
// Sp(EditorFontSize*LineHeightScale) physical px, which is
// EditorLineHeight()*fontScale density-dp. Every piece of geometry
// bookkeeping (window start, sub-line remainder, tap mapping, scroll
// clamping, cursor vertical move) must use this value rather than the raw
// EditorLineHeight; at a non-default font setting the two differ by the
// font-size setting AND the app-local pinch scale applied. The shaper draws
// baselines at Sp(EditorFontSize*appFontScale*LineHeightScale) physical px,
// which is EditorLineHeight()*effectiveFontScale density-dp. Every piece of
// geometry bookkeeping (window start, sub-line remainder, tap mapping,
// scroll clamping, cursor vertical move) must use this value rather than the
// raw EditorLineHeight; at a non-default font setting the two differ by the
// font scale, which would misplace taps by up to (fontScale-1) viewportfuls
// of lines and make scroll clamping stop short of (or run past) the file
// ends.
func EffectiveLineHeight() ui.Dp {
return EffectiveLineHeightAt(stateFontScale())
return EffectiveLineHeightAt(effectiveFontScale())
}
// effectiveFontScale is the TOTAL font scale of the rendered line pitch:
// the system user font scale times the app-local pinch scale. The system
// part is already folded into gtx.Metric on the render side; the logic side
// needs the product for its dp bookkeeping.
func effectiveFontScale() float32 {
fs := stateFontScale()
as := float32(1)
if TheState != nil && TheState.appFontScale > 0 {
as = TheState.appFontScale
}
return fs * as
}
// rescaleScrollAnchored scales the scroll offset by ratio while keeping the
// content point under app-local Y anchorY fixed on screen. The document is
// uniformly scaled by the font change (every line height and the sub-line
// offset scale by the same factor), so the content coordinate under the
// anchor scales by ratio; the new offset re-places that scaled coordinate
// under the same app point. With anchorY at the region top this degenerates
// to the plain top-anchor (new = old * ratio). A zero EditorRegion (not
// laid out yet) likewise degenerates to the top anchor.
func (s *State) rescaleScrollAnchored(ratio float64, anchorY float64) {
regionTop := float64(s.EditorRegion.Y)
dy := anchorY - regionTop
contentY := float64(s.ScrollOffset) + dy
s.ScrollOffset = ui.Dp(contentY*ratio - dy)
}
// glyphAtLocalPoint returns the index of the glyph a window-frame point
// (x, y in Dp; y relative to the window top, the same frame as GlyphLayout.Y)
// sits on: the display line identified from y, and on that line the last
// glyph whose X is at or before x. ok=false when the layout is empty, the
// line has no glyph, or x is in the left margin before the line's first
// glyph. (A point past the line's END still pins that line's last glyph:
// the content there is the line itself.)
func glyphAtLocalPoint(gl ui.GlyphLayout, x, y float64) (int, bool) {
lh := float64(gl.LineHeight)
if lh <= 0 || len(gl.ByteOffsets) == 0 {
return 0, false
}
line := int(y / lh)
if line < 0 {
line = 0
}
// The baseline of display line j sits in (j*lh, (j+1)*lh]; all glyphs on
// a line share one exact shaper value, so a range test finds the line's
// baseline.
base := -1.0
for _, gy := range gl.Y {
if f := float64(gy); f > float64(line)*lh && f <= float64(line+1)*lh {
base = f
break
}
}
if base < 0 {
return 0, false
}
best := -1
for i, gy := range gl.Y {
if float64(gy) != base {
continue
}
if float64(gl.X[i]) <= x+1e-9 {
best = i
}
}
return best, best >= 0
}
// contentPin is the anchor a pinch holds: a CONTENT point, not a layout
// point. Byte/Dy name the glyph (ABSOLUTE buffer byte) under the fingers and
// the point's offset from that glyph's baseline — both invariant under
// rewrap, where a visual line is not (a rewrapped fragment holds different
// text at the same fragment index). Line/Frag/Sub is the (logical line,
// fragment, sub-line) fallback anchor for frames without a shaped glyph
// under the point. EditSeq invalidates the pin on edits.
type contentPin struct {
Byte int
Dy float64
Line int
Frag int
Sub float64
HaveGlyph bool
EditSeq uint64
}
// invalidateShapedLayout drops the last shaped GlyphLayout. Call it whenever
// the SHAPING INPUTS change outside an edit (a font-scale change): the old
// layout's LineHeight/X/Y belong to the old size, and every consumer that
// falls back on it (window-start line, max scroll, tap mapping) would run
// the new scroll offset through the OLD line height for a frame or two —
// enough to put the shaped window ten thousand lines from the viewport.
// The next frame re-shapes and the feedback refills it; until then the
// logic-side geometry uses EffectiveLineHeight(), which tracks the scale.
func (s *State) invalidateShapedLayout() {
s.Editor.GlyphLayout = ui.GlyphLayout{}
}
// captureContentPin identifies the content point at region-relative (x, y)
// (dp from the editor region's left/top): the glyph under the point and the
// point's offset from its baseline, plus the (line, fragment, sub-line)
// fallback anchor. Must be called BEFORE the font change it will anchor.
func (s *State) captureContentPin(x, y float64) contentPin {
pin := contentPin{Byte: -1}
// Window-frame y (the GlyphLayout frame): the point's region-relative y
// plus the sub-line draw offset, the same convention as tapLocalY.
_, r := scrollVisualDecompose()
localY := y + r
gl := s.Editor.GlyphLayout
if i, ok := glyphAtLocalPoint(gl, x, localY); ok {
// ByteOffsets are window-relative; IMEWindowStartByte is the absolute
// byte of the window's first byte (the same value the Frame ships as
// WindowStartByte).
pin.Byte = s.Editor.IMEWindowStartByte + gl.ByteOffsets[i]
pin.Dy = localY - float64(gl.Y[i])
pin.HaveGlyph = true
}
if line, frag, sub, ok := s.captureFontPin(y); ok {
pin.Line, pin.Frag, pin.Sub = line, frag, sub
}
pin.EditSeq = s.Editor.EditSeq
return pin
}
// captureFontPin identifies the fallback layout anchor at region-relative Y
// m (dp from the top of the editor text region): the LOGICAL line under the
// point (through the current WrapIndex), the display line (wrap fragment) of
// that line the point is on, and the sub-line fraction within that fragment.
func (s *State) captureFontPin(m float64) (line, frag int, sub float64, ok bool) {
lh := float64(EffectiveLineHeight())
if lh <= 0 {
return 0, 0, 0, false
}
u := (float64(s.ScrollOffset) + m) / lh // continuous display-line coordinate under the point
k := int(u)
if k < 0 {
k = 0
}
sub = u - float64(k)
if cb := s.Editor.ChunkedBuffer; cb != nil && cb.WrapIndex != nil && k < cb.WrapIndex.Len() {
l := cb.WrapIndex.LineForVisual(int32(k))
base := int(cb.WrapIndex.VisualsBefore(l))
frag = k - base
if frag < 0 {
frag = 0
}
return l, frag, sub, true
}
return k, 0, sub, true // no wrap index: display line == logical line
}
// refineContentPin computes the scroll offset that places the pinned CONTENT
// point — absolute buffer byte byteOff, dy below its baseline — at
// region-relative Y m, given a freshly shaped layout (gl) for the window
// starting at windowStartByte (both from the same LayoutFeedback). The window
// top (layout y=0) sits at the top of the window's FIRST visual line, whose
// content coordinate is VisualsBefore(windowStartLine)*lh — the vk argument
// (NOT floor(shapedScroll/lh), which is a different line whenever the
// viewport top lands mid-way through a wrapped logical line) — so the glyph's
// content coordinate is vk*lh + Y + dy, and setting the offset to that minus
// m puts the point on the center exactly. This is what keeps the CHARACTER
// under the fingers when a rewrap moves it to a different fragment: the byte
// is invariant, its Y is read from the fresh layout. ok=false when the layout
// is unusable or the byte is not in the shaped window (then the caller falls
// back to the (line, frag, sub) anchor).
func refineContentPin(gl ui.GlyphLayout, vk, windowStartByte int, dy, m float64, byteOff int) (ui.Dp, bool) {
lh := gl.LineHeight
if lh <= 0 || len(gl.ByteOffsets) == 0 || byteOff < 0 || vk < 0 {
return 0, false
}
i := sort.Search(len(gl.ByteOffsets), func(i int) bool { return windowStartByte+gl.ByteOffsets[i] >= byteOff })
if i >= len(gl.ByteOffsets) || windowStartByte+gl.ByteOffsets[i] != byteOff {
return 0, false
}
contentY := float64(vk)*float64(lh) + float64(gl.Y[i]) + dy
return ui.Dp(contentY - m), true
}
// applyFontPin sets the scroll offset so the fallback anchor (logical line L,
// its frag-th display line, sub-line fraction sub within it) sits at
// region-relative Y m, under the CURRENT WrapIndex and line height. This is
// the pin's stand-in for frames without a shaped glyph under the point; it is
// re-runnable as wrap-count corrections land (the same mechanism as the
// restore line-pin, aimed at a point mid-viewport). No MaxScroll clamp here:
// the layout pass of the emitted frame clamps to the fresh value (a stale
// MaxScroll would under-clamp).
func (s *State) applyFontPin(line, frag int, sub, m float64) {
lh := float64(EffectiveLineHeight())
if lh <= 0 {
return
}
base := float64(line)
if cb := s.Editor.ChunkedBuffer; cb != nil && cb.WrapIndex != nil && line >= 0 && line < cb.WrapIndex.Len() {
base = float64(cb.WrapIndex.VisualsBefore(line))
// The line's fragment count may have shrunk (pinch out): keep the
// pinned fragment inside the line's new fragment range.
if line+1 < cb.WrapIndex.Len() {
count := int(cb.WrapIndex.VisualsBefore(line+1) - cb.WrapIndex.VisualsBefore(line))
if count > 0 && frag >= count {
frag = count - 1
}
}
}
s.ScrollOffset = ui.Dp((base+float64(frag)+sub)*lh - m)
}
// HandleFontPinch applies one frame's relative two-finger pinch factor to
// the app-local font scale (ui.FontPinchEvent, delivered by the renderer's
// pinch probe). The scale is a continuous float — multiplied by the
// per-frame distance ratio, clamped to [MinAppFontScale, MaxAppFontScale],
// never rounded — so the text size tracks the fingers smoothly. The content
// under the pinch CENTER (event's Center point) stays anchored: the Logic
// pins the logical line + fragment + sub-line under the center and
// re-derives the scroll offset under it as the font — and, a few frames
// later, the rewrap counts — change (see Logic.applyFontPinch).
func HandleFontPinch(data any) {
f, ok := data.(ui.FontPinchEvent)
log.Printf("PINCH logic HandleFontPinch scale=%.4f center=(%.0f,%.0f) ok=%v", f.Scale, f.Center.X, f.Center.Y, ok)
if !ok || f.Scale <= 0 {
return
}
s := TheState
// Region-relative (x, y) of the pinch center. Capture the CONTENT point
// the fingers are on (the glyph under the point + offset from its
// baseline) under the PRE-change layout; the scale change, then the
// re-derivation, keep that same content point under the center.
x := float64(f.Center.X - s.EditorRegion.X)
m := float64(f.Center.Y - s.EditorRegion.Y)
pin := s.captureContentPin(x, m)
old := s.appFontScale
if old <= 0 {
old = 1
}
ns := old * f.Scale
if ns < MinAppFontScale {
ns = MinAppFontScale
}
if ns > MaxAppFontScale {
ns = MaxAppFontScale
}
if ns == s.appFontScale {
return // no change: nothing to re-derive
}
s.appFontScale = ns
// The last shaped layout belongs to the old size (see
// invalidateShapedLayout): without this, the next frame computes its
// window start with the OLD line height and the new (rescaled) offset —
// a window ten thousand lines from the viewport — and the pin chases it.
s.invalidateShapedLayout()
ratio := float64(ns) / float64(old)
if TheLogic != nil {
TheLogic.releaseRestorePin() // the user takes over the viewport
TheLogic.setFontPin(pin, m, ratio) // re-derives the offset under it
} else {
// Test harness without a Logic: the immediate continuous anchor.
s.rescaleScrollAnchored(ratio, float64(s.EditorRegion.Y)+m)
}
}
// SetAppFontScale sets the app-local font scale to an absolute value
// (clamped to [MinAppFontScale, MaxAppFontScale]) with the VIEWPORT TOP as
// the anchor (no fingers are involved). Used by the one-shot debug command.
func SetAppFontScale(v float32) {
s := TheState
old := s.appFontScale
if old <= 0 {
old = 1
}
if v < MinAppFontScale {
v = MinAppFontScale
}
if v > MaxAppFontScale {
v = MaxAppFontScale
}
if v == s.appFontScale {
return // no change
}
s.appFontScale = v
// Same stale-layout hazard as HandleFontPinch: the window start for the
// next frame must be computed with the NEW line height.
s.invalidateShapedLayout()
ratio := float64(s.appFontScale) / float64(old)
s.rescaleScrollAnchored(ratio, float64(s.EditorRegion.Y))
if TheLogic != nil {
TheLogic.releaseRestorePin()
TheLogic.releaseFontPin()
}
}
// EffectiveLineHeightAt is EffectiveLineHeight for an explicit font scale
@ -352,6 +652,7 @@ func HandleScroll(data any) {
delta := data.(int) // pixels
if TheLogic != nil {
TheLogic.releaseRestorePin() // the user takes over the viewport
TheLogic.releaseFontPin()
}
TheState.ScrollOffset += ui.ToDp(ui.Px(delta), TheState.scale)
if TheState.ScrollOffset < 0 {
@ -2250,6 +2551,9 @@ func EditorLayout(screenWidth, screenHeight ui.Dp, wordWrap bool) []ui.Element {
}
}},
{Gesture: ui.SelDrag, Handler: HandleSelDragEvt},
// Two-finger pinch changes the app-local font scale continuously
// (the renderer owns the probe; see ui.Pinch).
{Gesture: ui.Pinch, Handler: HandleFontPinch},
},
)
// While the find bar is open, key focus belongs to the main-owned

18
internal/ui/element.go
View File

@ -900,6 +900,11 @@ const (
// The renderer registers the underlying gesture.Drag ops itself (it owns
// the handle geometry); this interaction just delivers the logic handler.
SelDrag
// Pinch is a two-finger pinch inside the element's text region. The
// renderer owns the probe (it needs raw two-pointer geometry that no
// single gesture primitive in Gio v0.10 provides); this interaction just
// delivers the logic handler, which receives FontPinchEvent.
Pinch
)
// Interaction pairs a gesture type with a handler function.
@ -950,6 +955,19 @@ type SelectionDragEvent struct {
// SelectionDragEnd is emitted when a selection/caret drag is released.
type SelectionDragEnd struct{}
// FontPinchEvent carries one frame's relative two-finger pinch factor
// (current inter-finger distance / previous frame's distance, both in px).
// The logic applies it as a multiplier to the app-local font scale; the
// value is a plain float32 ratio with no rounding, so the font size is
// continuous, never snapped to whole points. Center is the pinch midpoint
// (the average of the two fingers) in app-local window Dp — the same space
// as Point; the logic anchors the content under this point so it stays put
// while the font scales.
type FontPinchEvent struct {
Scale float32
Center Point
}
// MenuItem is one button in the selection menu.
// X/Y/W/H are relative to the Menu region.
type MenuItem struct {

510
internal/ui/pinch_test.go Normal file
View File

@ -0,0 +1,510 @@
package ui
import (
"math"
"testing"
"time"
"gioui.org/f32"
"gioui.org/io/pointer"
)
// pev builds a synthetic pointer event. Time is explicit: the Android driver
// supplies it per event and the freshness window runs on it.
func pev(kind pointer.Kind, id pointer.ID, x, y float32, at time.Duration) pointer.Event {
return pointer.Event{Kind: kind, PointerID: id, Position: f32.Point{X: x, Y: y}, Time: at}
}
// ms/s are time helpers so test timelines read as intended (a bare integer
// constant is nanoseconds — the first draft's "2000ms" was 2µs).
func ms(n int) time.Duration { return time.Duration(n) * time.Millisecond }
func s(n int) time.Duration { return time.Duration(n) * time.Second }
// runFrame feeds a batch of events (one frame's drain) through the tracker
// and returns the frame's factor, the grabs it asked for, and the
// survivor-finger scroll, mirroring consumePinchProbe.
func runFrame(t *pinchTracker, evts ...pointer.Event) (f float32, mid f32.Point, ok bool, grabs []pointer.ID, scroll int) {
for _, e := range evts {
if s := t.step(e); len(s.grabs) > 0 {
grabs = append(grabs, s.grabs...)
}
}
f, mid, ok, g2 := t.factor()
grabs = append(grabs, g2...)
scroll = t.survivorScroll()
return
}
// approx reports whether a and b are within 1e-3.
func approx(a, b float64) bool { return math.Abs(a-b) < 1e-3 }
// A clean two-finger pinch: the pair forms when BOTH fingers move (the
// two-mover rule), owes the spread that happened by then (baseline = press
// distance), then one factor per frame whose product telescopes to the total
// distance ratio — the property that makes the font track the fingers.
func TestTrackerPinchFactorSeries(t *testing.T) {
var tr pinchTracker
// Frame 1: both fingers down 200px apart. Pending: no pair, no grabs,
// no factor (a press alone is not a pinch).
_, _, ok, grabs, _ := runFrame(&tr,
pev(pointer.Press, 0, 100, 100, 0),
pev(pointer.Press, 1, 300, 100, 1),
)
if ok || len(grabs) != 0 {
t.Fatalf("press-only frame: ok=%v grabs=%v want nothing", ok, grabs)
}
// Frames 2-4: the pair spreads 200 -> 240 -> 300 -> 250. Frame 2 is the
// formation: both fingers moved 20px (movers), grabs issued, and the
// owed factor is measured against the PRESS distance (200).
f2, _, ok2, g2, _ := runFrame(&tr,
pev(pointer.Drag, 0, 80, 100, 10),
pev(pointer.Drag, 1, 320, 100, 11),
)
if !ok2 || !approx(float64(f2), 240.0/200) {
t.Fatalf("f2=%v ok=%v want %v", f2, ok2, 240/200)
}
if len(g2) != 2 {
t.Fatalf("formation frame must grab both fingers, got %v", g2)
}
f3, mid3, ok3, _, _ := runFrame(&tr,
pev(pointer.Drag, 0, 50, 100, 20),
pev(pointer.Drag, 1, 350, 100, 21),
)
if !ok3 || !approx(float64(f3), 300.0/240) {
t.Fatalf("f3=%v ok=%v want %v", f3, ok3, 300/240)
}
if mid3 != (f32.Point{X: 200, Y: 100}) {
t.Fatalf("mid3=%v want (200,100)", mid3)
}
f4, _, ok4, _, _ := runFrame(&tr,
pev(pointer.Drag, 0, 125, 100, 30),
pev(pointer.Drag, 1, 375, 100, 31),
)
if !ok4 || !approx(float64(f4), 250.0/300) {
t.Fatalf("f4=%v ok=%v want %v", f4, ok4, 250.0/300)
}
total := f2 * f3 * f4
if !approx(float64(total), 250.0/200) {
t.Fatalf("product=%v want total ratio %v", total, 250/200)
}
}
// A single finger — press, hold, long scroll — must never produce a factor
// or a grab. This is the device regression "one-finger scroll changes the
// font": the old code paired the scroll finger with whatever stale pointer
// was in the map.
func TestTrackerSingleFingerNeverScales(t *testing.T) {
var tr pinchTracker
// A pinch earlier in the session...
runFrame(&tr,
pev(pointer.Press, 0, 100, 100, 0),
pev(pointer.Press, 1, 300, 100, 1),
)
runFrame(&tr,
pev(pointer.Drag, 0, 80, 100, 10),
pev(pointer.Drag, 1, 320, 100, 11),
)
runFrame(&tr,
pev(pointer.Release, 0, 80, 100, 20),
pev(pointer.Release, 1, 320, 100, 21),
)
// ...now a single finger scrolls for many frames.
for i := 0; i < 20; i++ {
y := float32(400 - i*20)
_, _, ok, grabs, scroll := runFrame(&tr,
pev(pointer.Drag, 5, 200, y, time.Duration(100+i*10)))
if ok || len(grabs) != 0 || scroll != 0 {
t.Fatalf("frame %d: single finger produced factor ok=%v grabs=%v scroll=%v",
i, ok, grabs, scroll)
}
}
// Fresh pointer IDs (Android resets them after full release) — still nothing.
_, _, ok, grabs, _ := runFrame(&tr,
pev(pointer.Press, 0, 200, 500, 500),
)
if ok || len(grabs) != 0 {
t.Fatalf("fresh single finger: ok=%v grabs=%v", ok, grabs)
}
for i := 0; i < 10; i++ {
_, _, ok, grabs, _ = runFrame(&tr,
pev(pointer.Drag, 0, 200, float32(500-i*15), time.Duration(510+i*10)))
if ok || len(grabs) != 0 {
t.Fatalf("scroll frame %d: ok=%v grabs=%v", i, ok, grabs)
}
}
}
// Device regression "sudden dramatic zoom before pinching": a finger that
// has been RESTING (palm edge, parked pinky) must not be paired with a new
// one. The freshness window prunes it; the pair forms only from fresh
// fingers.
func TestTrackerRestingFingerNotPaired(t *testing.T) {
var tr pinchTracker
// Palm rests at t=0.
_, _, ok, grabs, _ := runFrame(&tr,
pev(pointer.Press, 3, 600, 800, 0))
if ok || len(grabs) != 0 {
t.Fatal("one resting finger must not pinch")
}
// At t=2s the thumb lands: the palm is stale (>300ms) and pruned.
_, _, ok, grabs, _ = runFrame(&tr,
pev(pointer.Press, 0, 100, 100, s(2)),
pev(pointer.Drag, 3, 605, 800, s(2)+ms(1)), // palm still there, drifting
)
if ok || len(grabs) != 0 {
t.Fatalf("resting palm + new finger must not form a pair (ok=%v grabs=%v)", ok, grabs)
}
// At t=2.1s the index lands: two fresh fingers -> pending (formation
// waits for a third finger or the first movement; the palm is stale).
_, _, _, grabs, _ = runFrame(&tr,
pev(pointer.Press, 1, 300, 100, s(2)+ms(100)))
if len(grabs) != 0 {
t.Fatalf("pending pair must not grab yet, grabs=%v", grabs)
}
// The palm keeps drifting and the pair starts moving: the pair is
// (thumb, index) and its distance is unaffected by the palm.
f, _, ok, grabs, _ := runFrame(&tr,
pev(pointer.Drag, 3, 640, 810, s(2)+ms(200)), // palm moves (pruned, ignored)
pev(pointer.Drag, 0, 80, 100, s(2)+ms(201)),
pev(pointer.Drag, 1, 320, 100, s(2)+ms(202)),
)
if len(grabs) != 2 {
t.Fatalf("thumb+index must form the pair on movement, grabs=%v", grabs)
}
if !ok || !approx(float64(f), 240.0/200) {
t.Fatalf("f=%v ok=%v want %v (palm must not enter the distance)", f, ok, 240/200)
}
}
// Device regression "pinch dies and becomes scroll": the pair must survive
// finger movement past the scroll slop. In the router this is guaranteed by
// the grabs (scroll is dropped from the pair's path); here we assert the
// state machine keeps emitting factors for a pair that moves a lot, and
// ignores drags of pointers that are not the pair.
func TestTrackerPinchSurvivesLargeMoves(t *testing.T) {
var tr pinchTracker
runFrame(&tr,
pev(pointer.Press, 0, 100, 100, 0),
pev(pointer.Press, 1, 300, 100, 1))
// Big outward moves (far past the ~30px scroll slop), 5 frames.
want := []float32{1.5, 1.4, 1.3, 1.2, 1.1}
d := 200.0
for i := 0; i < 5; i++ {
d *= float64(want[i])
half := d / 2
_, _, ok, _, _ := runFrame(&tr,
pev(pointer.Drag, 0, 200-float32(half), 100, time.Duration(10+i*10)),
pev(pointer.Drag, 1, 200+float32(half), 100, time.Duration(11+i*10)),
)
if !ok {
t.Fatalf("frame %d: factor stopped (pinch died)", i)
}
}
}
// Lifting one finger of the pair: no more factors (the pair is gone), the
// survivor's drags come out as scroll (forwarded), and returning a second
// finger re-forms the pair with a fresh baseline.
func TestTrackerSurvivorScrollAndReform(t *testing.T) {
var tr pinchTracker
runFrame(&tr,
pev(pointer.Press, 0, 100, 100, 0),
pev(pointer.Press, 1, 300, 100, 1))
// Pinch in a bit.
runFrame(&tr,
pev(pointer.Drag, 0, 80, 100, 10),
pev(pointer.Drag, 1, 320, 100, 11))
// Finger 0 lifts. Finger 1 survives.
_, _, ok, _, _ := runFrame(&tr,
pev(pointer.Release, 0, 80, 100, 20))
if ok {
t.Fatal("broken pair must not emit a factor")
}
// The survivor scrolls: 40px up over two frames -> +40, +25.
_, _, _, _, scroll := runFrame(&tr,
pev(pointer.Drag, 1, 320, 60, 30))
if scroll != 40 {
t.Fatalf("survivor scroll=%d want 40", scroll)
}
_, _, _, _, scroll = runFrame(&tr,
pev(pointer.Drag, 1, 320, 35, 40))
if scroll != 25 {
t.Fatalf("survivor scroll=%d want 25", scroll)
}
// A second finger returns: CANDIDATE for a re-form; the pair re-forms
// only when the new finger MOVES (two-mover rule — a palm resting on
// the survivor is not a pinch).
_, _, ok, grabs, _ := runFrame(&tr,
pev(pointer.Press, 2, 50, 35, 50))
if ok || len(grabs) != 0 {
t.Fatalf("candidate press must not re-form yet (ok=%v grabs=%v)", ok, grabs)
}
f, _, ok, grabs, _ := runFrame(&tr,
pev(pointer.Drag, 1, 320, 35, 60),
pev(pointer.Drag, 2, 30, 35, 61)) // 20px from press: a real second finger
if len(grabs) != 1 || grabs[0] != 2 {
t.Fatalf("re-form grabs=%v want [2] (only the new finger)", grabs)
}
// Baseline = the distance at the candidate's PRESS (270 = 320-50); the
// spread to 290 by re-form time is owed, not lost.
if !ok || !approx(float64(f), 290.0/270) {
t.Fatalf("post-reform f=%v ok=%v want %v", f, ok, 290/270)
}
// Survivor lifts: fully idle again.
_, _, ok, _, scroll = runFrame(&tr,
pev(pointer.Release, 1, 320, 35, 70))
if ok || scroll != 0 {
t.Fatal("idle state must emit nothing")
}
}
// Device regression "dramatic zoom before the pinch starts": a NEW pinch
// must never emit a factor on its formation frame, even if the previous
// pinch ended at a very different distance (stale baseline).
func TestTrackerFreshBaselineEachPinch(t *testing.T) {
var tr pinchTracker
// Pinch A at ~200px.
runFrame(&tr,
pev(pointer.Press, 0, 100, 100, 0),
pev(pointer.Press, 1, 300, 100, 1))
runFrame(&tr,
pev(pointer.Release, 0, 100, 100, 10),
pev(pointer.Release, 1, 300, 100, 11))
// Pinch B starts 500px apart (the fingers landed far apart). The old
// code would have emitted 500/200 = 2.5x on the first frame.
_, _, ok, _, _ := runFrame(&tr,
pev(pointer.Press, 0, 0, 100, 100),
pev(pointer.Press, 1, 500, 100, 101))
if ok {
t.Fatal("second pinch's formation frame must not emit a factor (stale baseline)")
}
f, _, ok, _, _ := runFrame(&tr,
pev(pointer.Drag, 0, -20, 100, 110),
pev(pointer.Drag, 1, 520, 100, 111))
if !ok || !approx(float64(f), 540.0/500) {
t.Fatalf("f=%v ok=%v want %v (baseline = this pinch's own start)", f, ok, 540/500)
}
}
// A third finger landing DURING an active pinch is ignored (palm rest): the
// pair is fixed, its distance is untouched, and no extra grabs are issued.
func TestTrackerExtraFingerDuringPinchIgnored(t *testing.T) {
var tr pinchTracker
runFrame(&tr,
pev(pointer.Press, 0, 100, 100, 0),
pev(pointer.Press, 1, 300, 100, 1))
// Palm lands and sits.
_, _, ok, grabs, _ := runFrame(&tr,
pev(pointer.Press, 4, 700, 900, 10))
if ok || len(grabs) != 0 {
t.Fatalf("extra finger during pinch: ok=%v grabs=%v", ok, grabs)
}
// Palm drifts a bit; the two pinch fingers move (forming the pair); the
// palm is not in the pair and its distance is untouched.
f, _, ok, grabs, _ := runFrame(&tr,
pev(pointer.Drag, 4, 710, 910, 20), // 14px: a mover, but its distance to the fingers changes little
pev(pointer.Drag, 0, 80, 100, 21),
pev(pointer.Drag, 1, 320, 100, 22))
if len(grabs) != 2 {
t.Fatalf("pinch fingers must form the pair, grabs=%v", grabs)
}
if !ok || !approx(float64(f), 240.0/200) {
t.Fatalf("f=%v ok=%v want %v", f, ok, 240/200)
}
// Palm lifts: still nothing.
_, _, ok, _, _ = runFrame(&tr,
pev(pointer.Release, 4, 750, 950, 30))
if ok {
t.Fatal("palm release must not emit a factor")
}
}
// A global cancel (app switch) breaks the pair with no survivor.
func TestTrackerCancelBreaksPair(t *testing.T) {
var tr pinchTracker
runFrame(&tr,
pev(pointer.Press, 0, 100, 100, 0),
pev(pointer.Press, 1, 300, 100, 1))
runFrame(&tr,
pev(pointer.Drag, 0, 80, 100, 10),
pev(pointer.Drag, 1, 320, 100, 11))
// App switch: cancels for both pointers.
_, _, ok, _, scroll := runFrame(&tr,
pev(pointer.Cancel, 0, 0, 0, 20),
pev(pointer.Cancel, 1, 0, 0, 21))
if ok || scroll != 0 {
t.Fatal("cancel must break the pair cleanly")
}
// The cancelled drags must not resurrect anything.
_, _, ok, grabs, _ := runFrame(&tr,
pev(pointer.Drag, 0, 70, 100, 30))
if ok || len(grabs) != 0 {
t.Fatal("post-cancel drags must be inert")
}
}
// A finger that leaves the region pre-pinch is no longer a candidate:
// press-leave-press must not form a pair.
func TestTrackerLeaveCancelsCandidate(t *testing.T) {
var tr pinchTracker
runFrame(&tr,
pev(pointer.Press, 0, 100, 100, 0))
runFrame(&tr,
pev(pointer.Drag, 0, 500, 1500, 10), // leaves the region
pev(pointer.Leave, 0, 500, 1500, 11))
_, _, ok, grabs, _ := runFrame(&tr,
pev(pointer.Press, 1, 300, 100, 20))
if ok || len(grabs) != 0 {
t.Fatalf("leaving finger must not pair with a new one (ok=%v grabs=%v)", ok, grabs)
}
}
// A pair broken by one finger lifting, then the survivor lifting too: the
// state must be fully idle (a later single-finger scroll emits nothing).
func TestTrackerFullyIdleAfterSurvivorLift(t *testing.T) {
var tr pinchTracker
runFrame(&tr,
pev(pointer.Press, 0, 100, 100, 0),
pev(pointer.Press, 1, 300, 100, 1))
runFrame(&tr,
pev(pointer.Release, 0, 100, 100, 10))
runFrame(&tr,
pev(pointer.Drag, 1, 300, 60, 20)) // survivor scrolls
runFrame(&tr,
pev(pointer.Release, 1, 300, 60, 30))
for i := 0; i < 5; i++ {
_, _, ok, grabs, scroll := runFrame(&tr,
pev(pointer.Drag, 1, 300, float32(60-i*20), time.Duration(40+i*10)))
if ok || len(grabs) != 0 || scroll != 0 {
t.Fatalf("frame %d: not idle (ok=%v grabs=%v scroll=%v)", i, ok, grabs, scroll)
}
}
}
// The sanity clamp: a factor outside 0.1..10 (teleporting pair) is dropped
// and the baseline advances, so one bad frame cannot jump the font.
func TestTrackerSanityClamp(t *testing.T) {
var tr pinchTracker
runFrame(&tr,
pev(pointer.Press, 0, 100, 100, 0),
pev(pointer.Press, 1, 300, 100, 1))
// Teleport: finger 0 jumps 3000px (ID-reuse noise) while finger 1 moves
// 20px: both are movers, the pair forms, distance 200 -> 2800, factor
// 14x — must be dropped, baseline advanced.
f, _, ok, _, _ := runFrame(&tr,
pev(pointer.Drag, 0, 3100, 100, ms(10)),
pev(pointer.Drag, 1, 320, 100, ms(11)))
if ok {
t.Fatalf("implausible factor emitted: %v", f)
}
// Next frame is sane relative to the NEW baseline (2800px).
f, _, ok, _, _ = runFrame(&tr,
pev(pointer.Drag, 0, 3050, 100, ms(20)),
pev(pointer.Drag, 1, 325, 100, ms(21)))
if !ok || !approx(float64(f), 2725.0/2780) {
t.Fatalf("f=%v ok=%v want %v", f, ok, 2725.0/2780)
}
}
// A slow frame: the whole pinch (formation + spread + both releases) can
// arrive in ONE frame's drain (the Android driver replays historical samples
// and a low frame rate batches events). The factors must still be owed:
// the formation-frame movement relative to the formation distance, and the
// final movement settled when the pair breaks.
func TestTrackerSlowFrameFullPinch(t *testing.T) {
var tr pinchTracker
// Frame 1: both Presses (200px apart) and drags out to 300px, all in
// one drain. The pair forms at 200 and moved to 300 this frame:
// factor 300/200 owed.
f, _, ok, grabs, _ := runFrame(&tr,
pev(pointer.Press, 0, 100, 100, 0),
pev(pointer.Press, 1, 300, 100, 1),
pev(pointer.Drag, 0, 50, 100, 2),
pev(pointer.Drag, 1, 350, 100, 3))
if !ok || !approx(float64(f), 300.0/200) {
t.Fatalf("f1=%v ok=%v want %v (formation-frame movement is owed)", f, ok, 300.0/200)
}
if len(grabs) != 2 {
t.Fatalf("grabs=%v want 2", grabs)
}
// Frame 2: spread to 400px, then both fingers lift — same drain. The
// 400/300 factor must be settled at the break, not lost.
f, _, ok, _, _ = runFrame(&tr,
pev(pointer.Drag, 0, 0, 100, 10),
pev(pointer.Drag, 1, 400, 100, 11),
pev(pointer.Release, 0, 0, 100, 12),
pev(pointer.Release, 1, 400, 100, 13))
if !ok || !approx(float64(f), 400.0/300) {
t.Fatalf("f2=%v ok=%v want %v (break settles the owed factor)", f, ok, 400.0/300)
}
// Fully idle after the break.
if _, _, ok, _, scroll := runFrame(&tr,
pev(pointer.Drag, 1, 400, 80, 20)); ok || scroll != 0 {
t.Fatal("tracker must be idle after both releases")
}
}
// A stationary pair that forms and then breaks without moving owes nothing.
func TestTrackerSlowFrameNoMovement(t *testing.T) {
var tr pinchTracker
if _, _, ok, _, _ := runFrame(&tr,
pev(pointer.Press, 0, 100, 100, 0),
pev(pointer.Press, 1, 300, 100, 1),
pev(pointer.Release, 0, 100, 100, 2),
pev(pointer.Release, 1, 300, 100, 3)); ok {
t.Fatal("stationary pair that breaks must not emit a factor")
}
}
// The worst case (a ~1fps emulator frame): the ENTIRE pinch — formation,
// spread, and both releases — lands in one frame's drain. The owed factor
// is settled at the break against the formation distance.
func TestTrackerBornAndDeadInOneFrame(t *testing.T) {
var tr pinchTracker
f, _, ok, grabs, _ := runFrame(&tr,
pev(pointer.Press, 0, 100, 100, 0),
pev(pointer.Press, 1, 300, 100, 1),
pev(pointer.Drag, 0, 0, 100, 2),
pev(pointer.Drag, 1, 400, 100, 3),
pev(pointer.Release, 0, 0, 100, 4),
pev(pointer.Release, 1, 400, 100, 5))
if !ok || !approx(float64(f), 400.0/200) {
t.Fatalf("f=%v ok=%v want %v", f, ok, 400.0/200)
}
if len(grabs) != 2 {
t.Fatalf("grabs=%v want 2", grabs)
}
if _, _, ok, _, _ := runFrame(&tr,
pev(pointer.Drag, 1, 400, 80, 10)); ok {
t.Fatal("idle tracker must not emit")
}
}
// Palm-first three-finger: the palm edge is resting and still, then the two
// pinch fingers land (all within the freshness window). The pair must be the
// two NEWEST fingers — the pinch pair — not the palm.
func TestTrackerPalmFirstThreeFingers(t *testing.T) {
var tr pinchTracker
// Palm rests at t=0 (still), pinch fingers land 80/120ms later.
// Pending: no pair yet (movement decides).
_, _, _, grabs, _ := runFrame(&tr,
pev(pointer.Press, 3, 720, 400, 0),
pev(pointer.Press, 0, 570, 1200, ms(80)),
pev(pointer.Press, 1, 870, 1200, ms(120)))
if len(grabs) != 0 {
t.Fatalf("pending: no grabs yet, got %v", grabs)
}
// The pinch spreads 300 -> 596 while the palm stays put: the two MOVERS
// (the pinch fingers) form the pair, and the factor must track the
// pinch pair — not the palm.
f, _, ok, grabs, _ := runFrame(&tr,
pev(pointer.Drag, 0, 422, 1200, ms(200)),
pev(pointer.Drag, 1, 1018, 1200, ms(201)),
pev(pointer.Drag, 3, 720, 400, ms(202))) // palm still
if len(grabs) != 2 || (grabs[0] != 0 && grabs[0] != 1) || (grabs[1] != 0 && grabs[1] != 1) {
t.Fatalf("pair must be the two pinch fingers, grabs=%v", grabs)
}
if !ok || !approx(float64(f), 596.0/300) {
t.Fatalf("f=%v ok=%v want %v (palm must not enter the distance)", f, ok, 596.0/300)
}
}

470
internal/ui/pinch_tracker.go Normal file
View File

@ -0,0 +1,470 @@
package ui
import (
"math"
"time"
"gioui.org/f32"
"gioui.org/io/pointer"
)
// pinchFreshWindow is how recently BOTH pair fingers must have pressed for a
// pinch to start. A finger that has been resting (palm edge, pinky parked on
// the screen) is not a pinch candidate; without this window a resting third
// finger could be paired with a new one and the font would follow a
// single-finger scroll.
const pinchFreshWindow = 300 * time.Millisecond
// pinchMoveEps is how far (window px, from the press position) a pending
// finger must travel to count as "moving" for pair formation. It filters
// touch jitter while still registering an intentional pinch start quickly.
const pinchMoveEps = 10.0
// Pair formation requires TWO MOVING fingers, and nothing else. A pinch is
// two fingers moving apart (or together); a scroll is one finger moving;
// a resting palm is a finger that never moves. No static rule about which
// finger is "the palm" (first? last? still?) survives both palm-first and
// palm-last hand lands — movement is the only signal that works for both.
// While 2-3 fresh fingers are down the tracker is "pending": it forms the
// pair the moment two of them have moved more than pinchMoveEps from where
// they pressed (the pair = those two). A lone mover is a scroll, never a
// pinch, and cannot form a pair; the pending state cancels on any release
// that leaves fewer than two fingers.
// pinchTracker is the pure state machine for the editor's two-finger pinch
// (font scaling). It is independent of the Gio input router so the whole
// gesture — including the failure modes found on device (pair switching,
// stale baselines, scroll-grab starvation) — is unit-testable with synthetic
// pointer-event sequences.
//
// Design: the pair is EXPLICIT and stable. When two fresh fingers are down
// inside the region, they are named as the pair and the adapter issues
// pointer.GrabCmds for both (exclusive event delivery: releases always
// arrive, even off-clip, and scroll/click are dropped with a Cancel). The
// per-frame factor is current pair distance / previous frame's pair
// distance. Neither the pair's composition nor its baseline is ever
// re-derived from whatever pointers happen to be present — that re-derivation
// (two lowest IDs + stale prevDist) is what made single-finger scrolls scale
// the font on the phone.
type pinchTracker struct {
// Pre-pinch ("pending"): fresh fingers down in the region, no pair yet.
// The pair forms when TWO of them have moved more than pinchMoveEps
// from where they pressed (see the comment above the constants).
observed map[pointer.ID]f32.Point
observedAt map[pointer.ID]time.Duration
// Position each pending finger had when it PRESSED: the formation
// baseline (formDist) is the press distance, so a pinch that has
// already spread by the time the pair starts owes that spread; and the
// displacement from here is the "mover" measurement.
pressPos map[pointer.ID]f32.Point
// Pending fingers that released in the CURRENT frame's drain, held lazy
// until factor() (which forms the pair — and may break it again — at
// the fingers' final positions). A normal active-pair release breaks
// immediately in step(); this map is only for releases that arrive
// while the pair is still pending.
released map[pointer.ID]bool
// Reform candidate: after a pair broke, the survivor is still grabbed
// and a NEW finger has landed. The pair re-forms (survivor + new) only
// when the new finger moves (pinchMoveEps) — the same two-mover rule;
// a palm landing on the survivor is not a pinch. reformPos is where
// the candidate PRESSED (the mover reference).
reformID pointer.ID
reformPos f32.Point
reformBase float32 // pair distance at the candidate's press
reformHave bool
// The active (grabbed) pair.
pair [2]pointer.ID
pos [2]f32.Point
on bool
prevDist float32
// The pair's distance at the moment of (re)formation: the distance of
// the two press positions. The Android driver replays
// historical samples, so drags of the forming pair can land in the same
// frame as the pair's formation; movement relative to the formation
// distance is real and owed.
formDist float32
// Set on the pair's formation frame (start or re-form): the baseline
// is being established, so factor() emits nothing unless the pair
// moved from its formation positions this frame.
fresh bool
// The pair broke (a finger lifted) during the current frame's drain.
// factor() runs AFTER the drain, so on a slow frame the pair's drags
// and the breaking release arrive together and the frame's factor
// would be lost (on==false by the time factor() runs). The factor is
// settled here instead.
brokeFactor float32
brokeMid f32.Point
// A pair finger was released while the other is still down (still
// grabbed by the probe; Gio v0.10 has no release-grab). The survivor's
// drags are forwarded as scroll so the finger is not dead.
survOn bool
survID pointer.ID
survPos f32.Point
// Accumulated survivor scroll for the current frame (window px,
// gesture.Scroll convention: positive = content scrolls up).
survScroll int
}
// pinchStep is the outcome of one pointer event.
type pinchStep struct {
// Grabs the adapter must issue (pointer.GrabCmd per ID). Only the
// survivor re-form path emits grabs per event; initial pair formation
// is decided in factor() (after the whole frame's events).
grabs []pointer.ID
}
// step feeds one pointer event into the tracker. The per-frame factor is not
// part of the step result: the adapter calls factor() once after draining
// the frame's events, because the Android driver replays historical samples
// (several drags per frame) and the font must see one factor per frame.
func (t *pinchTracker) step(pe pointer.Event) pinchStep {
var s pinchStep
id := pe.PointerID
switch pe.Kind {
case pointer.Press:
switch {
case t.on:
// Extra finger during an active pinch (palm rest, third
// finger): ignore — the pair is fixed.
case t.survOn && t.reformHave && id == t.reformID:
// The reform candidate lifted before moving: not a pinch.
t.reformHave = false
case t.survOn:
// A new finger lands while the survivor is down: CANDIDATE for
// a re-form; it must MOVE to become the pair (the same two-mover
// rule — a palm resting on the survivor is not a pinch). The
// formation baseline is the distance at this PRESS (any spread
// by re-form time is owed).
t.reformID = id
t.reformPos = pe.Position
t.reformBase = pairDist(t.survPos, pe.Position)
t.reformHave = true
default:
if t.observed == nil {
t.observed = make(map[pointer.ID]f32.Point)
t.observedAt = make(map[pointer.ID]time.Duration)
t.pressPos = make(map[pointer.ID]f32.Point)
}
t.observed[id] = pe.Position
t.observedAt[id] = pe.Time
t.pressPos[id] = pe.Position
// Prune stale fingers (resting palm/pinky); a pruned palm is
// out of the game entirely.
t.pruneStale(pe.Time)
// (Two or more fingers down is the PENDING state; the pair
// forms only when two of them are movers, decided in factor().)
}
case pointer.Drag:
switch {
case t.on:
if i := t.pairIndex(id); i >= 0 {
t.pos[i] = pe.Position
}
case t.survOn && id == t.survID:
t.survScroll += int(t.survPos.Y - pe.Position.Y)
t.survPos = pe.Position
case t.survOn && t.reformHave && id == t.reformID:
// The reform candidate moves: if it clears the epsilon from
// where it pressed it is a real second finger — re-form the
// pair (survivor + new).
dx, dy := pe.Position.X-t.reformPos.X, pe.Position.Y-t.reformPos.Y
if dx*dx+dy*dy > pinchMoveEps*pinchMoveEps {
t.pair = [2]pointer.ID{t.survID, id}
t.pos = [2]f32.Point{t.survPos, pe.Position}
t.survOn = false
t.reformHave = false
t.on = true
t.fresh = true
t.formDist = t.reformBase
s.grabs = []pointer.ID{id} // survivor already grabbed
}
default:
if _, ok := t.pressPos[id]; ok {
t.observed[id] = pe.Position
// Formation is NOT decided here: a per-event decision would
// lock in the first mover pair seen (e.g. (finger, drifting
// palm)) before the second pinch finger's drag lands in the
// same drain. factor() decides, after the full frame.
}
}
case pointer.Release, pointer.Cancel:
switch {
case t.on:
if i := t.pairIndex(id); i >= 0 {
t.breakPair(i)
}
case t.survOn && id == t.survID:
// The survivor lifts too: fully idle (the candidate, if any,
// was a lone finger — never a pair).
t.survOn = false
t.reformHave = false
default:
// Pending finger goes up: held LAZY until factor() — on a slow
// frame the whole pinch (drags AND releases) can land in one
// drain, and the pair must form at the fingers' final positions
// before the break settles the owed factor.
if _, ok := t.observed[id]; ok {
t.markReleased(id)
}
}
case pointer.Leave:
// Only meaningful pre-pinch: a finger that leaves the editor
// region is no longer a pinch candidate. (A grabbed pair finger
// keeps delivering through the grab; its Leave is ignored.)
if t.survOn && t.reformHave && id == t.reformID {
t.reformHave = false
} else if t.observed != nil {
if _, ok := t.observed[id]; ok {
t.markReleased(id)
}
}
}
return s
}
// breakPair settles the frame's owed factor (if any) and breaks the pair.
// The surviving finger stays grabbed by the probe (no release-grab in v0.10)
// and becomes the survivor — UNLESS it too released in this same drain
// (slow frame: a born-and-dead pair), in which case the tracker goes fully
// idle. Settling here (before factor() runs) is what keeps the factor on a
// frame where the drags and the breaking release arrive together.
func (t *pinchTracker) breakPair(i int) {
base := t.prevDist
if base == 0 && t.fresh {
base = t.formDist
}
if d := pairDist(t.pos[0], t.pos[1]); d > 0 && base > 0 && d != base {
if f := d / base; f >= 0.1 && f <= 10 {
t.brokeFactor = f
t.brokeMid = pairMid(t.pos[0], t.pos[1])
}
}
t.on = false
t.prevDist = 0
other := 1 - i
if !t.released[t.pair[other]] {
t.survOn = true
t.survID = t.pair[other]
t.survPos = t.pos[other]
}
}
// markReleased lazily records a pending finger's release (see the Release
// case). Cleared by factor() after it has formed (and possibly broken) the
// pair.
func (t *pinchTracker) markReleased(id pointer.ID) {
if t.released == nil {
t.released = make(map[pointer.ID]bool)
}
t.released[id] = true
}
// dropObserved removes a pending finger for good.
func (t *pinchTracker) dropObserved(id pointer.ID) {
if t.observed == nil {
return
}
delete(t.observed, id)
delete(t.observedAt, id)
delete(t.pressPos, id)
delete(t.released, id)
}
// movers returns the pending fingers that moved more than pinchMoveEps from
// where they pressed, most-displaced first.
func (t *pinchTracker) movers() []pointer.ID {
type dm struct {
id pointer.ID
d float64
}
var list []dm
for id, pp := range t.pressPos {
dx := float64(t.observed[id].X - pp.X)
dy := float64(t.observed[id].Y - pp.Y)
if d := math.Sqrt(dx*dx + dy*dy); d > pinchMoveEps {
list = append(list, dm{id, d})
}
}
for i := 0; i < len(list); i++ {
for j := i + 1; j < len(list); j++ {
if list[j].d > list[i].d {
list[i], list[j] = list[j], list[i]
}
}
}
out := make([]pointer.ID, 0, len(list))
for _, e := range list {
out = append(out, e.id)
}
return out
}
// tryFormPair forms the pair from the pending fingers if at least TWO are
// movers (two moving fingers = a pinch; one = a scroll, never a pair). Of
// the mover pairs, the one whose DISTANCE changed most wins: a pinch's pair
// distance changes, while a drifting palm's distance to a finger changes
// little. A mover pair whose distance does not change (fingers moving in
// unison) is a slide, not a pinch. Returns the IDs to grab, or nil when no
// pair forms.
func (t *pinchTracker) tryFormPairAny() []pointer.ID {
m := t.movers()
if len(m) < 2 {
return nil
}
bestA, bestB := m[0], m[1]
bestChange := -1.0
for i := 0; i < len(m); i++ {
for j := i + 1; j < len(m); j++ {
a, b := m[i], m[j]
now := float64(pairDist(t.observed[a], t.observed[b]))
press := float64(pairDist(t.pressPos[a], t.pressPos[b]))
if chg := math.Abs(now - press); chg > bestChange {
bestChange = chg
bestA, bestB = a, b
}
}
}
if bestChange <= pinchMoveEps {
return nil // unison movement: a slide, not a pinch
}
a, b := bestA, bestB
t.pair = [2]pointer.ID{a, b}
t.pos = [2]f32.Point{t.observed[a], t.observed[b]}
t.on = true
t.fresh = true
// Baseline = the PRESS distance of the pair: any spread that happened
// before the pair started is owed, not lost.
t.formDist = pairDist(t.pressPos[a], t.pressPos[b])
t.observed = nil
t.observedAt = nil
t.pressPos = nil
return []pointer.ID{a, b}
}
// pairIndex returns 0/1 if id is a member of the active pair, else -1.
func (t *pinchTracker) pairIndex(id pointer.ID) int {
if !t.on {
return -1
}
if t.pair[0] == id {
return 0
}
if t.pair[1] == id {
return 1
}
return -1
}
// pruneStale drops observed fingers that pressed more than
// pinchFreshWindow before now.
func (t *pinchTracker) pruneStale(now time.Duration) {
for id, at := range t.observedAt {
if now-at > pinchFreshWindow {
delete(t.observed, id)
delete(t.observedAt, id)
delete(t.pressPos, id)
}
}
}
// factor returns the current frame's relative pinch scale and the pair
// midpoint (window px), or ok=false when no factor applies (no active pair,
// first frame of a pinch, or an implausible jump). It must be called once
// per frame, after all of the frame's events have been stepped.
func (t *pinchTracker) factor() (f float32, mid f32.Point, ok bool, grabs []pointer.ID) {
// Pending pair formation is decided HERE, after the whole frame's
// events (not per event): a per-event decision would lock in the first
// mover pair seen — e.g. (finger, drifting palm) — before the second
// pinch finger's drag lands in the same drain.
if !t.on && !t.survOn && t.observed != nil {
if g := t.tryFormPairAny(); g != nil {
grabs = g
// Slow frame: a pair member may have released in this same
// drain (the release arrived while the pair was still pending):
// break it now at the fingers' final positions.
for i, id := range t.pair {
if t.released[id] {
t.breakPair(i)
break
}
}
}
}
// Pending fingers that released this frame are dropped for good.
if t.released != nil {
for id := range t.released {
t.dropObserved(id)
}
t.released = nil
}
if !t.on {
// The pair broke during this frame's drain: emit the settled
// factor (on a slow frame the drags and the breaking release can
// arrive together, and factor() runs only after the drain).
if t.brokeFactor > 0 {
f, mid, ok = t.brokeFactor, t.brokeMid, true
t.brokeFactor = 0
t.brokeMid = f32.Point{}
}
return f, mid, ok, grabs
}
d := pairDist(t.pos[0], t.pos[1])
if t.fresh {
// Formation frame: establish the baseline. Emit only if the pair
// already moved from its formation positions this frame (the
// Android driver replays historical samples, so drags can batch
// with the formation Press).
t.fresh = false
t.prevDist = d
if t.formDist > 0 && d != t.formDist {
f = d / t.formDist
if f < 0.1 || f > 10 {
return 0, f32.Point{}, false, grabs
}
return f, pairMid(t.pos[0], t.pos[1]), true, grabs
}
return 0, f32.Point{}, false, grabs
}
if t.prevDist > 0 && d > 0 && d != t.prevDist {
// No-movement guard: a stationary pair emits nothing (an f=1.0
// factor would churn the font pin's re-layout for no change).
f = d / t.prevDist
// Sanitize: a real pinch moves millimeters between frames; a
// factor this far off 1 is noise, not a finger.
if f < 0.1 || f > 10 {
t.prevDist = d
return 0, f32.Point{}, false, grabs
}
t.prevDist = d
return f, pairMid(t.pos[0], t.pos[1]), true, grabs
}
t.prevDist = d
return 0, f32.Point{}, false, grabs
}
// survivorScroll returns and resets the frame's accumulated survivor-finger
// scroll (window px, gesture.Scroll convention: positive = content scrolls
// up).
func (t *pinchTracker) survivorScroll() int {
d := t.survScroll
t.survScroll = 0
return d
}
// reset clears all state (the editor left the screen).
func (t *pinchTracker) reset() {
*t = pinchTracker{}
}
// pairDist is the distance between two pair points (window px).
func pairDist(a, b f32.Point) float32 {
dx, dy := a.X-b.X, a.Y-b.Y
return float32(math.Sqrt(float64(dx*dx + dy*dy)))
}
// pairMid is the midpoint of two pair points (window px).
func pairMid(a, b f32.Point) f32.Point {
return f32.Point{X: (a.X + b.X) / 2, Y: (a.Y + b.Y) / 2}
}

281
internal/ui/real_draw_probe_test.go Normal file
View File

@ -0,0 +1,281 @@
package ui
// Definitive host test: run the app's REAL Renderer.Draw op stream (a
// realistic editor frame: status bar, editor TextField, bottom bar) through
// the real input.Router with the app's per-frame protocol (drain before
// commit), queue a touch press inside the editor text, and check which tags
// receive it: the scroll tag (known-good on the phone) as control, plus the
// pressProbe/pinchProbe tags (dead on the phone).
import (
"image"
"testing"
"time"
"gioui.org/f32"
"gioui.org/font/gofont"
"gioui.org/io/event"
"gioui.org/io/input"
"gioui.org/io/pointer"
"gioui.org/layout"
"gioui.org/op"
"gioui.org/text"
"gioui.org/unit"
)
func TestRealDrawOpsProbeHit(t *testing.T) {
shp := text.NewShaper(text.WithCollection(gofont.Collection()))
r := New(Theme{FontSize: 14}, shp)
noop := func(any) {}
const (
wW = 411
wH = 914
)
editorRegion := Region{X: 10, Y: 52, W: wW - 20, H: 700}
editor := NewTextField("editor_text", "hello world\nsecond line\nthird line", editorRegion, true, editorRegion.W, 0, 5, -1, -1, []Interaction{
{Gesture: Scroll, Handler: noop},
{Gesture: Tap, Handler: noop},
{Gesture: Pinch, Handler: noop},
})
editor.Focused = true
editor.ShowIMESeq = 7
statusBar := NewContainer(Region{X: 0, Y: 0, W: wW, H: 52}, Color{R: 240, G: 240, B: 240, A: 255}, []Element{
NewLabel("←", 16, Region{X: 5, Y: 5, W: 40, H: 42}, AlignStart, "back_id", nil),
NewLabel("/storage/emulated/0/Notes/test.txt", 14, Region{X: 50, Y: 5, W: 300, H: 42}, AlignStart, "path_id", nil),
})
bottomBar := NewContainer(Region{X: 0, Y: wH - 40, W: wW, H: 40}, Color{R: 240, G: 240, B: 240, A: 255}, []Element{
NewLabel("Saved", 14, Region{X: 5, Y: wH - 35, W: 100, H: 30}, AlignStart, "saved_id", nil),
})
elems := []Element{statusBar, editor, bottomBar}
m := unit.Metric{PxPerDp: 1, PxPerSp: 1}
gtxFor := func(ops *op.Ops) layout.Context {
return layout.Context{
Ops: ops,
Metric: m,
Constraints: layout.Constraints{Min: image.Point{}, Max: image.Point{X: wW, Y: wH}},
}
}
var rtr input.Router
probeK := pointer.Press | pointer.Drag | pointer.Release | pointer.Cancel
pinchF := pointer.Filter{Target: event.Tag(r.pinchProbe), Kinds: probeK}
pressF := pointer.Filter{Target: event.Tag(r.pressProbe), Kinds: probeK}
// Frame 1: record the real app ops and commit.
{
var ops op.Ops
r.Draw(gtxFor(&ops), elems, 1.0)
if _, ok := r.scrolls["editor_text"]; !ok {
t.Fatal("scroll reg not created for editor_text")
}
scroll := r.scrolls["editor_text"].scroll
t.Logf("scroll tag = %p", scroll)
// Pre-merge filters (app calls Event every frame; first call here).
rtr.Source().Event(pinchF)
rtr.Source().Event(pressF)
rtr.Source().Event(pointer.Filter{Target: scroll, Kinds: probeK})
rtr.Frame(&ops)
}
// A touch press lands in the editor text region.
pos := f32.Pt(100, 100)
rtr.Queue(pointer.Event{Kind: pointer.Press, Source: pointer.Touch, Position: pos})
t.Logf("queued press at %v", pos)
// Frame 2: draw again, then drain (app protocol: consume before commit).
{
var ops op.Ops
r.Draw(gtxFor(&ops), elems, 1.0)
drain := func(name string, f pointer.Filter) {
for {
e, ok := rtr.Source().Event(f)
if !ok {
break
}
if pe, ok := e.(pointer.Event); ok {
t.Logf("frame2: %-12s got kind=%v pos=%v", name, pe.Kind, pe.Position)
}
}
}
drain("pressProbe", pressF)
drain("pinchProbe", pinchF)
drain("scroll", pointer.Filter{Target: r.scrolls["editor_text"].scroll, Kinds: probeK})
rtr.Frame(&ops)
}
}
// TestRealDrawPinchGrabLifecycle runs the REAL app frame loop (draw ops ->
// CheckGestures -> commit) through the real input.Router with a real
// two-finger pinch and verifies the grab semantics that fix the on-device
// jank: the pair is tracked exclusively (the probe keeps getting drags even
// off-clip, scroll gets nothing), a factor is emitted per moved frame, a
// release breaks the pair, and the survivor finger's drags come out as plain
// scroll (forwarded), so the finger is not dead after a pinch.
func TestRealDrawPinchGrabLifecycle(t *testing.T) {
shp := text.NewShaper(text.WithCollection(gofont.Collection()))
r := New(Theme{FontSize: 14}, shp)
var pinchEvents []any
var scrollEvents []any
editor := NewTextField("editor_text", "hello world\nsecond line\nthird line",
Region{X: 10, Y: 52, W: 391, H: 700}, true, 391, 0, 5, -1, -1, []Interaction{
{Gesture: Scroll, Handler: func(d any) { scrollEvents = append(scrollEvents, d) }},
{Gesture: Tap, Handler: func(any) {}},
{Gesture: Pinch, Handler: func(d any) { pinchEvents = append(pinchEvents, d) }},
})
editor.Focused = true
elems := []Element{editor}
m := unit.Metric{PxPerDp: 1, PxPerSp: 1}
gtxFor := func(ops *op.Ops) layout.Context {
return layout.Context{
Ops: ops,
Metric: m,
Constraints: layout.Constraints{Min: image.Point{}, Max: image.Point{X: 411, Y: 914}},
}
}
var rtr input.Router
var prevOps op.Ops
havePrev := false
// ptEv builds a router-queueable pointer event. "Drag" is written as
// Move: the router only accepts Press/Move/Release/Cancel/Scroll and
// converts a pressed pointer's Move into a Drag before delivery.
ptEv := func(kind pointer.Kind, id pointer.ID, x, y float32, at time.Duration) pointer.Event {
if kind == pointer.Drag {
kind = pointer.Move
}
return pointer.Event{Kind: kind, Source: pointer.Touch, PointerID: id, Position: f32.Point{X: x, Y: y}, Time: at}
}
// runFrame mirrors the app's loop: commit the previous frame's ops
// (w.Event), queue this frame's pointer events, draw, CheckGestures.
// It also observes what the SCROLL tag receives this frame (drained
// before CheckGestures so the observation is lossless).
runFrame := func(evts ...pointer.Event) (events []InputEvent, scrollKinds []pointer.Kind) {
if havePrev {
rtr.Frame(&prevOps)
}
for _, e := range evts {
rtr.Queue(e)
}
var ops op.Ops
r.Draw(gtxFor(&ops), elems, 1.0)
for {
e, ok := rtr.Source().Event(pointer.Filter{
Target: r.scrolls["editor_text"].scroll,
Kinds: pointer.Press | pointer.Drag | pointer.Release | pointer.Cancel,
})
if !ok {
break
}
if pe, ok := e.(pointer.Event); ok {
scrollKinds = append(scrollKinds, pe.Kind)
}
}
events = r.CheckGestures(rtr.Source(), m)
// The app's main loop dispatches each event to its logic handler;
// mirror that so the capture handlers see them.
for _, e := range events {
e.Handler(e.Data)
}
prevOps, havePrev = ops, true
return events, scrollKinds
}
// Setup frame: register the ops.
runFrame()
hasKind := func(kinds []pointer.Kind, k pointer.Kind) bool {
for _, x := range kinds {
if x == k {
return true
}
}
return false
}
// Two fingers press 200px apart inside the editor text. Pending: no
// pair yet (the pair forms when BOTH move), no pinch events, no grabs.
_, scrollKinds := runFrame(
ptEv(pointer.Press, 0, 100, 100, time.Millisecond),
ptEv(pointer.Press, 1, 300, 100, 2*time.Millisecond))
if len(pinchEvents) != 0 {
t.Fatalf("press frame emitted a pinch event: %v", pinchEvents)
}
if hasKind(scrollKinds, pointer.Drag) {
t.Fatalf("scroll saw a drag on the press frame: %v", scrollKinds)
}
// The pair spreads 200 -> 240: this is the FORMATION frame — both
// fingers moved (the two-mover rule), the pair forms and the grabs are
// issued. One factor (1.2) is owed against the press distance. The
// pair's drags of THIS frame still reach scroll (the grabs commit on
// the next frame): a one-frame leak bounded by the scroll slop — the
// cost of not grabbing on press (which would kill single-finger
// scrolls). From the NEXT frame on, scroll must see nothing of the pair.
runFrame(
ptEv(pointer.Drag, 0, 80, 100, 10*time.Millisecond),
ptEv(pointer.Drag, 1, 320, 100, 11*time.Millisecond))
if len(pinchEvents) != 1 {
t.Fatalf("formation frame: pinch events=%d want 1 (%v)", len(pinchEvents), pinchEvents)
}
fpe, ok := pinchEvents[0].(FontPinchEvent)
if !ok || fpe.Scale < 1.19 || fpe.Scale > 1.21 {
t.Fatalf("scale=%v want ~1.2", pinchEvents[0])
}
// The pair keeps spreading: from here the grabs are active and SCROLL
// sees no drag of the pair (a Cancel for the dropped press may arrive).
_, scrollKinds = runFrame(
ptEv(pointer.Drag, 0, 50, 100, 15*time.Millisecond),
ptEv(pointer.Drag, 1, 350, 100, 16*time.Millisecond))
if len(pinchEvents) != 2 {
t.Fatalf("mid-pinch frame: pinch events=%d want 2 (%v)", len(pinchEvents), pinchEvents)
}
if hasKind(scrollKinds, pointer.Drag) {
t.Fatalf("scroll saw the pair's drags after formation: %v (grab failed)", scrollKinds)
}
// Finger 0 drags FAR OUTSIDE the editor region: the grab keeps it
// delivering to the probe (no stale pointer, no lost release).
_, scrollKinds = runFrame(
ptEv(pointer.Drag, 0, 2, 890, 20*time.Millisecond), // off-clip
ptEv(pointer.Drag, 1, 330, 100, 21*time.Millisecond))
if len(pinchEvents) != 3 {
t.Fatalf("off-clip frame: pinch events=%d want 3 (pair must survive off-clip)", len(pinchEvents))
}
if hasKind(scrollKinds, pointer.Drag) {
t.Fatalf("scroll saw the off-clip drag: %v", scrollKinds)
}
// Finger 0 lifts (off-clip): the release still arrives via the grab.
// The pair breaks; no pinch event.
before := len(pinchEvents)
runFrame(ptEv(pointer.Release, 0, 2, 890, 30*time.Millisecond))
if len(pinchEvents) != before {
t.Fatal("broken pair emitted a pinch event")
}
// The survivor (finger 1) scrolls 30px up: forwarded as a plain scroll
// delta to the editor's scroll handler — the finger is not dead.
scrollBefore := len(scrollEvents)
runFrame(ptEv(pointer.Drag, 1, 330, 70, 40*time.Millisecond))
if len(scrollEvents) != scrollBefore+1 {
t.Fatalf("survivor scroll not forwarded: events=%d want %d", len(scrollEvents), scrollBefore+1)
}
if d, ok := scrollEvents[scrollBefore].(int); !ok || d != 30 {
t.Fatalf("survivor delta=%v want 30 (px, scroll-up positive)", scrollEvents[scrollBefore])
}
// A second finger returns: candidate for a re-form (no event on the
// press); it must MOVE to become the pair (two-mover rule).
runFrame(ptEv(pointer.Press, 2, 50, 70, 50*time.Millisecond))
before = len(pinchEvents)
runFrame(
ptEv(pointer.Drag, 1, 340, 70, 60*time.Millisecond),
ptEv(pointer.Drag, 2, 30, 70, 61*time.Millisecond)) // 20px from press
if len(pinchEvents) != before+1 {
t.Fatalf("re-formed pair: pinch events=%d want %d", len(pinchEvents), before+1)
}
}

182
internal/ui/render.go
View File

@ -62,6 +62,17 @@ type scrollReg struct {
handler func(any)
}
// Probe tags for the raw-pointer probes (long-press, pinch). Each probe
// needs its OWN named type: the unnamed fieldless struct{} is a single
// canonical Go type, so distinct `struct{}` fields are the SAME value.
// Gio's router keys handlers by the tag value, so two `struct{}` probes
// collapse into one handler and whichever probe drains first (the press
// probe, which is consumed before the pinch probe) consumes every event,
// starving the other — this is why pinch received nothing on device while
// long-press appeared to work.
type pressProbeTag struct{}
type pinchProbeTag struct{}
// Renderer consumes a slice of elements and draws them.
//
// The Renderer is owned by the main goroutine. It is the home of any state
@ -80,6 +91,17 @@ type Renderer struct {
lastLineY Dp // last line baseline offset from text origin, in Dp (derived from GlyphLayout)
glyphLayout GlyphLayout // captured per-glyph layout from last drawWrappedText
// ZeroWheelScroll, set by main on a frame whose window size shrank, makes
// CheckGestures drain any pointer.Scroll events queued for the editor's
// scroll gesture before consuming gestures. Gio's window calls RevealFocus
// on any frame the viewport shrinks (e.g. the IME opening under
// adjustResize) and synthesizes a pointer.Scroll nudge to bring the focused
// field's (stale, pre-resize) bounds into view; consumed by gesture.Scroll
// that nudge shifts the editor content. The drain kills only the
// synthesized event: finger scroll (pointer.Drag) and the flinger are
// untouched, and normal frames consume pointer.Scroll as before.
ZeroWheelScroll bool
// IME dedup (main-owned, persistent across frames). Re-pushing an unchanged
// snippet or selection every frame resets the IME's composition and caret,
// which desyncs fast commits; push only on change, as widget.Editor does in
@ -114,12 +136,40 @@ type Renderer struct {
// scroll/drag then, not a press). longPressID gates the long-press to the
// editor's click reg (browser rows etc. don't long-press). ppLast is in
// f32.Point because pointer.Event.Position is window-space f32.
pressProbe struct{}
pressProbe pressProbeTag
ppLast f32.Point
ppActive bool
ppMoved bool
longPressID string
// Pinch-to-change-font-size (editor text region). Gio v0.10 has no
// two-finger pinch primitive: pinchProbe is a raw event tag inside the
// editor clip, and pinchT (pinch_tracker.go) is the gesture's state
// machine. When two fresh fingers are down the tracker names them as an
// EXPLICIT pair and the adapter grabs both (pointer.GrabCmd): exclusive
// event delivery (releases always arrive, even off-clip; scroll/click
// are dropped with a Cancel, which also stops the first finger dragging
// the text mid-pinch). The per-frame factor is the pair-distance ratio
// (FontPinchEvent to pinchHandler). The pair is never re-derived from
// whatever pointers happen to be present: that re-derivation (two
// lowest IDs) let a resting third finger pair with a live one and made
// single-finger scrolls scale the font on the phone. When one pair
// finger lifts, the survivor stays grabbed (v0.10 has no release-grab)
// and its drags are forwarded as scroll via pinchScrollHandler. State
// is dropped when the editor leaves the screen (see Draw).
pinchProbe pinchProbeTag
pinchT pinchTracker
pinchHandler func(any)
pinchScrollHandler func(any)
pinchProbeOn bool
// appFontScale is the app-local font-size multiplier (pinch zoom;
// 1.0 = default, 0 = not set yet). The main goroutine feeds it from the
// frame's snapshot via SetAppFontScale before Draw; drawWrappedText
// multiplies the editor font size by it. The system user font scale is
// separate and already folded into gtx.Metric/gtx.Sp.
appFontScale float32
// Selection / caret drag handles (0 = start, 1 = end, 2 = body, 3 = caret
// handle). Registered clipped in drawWrappedText only while a selection or
// caret handle is visible; a gesture.Drag grabs the pointer once movement
@ -155,6 +205,17 @@ type Renderer struct {
gestureExclusions [][4]int
}
// SetAppFontScale sets the app-local font-size multiplier for subsequent
// draw passes (1.0 = default; <= 0 is treated as 1). Main-goroutine-only;
// call before Draw (see appFontScale).
func (r *Renderer) SetAppFontScale(v float32) {
if v > 0 {
r.appFontScale = v
} else {
r.appFontScale = 1
}
}
// GestureExclusions returns the handle grab boxes collected for the last
// frame (see gestureExclusions).
func (r *Renderer) GestureExclusions() [][4]int { return r.gestureExclusions }
@ -238,6 +299,12 @@ func (r *Renderer) toDp(px Px) Dp {
func (r *Renderer) Draw(gtx layout.Context, elems []Element, scale float32) {
r.scale = scale
r.focusSeenThisFrame = false // per-frame reset for FocusCmd dedup
if !r.pinchProbeOn {
// No editor text in the previous frame: drop all pinch state so a
// later pinch starts clean.
r.pinchT.reset()
}
r.pinchProbeOn = false
// Gio sets constraints to layout.Exact(windowSize), so Min==Max. Use (0,0) as Min.
winW := gtx.Constraints.Max.X
winH := gtx.Constraints.Max.Y
@ -343,6 +410,12 @@ func (r *Renderer) CheckGestures(q input.Source, m unit.Metric) []InputEvent {
// Long-press motion probe first: it must see the press/drag events before
// the click loop decides about a long press.
r.consumePressProbe(q)
// Pinch probe: drain the raw pointer events, grab/release the pair,
// and emit the frame's scale factor and any survivor-finger scroll.
// Runs before the click/scroll loops so the pinch is applied in the
// same input batch that carried the finger moves, and the pair's
// grabs are queued ahead of any competing scroll grab.
events = append(events, r.consumePinchProbe(q)...)
for id, reg := range r.clicks {
// Drain every queued event for this gesture in this frame.
// gesture.Click returns one event per Update call, but on Android a
@ -467,9 +540,25 @@ func (r *Renderer) CheckGestures(q input.Source, m unit.Metric) []InputEvent {
// gesture.Scroll.Update returns scroll delta in pixels.
// ScrollY range: Min = -scrollOffset (remaining above), Max = large (content height unknown yet).
// With Min==Max==0, clampSplit consumes zero scroll.
// Update runs unconditionally (it keeps the gesture's flinger state
// healthy). Emission is suppressed while a pinch owns the pair: the
// pair is grabbed (scroll is dropped from its path) once the grabs
// commit, and this guard covers the one-frame window before that.
if r.ZeroWheelScroll {
// RevealFocus (see ZeroWheelScroll) queued a synthetic
// pointer.Scroll on this shrink frame; consume it here so the
// gesture never sees it. Scroll-range clamping is no use: the
// router UNIONs ranges across frames and the historical max
// can never shrink back to zero.
for {
if _, ok := q.Event(pointer.Filter{Target: reg.scroll, Kinds: pointer.Scroll}); !ok {
break
}
}
}
delta := reg.scroll.Update(m, q, time.Now(), gesture.Vertical,
pointer.ScrollRange{}, pointer.ScrollRange{Min: -(1 << 30), Max: 1 << 30})
if delta != 0 {
if delta != 0 && !r.pinchT.on {
events = append(events, InputEvent{
Handler: reg.handler,
Data: delta,
@ -480,11 +569,63 @@ func (r *Renderer) CheckGestures(q input.Source, m unit.Metric) []InputEvent {
return events
}
// consumePinchProbe drains the pinch probe's raw pointer events into the
// tracker, issues the pair's grabs, and returns this frame's events: the
// scale factor (when the active pair moved) and the survivor finger's
// forwarded scroll (when a pair broke with one finger still down).
func (r *Renderer) consumePinchProbe(q input.Source) []InputEvent {
var events []InputEvent
for {
evt, ok := q.Event(pointer.Filter{Target: r.pinchProbe, Kinds: pointer.Press | pointer.Drag | pointer.Release | pointer.Cancel | pointer.Leave})
if !ok {
break
}
pe, ok := evt.(pointer.Event)
if !ok {
continue
}
if s := r.pinchT.step(pe); len(s.grabs) > 0 {
for _, id := range s.grabs {
q.Execute(pointer.GrabCmd{Tag: r.pinchProbe, ID: id})
}
}
}
// The tracker may have formed the pair during factor() (after the full
// frame's events); issue those grabs now. Either way they commit
// before any scroll grab queued later in this frame (FIFO command
// queue), so the pair wins the race even if a finger is already past
// the scroll slop.
f, mid, ok, grabs := r.pinchT.factor()
for _, id := range grabs {
q.Execute(pointer.GrabCmd{Tag: r.pinchProbe, ID: id})
}
if ok && r.pinchHandler != nil {
events = append(events, InputEvent{
Handler: r.pinchHandler,
Data: FontPinchEvent{
Scale: f,
Center: Point{X: r.toDp(Px(mid.X)), Y: r.toDp(Px(mid.Y))},
},
})
}
if d := r.pinchT.survivorScroll(); d != 0 && r.pinchScrollHandler != nil {
events = append(events, InputEvent{
Handler: r.pinchScrollHandler,
Data: d,
})
}
return events
}
// consumePressProbe drains the raw pointer events of the long-press probe
// and updates ppLast/ppMoved. It runs before the click loop each frame.
// Leave ends the pending press: a finger that drifts off the editor region
// must not keep a long press armed (its release would never come to the
// probe if it was grabbed by scroll, so without this the timer could fire
// for a finger that is long gone).
func (r *Renderer) consumePressProbe(q input.Source) {
for {
evt, ok := q.Event(pointer.Filter{Target: r.pressProbe, Kinds: pointer.Press | pointer.Drag | pointer.Release | pointer.Cancel})
evt, ok := q.Event(pointer.Filter{Target: r.pressProbe, Kinds: pointer.Press | pointer.Drag | pointer.Release | pointer.Cancel | pointer.Leave})
if !ok {
return
}
@ -505,7 +646,7 @@ func (r *Renderer) consumePressProbe(q input.Source) {
}
}
r.ppLast = pe.Position
case pointer.Release, pointer.Cancel:
case pointer.Release, pointer.Cancel, pointer.Leave:
r.ppMoved = false
r.ppActive = false
}
@ -595,6 +736,18 @@ func (r *Renderer) drawElement(gtx layout.Context, e Element) {
// clipped in drawWrappedText where the handle geometry is known.
r.registerInteraction(interactive.ID(), interaction, gtx)
}
if interaction.Gesture == Pinch {
// The renderer owns the probe (raw two-pointer geometry);
// this records the logic handlers. The scroll handler is
// the survivor-finger's forwarding target: after a pinch
// breaks with one finger still down, that finger stays
// grabbed by the probe (v0.10 has no release-grab), so its
// drags are emitted as plain scroll deltas.
r.pinchHandler = interaction.Handler
if reg, ok := r.scrolls[interactive.ID()]; ok {
r.pinchScrollHandler = reg.handler
}
}
}
}
e.Draw(gtx, r)
@ -783,19 +936,28 @@ func (r *Renderer) drawWrappedText(gtx layout.Context, str string, reg Region, w
}
r.gestureExclusions = nil // rebuilt from this frame's handle boxes
// App-local font-size multiplier (pinch zoom; SetAppFontScale keeps it
// > 0). It multiplies the sp font size directly, so the value is a
// continuous float — no rounding to whole points anywhere.
appScale := r.appFontScale
if appScale <= 0 {
appScale = 1
}
size := float32(r.theme.FontSize) * appScale
// Fixed line height based on font size, not glyph metrics.
lineHeightSp := unit.Sp(float32(r.theme.FontSize) * LineHeightScale)
lineHeightSp := unit.Sp(size * LineHeightScale)
// User font-size setting (sp per dp). The shaper draws baselines at
// Sp(...) physical px, so the RENDERED line pitch in density-dp is
// lineHeightSp × fontScale. Every dp-space value below (line height,
// ascent) uses the scaled form so caret/handles/highlight follow the
// drawn glyphs; the logic side tracks the same factor via
// EffectiveLineHeight (ScaleEvent.FontScale).
// EffectiveLineHeight (ScaleEvent.FontScale × app font scale).
fontScale := float32(1)
if gtx.Metric.PxPerDp > 0 && gtx.Metric.PxPerSp > 0 {
fontScale = gtx.Metric.PxPerSp / gtx.Metric.PxPerDp
}
ascent := Dp(float32(r.theme.FontSize) * fontScale)
ascent := Dp(size * fontScale)
lineH := Dp(float32(lineHeightSp) * fontScale)
// Wrap disabled: shape with unlimited width so lines extend past the
// region (clipped by textClip below) instead of wrapping.
@ -804,7 +966,7 @@ func (r *Renderer) drawWrappedText(gtx layout.Context, str string, reg Region, w
maxWidthPx = int(r.toPx(wrapWidth))
}
params := text.Parameters{
PxPerEm: fixed.I(gtx.Sp(r.theme.FontSize)),
PxPerEm: fixed.I(gtx.Sp(unit.Sp(size))),
MinWidth: 0,
MaxWidth: maxWidthPx,
MaxLines: 0, // unlimited - wrap at MaxWidth
@ -952,6 +1114,10 @@ func (r *Renderer) drawWrappedText(gtx layout.Context, str string, reg Region, w
// inside the editor region.
r.selDragsOn = [4]bool{}
event.Op(gtx.Ops, r.pressProbe)
// Pinch probe: same clip as the press probe, so a pinch only registers
// when both fingers' presses/moves land in the editor text region.
event.Op(gtx.Ops, r.pinchProbe)
r.pinchProbeOn = true
r.longPressID = "editor_text"
if r.selDragHandler != nil && (selStart >= 0 && selEnd > selStart || caretDrag) {
// handleAt mirrors the cursor computation above: window-relative byte

133
internal/ui/reveal_focus_drain_test.go Normal file
View File

@ -0,0 +1,133 @@
package ui
import (
"image"
"testing"
"time"
"gioui.org/font/gofont"
"gioui.org/gesture"
"gioui.org/io/event"
"gioui.org/io/input"
"gioui.org/io/key"
"gioui.org/io/pointer"
"gioui.org/layout"
"gioui.org/op"
"gioui.org/text"
"gioui.org/unit"
)
// TestRevealFocusDrainTermination reproduces the IME-open scenario at the
// router level: the focused editor field has registered (stale, taller)
// bounds, the window's viewport shrinks, and the framework calls
// Router.RevealFocus which synthesizes a pointer.Scroll for the focused
// field's scroll handler. The app's shrink-frame fix (Renderer
// ZeroWheelScroll) drains pointer.Scroll events for the gesture's tag before
// gesture.Scroll.Update consumes them. This test verifies:
// 1. RevealFocus really does queue a scroll event for the gesture tag,
// 2. the drain loop terminates (bounded iteration count),
// 3. after the drain, gesture.Scroll.Update returns 0 (content does not
// move), and
// 4. on a normal (non-shrink) frame the drain is not needed and the
// gesture consumes whatever scroll it would have consumed anyway.
func TestRevealFocusDrainTermination(t *testing.T) {
shp := text.NewShaper(text.WithCollection(gofont.Collection()))
r := New(Theme{FontSize: 14}, shp)
noop := func(any) {}
const (
wW = 411
wH = 914
)
editorRegion := Region{X: 10, Y: 52, W: wW - 20, H: wH - 92}
// KeyDown is required: it is what makes drawElement record the
// event.Op reference for the tag, without which the key queue clears
// the FocusCmd target ("tag has no event.Op references").
editor := NewTextField("editor_text", "hello world\nsecond line\nthird line", editorRegion, true, editorRegion.W, 0, 5, -1, -1, []Interaction{
{Gesture: Scroll, Handler: noop},
{Gesture: KeyDown, Handler: noop},
})
editor.Focused = true
elems := []Element{editor}
m := unit.Metric{PxPerDp: 1, PxPerSp: 1}
var rtr input.Router
gtxFor := func(ops *op.Ops) layout.Context {
return layout.Context{
Ops: ops,
Metric: m,
Source: rtr.Source(),
Constraints: layout.Constraints{Min: image.Point{}, Max: image.Point{X: wW, Y: wH}},
}
}
// The app consumes a key.FocusFilter for the focused field every frame
// (main.go); that is what marks the handler focusable so keyQueue.Frame
// keeps the focus across frames.
drainFocus := func(q input.Source) {
for {
if _, ok := q.Event(key.FocusFilter{Target: event.Tag("editor_text")}); !ok {
break
}
}
}
// Frame 1: register handlers, focus the field (Draw issues the FocusCmd).
{
var ops op.Ops
gtx := gtxFor(&ops)
r.Draw(gtx, elems, 1.0)
drainFocus(gtx.Source)
rtr.Frame(&ops)
}
scroll := r.scrolls["editor_text"].scroll
t.Logf("focused(editor_text) after frame1: %v", rtr.Source().Focused(event.Tag("editor_text")))
// Frame 2: the app consumes gestures every frame; gesture.Scroll.Update
// registers the Scroll kind in the handler's filter, which Router.Deliver
// (used by RevealFocus/ScrollFocus) requires to match.
{
var ops op.Ops
gtx := gtxFor(&ops)
r.Draw(gtx, elems, 1.0)
drainFocus(gtx.Source)
_ = scroll.Update(m, gtx.Source, time.Now(), gesture.Vertical,
pointer.ScrollRange{}, pointer.ScrollRange{Min: -(1 << 30), Max: 1 << 30})
rtr.Frame(&ops)
}
// Simulate the keyboard opening: the framework shrinks the viewport and
// calls RevealFocus with the new (smaller) viewport.
shrunk := image.Rectangle{Min: image.Point{}, Max: image.Point{X: wW, Y: wH - 400}}
rtr.RevealFocus(shrunk)
// The app's shrink-frame fix: drain pointer.Scroll for the gesture tag.
q := rtr.Source()
drained := 0
for {
evt, ok := q.Event(pointer.Filter{Target: scroll, Kinds: pointer.Scroll})
if !ok {
break
}
drained++
if pe, ok := evt.(pointer.Event); ok {
t.Logf("drained synthetic scroll: %+v", pe.Scroll)
}
if drained > 10 {
t.Fatalf("drain loop did not terminate within 10 iterations")
}
}
if drained == 0 {
t.Fatalf("RevealFocus did not queue a scroll event for the scroll gesture tag")
}
// The gesture must now see no scroll (content stays put).
var ops op.Ops
r.Draw(gtxFor(&ops), elems, 1.0)
delta := scroll.Update(m, q, time.Now(), gesture.Vertical,
pointer.ScrollRange{}, pointer.ScrollRange{Min: -(1 << 30), Max: 1 << 30})
if delta != 0 {
t.Fatalf("gesture consumed %d px of scroll after drain; content would shift", delta)
}
rtr.Frame(&ops)
}

29
internal/ui/tag_identity_test.go Normal file
View File

@ -0,0 +1,29 @@
package ui
import (
"testing"
"gioui.org/font/gofont"
"gioui.org/text"
)
// Regression test for the pinch-on-device bug: the probes were declared as
// `struct{}` fields. The unnamed fieldless struct{} is a single canonical Go
// type, so all three probes were the SAME tag value and Gio's router merged
// them into one handler — the press probe (drained first) consumed every
// event and the pinch probe was starved. Each probe now has its own named
// type and must be a distinct map key.
func TestProbeTagIdentity(t *testing.T) {
r := New(Theme{FontSize: 14}, text.NewShaper(text.WithCollection(gofont.Collection())))
m := map[interface{}]int{}
m[r.pressProbe] = 1
m[r.pinchProbe] = 2
t.Logf("distinct probe tag keys: %d", len(m))
if len(m) != 2 {
t.Fatalf("probe tags must be distinct values, got %d distinct keys", len(m))
}
var a, b interface{} = r.pressProbe, r.pinchProbe
if a == b {
t.Fatal("pressProbe and pinchProbe must not be equal interface values")
}
}

5
internal/ui/unit.go
View File

@ -91,6 +91,11 @@ type LayoutFeedback struct {
WindowStartByte int // absolute byte offset of the window's first byte
WindowStartLine int // logical line the window starts at (-1: none)
EditSeq uint64 // editor content-edit counter at frame time
// ScrollOffset is the editor scroll offset (Dp) the frame this layout
// was shaped from carried. The logic needs it to express layout positions
// (window-relative) in content coordinates: the window top is the
// shaped scroll's sub-line remainder above the region top.
ScrollOffset Dp
}
type GlyphLayout struct {

13
scripts/emu.sh
View File

@ -16,7 +16,11 @@
# # first tap right after launch/open is
# # sometimes swallowed — re-tap.
# scripts/emu.sh type TEXT # type into the focused field (spaces ok)
# scripts/emu.sh cmd <top|bottom|frac F|dp N> # one-shot editor debug command
# scripts/emu.sh cmd <top|bottom|frac F|dp N|pinch F|fontsize F>
# # one-shot editor debug command
# # (pinch F = relative app font scale
# # anchored at the region center;
# # fontsize F = absolute, top-anchored)
# scripts/emu.sh perf on|off # enable/disable the in-app profiler
# scripts/emu.sh perf pull [FILE] # pull logic_frames.csv (default ./logic_frames.csv)
# scripts/emu.sh push FILE # push FILE -> /storage/emulated/0/Notes/
@ -30,7 +34,8 @@
# the CSV is truncated on app relaunch, so pull it before restarting the
# app if you are accumulating data.
# - One-shot editor debug commands are read from /storage/emulated/0/PadPerf/cmd
# (scroll: top | bottom | "frac 0.5" | "dp 1234").
# (scroll: top | bottom | "frac 0.5" | "dp 1234"; app font scale:
# "pinch 1.1" relative | "fontsize 1.5" absolute).
# - Test files go under /storage/emulated/0/Notes/.
# - The emulator OOMs above ~2.5 GB RSS on this VM; kill it if it wedges.
#
@ -138,7 +143,7 @@ cmd_up() {
kill -9 "$qp" 2>/dev/null || true
sleep 2
fi
start_emulator
start_emulator "$@"
local rc=0
wait_device_online "$ONLINE_TIMEOUT" || rc=$?
if [ "$rc" -ne 0 ]; then
@ -163,7 +168,7 @@ cmd_up() {
sleep 1
j=$((j + 1))
done
start_emulator -no-snapshot-load
start_emulator -no-snapshot-load "$@"
# Note: full BOOT_TIMEOUT here — a cold boot legitimately takes far
# longer than the ONLINE_TIMEOUT used for the first (snapshot) attempt.
wait_device_online "$BOOT_TIMEOUT" || { tail -n 5 "$EMU_LOG" >&2; die "device never came online (cold boot); see $EMU_LOG"; }

11
tools/touchinject/AndroidManifest.xml Normal file
View File

@ -0,0 +1,11 @@
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
package="touch.inject">
<uses-permission android:name="android.permission.INJECT_EVENTS"/>
<application>
<service android:name=".Injector" android:exported="true"/>
<receiver android:name=".Injector$ScriptReceiver" android:exported="true"/>
</application>
</manifest>

223
tools/touchinject/Injector.java Normal file
View File

@ -0,0 +1,223 @@
package touch.inject;
import android.app.Service;
import android.content.BroadcastReceiver;
import android.content.Context;
import android.content.Intent;
import android.hardware.input.InputManager;
import android.os.IBinder;
import android.os.SystemClock;
import android.util.Log;
import android.view.MotionEvent;
import java.util.LinkedHashMap;
import java.util.Map;
/**
* Multi-touch gesture injector for the Pad emulator.
*
* Runs as a privileged system app (/system/priv-app) so that the
* signature|privileged INJECT_EVENTS permission is granted; it then injects
* real MotionEvents through InputManager — the exact path a physical touch
* screen takes — which is what the adb `input` applet cannot do (it has no
* multi-touch support).
*
* Usage:
* adb shell am startservice -n touch.inject/.Injector -e script "SCRIPT"
*
* SCRIPT is a whitespace-separated sequence of:
* down <finger> <x> <y> finger (1-based) touches at (x,y) px
* move <finger> <x> <y> finger moves to (x,y) px
* up <finger> finger lifts
* wait <ms> pause
*
* Progress is logged to logcat under the "TouchInject" tag.
*/
public class Injector extends Service {
public static final String TAG = "TouchInject";
/** Shell-triggerable entry point: works even while the app is stopped
* (the shell can broadcast to an explicit component).
* KNOWN FLAKE: the process is "cached" while the thread runs and the
* 1.5GB emulator OOM-killed it once mid-script (during a 200ms wait),
* losing the final UP. Service routing is NOT an alternative — Android
* 12+ blocks background startService from a receiver, and the AVD's
* locked bootloader blocks the system-app escalation. The harness
* therefore verifies "=== done" in logcat after every script and
* re-runs on failure; scripts should stay short. */
public static class ScriptReceiver extends BroadcastReceiver {
@Override
public void onReceive(Context context, Intent intent) {
String script = intent.getStringExtra("script");
if (script == null) {
return;
}
InputManager im = (InputManager)
context.getSystemService(Context.INPUT_SERVICE);
new Thread(() -> execute(im, script), "TouchInject").start();
}
}
@Override
public int onStartCommand(Intent intent, int flags, int startId) {
String script = intent != null ? intent.getStringExtra("script") : null;
if (script == null) {
stopSelf();
return START_NOT_STICKY;
}
InputManager im = (InputManager) getSystemService(INPUT_SERVICE);
new Thread(() -> execute(im, script), "TouchInject").start();
return START_NOT_STICKY;
}
@Override
public IBinder onBind(Intent intent) {
return null;
}
private static void execute(InputManager im, String script) {
Log.i(TAG, "=== script: " + script);
// finger number (1-based) -> [x, y]; insertion order = pointer index.
LinkedHashMap<Integer, float[]> fingers = new LinkedHashMap<>();
long downTime = 0;
String[] toks = script.split("\\s+");
int i = 0;
while (i < toks.length) {
String cmd = toks[i++];
try {
switch (cmd) {
case "down": {
int f = Integer.parseInt(toks[i++]);
float x = Float.parseFloat(toks[i++]);
float y = Float.parseFloat(toks[i++]);
int action = fingers.isEmpty()
? MotionEvent.ACTION_DOWN
: MotionEvent.ACTION_POINTER_DOWN;
int idx = fingers.size();
fingers.put(f, new float[]{x, y});
inject(im, action, idx, fingers, downTime);
if (downTime == 0) {
downTime = SystemClock.uptimeMillis();
}
break;
}
case "move": {
int f = Integer.parseInt(toks[i++]);
float x = Float.parseFloat(toks[i++]);
float y = Float.parseFloat(toks[i++]);
float[] p = fingers.get(f);
if (p == null) {
Log.w(TAG, "move of unknown finger " + f + " (skipped)");
break;
}
p[0] = x;
p[1] = y;
// Generic ACTION_MOVE for all moves (1 or N pointers):
// InputFlinger accepts it and GioView treats every
// pointer as a MOVE.
inject(im, MotionEvent.ACTION_MOVE, 0, fingers, downTime);
break;
}
case "up": {
int f = Integer.parseInt(toks[i++]);
if (!fingers.containsKey(f)) {
Log.w(TAG, "up of unknown finger " + f + " (skipped)");
break;
}
boolean last = fingers.size() == 1;
int action = last
? MotionEvent.ACTION_UP
: MotionEvent.ACTION_POINTER_UP;
int idx = indexOf(fingers, f);
// The lifted pointer must still be part of the event.
inject(im, action, idx, fingers, downTime);
fingers.remove(f);
if (fingers.isEmpty()) {
downTime = 0;
}
break;
}
case "wait": {
int ms = Integer.parseInt(toks[i++]);
Thread.sleep(ms);
break;
}
default:
Log.e(TAG, "unknown command: " + cmd);
}
} catch (Exception e) {
Log.e(TAG, "script error at '" + cmd + "': " + e);
break;
}
}
Log.i(TAG, "=== done");
}
private static int indexOf(LinkedHashMap<Integer, float[]> m, int f) {
int idx = 0;
for (Integer k : m.keySet()) {
if (k == f) {
return idx;
}
idx++;
}
return 0;
}
// INJECT_INPUT_EVENT_MODE_WAIT_FOR_RESULT. The SDK stub jar omits the
// constant (and injectInputEvent itself), so both go through reflection;
// the method is a public API at runtime on the device.
private static final int INJECT_WAIT_FOR_RESULT = 1;
private static void inject(InputManager im, int action, int actionIndex,
Map<Integer, float[]> fingers, long downTime) {
int n = fingers.size();
int[] ids = new int[n];
MotionEvent.PointerCoords[] coords = new MotionEvent.PointerCoords[n];
int j = 0;
for (Map.Entry<Integer, float[]> e : fingers.entrySet()) {
ids[j] = e.getKey() - 1; // pointer id (0-based)
MotionEvent.PointerCoords c = new MotionEvent.PointerCoords();
c.x = e.getValue()[0];
c.y = e.getValue()[1];
c.pressure = 1f;
c.size = 1f;
// TOOL_TYPE_FINGER. The compile-time android.jar lacks
// PointerCoords.setToolType, but the device runtime (API 24+)
// has it, so call it reflectively.
try {
java.lang.reflect.Method stt = MotionEvent.PointerCoords.class
.getMethod("setToolType", int.class);
stt.invoke(c, 1);
} catch (Exception ignored) {
}
coords[j] = c;
j++;
}
long now = SystemClock.uptimeMillis();
int fullAction = action == MotionEvent.ACTION_MOVE
? action
: action | (actionIndex << MotionEvent.ACTION_POINTER_INDEX_SHIFT);
MotionEvent ev = MotionEvent.obtain(downTime, now, fullAction, n, ids, coords,
0 /*edgeFlags*/, 1f /*xPrecision*/, 1f /*yPrecision*/,
0 /*metaState*/, 0 /*deviceId*/,
0x4000003 /*SOURCE_TOUCH|SOURCE_CLASS_MASK*/, 0 /*displayId*/);
boolean ok = doInject(im, ev);
Log.i(TAG, String.format("inject action=%d idx=%d n=%d ids=%s ok=%b",
action, actionIndex, n, java.util.Arrays.toString(ids), ok));
ev.recycle();
}
private static boolean doInject(InputManager im, MotionEvent ev) {
try {
java.lang.reflect.Method m = InputManager.class.getMethod(
"injectInputEvent", android.view.InputEvent.class, int.class);
Object r = m.invoke(im, ev, INJECT_WAIT_FOR_RESULT);
return r instanceof Boolean && (Boolean) r;
} catch (Exception e) {
Throwable c = e.getCause() != null ? e.getCause() : e;
Log.e(TAG, "injectInputEvent failed: " + c);
return false;
}
}
}

48
tools/touchinject/build.sh Executable file
View File

@ -0,0 +1,48 @@
#!/usr/bin/env bash
# Builds touchinject.apk (a privileged multi-touch gesture injector) from
# Injector.java using the local Android SDK. The APK must be installed under
# /system/priv-app for the INJECT_EVENTS (signature|privileged) permission to
# be granted — see the bottom of this file.
set -euo pipefail
cd "$(dirname "$0")"
SDK=/home/gmp/android-sdk
BT="$SDK/build-tools/35.0.0"
PLAT="$SDK/platforms/android-35/android.jar"
rm -rf build
mkdir -p build/classes
javac -nowarn -source 1.8 -target 1.8 -classpath "$PLAT" -d build/classes Injector.java
"$BT/d8" --release --min-api 24 --lib "$PLAT" --output build build/classes/touch/inject/*.class
# aapt2 resolves the android: namespace against the framework resource
# table, which is shipped inside android.jar (resources.arsc).
"$BT/aapt2" link --manifest AndroidManifest.xml -I "$PLAT" \
--min-sdk-version 24 --target-sdk-version 35 -o build/base.apk
cp build/base.apk build/touchinject-unsigned.apk
jar uf build/touchinject-unsigned.apk -C build classes.dex
if [ ! -f build/ts.jks ]; then
keytool -genkeypair -keystore build/ts.jks -storepass android -keypass android \
-alias ts -dname "CN=TouchInject, OU=Dev" -keyalg RSA -keysize 2048 -validity 10000
fi
"$BT/apksigner" sign --ks build/ts.jks --ks-pass pass:android --key-pass pass:android \
--out touchinject.apk build/touchinject-unsigned.apk
"$BT/apksigner" verify --print-certs touchinject.apk | head -1
echo "=== DONE: $(pwd)/touchinject.apk"
# --- Install as a privileged system app (emulator, rooted) ---------------
# SER=emulator-5554
# adb -s $SER root && adb -s $SER remount
# adb -s $SER shell mkdir -p /system/priv-app/TouchInject
# adb -s $SER push touchinject.apk /system/priv-app/TouchInject/TouchInject.apk
# adb -s $SER shell chmod 644 /system/priv-app/TouchInject/TouchInject.apk
# adb -s $SER reboot # permission grant is evaluated at install/boot
# adb -s $SER wait-for-device && adb -s $SER shell 'while [ -z $(getprop sys.boot_completed) ]; do sleep 1; done'
# adb -s $SER shell dumpsys package touch.inject | grep -A2 INJECT_EVENTS # granted=true
#
# --- Inject a gesture ------------------------------------------------------
# adb -s $SER shell am startservice -n touch.inject/.Injector \
# -e script "down 1 400 1000 wait 100 down 2 700 1200 wait 100 \
# move 1 380 980 move 2 720 1220 wait 50 \
# move 1 340 940 move 2 760 1260 wait 50 up 1 up 2"
# adb -s $SER logcat -d -s TouchInject

128
tools/touchinject/run_tests.sh Executable file
View File

@ -0,0 +1,128 @@
#!/usr/bin/env bash
# Pinch e2e test harness for the Pad emulator (aosp_atd AVD).
#
# The platform-signed touch.inject app (real MotionEvents through
# InputManager — the only multi-touch path that works in the emulator)
# drives the renderer's pinch probe; each gesture below is ONE script in
# ONE process (the injector's finger map is per-process).
#
# KNOWN FLAKES (both verified, both handled below):
# 1. The injector process is "cached" while its script thread runs and the
# 1.5 GB emulator OOM-kills it mid-script occasionally — the script
# never reaches "=== done". ti() checks logcat for the done line and
# re-runs (a re-run starts a fresh gesture; a stuck pointer from a
# half-run is replaced by the next ACTION_DOWN).
# 2. The app is on-demand-rendering at ~1 fps: a burst of moves within
# one frame's drain does not scroll (single-frame delta pattern).
# Scroll tests therefore space the moves >= 40 ms apart — which is
# also what a real finger produces over several frames.
#
# READING RESULTS: AppFontScale in the session file is the reliable pinch
# signal. Scroll in the session is NOT (the restore re-derives it from the
# pinned line anchor); for scroll tests compare the before/after
# screenshots instead.
SER=emulator-5554
export ANDROID_SERIAL=$SER
ti() {
local script="$1"
for attempt in 1 2 3; do
adb logcat -c 2>/dev/null
adb shell 'am broadcast -n "touch.inject/.Injector$ScriptReceiver" --es script "'"$script"'"' >/dev/null 2>&1
sleep 4
if adb shell logcat -d 2>/dev/null | grep -aq "TouchInject: === done"; then
return 0
fi
echo " (injection incomplete, attempt $attempt — retrying)"
done
echo " !! injection failed after 3 attempts"
return 1
}
# ensure Pad is in the foreground (the ATD launcher intermittently holds
# focus after a cold start; a backgrounded app receives no touch)
fg() {
for i in 1 2 3 4 5; do
FOC=$(adb shell 'dumpsys window | grep mCurrentFocus')
case "$FOC" in *pad.pad*) return 0;; esac
adb shell am start -n pad.pad/org.gioui.GioActivity >/dev/null 2>&1
sleep 8
done
echo " !! Pad never gained focus: $FOC"
return 1
}
dbg() { adb shell "echo '$1' > /sdcard/PadPerf/cmd"; sleep 1.5; }
shot() { adb exec-out screencap -p > "$1"; }
# force-stop (flushes the session), print it, relaunch + reopen the file
measure() {
adb shell am force-stop pad.pad
sleep 4
echo "session: $(adb shell cat /sdcard/Pad/session.json 2>/dev/null)"
adb shell am start -n pad.pad/org.gioui.GioActivity >/dev/null 2>&1
sleep 4
}
case "$1" in
t1) # single-finger scroll must NEVER change the font (device regression)
echo "=== T1: single-finger scroll (font must not change) ==="
fg || exit 1
shot /tmp/t1_before.png
S="down 1 720 1400 wait 80"
for y in 1320 1240 1160 1080 1000 920 840 760 680 600 520 440 360 280 200; do
S="$S move 1 720 $y wait 40"
done
S="$S wait 100 up 1"
ti "$S"
sleep 2
shot /tmp/t1_after.png # compare to before: content moved, same font size
measure
;;
t2) # two-finger pinch-out: font must scale, center anchored
echo "=== T2: two-finger pinch-out (font must grow) ==="
fg || exit 1
shot /tmp/t2_before.png
S="down 1 570 1200 wait 40 down 2 870 1200 wait 60"
for i in 1 2 3 4 5 6 7 8; do
x1=$((570 - i*37)); x2=$((870 + i*37))
S="$S move 1 $x1 1200 move 2 $x2 1200 wait 50"
done
S="$S wait 150 up 1 up 2"
ti "$S"
sleep 1
shot /tmp/t2_after.png
measure
;;
t3) # palm-first: the resting finger lands FIRST, the pinch pair is the
# two that move — the palm must never enter the distance
echo "=== T3: palm-first three fingers (pinch pair = the two movers) ==="
fg || exit 1
shot /tmp/t3_before.png
S="down 1 720 400 wait 100 down 2 570 1200 wait 60 down 3 870 1200 wait 150"
for i in 1 2 3 4 5 6 7 8; do
x2=$((570 - i*37)); x3=$((870 + i*37))
S="$S move 2 $x2 1200 move 3 $x3 1200 wait 50"
done
S="$S wait 150 up 2 up 3 wait 50 up 1"
ti "$S"
sleep 1
shot /tmp/t3_after.png
measure
;;
t4) # lift one finger mid-pinch: survivor must SCROLL, not zoom
echo "=== T4: lift finger mid-pinch (survivor scrolls, no more zoom) ==="
fg || exit 1
shot /tmp/t4_before.png
S="down 1 570 1200 wait 40 down 2 870 1200 wait 60 move 1 533 1200 move 2 907 1200 wait 80 up 2 wait 80"
for y in 1280 1360 1440 1520 1600 1680 1760 1840 1920 2000; do
S="$S move 1 533 $y wait 40" # survivor (finger 1) keeps its own x
done
S="$S wait 100 up 1"
ti "$S"
sleep 1
shot /tmp/t4_after.png # compare to before: content scrolled, font frozen
measure
;;
reset) # back to a known state (one command per file — no pipes)
dbg "fontsize 1"; dbg "top"
;;
*) echo "usage: $0 t1|t2|t3|t4|reset" ;;
esac