From e4c5804170e92933764e527cfe447191801bb02c Mon Sep 17 00:00:00 2001 From: Greg Pomerantz Date: Fri, 8 May 2026 12:01:06 -0400 Subject: [PATCH] Add touch and input handling specification (doc/touch.md) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Gioui primitives: pointer.InputOp, key.InputOp, key.FocusOp, SoftKeyboardOp - Op/Event flow: Logic computes elements, Renderer submits ops, Main batches events - Coordinate mapping: UI space → text space → byte space, with hit-region tags - Gesture state machine: tap, double-tap, long-press, drag-select, scroll - Gesture disambiguation: timing thresholds (500ms long-press, 300ms double-tap, 16DP drag) - Selection logic: selection_start/selection_end byte offsets - Input batching: Main accumulates events, sends batch to Logic per frame - Keyboard interaction: focus acquisition/loss, key event table, IME composition - Scroll momentum: fling logic with exponential decay, scroll clamping Co-authored-by: Qwen-Coder --- doc/touch.md | 197 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 197 insertions(+) create mode 100644 doc/touch.md diff --git a/doc/touch.md b/doc/touch.md new file mode 100644 index 0000000..2abf655 --- /dev/null +++ b/doc/touch.md @@ -0,0 +1,197 @@ +# Touch and Input Handling + +This document specifies how Pad handles low-level pointer and keyboard events using Gioui primitives, and how these are translated into editor actions (cursor movement, selection, scrolling). + +## 1. Gioui Primitives + +Pad avoids high-level widgets and uses the following low-level Gioui ops and events. Ops are submitted in the paint function; Events are returned by `w.Event()`. + +| Op (submitted in paint) | Event (returned by w.Event()) | Use Case | +|---|---|---| +| `pointer.InputOp` | `pointer.Event` | Defines hit-regions (editor, buttons, search bar). Captures `Press`, `Release`, `Move`, `Drag`, `Scroll`. | +| `key.InputOp` | `key.Event` | Enables keyboard focus. Captures `Edit` (text insertion), `Press` (Backspace, Enter, Arrows). | +| `key.FocusOp` | — | Requests keyboard focus for the active element. | +| `key.SoftKeyboardOp` | — | Explicitly shows/hides the Android soft keyboard. | + +### 1.1 Op/Event Flow + +Because Logic does not have access to `*op.Ops`, the Op/Event flow is split: + +1. **Logic → Ops**: Logic computes `[]Element`. Each element carries metadata about what input ops it needs (e.g., "this TextField needs `pointer.InputOp` with `Kind: Tap` on region X"). +2. **Renderer → Ops**: During the paint function, the Renderer iterates `[]Element` and submits the appropriate `pointer.InputOp` / `key.InputOp` calls into `*op.Ops`. +3. **Gioui → Events**: Gioui returns `pointer.Event` / `key.Event` via `w.Event()` in subsequent cycles. +4. **Main → Logic**: The Main goroutine batches these events into `[]InputEvent` and sends them to Logic. +5. **Logic → State**: Logic processes events, updates state, and computes a new frame. + +This ensures that hit-regions are always defined in the same frame where the elements are drawn, and Logic remains testable without the SDK. + +## 2. Coordinate Mapping + +The editor operates in three coordinate spaces: + +1. **UI Space (DP)**: The raw (X, Y) coordinates from Gioui events, relative to the window origin. +2. **Text Space (Lines/Cols)**: The logical position within the text file, accounting for scroll offset and word wrap. +3. **Byte Space (Offsets)**: The raw byte offset in the UTF-8 file buffer. + +### 2.1 Hit-Region Tags + +Gioui `pointer.InputOp` takes a `tag` (an `op.Tag`). When an event fires, `event.Tag` identifies which region was touched. + +**Tagging Strategy**: +- The Renderer assigns a unique tag per element (e.g., `editor`, `search_bar`, `status_bar`, `alpha_index`, `merge_hunk_accept_ours`). +- When Logic receives a `pointer.Event`, it first checks the tag to determine which element was touched, then applies element-specific coordinate logic. + +### 2.2 Mapping UI → Byte Offset (Editor/TextField) + +When a `pointer.Event` occurs at `(Ex, Ey)` on the `editor` tag: + +1. **Adjust for Scroll**: `Y = Ey + scroll_offset`. +2. **Identify Visual Line**: `visual_line = floor(Y / line_height)`. +3. **Resolve to Logical Line**: If word wrap is enabled, use the cached wrap positions to map `visual_line` → `logical_line`. Otherwise, they are the same. +4. **Identify Column**: + - Retrieve text for `logical_line` from the chunked buffer. + - Use the font shaper to measure glyph widths until the cumulative width exceeds `Ex`. + - Clamp to the line length. +5. **Result**: Convert `(logical_line, col_index)` to a byte offset using the **Line Index**. + +### 2.3 Mapping UI → Element (Non-Text Elements) + +For buttons, search bar, alpha index, merge hunks: +- Each element has a `Region()` (rectangle). The Main goroutine checks if `(Ex, Ey)` falls within the region. +- For the **Alpha Index**, the Y-coordinate maps to a letter (e.g., `letter = alphabet[floor((Y - alpha_top) / (alpha_height / 26))]`). +- For **Merge Hunks**, the tag identifies which hunk button (accept-ours / accept-theirs) was tapped. + +## 3. Gestures and State Machine + +The Logic goroutine maintains a small state machine for the active gesture. The state is transient (does not survive process death) and is reset on gesture completion. + +### 3.1 Gesture States + +| State | Entry Condition | Exit Condition | Action | +|---|---|---|---| +| `Idle` | Default state | `Press` received | — | +| `Pressed` | `Press` at (X, Y) | `Release` or `Drag` | Record press time and position | +| `Tapping` | `Release` received (short duration, no move) | Timeout or next `Press` | Move cursor, clear selection | +| `DoubleTap` | Second `Tapping` within 300ms | Gesture complete | Select word at (X, Y) | +| `LongPress` | `Press` held > 500ms | `Release` | Select word (future: magnifier) | +| `Selecting` | `Drag` received | `Release` | Update `selection_end` to current (X, Y) | +| `Scrolling` | `Drag` on non-focused area or two-finger drag | `Release` | Update `scroll_offset` | + +### 3.2 Gesture Disambiguation + +The Logic goroutine distinguishes gestures using timing and movement thresholds: + +- **Tap vs Long Press**: If `Press` → `Release` occurs within 500ms, it's a tap. Otherwise, it's a long press. +- **Tap vs Drag**: If the finger moves more than 16 DP between `Press` and `Release`, it's a drag. Otherwise, it's a tap. +- **Single vs Double Tap**: If two taps occur within 300ms, the second tap triggers a word selection. +- **Select vs Scroll**: If the keyboard is visible (focused `TextField`), a single-finger drag is a selection. If the keyboard is hidden, a single-finger drag is a scroll. Two-finger drags are always scrolls. + +### 3.3 Selection Logic + +Selection is defined by two byte offsets: `selection_start` and `selection_end`. +- If `selection_start == selection_end`, there is no selection (just a cursor). +- During a **Drag** gesture: + - On `Press`: Set `selection_start = selection_end = offset_at(X, Y)`. + - On `Drag`: Update `selection_end = offset_at(X, Y)`. + - The UI renders a `TextField` with highlighted regions between the two offsets. + +### 3.4 Double Click (Word Selection) + +When a double-click is detected: +1. Identify the character at the click offset. +2. Expand left and right until a non-word character (space, punctuation, newline) is hit. +3. Set `selection_start` and `selection_end` to these boundaries. + +## 4. Input Batching and Latency + +As specified in `architecture.md`, the Main goroutine batches events. For touch, this is critical: + +### 4.1 Event Compression in the Main Loop + +The Main goroutine collects events during its event cycle. For a drag gesture, multiple `pointer.Event` (Type: `Move`) may fire before the next `FrameEvent`. The Main goroutine: + +1. Accumulates all events into a `[]InputEvent` slice. +2. On the next `FrameEvent`, it sends the entire batch to Logic via `inputChan`. +3. Logic processes the batch sequentially, updating the gesture state and cursor/selection. +4. Logic produces a single frame reflecting the final state after all events. + +This means that if 3 `Drag` events fire in one frame, Logic sees all 3 and produces one frame with the cursor/selection at the final position. There is no "churn" of intermediate frames. + +### 4.2 Latency Budget + +- **Event Capture**: Gioui captures the touch event immediately (sub-ms). +- **Batching**: Main goroutine holds events until the next `FrameEvent` (up to 16ms, but typically < 8ms). +- **Processing**: Logic processes the batch and computes layout (< 16ms). +- **Rendering**: Frame receiver stores and invalidates (sub-ms). +- **Total**: < 32ms from touch to visual feedback (typically < 16ms). + +### 4.3 Gesture Continuity + +Because Logic maintains the gesture state machine across frames, a multi-frame drag gesture is continuous: +- Frame 1: `Press` at A → Logic enters `Pressed` state. +- Frame 2: `Drag` to B → Logic enters `Selecting` state, updates `selection_end`. +- Frame 3: `Drag` to C → Logic updates `selection_end` again. +- Frame 4: `Release` → Logic finalizes selection, enters `Idle` state. + +Each frame produces a new `[]Element` with the updated selection highlight. + +## 5. Keyboard Interaction + +The `TextField` element must be "focused" to receive keyboard events. Focus is a transient state managed by Logic. + +### 5.1 Focus Acquisition + +1. **Tap on Editor**: User taps the `TextField` region. Main goroutine sends `pointer.Event` (Type: `Press`) to Logic. +2. **Focus Request**: Logic sets `focused = true` and computes a frame with `key.FocusOp` and `key.SoftKeyboardOp(true)`. +3. **Keyboard Appears**: Gioui shows the Android soft keyboard. Subsequent `w.Event()` calls return `key.Event`. + +### 5.2 Focus Loss + +1. **Tap Outside Editor**: User taps a non-editor region (e.g., status bar, directory browser). Logic sets `focused = false` and computes a frame with `key.SoftKeyboardOp(false)`. +2. **Keyboard Disappears**: Gioui hides the soft keyboard. + +### 5.3 Key Events + +| Event | Type | Name | Action | +|---|---|---|---| +| Text insertion | `Edit` | — | Insert `event.Text` at cursor, append to undo chain | +| Backspace | `Press` | `Backspace` | Delete character before cursor, append to delete chain | +| Enter | `Press` | `Enter` | Insert newline at cursor | +| Tab | `Press` | `Tab` | Insert spaces (configurable, e.g., 4 spaces) | +| Arrow Up/Down | `Press` | `Up`/`Down` | Move cursor up/down one line | +| Arrow Left/Right | `Press` | `Left`/`Right` | Move cursor left/right one character | +| Ctrl+A | `Press` | `A` (with Ctrl) | Select all text | +| Ctrl+Z | `Press` | `Z` (with Ctrl) | Undo | +| Ctrl+Y | `Press` | `Y` (with Ctrl) | Redo | +| Ctrl+F | `Press` | `F` (with Ctrl) | Show search bar | + +### 5.4 IME and Composition + +Android soft keyboards use IME composition for languages like Chinese, Japanese, or Korean. Gioui's `key.Event` (Type: `Edit`) handles the final committed text. Logic treats the committed text as a single insertion at the cursor. + +## 6. Scroll Momentum + +Pad implements a momentum model in the Logic goroutine to provide smooth scrolling. + +### 6.1 Scroll Sources + +| Source | Event | Behavior | +|---|---|---| +| **Mouse Wheel** | `pointer.Event` (Source: `MouseWheel`) | Direct scroll by `event.Scroll.Y` DP | +| **Finger Drag (No Focus)** | `pointer.Event` (Type: `Move`, no keyboard) | Direct 1:1 mapping of finger movement to `scroll_offset` | +| **Fling** | `pointer.Event` (Type: `Release`, high velocity) | Momentum-based scroll with exponential decay | + +### 6.2 Fling Logic + +When a `Release` event occurs with a high vertical velocity: + +1. **Velocity Calculation**: Logic tracks the position and timestamp of the last few `Move` events. On `Release`, it calculates the velocity (DP/ms). +2. **Momentum Start**: If velocity > threshold (e.g., 0.5 DP/ms), Logic enters a `Flinging` state. +3. **Decay**: On each subsequent frame (triggered by a timer or periodic `sendFrame()`), Logic updates `scroll_offset` by `velocity * elapsed_ms` and multiplies `velocity` by a decay factor (e.g., 0.95). +4. **Stop**: When velocity drops below a minimum (e.g., 0.01 DP/ms), Logic stops the fling and returns to `Idle`. + +### 6.3 Scroll Clamping + +`scroll_offset` is clamped to ensure the viewport never scrolls past the top or bottom of the file: +- **Min**: 0 (top of file) +- **Max**: `total_file_height - viewport_height` (bottom of file)