Pad/doc/touch.md
Greg Pomerantz e4c5804170 Add touch and input handling specification (doc/touch.md)
- Gioui primitives: pointer.InputOp, key.InputOp, key.FocusOp, SoftKeyboardOp
- Op/Event flow: Logic computes elements, Renderer submits ops, Main batches events
- Coordinate mapping: UI space → text space → byte space, with hit-region tags
- Gesture state machine: tap, double-tap, long-press, drag-select, scroll
- Gesture disambiguation: timing thresholds (500ms long-press, 300ms double-tap, 16DP drag)
- Selection logic: selection_start/selection_end byte offsets
- Input batching: Main accumulates events, sends batch to Logic per frame
- Keyboard interaction: focus acquisition/loss, key event table, IME composition
- Scroll momentum: fling logic with exponential decay, scroll clamping

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-05-08 12:01:06 -04:00

198 lines
11 KiB
Markdown

# Touch and Input Handling
This document specifies how Pad handles low-level pointer and keyboard events using Gioui primitives, and how these are translated into editor actions (cursor movement, selection, scrolling).
## 1. Gioui Primitives
Pad avoids high-level widgets and uses the following low-level Gioui ops and events. Ops are submitted in the paint function; Events are returned by `w.Event()`.
| Op (submitted in paint) | Event (returned by w.Event()) | Use Case |
|---|---|---|
| `pointer.InputOp` | `pointer.Event` | Defines hit-regions (editor, buttons, search bar). Captures `Press`, `Release`, `Move`, `Drag`, `Scroll`. |
| `key.InputOp` | `key.Event` | Enables keyboard focus. Captures `Edit` (text insertion), `Press` (Backspace, Enter, Arrows). |
| `key.FocusOp` | — | Requests keyboard focus for the active element. |
| `key.SoftKeyboardOp` | — | Explicitly shows/hides the Android soft keyboard. |
### 1.1 Op/Event Flow
Because Logic does not have access to `*op.Ops`, the Op/Event flow is split:
1. **Logic → Ops**: Logic computes `[]Element`. Each element carries metadata about what input ops it needs (e.g., "this TextField needs `pointer.InputOp` with `Kind: Tap` on region X").
2. **Renderer → Ops**: During the paint function, the Renderer iterates `[]Element` and submits the appropriate `pointer.InputOp` / `key.InputOp` calls into `*op.Ops`.
3. **Gioui → Events**: Gioui returns `pointer.Event` / `key.Event` via `w.Event()` in subsequent cycles.
4. **Main → Logic**: The Main goroutine batches these events into `[]InputEvent` and sends them to Logic.
5. **Logic → State**: Logic processes events, updates state, and computes a new frame.
This ensures that hit-regions are always defined in the same frame where the elements are drawn, and Logic remains testable without the SDK.
## 2. Coordinate Mapping
The editor operates in three coordinate spaces:
1. **UI Space (DP)**: The raw (X, Y) coordinates from Gioui events, relative to the window origin.
2. **Text Space (Lines/Cols)**: The logical position within the text file, accounting for scroll offset and word wrap.
3. **Byte Space (Offsets)**: The raw byte offset in the UTF-8 file buffer.
### 2.1 Hit-Region Tags
Gioui `pointer.InputOp` takes a `tag` (an `op.Tag`). When an event fires, `event.Tag` identifies which region was touched.
**Tagging Strategy**:
- The Renderer assigns a unique tag per element (e.g., `editor`, `search_bar`, `status_bar`, `alpha_index`, `merge_hunk_accept_ours`).
- When Logic receives a `pointer.Event`, it first checks the tag to determine which element was touched, then applies element-specific coordinate logic.
### 2.2 Mapping UI → Byte Offset (Editor/TextField)
When a `pointer.Event` occurs at `(Ex, Ey)` on the `editor` tag:
1. **Adjust for Scroll**: `Y = Ey + scroll_offset`.
2. **Identify Visual Line**: `visual_line = floor(Y / line_height)`.
3. **Resolve to Logical Line**: If word wrap is enabled, use the cached wrap positions to map `visual_line``logical_line`. Otherwise, they are the same.
4. **Identify Column**:
- Retrieve text for `logical_line` from the chunked buffer.
- Use the font shaper to measure glyph widths until the cumulative width exceeds `Ex`.
- Clamp to the line length.
5. **Result**: Convert `(logical_line, col_index)` to a byte offset using the **Line Index**.
### 2.3 Mapping UI → Element (Non-Text Elements)
For buttons, search bar, alpha index, merge hunks:
- Each element has a `Region()` (rectangle). The Main goroutine checks if `(Ex, Ey)` falls within the region.
- For the **Alpha Index**, the Y-coordinate maps to a letter (e.g., `letter = alphabet[floor((Y - alpha_top) / (alpha_height / 26))]`).
- For **Merge Hunks**, the tag identifies which hunk button (accept-ours / accept-theirs) was tapped.
## 3. Gestures and State Machine
The Logic goroutine maintains a small state machine for the active gesture. The state is transient (does not survive process death) and is reset on gesture completion.
### 3.1 Gesture States
| State | Entry Condition | Exit Condition | Action |
|---|---|---|---|
| `Idle` | Default state | `Press` received | — |
| `Pressed` | `Press` at (X, Y) | `Release` or `Drag` | Record press time and position |
| `Tapping` | `Release` received (short duration, no move) | Timeout or next `Press` | Move cursor, clear selection |
| `DoubleTap` | Second `Tapping` within 300ms | Gesture complete | Select word at (X, Y) |
| `LongPress` | `Press` held > 500ms | `Release` | Select word (future: magnifier) |
| `Selecting` | `Drag` received | `Release` | Update `selection_end` to current (X, Y) |
| `Scrolling` | `Drag` on non-focused area or two-finger drag | `Release` | Update `scroll_offset` |
### 3.2 Gesture Disambiguation
The Logic goroutine distinguishes gestures using timing and movement thresholds:
- **Tap vs Long Press**: If `Press``Release` occurs within 500ms, it's a tap. Otherwise, it's a long press.
- **Tap vs Drag**: If the finger moves more than 16 DP between `Press` and `Release`, it's a drag. Otherwise, it's a tap.
- **Single vs Double Tap**: If two taps occur within 300ms, the second tap triggers a word selection.
- **Select vs Scroll**: If the keyboard is visible (focused `TextField`), a single-finger drag is a selection. If the keyboard is hidden, a single-finger drag is a scroll. Two-finger drags are always scrolls.
### 3.3 Selection Logic
Selection is defined by two byte offsets: `selection_start` and `selection_end`.
- If `selection_start == selection_end`, there is no selection (just a cursor).
- During a **Drag** gesture:
- On `Press`: Set `selection_start = selection_end = offset_at(X, Y)`.
- On `Drag`: Update `selection_end = offset_at(X, Y)`.
- The UI renders a `TextField` with highlighted regions between the two offsets.
### 3.4 Double Click (Word Selection)
When a double-click is detected:
1. Identify the character at the click offset.
2. Expand left and right until a non-word character (space, punctuation, newline) is hit.
3. Set `selection_start` and `selection_end` to these boundaries.
## 4. Input Batching and Latency
As specified in `architecture.md`, the Main goroutine batches events. For touch, this is critical:
### 4.1 Event Compression in the Main Loop
The Main goroutine collects events during its event cycle. For a drag gesture, multiple `pointer.Event` (Type: `Move`) may fire before the next `FrameEvent`. The Main goroutine:
1. Accumulates all events into a `[]InputEvent` slice.
2. On the next `FrameEvent`, it sends the entire batch to Logic via `inputChan`.
3. Logic processes the batch sequentially, updating the gesture state and cursor/selection.
4. Logic produces a single frame reflecting the final state after all events.
This means that if 3 `Drag` events fire in one frame, Logic sees all 3 and produces one frame with the cursor/selection at the final position. There is no "churn" of intermediate frames.
### 4.2 Latency Budget
- **Event Capture**: Gioui captures the touch event immediately (sub-ms).
- **Batching**: Main goroutine holds events until the next `FrameEvent` (up to 16ms, but typically < 8ms).
- **Processing**: Logic processes the batch and computes layout (< 16ms).
- **Rendering**: Frame receiver stores and invalidates (sub-ms).
- **Total**: < 32ms from touch to visual feedback (typically < 16ms).
### 4.3 Gesture Continuity
Because Logic maintains the gesture state machine across frames, a multi-frame drag gesture is continuous:
- Frame 1: `Press` at A Logic enters `Pressed` state.
- Frame 2: `Drag` to B Logic enters `Selecting` state, updates `selection_end`.
- Frame 3: `Drag` to C Logic updates `selection_end` again.
- Frame 4: `Release` Logic finalizes selection, enters `Idle` state.
Each frame produces a new `[]Element` with the updated selection highlight.
## 5. Keyboard Interaction
The `TextField` element must be "focused" to receive keyboard events. Focus is a transient state managed by Logic.
### 5.1 Focus Acquisition
1. **Tap on Editor**: User taps the `TextField` region. Main goroutine sends `pointer.Event` (Type: `Press`) to Logic.
2. **Focus Request**: Logic sets `focused = true` and computes a frame with `key.FocusOp` and `key.SoftKeyboardOp(true)`.
3. **Keyboard Appears**: Gioui shows the Android soft keyboard. Subsequent `w.Event()` calls return `key.Event`.
### 5.2 Focus Loss
1. **Tap Outside Editor**: User taps a non-editor region (e.g., status bar, directory browser). Logic sets `focused = false` and computes a frame with `key.SoftKeyboardOp(false)`.
2. **Keyboard Disappears**: Gioui hides the soft keyboard.
### 5.3 Key Events
| Event | Type | Name | Action |
|---|---|---|---|
| Text insertion | `Edit` | | Insert `event.Text` at cursor, append to undo chain |
| Backspace | `Press` | `Backspace` | Delete character before cursor, append to delete chain |
| Enter | `Press` | `Enter` | Insert newline at cursor |
| Tab | `Press` | `Tab` | Insert spaces (configurable, e.g., 4 spaces) |
| Arrow Up/Down | `Press` | `Up`/`Down` | Move cursor up/down one line |
| Arrow Left/Right | `Press` | `Left`/`Right` | Move cursor left/right one character |
| Ctrl+A | `Press` | `A` (with Ctrl) | Select all text |
| Ctrl+Z | `Press` | `Z` (with Ctrl) | Undo |
| Ctrl+Y | `Press` | `Y` (with Ctrl) | Redo |
| Ctrl+F | `Press` | `F` (with Ctrl) | Show search bar |
### 5.4 IME and Composition
Android soft keyboards use IME composition for languages like Chinese, Japanese, or Korean. Gioui's `key.Event` (Type: `Edit`) handles the final committed text. Logic treats the committed text as a single insertion at the cursor.
## 6. Scroll Momentum
Pad implements a momentum model in the Logic goroutine to provide smooth scrolling.
### 6.1 Scroll Sources
| Source | Event | Behavior |
|---|---|---|
| **Mouse Wheel** | `pointer.Event` (Source: `MouseWheel`) | Direct scroll by `event.Scroll.Y` DP |
| **Finger Drag (No Focus)** | `pointer.Event` (Type: `Move`, no keyboard) | Direct 1:1 mapping of finger movement to `scroll_offset` |
| **Fling** | `pointer.Event` (Type: `Release`, high velocity) | Momentum-based scroll with exponential decay |
### 6.2 Fling Logic
When a `Release` event occurs with a high vertical velocity:
1. **Velocity Calculation**: Logic tracks the position and timestamp of the last few `Move` events. On `Release`, it calculates the velocity (DP/ms).
2. **Momentum Start**: If velocity > threshold (e.g., 0.5 DP/ms), Logic enters a `Flinging` state.
3. **Decay**: On each subsequent frame (triggered by a timer or periodic `sendFrame()`), Logic updates `scroll_offset` by `velocity * elapsed_ms` and multiplies `velocity` by a decay factor (e.g., 0.95).
4. **Stop**: When velocity drops below a minimum (e.g., 0.01 DP/ms), Logic stops the fling and returns to `Idle`.
### 6.3 Scroll Clamping
`scroll_offset` is clamped to ensure the viewport never scrolls past the top or bottom of the file:
- **Min**: 0 (top of file)
- **Max**: `total_file_height - viewport_height` (bottom of file)