Pad/doc/touch.md

# Touch and Input Handling

This document specifies how Pad handles low-level pointer and keyboard events using Gioui primitives, and how these are translated into editor actions (cursor movement, selection, scrolling).

## 1. Gioui Primitives

Pad avoids high-level widgets and uses the following low-level Gioui ops and events. Ops are submitted in the paint function; Events are returned by `w.Event()`.

| Op (submitted in paint) | Event (returned by w.Event()) | Use Case |
|---|---|---|
| `pointer.InputOp` | `pointer.Event` | Defines hit-regions (editor, buttons, search bar). Captures `Press`, `Release`, `Move`, `Drag`, `Scroll`. |
| `key.InputOp` | `key.Event` | Enables keyboard focus. Captures `Edit` (text insertion), `Press` (Backspace, Enter, Arrows). |
| `key.FocusOp` | — | Requests keyboard focus for the active element. |
| `key.SoftKeyboardOp` | — | Explicitly shows/hides the Android soft keyboard. |

### 1.1 Op/Event Flow

Because Logic does not have access to `*op.Ops`, the Op/Event flow is split:

1. **Logic → Ops**: Logic computes `[]Element`. Each element carries metadata about what input ops it needs (e.g., "this TextField needs `pointer.InputOp` with `Kind: Tap` on region X").
2. **Renderer → Ops**: During the paint function, the Renderer iterates `[]Element` and submits the appropriate `pointer.InputOp` / `key.InputOp` calls into `*op.Ops`.
3. **Gioui → Events**: Gioui returns `pointer.Event` / `key.Event` via `w.Event()` in subsequent cycles.
4. **Main → Logic**: The Main goroutine batches these events into `[]InputEvent` and sends them to Logic.
5. **Logic → State**: Logic processes events, updates state, and computes a new frame.

This ensures that hit-regions are always defined in the same frame where the elements are drawn, and Logic remains testable without the SDK.

## 2. Coordinate Mapping

The editor operates in three coordinate spaces:

1.  **UI Space (DP)**: The raw (X, Y) coordinates from Gioui events, relative to the window origin.
2.  **Text Space (Lines/Cols)**: The logical position within the text file, accounting for scroll offset and word wrap.
3.  **Byte Space (Offsets)**: The raw byte offset in the UTF-8 file buffer.

### 2.1 Hit-Region Tags

Gioui `pointer.InputOp` takes a `tag` (an `op.Tag`). When an event fires, `event.Tag` identifies which region was touched.

**Tagging Strategy**:
- The Renderer assigns a unique tag per element (e.g., `editor`, `search_bar`, `status_bar`, `alpha_index`, `merge_hunk_accept_ours`).
- When Logic receives a `pointer.Event`, it first checks the tag to determine which element was touched, then applies element-specific coordinate logic.

### 2.2 Mapping UI → Byte Offset (Editor/TextField)

When a `pointer.Event` occurs at `(Ex, Ey)` on the `editor` tag:

1.  **Adjust for Scroll**: `Y = Ey + scroll_offset`.
2.  **Identify Visual Line**: `visual_line = floor(Y / line_height)`.
3.  **Resolve to Logical Line**: If word wrap is enabled, use the cached wrap positions to map `visual_line` → `logical_line`. Otherwise, they are the same.
4.  **Identify Column**:
    - Retrieve text for `logical_line` from the chunked buffer.
    - Use the font shaper to measure glyph widths until the cumulative width exceeds `Ex`.
    - Clamp to the line length.
5.  **Result**: Convert `(logical_line, col_index)` to a byte offset using the **Line Index**.

### 2.3 Mapping UI → Element (Non-Text Elements)

For buttons, search bar, alpha index, merge hunks:
- Each element has a `Region()` (rectangle). The Main goroutine checks if `(Ex, Ey)` falls within the region.
- For the **Alpha Index**, the Y-coordinate maps to a letter (e.g., `letter = alphabet[floor((Y - alpha_top) / (alpha_height / 26))]`).
- For **Merge Hunks**, the tag identifies which hunk button (accept-ours / accept-theirs) was tapped.

## 3. Gestures and State Machine

The Logic goroutine maintains a small state machine for the active gesture. The state is transient (does not survive process death) and is reset on gesture completion.

### 3.1 Gesture States

| State | Entry Condition | Exit Condition | Action |
|---|---|---|---|
| `Idle` | Default state | `Press` received | — |
| `Pressed` | `Press` at (X, Y) | `Release` or `Drag` | Record press time and position |
| `Tapping` | `Release` received (short duration, no move) | Timeout or next `Press` | Move cursor, clear selection |
| `DoubleTap` | Second `Tapping` within 300ms | Gesture complete | Select word at (X, Y) |
| `LongPress` | `Press` held > 500ms | `Release` | Select word (future: magnifier) |
| `Selecting` | `Drag` received | `Release` | Update `selection_end` to current (X, Y) |
| `Scrolling` | `Drag` on non-focused area or two-finger drag | `Release` | Update `scroll_offset` |

### 3.2 Gesture Disambiguation

The Logic goroutine distinguishes gestures using timing and movement thresholds:

- **Tap vs Long Press**: If `Press` → `Release` occurs within 500ms, it's a tap. Otherwise, it's a long press.
- **Tap vs Drag**: If the finger moves more than 16 DP between `Press` and `Release`, it's a drag. Otherwise, it's a tap.
- **Single vs Double Tap**: If two taps occur within 300ms, the second tap triggers a word selection.
- **Select vs Scroll**: If the keyboard is visible (focused `TextField`), a single-finger drag is a selection. If the keyboard is hidden, a single-finger drag is a scroll. Two-finger drags are always scrolls.

### 3.3 Selection Logic

Selection is defined by two byte offsets: `selection_start` and `selection_end`.
- If `selection_start == selection_end`, there is no selection (just a cursor).
- During a **Drag** gesture:
    - On `Press`: Set `selection_start = selection_end = offset_at(X, Y)`.
    - On `Drag`: Update `selection_end = offset_at(X, Y)`.
    - The UI renders a `TextField` with highlighted regions between the two offsets.

### 3.4 Double Click (Word Selection)

When a double-click is detected:
1.  Identify the character at the click offset.
2.  Expand left and right until a non-word character (space, punctuation, newline) is hit.
3.  Set `selection_start` and `selection_end` to these boundaries.

## 4. Input Batching and Latency

As specified in `architecture.md`, the Main goroutine batches events. For touch, this is critical:

### 4.1 Event Compression in the Main Loop

The Main goroutine collects events during its event cycle. For a drag gesture, multiple `pointer.Event` (Type: `Move`) may fire before the next `FrameEvent`. The Main goroutine:

1.  Accumulates all events into a `[]InputEvent` slice.
2.  On the next `FrameEvent`, it sends the entire batch to Logic via `inputChan`.
3.  Logic processes the batch sequentially, updating the gesture state and cursor/selection.
4.  Logic produces a single frame reflecting the final state after all events.

This means that if 3 `Drag` events fire in one frame, Logic sees all 3 and produces one frame with the cursor/selection at the final position. There is no "churn" of intermediate frames.

### 4.2 Latency Budget

- **Event Capture**: Gioui captures the touch event immediately (sub-ms).
- **Batching**: Main goroutine holds events until the next `FrameEvent` (up to 16ms, but typically < 8ms).
- **Processing**: Logic processes the batch and computes layout (< 16ms).
- **Rendering**: Frame receiver stores and invalidates (sub-ms).
- **Total**: < 32ms from touch to visual feedback (typically < 16ms).

### 4.3 Gesture Continuity

Because Logic maintains the gesture state machine across frames, a multi-frame drag gesture is continuous:
- Frame 1: `Press` at A → Logic enters `Pressed` state.
- Frame 2: `Drag` to B → Logic enters `Selecting` state, updates `selection_end`.
- Frame 3: `Drag` to C → Logic updates `selection_end` again.
- Frame 4: `Release` → Logic finalizes selection, enters `Idle` state.

Each frame produces a new `[]Element` with the updated selection highlight.

## 5. Keyboard Interaction

The `TextField` element must be "focused" to receive keyboard events. Focus is a transient state managed by Logic.

### 5.1 Focus Acquisition

1.  **Tap on Editor**: User taps the `TextField` region. Main goroutine sends `pointer.Event` (Type: `Press`) to Logic.
2.  **Focus Request**: Logic sets `focused = true` and computes a frame with `key.FocusOp` and `key.SoftKeyboardOp(true)`.
3.  **Keyboard Appears**: Gioui shows the Android soft keyboard. Subsequent `w.Event()` calls return `key.Event`.

### 5.2 Focus Loss

1.  **Tap Outside Editor**: User taps a non-editor region (e.g., status bar, directory browser). Logic sets `focused = false` and computes a frame with `key.SoftKeyboardOp(false)`.
2.  **Keyboard Disappears**: Gioui hides the soft keyboard.

### 5.3 Key Events

| Event | Type | Name | Action |
|---|---|---|---|
| Text insertion | `Edit` | — | Insert `event.Text` at cursor, append to undo chain |
| Backspace | `Press` | `Backspace` | Delete character before cursor, append to delete chain |
| Enter | `Press` | `Enter` | Insert newline at cursor |
| Tab | `Press` | `Tab` | Insert spaces (configurable, e.g., 4 spaces) |
| Arrow Up/Down | `Press` | `Up`/`Down` | Move cursor up/down one line |
| Arrow Left/Right | `Press` | `Left`/`Right` | Move cursor left/right one character |
| Ctrl+A | `Press` | `A` (with Ctrl) | Select all text |
| Ctrl+Z | `Press` | `Z` (with Ctrl) | Undo |
| Ctrl+Y | `Press` | `Y` (with Ctrl) | Redo |
| Ctrl+F | `Press` | `F` (with Ctrl) | Show search bar |

### 5.4 IME and Composition (Pure Gioui)

Android soft keyboards use IME composition for languages like Chinese, Japanese, or Korean. Gioui's `key.Event` (Type: `Edit`) handles the final committed text. Logic treats the committed text as a single insertion at the cursor.

**Limitation**: Gioui's `key.InputOp` on Android does not provide a full `InputConnection`. The IME has no context — it cannot read surrounding text for autocorrect, cannot anchor swipe gestures to our cursor, and cannot provide predictive suggestions. Only raw character commits are received.

## 5.5 Android IME Bridge via Fragment

To access the full suite of Android IME features (autocorrect, swipe typing, voice input, predictions), Pad uses a **hidden `EditText`** hosted in a native `Fragment`. This is the same pattern used by Passgo (see `cmd/passgo-gui/impl_android.go`, `PgpConnect.java`).

### 5.5.1 Fragment Registration

The Fragment is registered on app startup using `app.ViewEvent`:

1. **Go side**: `handleEvent` receives `app.ViewEvent` on app start. `e.View` is a `uintptr` pointing to the `GioView` (`android.view.View`).
2. **C side (JNI)**: `registerFragment()` receives the `jobject view`:
   - Gets the `Context` from the View via `getContext()`
   - Gets the `ClassLoader` from the Context via `getClassLoader()`
   - Loads the Fragment class via `findClass("pad/ime/ImeFragment")`
   - Creates an instance by calling the constructor with the View
3. **Java side (`ImeFragment.java`)**:
   - `ImeFragment` extends `Fragment`
   - Constructor receives the `View` (GioView), extracts `Context`
   - In `onAttach()`, casts `Context` → `Activity`
   - Uses `act.getFragmentManager().beginTransaction().add(inst, "ImeFragment").commitNow()`
   - Inflates a layout containing a transparent, zero-size `EditText`

### 5.5.2 IME Fragment Architecture

```
┌─────────────────────────────────────────────────────────────┐
│  Android Activity                                            │
│  ┌───────────────────────────────────────────────────────┐  │
│  │  GioView (OpenGL rendering)                           │  │
│  │  - pointer.InputOp → pointer.Event                    │  │
│  │  - key.InputOp → key.Event                            │  │
│  └───────────────────────────────────────────────────────┘  │
│  ┌───────────────────────────────────────────────────────┐  │
│  │  ImeFragment (transparent overlay)                     │  │
│  │  ┌─────────────────────────────────────────────────┐  │  │
│  │  │  EditText (hidden, zero-size)                    │  │  │
│  │  │  - Holds text window around cursor               │  │  │
│  │  │  - Selection synced to cursor position           │  │  │
│  │  │  - IME receives all keyboard events              │  │  │
│  │  └─────────────────────────────────────────────────┘  │  │
│  └───────────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────┘
```

### 5.5.3 Two-Way Sync: Gio ↔ EditText

The IME bridge maintains a tight sync loop between the Gio editor state and the hidden `EditText`:

**Gio → EditText (Cursor Sync)**:
1. User taps to move cursor in Gio editor.
2. Logic updates `cursor_pos` in state.
3. Main goroutine calls a JNI function: `SetEditTextSelection(byteOffset)`.
4. C side calls `editText.setSelection(byteOffset)` on the `EditText`.
5. IME now knows the cursor position and can provide context-aware suggestions.

**EditText → Gio (IME Events)**:
1. User types/swipes/uses voice input on the soft keyboard.
2. IME commits text to the `EditText` via `EditText.onTextChanged()`.
3. A `TextWatcher` on the `EditText` captures the change.
4. The change is sent via JNI callback to Go: `ImeTextChanged(replacementText, start, before, count)`.
5. C side allocates memory for the string and calls a Go export function.
6. Go side creates a synthetic `key.Event` (Type: `Edit`) and sends it to the Main goroutine.
7. Main goroutine batches it into `[]InputEvent` alongside any pointer/keyboard events.
8. Logic processes the synthetic event as a text insertion at the cursor.

### 5.5.4 Text Window Management

The `EditText` does not hold the entire file — only a **text window** around the cursor:

- **Window size**: ±1KB of text around the cursor (configurable).
- **On cursor move**: The Main goroutine sends a JNI call to update the `EditText`'s text and selection.
- **On text insertion/deletion**: The Logic goroutine updates the in-memory buffer, then sends a JNI call to update the `EditText`'s text and selection.

This ensures the IME always has context for suggestions, even for multi-gigabyte files.

### 5.5.5 IME Feature Support

| Feature | Supported | How |
|---|---|---|
| Autocorrect | Yes | IME reads text window from `EditText`, suggests corrections |
| Swipe typing | Yes | IME anchors swipe to `EditText` selection (cursor position) |
| Voice input | Yes | IME commits voice text to `EditText` |
| Predictive bar | Yes | IME reads text window for context |
| Emoji keyboard | Yes | IME commits emoji to `EditText` |
| CJK composition | Yes | IME composition buffer → `EditText` → Gio |

### 5.5.6 Event Merging in the Main Loop

IME events from the Fragment are merged with Gioui events in the Main goroutine:

```
Main goroutine event cycle:
  1. w.Event() → returns next event (FrameEvent, pointer.Event, key.Event)
  2. Check for pending IME events (non-blocking channel read)
  3. If IME event exists → create InputEvent, add to batch
  4. If Gioui event exists → create InputEvent, add to batch
  5. Send entire batch to Logic via inputChan
```

IME events are **not** returned by `w.Event()` — they arrive asynchronously via a JNI callback channel. The Main goroutine drains this channel in every cycle, ensuring IME events are batched alongside Gioui events and delivered to Logic atomically.

### 5.5.7 Implementation Notes

- The `ImeFragment` lives in `cmd/pad/impl_android.go` (Go), `jni_android.c` (C), and `ime/ImeFragment.java` (Java).
- The JNI callback uses a `BlockingQueue` or unbuffered channel to pass events from the Java main looper to the Main goroutine.
- The `EditText` is transparent (`android:background="@android:color/transparent"`) and zero-size (`android:layout_width="0dp" android:layout_height="0dp"`), so it does not affect the UI.
- The `EditText` is **not** focusable by touch — it only receives focus programmatically when the Gio editor needs the keyboard.

## 6. Scroll Momentum

Pad implements a momentum model in the Logic goroutine to provide smooth scrolling.

### 6.1 Scroll Sources

| Source | Event | Behavior |
|---|---|---|
| **Mouse Wheel** | `pointer.Event` (Source: `MouseWheel`) | Direct scroll by `event.Scroll.Y` DP |
| **Finger Drag (No Focus)** | `pointer.Event` (Type: `Move`, no keyboard) | Direct 1:1 mapping of finger movement to `scroll_offset` |
| **Fling** | `pointer.Event` (Type: `Release`, high velocity) | Momentum-based scroll with exponential decay |

### 6.2 Fling Logic

When a `Release` event occurs with a high vertical velocity:

1.  **Velocity Calculation**: Logic tracks the position and timestamp of the last few `Move` events. On `Release`, it calculates the velocity (DP/ms).
2.  **Momentum Start**: If velocity > threshold (e.g., 0.5 DP/ms), Logic enters a `Flinging` state.
3.  **Decay**: On each subsequent frame (triggered by a timer or periodic `sendFrame()`), Logic updates `scroll_offset` by `velocity * elapsed_ms` and multiplies `velocity` by a decay factor (e.g., 0.95).
4.  **Stop**: When velocity drops below a minimum (e.g., 0.01 DP/ms), Logic stops the fling and returns to `Idle`.

### 6.3 Scroll Clamping

`scroll_offset` is clamped to ensure the viewport never scrolls past the top or bottom of the file:
- **Min**: 0 (top of file)
- **Max**: `total_file_height - viewport_height` (bottom of file)