Pad/doc/touch.md

Touch and Input Handling

This document specifies how Pad handles low-level pointer and keyboard events using Gioui primitives, and how these are translated into editor actions (cursor movement, selection, scrolling).

1. Gioui Primitives

Pad avoids high-level widgets and uses the following low-level Gioui ops and events. Ops are submitted in the paint function; Events are returned by w.Event().

Op (submitted in paint)	Event (returned by w.Event())	Use Case
pointer.InputOp	pointer.Event	Defines hit-regions (editor, buttons, search bar). Captures Press, Release, Move, Drag, Scroll.
key.InputOp	key.Event	Enables keyboard focus. Captures Edit (text insertion), Press (Backspace, Enter, Arrows).
key.FocusOp	—	Requests keyboard focus for the active element.
key.SoftKeyboardOp	—	Explicitly shows/hides the Android soft keyboard.

1.1 Op/Event Flow

Because Logic does not have access to *op.Ops, the Op/Event flow is split:

1. Logic → Ops: Logic computes []Element. Each element carries metadata about what input ops it needs (e.g., "this TextField needs pointer.InputOp with Kind: Tap on region X").
2. Renderer → Ops: During the paint function, the Renderer iterates []Element and submits the appropriate pointer.InputOp / key.InputOp calls into *op.Ops.
3. Gioui → Events: Gioui returns pointer.Event / key.Event via w.Event() in subsequent cycles.
4. Main → Logic: The Main goroutine batches these events into []InputEvent and sends them to Logic.
5. Logic → State: Logic processes events, updates state, and computes a new frame.

This ensures that hit-regions are always defined in the same frame where the elements are drawn, and Logic remains testable without the SDK.

2. Coordinate Mapping

The editor operates in three coordinate spaces:

1. UI Space (DP): The raw (X, Y) coordinates from Gioui events, relative to the window origin.
2. Text Space (Lines/Cols): The logical position within the text file, accounting for scroll offset and word wrap.
3. Byte Space (Offsets): The raw byte offset in the UTF-8 file buffer.

2.1 Hit-Region Tags

Gioui pointer.InputOp takes a tag (an op.Tag). When an event fires, event.Tag identifies which region was touched.

Tagging Strategy:

• The Renderer assigns a unique tag per element (e.g., editor, search_bar, status_bar, alpha_index, merge_hunk_accept_ours).
• When Logic receives a pointer.Event, it first checks the tag to determine which element was touched, then applies element-specific coordinate logic.

2.2 Mapping UI → Byte Offset (Editor/TextField)

When a pointer.Event occurs at (Ex, Ey) on the editor tag:

1. Adjust for Scroll: Y = Ey + scroll_offset.
2. Identify Visual Line: visual_line = floor(Y / line_height).
3. Resolve to Logical Line: If word wrap is enabled, use the cached wrap positions to map visual_line → logical_line. Otherwise, they are the same.
4. Identify Column:
   • Retrieve text for logical_line from the chunked buffer.
   • Use the font shaper to measure glyph widths until the cumulative width exceeds Ex.
   • Clamp to the line length.
5. Result: Convert (logical_line, col_index) to a byte offset using the Line Index.

2.3 Mapping UI → Element (Non-Text Elements)

For buttons, search bar, alpha index, merge hunks:

• Each element has a Region() (rectangle). The Main goroutine checks if (Ex, Ey) falls within the region.
• For the Alpha Index, the Y-coordinate maps to a letter (e.g., letter = alphabet[floor((Y - alpha_top) / (alpha_height / 26))]).
• For Merge Hunks, the tag identifies which hunk button (accept-ours / accept-theirs) was tapped.

3. Gestures and State Machine

The Logic goroutine maintains a small state machine for the active gesture. The state is transient (does not survive process death) and is reset on gesture completion.

3.1 Gesture States

State	Entry Condition	Exit Condition	Action
Idle	Default state	Press received	—
Pressed	Press at (X, Y)	Release or Drag	Record press time and position
Tapping	Release received (short duration, no move)	Timeout or next Press	Move cursor, clear selection
DoubleTap	Second Tapping within 300ms	Gesture complete	Select word at (X, Y)
LongPress	Press held > 500ms	Release	Select word (future: magnifier)
Selecting	Drag received	Release	Update selection_end to current (X, Y)
Scrolling	Drag on non-focused area or two-finger drag	Release	Update scroll_offset

3.2 Gesture Disambiguation

The Logic goroutine distinguishes gestures using timing and movement thresholds:

• Tap vs Long Press: If Press → Release occurs within 500ms, it's a tap. Otherwise, it's a long press.
• Tap vs Drag: If the finger moves more than 16 DP between Press and Release, it's a drag. Otherwise, it's a tap.
• Single vs Double Tap: If two taps occur within 300ms, the second tap triggers a word selection.
• Select vs Scroll: If the keyboard is visible (focused TextField), a single-finger drag is a selection. If the keyboard is hidden, a single-finger drag is a scroll. Two-finger drags are always scrolls.

3.3 Selection Logic

Selection is defined by two byte offsets: selection_start and selection_end.

• If selection_start == selection_end, there is no selection (just a cursor).
• During a Drag gesture:
  • On Press: Set selection_start = selection_end = offset_at(X, Y).
  • On Drag: Update selection_end = offset_at(X, Y).
  • The UI renders a TextField with highlighted regions between the two offsets.

3.4 Double Click (Word Selection)

When a double-click is detected:

1. Identify the character at the click offset.
2. Expand left and right until a non-word character (space, punctuation, newline) is hit.
3. Set selection_start and selection_end to these boundaries.

4. Input Batching and Latency

As specified in architecture.md, the Main goroutine batches events. For touch, this is critical:

4.1 Event Compression in the Main Loop

The Main goroutine collects events during its event cycle. For a drag gesture, multiple pointer.Event (Type: Move) may fire before the next FrameEvent. The Main goroutine:

1. Accumulates all events into a []InputEvent slice.
2. On the next FrameEvent, it sends the entire batch to Logic via inputChan.
3. Logic processes the batch sequentially, updating the gesture state and cursor/selection.
4. Logic produces a single frame reflecting the final state after all events.

This means that if 3 Drag events fire in one frame, Logic sees all 3 and produces one frame with the cursor/selection at the final position. There is no "churn" of intermediate frames.

4.2 Latency Budget

• Event Capture: Gioui captures the touch event immediately (sub-ms).
• Batching: Main goroutine holds events until the next FrameEvent (up to 16ms, but typically < 8ms).
• Processing: Logic processes the batch and computes layout (< 16ms).
• Rendering: Frame receiver stores and invalidates (sub-ms).
• Total: < 32ms from touch to visual feedback (typically < 16ms).

4.3 Gesture Continuity

Because Logic maintains the gesture state machine across frames, a multi-frame drag gesture is continuous:

• Frame 1: Press at A → Logic enters Pressed state.
• Frame 2: Drag to B → Logic enters Selecting state, updates selection_end.
• Frame 3: Drag to C → Logic updates selection_end again.
• Frame 4: Release → Logic finalizes selection, enters Idle state.

Each frame produces a new []Element with the updated selection highlight.

5. Keyboard Interaction

The TextField element must be "focused" to receive keyboard events. Focus is a transient state managed by Logic.

5.1 Focus Acquisition

1. Tap on Editor: User taps the TextField region. Main goroutine sends pointer.Event (Type: Press) to Logic.
2. Focus Request: Logic sets focused = true and computes a frame with key.FocusOp and key.SoftKeyboardOp(true).
3. Keyboard Appears: Gioui shows the Android soft keyboard. Subsequent w.Event() calls return key.Event.

5.2 Focus Loss

1. Tap Outside Editor: User taps a non-editor region (e.g., status bar, directory browser). Logic sets focused = false and computes a frame with key.SoftKeyboardOp(false).
2. Keyboard Disappears: Gioui hides the soft keyboard.

5.3 Key Events

Event	Type	Name	Action
Text insertion	Edit	—	Insert event.Text at cursor, append to undo chain
Backspace	Press	Backspace	Delete character before cursor, append to delete chain
Enter	Press	Enter	Insert newline at cursor
Tab	Press	Tab	Insert spaces (configurable, e.g., 4 spaces)
Arrow Up/Down	Press	Up/Down	Move cursor up/down one line
Arrow Left/Right	Press	Left/Right	Move cursor left/right one character
Ctrl+A	Press	A (with Ctrl)	Select all text
Ctrl+Z	Press	Z (with Ctrl)	Undo
Ctrl+Y	Press	Y (with Ctrl)	Redo
Ctrl+F	Press	F (with Ctrl)	Show search bar

5.4 IME and Composition (Pure Gioui)

Android soft keyboards use IME composition for languages like Chinese, Japanese, or Korean. Gioui's key.Event (Type: Edit) handles the final committed text. Logic treats the committed text as a single insertion at the cursor.

Limitation: Gioui's key.InputOp on Android does not provide a full InputConnection. The IME has no context — it cannot read surrounding text for autocorrect, cannot anchor swipe gestures to our cursor, and cannot provide predictive suggestions. Only raw character commits are received.

5.5 Android IME Bridge via Fragment

To access the full suite of Android IME features (autocorrect, swipe typing, voice input, predictions), Pad uses a hidden EditText hosted in a native Fragment. This is the same pattern used by Passgo (see cmd/passgo-gui/impl_android.go, PgpConnect.java).

5.5.1 Fragment Registration

The Fragment is registered on app startup using app.ViewEvent:

1. Go side: handleEvent receives app.ViewEvent on app start. e.View is a uintptr pointing to the GioView (android.view.View).
2. C side (JNI): registerFragment() receives the jobject view:
   • Gets the Context from the View via getContext()
   • Gets the ClassLoader from the Context via getClassLoader()
   • Loads the Fragment class via findClass("pad/ime/ImeFragment")
   • Creates an instance by calling the constructor with the View
3. Java side (ImeFragment.java):
   • ImeFragment extends Fragment
   • Constructor receives the View (GioView), extracts Context
   • In onAttach(), casts Context → Activity
   • Uses act.getFragmentManager().beginTransaction().add(inst, "ImeFragment").commitNow()
   • Inflates a layout containing a transparent, zero-size EditText

5.5.2 IME Fragment Architecture

┌─────────────────────────────────────────────────────────────┐
│  Android Activity                                            │
│  ┌───────────────────────────────────────────────────────┐  │
│  │  GioView (OpenGL rendering)                           │  │
│  │  - pointer.InputOp → pointer.Event                    │  │
│  │  - key.InputOp → key.Event                            │  │
│  └───────────────────────────────────────────────────────┘  │
│  ┌───────────────────────────────────────────────────────┐  │
│  │  ImeFragment (transparent overlay)                     │  │
│  │  ┌─────────────────────────────────────────────────┐  │  │
│  │  │  EditText (hidden, zero-size)                    │  │  │
│  │  │  - Holds text window around cursor               │  │  │
│  │  │  - Selection synced to cursor position           │  │  │
│  │  │  - IME receives all keyboard events              │  │  │
│  │  └─────────────────────────────────────────────────┘  │  │
│  └───────────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────┘

5.5.3 Two-Way Sync: Gio ↔ EditText

The IME bridge maintains a tight sync loop between the Gio editor state and the hidden EditText:

Gio → EditText (Cursor Sync):

1. User taps to move cursor in Gio editor.
2. Logic updates cursor_pos in state.
3. Main goroutine calls a JNI function: SetEditTextSelection(byteOffset).
4. C side calls editText.setSelection(byteOffset) on the EditText.
5. IME now knows the cursor position and can provide context-aware suggestions.

EditText → Gio (IME Events):

1. User types/swipes/uses voice input on the soft keyboard.
2. IME commits text to the EditText via EditText.onTextChanged().
3. A TextWatcher on the EditText captures the change.
4. The change is sent via JNI callback to Go: ImeTextChanged(replacementText, start, before, count).
5. C side allocates memory for the string and calls a Go export function.
6. Go side creates a synthetic key.Event (Type: Edit) and sends it to the Main goroutine.
7. Main goroutine batches it into []InputEvent alongside any pointer/keyboard events.
8. Logic processes the synthetic event as a text insertion at the cursor.

5.5.4 Text Window Management

The EditText does not hold the entire file — only a text window around the cursor:

• Window size: ±1KB of text around the cursor (configurable).
• On cursor move: The Main goroutine sends a JNI call to update the EditText's text and selection.
• On text insertion/deletion: The Logic goroutine updates the in-memory buffer, then sends a JNI call to update the EditText's text and selection.

This ensures the IME always has context for suggestions, even for multi-gigabyte files.

5.5.5 IME Feature Support

Feature	Supported	How
Autocorrect	Yes	IME reads text window from EditText, suggests corrections
Swipe typing	Yes	IME anchors swipe to EditText selection (cursor position)
Voice input	Yes	IME commits voice text to EditText
Predictive bar	Yes	IME reads text window for context
Emoji keyboard	Yes	IME commits emoji to EditText
CJK composition	Yes	IME composition buffer → EditText → Gio

5.5.6 Event Merging in the Main Loop

IME events from the Fragment are merged with Gioui events in the Main goroutine:

Main goroutine event cycle:
  1. w.Event() → returns next event (FrameEvent, pointer.Event, key.Event)
  2. Check for pending IME events (non-blocking channel read)
  3. If IME event exists → create InputEvent, add to batch
  4. If Gioui event exists → create InputEvent, add to batch
  5. Send entire batch to Logic via inputChan

IME events are not returned by w.Event() — they arrive asynchronously via a JNI callback channel. The Main goroutine drains this channel in every cycle, ensuring IME events are batched alongside Gioui events and delivered to Logic atomically.

5.5.7 Implementation Notes

• The ImeFragment lives in cmd/pad/impl_android.go (Go), jni_android.c (C), and ime/ImeFragment.java (Java).
• The JNI callback uses a BlockingQueue or unbuffered channel to pass events from the Java main looper to the Main goroutine.
• The EditText is transparent (android:background="@android:color/transparent") and zero-size (android:layout_width="0dp" android:layout_height="0dp"), so it does not affect the UI.
• The EditText is not focusable by touch — it only receives focus programmatically when the Gio editor needs the keyboard.

5.6 Cut, Copy, and Paste

Pad implements its own clipboard (not Android's ClipboardManager) with persistent storage — the clipboard survives process death and app restart.

5.6.1 Clipboard Storage

The clipboard is stored as a string field in the Logic goroutine's state and persisted to disk:

• File: .pad/clipboard.json
• Format: {"text": "..."} (UTF-8 string)
• Write trigger: Every cut or copy operation triggers an async write to disk (debounced, 100ms).
• Load trigger: On app start, Logic reads .pad/clipboard.json and restores the clipboard.
• Atomic write: Written via temp file + rename in .pad/tmp/ (same pattern as auto-save).

This ensures the clipboard is available even if the system kills the app while it is in the background.

5.6.2 StatusBar Layout with Cut/Copy/Paste Icons

The StatusBar (top bar) has fixed icon slots for cut, copy, and paste. Icons appear/disappear without reflowing other elements.

Layout (vertical stack, top to bottom):

┌──────────────────────────────────────────────────────────────┐
│  filename.txt                                                │  ← Line 1: filename (truncated with ellipsis)
│  [Cut] [Copy]  Ln 10, Col 5  [Paste]  1024 / 50000  [Conf] │  ← Line 2: action icons + info
└──────────────────────────────────────────────────────────────┘

Fixed slot positions (in Dp, from the edges):

• Cut icon: leftmost action icon (24×24 DP, fixed position)
• Copy icon: second from left (24×24 DP, fixed position, 36 DP from Cut icon)
• Paste icon: second from right (24×24 DP, fixed position)
• Conflict icon: rightmost (24×24 DP, fixed position)
• Filename: top line, truncated with ellipsis if too long
• Line/col info: between Cut/Copy and Paste icons
• File size: between Paste and Conflict icons

Visibility rules:

Icon	Visible When
Cut	selection_start != selection_end
Copy	selection_start != selection_end
Paste	clipboard != ""
Conflict	sync conflict detected for active file

Icons are rendered as Button elements with fixed sizes. The StatusBar's Region height is computed dynamically based on whether the filename line is visible (2 lines when filename is shown, 1 line when in merge/search mode).

5.6.3 Cut Operation

1. User taps the Cut icon.
2. Logic extracts text: clipboard = buffer[selection_start:selection_end].
3. Logic writes clipboard to .pad/clipboard.json (async).
4. Logic deletes selected text from buffer (append to delete chain).
5. Logic sets selection_start = selection_end = cursor_pos (clears selection).
6. Logic produces a new frame (Cut/Copy icons hidden, Paste icon may appear).

5.6.4 Copy Operation

1. User taps the Copy icon.
2. Logic extracts text: clipboard = buffer[selection_start:selection_end].
3. Logic writes clipboard to .pad/clipboard.json (async).
4. Selection is not cleared (text remains in buffer).
5. Logic produces a new frame (Cut/Copy icons still visible, Paste icon appears).

5.6.5 Paste Operation

1. User taps the Paste icon.
2. Logic inserts clipboard text at cursor position.
3. Logic appends an insert operation to the undo chain.
4. Logic advances cursor past the inserted text.
5. Logic produces a new frame (no change to Cut/Copy/Paste visibility).

5.6.6 Input Routing

Cut, Copy, and Paste icons are tagged with unique op.Tag values. When the user taps an icon:

1. Main goroutine receives pointer.Event (Type: Press) with the icon's tag.
2. Main goroutine sends InputEvent{ElementID: "cut"}, InputEvent{ElementID: "copy"}, or InputEvent{ElementID: "paste"} to Logic.
3. Logic processes the event and updates state.

5.6.7 Clipboard State Machine

The clipboard is a simple string field. It has no state machine — it is either empty or contains text. On cut/copy, the text is replaced. On paste, the text is unchanged.

Edge cases:

• Paste into empty clipboard: No-op (Paste icon is hidden when clipboard is empty).
• Cut with no selection: No-op (Cut icon is hidden when there is no selection).
• Copy with no selection: No-op (Copy icon is hidden when there is no selection).
• Multiple cuts/copies: Each cut/copy replaces the previous clipboard content.
• Clipboard persistence: If the app is killed, the clipboard is restored from .pad/clipboard.json on restart.

5.7 Filename Display with Ellipsis Toggle

The filename is displayed on a separate line at the top of the StatusBar. When the filename is too long to fit the screen width, it is truncated with an ellipsis (...). Tapping the ellipsis toggles between the truncated single-line view and a full multi-line view.

5.7.1 Truncation Behavior

• Truncated view: Filename is truncated to fit the screen width, with ... at the end. Example: very_long_filename...
• Full view: Filename is displayed in full, potentially spanning multiple lines. Example: very_long_filename_that_does_not_fit_on_a_single_line.txt

5.7.2 Toggle Interaction

1. Tap on ellipsis: Toggling between truncated and full views.
2. Tap elsewhere on filename: No action (filename text is not interactive).
3. Ellipsis region: The ellipsis (...) is a separate Button element with its own op.Tag. Tapping it triggers the toggle.

5.7.3 Input Routing

• The ellipsis is tagged with op.Tag value "filename_toggle".
• When the user taps the ellipsis, the Main goroutine sends InputEvent{ElementID: "filename_toggle"} to Logic.
• Logic toggles the filename_expanded state field.
• Logic produces a new frame with the updated filename display.

5.7.4 State Field

The Logic goroutine maintains a boolean field filename_expanded:

• false (default): truncated view with ellipsis.
• true: full multi-line view.

This field is not persisted to disk — it is reset on app restart (same as gesture state).

5.7.5 Layout

In the truncated view, the StatusBar has 2 lines:

Line 1: filename.txt...
Line 2: [Cut] [Copy]  Ln 10, Col 5  [Paste]  1024 / 50000  [Conf]

In the full view, the StatusBar has 3+ lines (depending on filename length):

Line 1: very_long_filename_that_does_not_fit_on_a_single_line.txt
Line 2: [Cut] [Copy]  Ln 10, Col 5  [Paste]  1024 / 50000  [Conf]

The Region.H of the StatusBar is computed dynamically based on the number of lines.

6. Scroll Momentum

Pad implements a momentum model in the Logic goroutine to provide smooth scrolling.

6.1 Scroll Sources

Source	Event	Behavior
Mouse Wheel	pointer.Event (Source: MouseWheel)	Direct scroll by event.Scroll.Y DP
Finger Drag (No Focus)	pointer.Event (Type: Move, no keyboard)	Direct 1:1 mapping of finger movement to scroll_offset
Fling	pointer.Event (Type: Release, high velocity)	Momentum-based scroll with exponential decay

6.2 Fling Logic

When a Release event occurs with a high vertical velocity:

1. Velocity Calculation: Logic tracks the position and timestamp of the last few Move events. On Release, it calculates the velocity (DP/ms).
2. Momentum Start: If velocity > threshold (e.g., 0.5 DP/ms), Logic enters a Flinging state.
3. Decay: On each subsequent frame (triggered by a timer or periodic sendFrame()), Logic updates scroll_offset by velocity * elapsed_ms and multiplies velocity by a decay factor (e.g., 0.95).
4. Stop: When velocity drops below a minimum (e.g., 0.01 DP/ms), Logic stops the fling and returns to Idle.

6.3 Scroll Clamping

scroll_offset is clamped to ensure the viewport never scrolls past the top or bottom of the file:

• Min: 0 (top of file)
• Max: total_file_height - viewport_height (bottom of file)