Pad/doc/SPEC.md

# Pad — Text Editor Specification

## 1. Overview

Pad is a minimal, high-performance plain text editor for Android. It is configured with a single directory of text files and provides instant file access, automatic persistence, and seamless recovery from process death. The editor is designed for a directory synced across devices, with reactive awareness of external file modifications.

**Design philosophy:**

- **No practical limits** — files and directories of arbitrary size must work without degradation
- **Minimal surface area** — plain text only, no formatting, no file management UI
- **Automatic everything** — save, state persistence, sync awareness happen without user intervention
- **Survive anything** — process kill, battery death, device restart: the editor restores exactly where the user left off

## 2. Core Requirements

| Requirement | Detail |
|---|---|
| Instant open | Files open immediately regardless of size; content loads on demand |
| Auto-save | Every edit is persisted to disk with a short debounce; no save button exists |
| Unlimited scale | No caps on file size, line count, or directory entry count |
| External change detection | The editor monitors the configured directory for modifications from sync tools |
| Cross-session undo | Full undo history survives process death, file close, and device restart |
| State restoration | On relaunch, the editor restores the exact file, cursor position, and scroll state |
| Per-file cursor memory | Reopening any previously visited file restores the last cursor position |
| Plain text only | No formatting, no syntax highlighting, no markdown rendering |

## 3. Architecture

```
cmd/
  pad/                  # Mobile entry point (Gioui window + frame loop)

internal/
  browser/              # Directory listing, pagination, search, alphabetical index
  editor/               # Virtual scroll renderer, chunked edit buffer, input handling
  filesystem/           # Chunked file I/O, debounced writes, atomic operations
  watcher/              # File system monitoring, change detection, notification
  conflict/             # External change reconciliation, merge strategies
  state/                # App state persistence, per-file cursor map, restore logic
  undo/                 # Undo operation log, context anchoring, rebasing
  ui/                   # Element types, renderer, Gioui drawing
```

## 4. File Browser

The browser presents the configured directory's contents. It must handle directories with hundreds of thousands of entries.

### Behavior

- **Lazy loading** — directory entries are loaded in pages, not all at once
- **Alphabetical index** — a tap-on-letter sidebar jumps to the corresponding section
- **Search** — incremental text filter over entry names
- **Single tap to open** — no long-press menus, no file management actions

### Performance

- First paint: directory root visible before all entries are read
- Scroll: smooth at 60 fps with virtualized list (only visible rows rendered)
- Search: results filter in real time, no debounce on keystrokes

## 5. Text Editor

The editor renders plain text with virtual scrolling and chunked loading. The entire file is never loaded into memory.

### Virtual Scrolling

Only lines visible on screen (plus a small prefetch buffer) are rendered. Scrolling triggers on-demand loading of adjacent chunks.

### Chunked Buffer

- Files are divided into fixed-size chunks (configurable, e.g., 64 KB)
- Only chunks near the cursor or visible region reside in memory
- Chunks are evicted under memory pressure, reloaded from disk on demand
- Edits at chunk boundaries are handled by splitting/merging chunks

### Line Index

The editor maintains a line-offset index (line number → byte offset in file) for every file it has opened. This index is:

- **Built** in a single streaming pass over the file (O(n) time, O(lines) space for the index array)
- **Cached** on disk inside `.pad/indices/` (one file per indexed file, named by the hash of the original path)
- **Invalidated** when the file's modification time or size changes (detected on open)
- **Updated incrementally** during editing sessions (insert/delete shifts subsequent offsets in the in-memory copy)
- **Evicted** under disk pressure (LRU by last access time)

The index enables O(log n) jumps to any line number via binary search. Without it, finding line N in a large file requires scanning from the start — O(N) in bytes. For a 1 GB file with average 50-byte lines, that's ~20M lines and 20 MB of index data (4 bytes per line as uint32). This is acceptable on modern devices.

During an editing session, the in-memory index is updated with each edit. On file close, the updated index is written back. On next open, if the disk file matches the index's mtime/size stamp, the cached index is reused.

### Input

- Standard text input: insert characters, backspace, delete
- Selection: tap-and-drag or tap-hold to select a range
- Paste: clipboard content inserted at cursor or replacing selection
- No keyboard shortcuts beyond what the OS input method provides

### Word Wrap

Long lines are wrapped visually at word boundaries within the viewport width. This is a display-only feature — no newlines are inserted into the underlying text or written to disk. The logic layer computes wrapped display lines from the raw text, region width, and estimated character advance.

Word wrap is **on by default**. Users can toggle it off for horizontal scrolling instead.

### Search

The editor has an in-editor search bar (appears below the header when activated). It contains a text field and up/down arrow buttons. Behavior:

1. As the user types, the logic layer finds all occurrences of the query in the loaded buffer
2. The cursor jumps to the first match immediately
3. Up/down arrows cycle through matches (up = previous, down = next)
4. Each keystroke that changes the query re-runs the search and jumps to the first match
5. The search bar displays `Match N of M` so the user knows their position

For large files, search runs over the currently loaded chunks. If a match exists in an unloaded region, the editor loads that chunk and scrolls to it.

## 6. Undo System

The undo system records reversible edit operations with context anchoring, persisted to disk for cross-session survival.

### 6.1 Operation Model

Edits are recorded as a **stack of chained operations**. A chain is a contiguous sequence of operations sharing a common anchor (position and context). A new chain begins whenever the cursor moves or a new selection is made.

### 6.2 Entry Formats

**Insert chain** (typing at a fixed cursor position):

```
Head:   {pos: 1024, before: "<N bytes before>", after: "<M bytes after>", inserted: "h"}
Tail:   {inserted: "e"}
Tail:   {inserted: "l"}
Tail:   {inserted: "l"}
Tail:   {inserted: "o"}
```

Entries without `pos`, `before`, or `after` inherit from the head. Position advances by the length of the previous entry's `inserted` text.

**Delete chain** (backspacing without hitting original text):

```
Head:   {pos: 1023, after: "<M bytes after>", deleted: "o"}
Tail:   {deleted: "l"}
Tail:   {deleted: "l"}
```

Entries without `pos` or `after` inherit from the head. Position for entry `i` is `head.pos + i`. The `before` field is omitted because it shifts with each deletion; `after` is the stable anchor.

**Selection replacement** (select text, type to replace):

```
Head:   {pos: 5000, before: "<N>", after: "<M>", deleted: "<selected text>", inserted: "x"}
Tail:   {inserted: "y"}
Tail:   {inserted: "z"}
```

The head records both the deleted selection and the first inserted character. Continuation typing appends to the chain.

**Field semantics:**

| Field | Present on | Meaning |
|---|---|---|
| `pos` | Head only | Byte offset of the first operation in the chain |
| `before` | Head of insert/replace chains | N bytes immediately before `pos`, used for re-anchoring |
| `after` | Head of all chains | M bytes immediately after the affected range, used for re-anchoring |
| `deleted` | Head of delete chains, selection replace | Text that was removed |
| `inserted` | Every entry | Text that was added (empty string for pure deletion entries) |

Fields absent from an entry are inherited from the chain head.

### 6.3 Backspace Semantics

Backspace interacts with the undo stack:

1. **If the last stack entry is an `inserted` entry** and the character immediately before the cursor matches that entry's `inserted` text: **pop** that entry from the stack. No new operation is recorded. The character is removed from the file.

2. **Otherwise** (stack empty, or last entry is a delete): record a **new chain head** with `deleted` set to the backspaced character, `pos` set to the byte offset of that character, and `after` set to the context after the cursor.

3. **Subsequent backspaces** (after entering delete mode): each appends a `{deleted: "<char>"}` entry to the current chain, inheriting `pos` and `after` from the head.

This means backspacing over text the user just typed is "free" — it removes operations from the stack rather than adding new ones. Undo after backspace naturally restores the popped character.

### 6.4 Undo Replay

To undo, pop the top chain from the stack and replay in reverse:

1. **Re-anchor**: search the file for the head's context (`before` + `after` for insert chains, `after` for delete chains). If found, update `pos` to the new location. If not found, skip the entire chain.

2. **Apply in reverse**: for each entry from last to first:
   - **Insert entry**: delete `inserted` text at the current position
   - **Delete entry**: insert `deleted` text at the current position
   - Advance/retreat position by the length of the text affected

3. For delete chains, replay right-to-left (last entry first) so that each insert shifts subsequent text right without affecting earlier positions.

### 6.5 Stack Limits

| Parameter | Value | Rationale |
|---|---|---|
| Max operations | 1000 | Sufficient for ~1000 typing sessions or individual keystrokes |
| Context window | 50 bytes each side | Unique enough for prose/code, small enough for persistence |
| `deleted` field size | **1 MB inline** | Below: stored directly in the operation. Above: stored in a separate backup file (see below) |
| Total undo disk budget | 10 MB | Per `.pad/undo/` directory; oldest files evicted on excess |
| Redo | Not supported | Can be added later; keeps initial implementation minimal |

**Large deletion handling**: When a single deletion exceeds 1 MB, the deleted text is written to `.pad/undo/<hash>_backup_<seq>.bin` and the operation records a `deleted_file` path instead of inline text. The backup file is cleaned up when the operation is popped (by undo or eviction). This preserves the "no practical limits" philosophy — even a 4 GB file deletion is undoable — while keeping the serialized undo stack small enough for fast persistence.

```json
{
  "type": "delete_chain",
  "pos": 0,
  "after": "...",
  "deleted_file": ".pad/undo/abc123_backup_001.bin",
  "deleted_size": 4294967296,
  "entries": []
}
```

The `deleted_size` field is used for display (e.g., toast: "Undo: 4.0 GB restored") and for eviction decisions.

### 6.6 Persistence

The undo stack for the active file is serialized to `.pad/undo/<hash>.json` (see §8) and persisted on every edit, debounced separately from auto-save. On restore, each operation's context is verified; operations that cannot be anchored are silently dropped from the stack.

The stack survives:
- Process kill by the OS
- Device restart
- File close and reopen
- App restart

The stack does **not** survive:
- Explicit user action to clear it (if such a feature is added)
- External file changes that make anchoring impossible (text context no longer exists)
- Eviction due to disk budget exceeded (oldest undo files pruned first)

## 7. Auto-save

Every edit to a file is written to disk automatically.

### Behavior

- **Debounce**: 500–1000 ms after the last keystroke (configurable)
- **Atomic write**: write to a temporary file inside `.pad/tmp/`, then rename over the target. The `.pad/` directory is excluded from Syncthing sync (via `.stignore`), so temp files are invisible to the sync daemon and do not generate spurious sync events
- **No UI**: there is no save button, no unsaved indicator, no "do you want to save?" dialog

### Interaction with Undo

Auto-save writes the current file state. The undo stack is persisted separately. Undo works transparently across saves — saving does not "commit" or "clear" undo history.

### Syncthing Compatibility

The `.pad/` directory contains editor-internal data (state, indices, temp files) and is excluded from sync. Syncthing ignores it via a `.stignore` file placed in the configured directory:

```
.pad/
```

This ensures:
- Temp files used for atomic writes are never synced
- State and index files remain local to the device
- Only the user's text files are synchronized across devices

## 8. State Management

The editor persists its state across multiple files inside `.pad/`. This multi-file approach avoids the overhead of a database dependency and keeps each file small enough for fast atomic writes.

### Directory Layout

```
.pad/
  state.json          # App state (active file, cursor, scroll, browser)
  cursors.json        # Per-file cursor map
  undo/
    <file_hash>.json  # Undo stack for a specific file (one per actively edited file)
  indices/
    <file_hash>.bin   # Line-offset index (see §5)
  tmp/
                      # Temp files for atomic writes
```

All `.pad/` contents are excluded from Syncthing sync (see §7).

### App State (`state.json`)

```json
{
  "active_file": "path/to/open/file.txt",
  "cursor_pos": 1024,
  "scroll_offset": 500,
  "browser_scroll": 1200
}
```

| Field | Description |
|---|---|
| `active_file` | Path of the currently open file (relative to configured directory) |
| `cursor_pos` | Byte offset of the cursor in the active file |
| `scroll_offset` | Vertical scroll position (line or pixel offset) |
| `browser_scroll` | Scroll position in the directory browser |

### Cursor Map (`cursors.json`)

```json
{
  "path/to/file1.txt": 2048,
  "path/to/file2.txt": 512
}
```

Map of file path → last cursor position. Evicts oldest entries to stay under 1000 entries.

### Undo Stacks (`undo/<hash>.json`)

Each actively edited file gets its own undo stack file, named by the SHA-256 hash of the file path. This keeps undo data isolated per-file and avoids serializing all undo stacks into one large blob.

```json
{
  "file": "path/to/open/file.txt",
  "file_mtime": 1715000000,
  "file_size": 50000,
  "operations": [
    {
      "type": "insert_chain",
      "pos": 1024,
      "before": "...",
      "after": "...",
      "entries": [
        {"inserted": "h"},
        {"inserted": "e"},
        {"inserted": "l"},
        {"inserted": "l"},
        {"inserted": "o"}
      ]
    }
  ]
}
```

The `file_mtime` and `file_size` fields are used to detect if the on-disk file has changed since the undo stack was last used (external update or sync conflict). If they differ, the stack is discarded.

Old undo stack files are evicted when disk usage exceeds a budget (e.g., 10 MB total), using LRU by last access time.

### Why Not SQLite?

SQLite (via `modernc.org/sqlite`) was considered but rejected for the initial implementation:

- **Simplicity**: JSON files are human-readable, trivially debuggable, and require no schema migrations
- **Workload**: The editor performs sequential appends to undo stacks and point reads for state — JSON files handle this well
- **Atomicity**: Temp+rename gives us crash-safe writes without a transaction log
- **Dependency**: One fewer dependency in the Go module

If profiling reveals that JSON serialization or file I/O is a bottleneck (unlikely for the expected workload), SQLite can be adopted later with minimal API changes.

### Write Strategy

- **Continuous**: state is written on every navigation, edit, or cursor movement
- **Debounced**: writes are batched with a short debounce (separate from auto-save debounce)
- **Atomic**: write to temp file in `.pad/tmp/`, then rename (same as auto-save)
- **Bounded**: `cursors.json` evicts oldest entries (max 1000). Undo stacks are capped at 1000 operations per file and evicted by total disk size.

### Restore Flow

1. App launches → read `state.json`
2. If `active_file` exists on disk → open it, load chunk around `cursor_pos`, place cursor
3. If `active_file` was deleted → show browser at `browser_scroll` position
4. If no state file exists → show browser at root
5. Load undo stack from `undo/<hash>.json`; verify file mtime/size match; drop stack if mismatched

## 9. File System Monitor

The editor watches the configured directory for external modifications (e.g., from a sync tool on another device).

### Detection

- Uses the platform's file system watch API (inotify on Android/Linux equivalent)
- Detects: file content changes, file creation, file deletion, file rename
- Watches the entire directory tree recursively

### Response

| Event | Action |
|---|---|
| File modified externally (editor not open) | Update browser listing (timestamp, size) |
| File modified externally (editor is open) | Trigger conflict resolution (§10) |
| File created | Appear in browser on next scroll/search |
| File deleted | Remove from browser; if open, close and show browser |
| File renamed | Update browser listing |

## 10. Conflict Resolution

When the file system monitor detects an external modification to the currently open file:

### Detection

Compare the file's last-modified timestamp and size against the editor's known state. If either differs, the file has changed externally.

### Strategies

| Strategy | Behavior |
|---|---|
| **Discard local** | Replace buffer with disk content, preserve cursor position if possible |
| **Keep local** | Ignore external change, continue editing (next sync may overwrite) |
| **Manual** | Show a brief prompt asking the user to choose |

The default strategy is configurable. The prompt (if shown) is the only modal UI in the application.

### Interaction with Undo

When an external change is applied to the buffer:

- The undo stack is **preserved** (not flushed)
- Operations are re-anchored at undo time using context search (§6.4)
- Operations that cannot be anchored are silently skipped
- New operations recorded after the external change use positions in the new coordinate space

## 11. Performance Guarantees

| Operation | Target | Mechanism |
|---|---|---|
| Open any file | < 100 ms | Chunked load: only the chunk around the cursor is read |
| Scroll | 60 fps | Virtual rendering: only visible lines + prefetch buffer |
| Type | < 16 ms per keystroke | In-memory chunk edit, async chunk flush |
| Search in directory | < 100 ms for 100k files | Lazy index built on first browse, cached on disk |
| Search in file | < 50 ms for 1M lines | In-memory scan of loaded chunks; line index for navigation |
| Jump to line | < 10 ms | Binary search on cached line-offset index |
| Undo | < 100 ms | Context search is O(n) in file size but bounded by context window |
| State restore | < 500 ms | Read state file, open file, load chunk, place cursor |
| Auto-save | < 100 ms | Debounced atomic write |

### Memory

| Component | Budget |
|---|---|
| Chunk cache | Configurable, e.g., 16–64 MB |
| Undo stack (in memory) | ~200 KB for 1000 ops with 50B context (excluding large `deleted` fields) |
| Undo stack (on disk) | 10 MB total budget across all files |
| State files | < 1 KB each (`state.json`, `cursors.json`) |
| Line index | ~4 bytes per line (stored on disk, loaded fully for active file) |

## 12. Out of Scope

- Syntax highlighting, themes, or formatting
- Multiple tabs or split views
- Find and replace (search is in scope; replace may be added later)
- File management (create, delete, rename, move) — handled by external tools
- Cloud sync — handled by external tools (Syncthing, Dropbox, etc.)
- Collaborative editing
- Redo (may be added later)
- Keyboard shortcut configuration
- Plugins or extensions