diff --git a/doc/SPEC.md b/doc/SPEC.md deleted file mode 100644 index 09b7041..0000000 --- a/doc/SPEC.md +++ /dev/null @@ -1,465 +0,0 @@ -# Pad — Text Editor Specification - -## 1. Overview - -Pad is a minimal, high-performance plain text editor for Android. It is configured with a single directory of text files and provides instant file access, automatic persistence, and seamless recovery from process death. The editor is designed for a directory synced across devices, with reactive awareness of external file modifications. - -**Design philosophy:** - -- **No practical limits** — files and directories of arbitrary size must work without degradation -- **Minimal surface area** — plain text only, no formatting, no file management UI -- **Automatic everything** — save, state persistence, sync awareness happen without user intervention -- **Survive anything** — process kill, battery death, device restart: the editor restores exactly where the user left off - -## 2. Core Requirements - -| Requirement | Detail | -|---|---| -| Instant open | Files open immediately regardless of size; content loads on demand | -| Auto-save | Every edit is persisted to disk with a short debounce; no save button exists | -| Unlimited scale | No caps on file size, line count, or directory entry count | -| External change detection | The editor monitors the configured directory for modifications from sync tools | -| Cross-session undo | Full undo history survives process death, file close, and device restart | -| State restoration | On relaunch, the editor restores the exact file, cursor position, and scroll state | -| Per-file cursor memory | Reopening any previously visited file restores the last cursor position | -| Plain text only | No formatting, no syntax highlighting, no markdown rendering | - -## 3. Architecture - -### 3.1 Code Organization - -``` -cmd/ - pad/ # Mobile entry point (Gioui window + frame loop) - -internal/ - browser/ # Directory listing, pagination, search, alphabetical index - editor/ # Virtual scroll renderer, chunked edit buffer, input handling - filesystem/ # Chunked file I/O, debounced writes, atomic operations - watcher/ # File system monitoring, change detection, notification - conflict/ # External change reconciliation, merge strategies - state/ # App state persistence, per-file cursor map, restore logic - undo/ # Undo operation log, context anchoring, rebasing - ui/ # Element types, renderer, Gioui drawing -``` - -### 3.2 Runtime Architecture - -The runtime uses four concurrent components coordinated by channels and a single mutex: - -- **Main goroutine** — Gioui event loop; renders frames under mutex, collects user input -- **Frame receiver goroutine** — tight loop that stores frames from logic and triggers redraws -- **Logic goroutine** — sole owner of application state; processes input, computes layout, dispatches async work -- **Worker pool** — fixed goroutines for file I/O, diff computation, undo replay - -Details: [`architecture.md`](./architecture.md) — concurrency model, channel topology, deadlock analysis, sync/async partitioning, persistence strategy. - -## 4. File Browser - -The browser presents the configured directory's contents. It must handle directories with hundreds of thousands of entries. - -### Behavior - -- **Lazy loading** — directory entries are loaded in pages, not all at once -- **Alphabetical index** — a tap-on-letter sidebar jumps to the corresponding section -- **Search** — incremental text filter over entry names -- **Single tap to open** — no long-press menus, no file management actions - -### Performance - -- First paint: directory root visible before all entries are read -- Scroll: smooth at 60 fps with virtualized list (only visible rows rendered) -- Search: results filter in real time, no debounce on keystrokes - -## 5. Text Editor - -The editor renders plain text with virtual scrolling and chunked loading. The entire file is never loaded into memory. - -### Virtual Scrolling - -Only lines visible on screen (plus a small prefetch buffer) are rendered. Scrolling triggers on-demand loading of adjacent chunks. - -### Chunked Buffer - -- Files are divided into fixed-size chunks (configurable, e.g., 64 KB) -- Only chunks near the cursor or visible region reside in memory -- Chunks are evicted under memory pressure, reloaded from disk on demand -- Edits at chunk boundaries are handled by splitting/merging chunks - -### Line Index - -The editor maintains a line-offset index (line number → byte offset in file) for every file it has opened. This index is: - -- **Built** in a single streaming pass over the file (O(n) time, O(lines) space for the index array) -- **Cached** on disk inside `.pad/indices/` (one file per indexed file, named by the hash of the original path) -- **Invalidated** when the file's modification time or size changes (detected on open) -- **Updated incrementally** during editing sessions (insert/delete shifts subsequent offsets in the in-memory copy) -- **Evicted** under disk pressure (LRU by last access time) - -The index enables O(log n) jumps to any line number via binary search. Without it, finding line N in a large file requires scanning from the start — O(N) in bytes. For a 1 GB file with average 50-byte lines, that's ~20M lines and 20 MB of index data (4 bytes per line as uint32). This is acceptable on modern devices. - -During an editing session, the in-memory index is updated with each edit. On file close, the updated index is written back. On next open, if the disk file matches the index's mtime/size stamp, the cached index is reused. - -### Input - -- Standard text input: insert characters, backspace, delete -- Selection: tap-and-drag or tap-hold to select a range -- Paste: clipboard content inserted at cursor or replacing selection -- No keyboard shortcuts beyond what the OS input method provides - -### Word Wrap - -Long lines are wrapped visually at word boundaries within the viewport width. This is a display-only feature — no newlines are inserted into the underlying text or written to disk. The logic layer computes wrapped display lines from the raw text, region width, and estimated character advance. - -Word wrap is **on by default**. Users can toggle it off for horizontal scrolling instead. - -### Search - -The editor has an in-editor search bar (appears below the header when activated). It contains a text field and up/down arrow buttons. Behavior: - -1. As the user types, the logic layer finds all occurrences of the query in the loaded buffer -2. The cursor jumps to the first match immediately -3. Up/down arrows cycle through matches (up = previous, down = next) -4. Each keystroke that changes the query re-runs the search and jumps to the first match -5. The search bar displays `Match N of M` so the user knows their position - -For large files, search runs over the currently loaded chunks. If a match exists in an unloaded region, the editor loads that chunk and scrolls to it. - -## 6. Undo System - -The undo system records reversible edit operations with context anchoring, persisted to disk for cross-session survival. - -### 6.1 Operation Model - -Edits are recorded as a **stack of chained operations**. A chain is a contiguous sequence of operations sharing a common anchor (position and context). A new chain begins whenever the cursor moves or a new selection is made. - -### 6.2 Entry Formats - -**Insert chain** (typing at a fixed cursor position): - -``` -Head: {pos: 1024, before: "", after: "", inserted: "h"} -Tail: {inserted: "e"} -Tail: {inserted: "l"} -Tail: {inserted: "l"} -Tail: {inserted: "o"} -``` - -Entries without `pos`, `before`, or `after` inherit from the head. Position advances by the length of the previous entry's `inserted` text. - -**Delete chain** (backspacing without hitting original text): - -``` -Head: {pos: 1023, after: "", deleted: "o"} -Tail: {deleted: "l"} -Tail: {deleted: "l"} -``` - -Entries without `pos` or `after` inherit from the head. Position for entry `i` is `head.pos + i`. The `before` field is omitted because it shifts with each deletion; `after` is the stable anchor. - -**Selection replacement** (select text, type to replace): - -``` -Head: {pos: 5000, before: "", after: "", deleted: "", inserted: "x"} -Tail: {inserted: "y"} -Tail: {inserted: "z"} -``` - -The head records both the deleted selection and the first inserted character. Continuation typing appends to the chain. - -**Field semantics:** - -| Field | Present on | Meaning | -|---|---|---| -| `pos` | Head only | Byte offset of the first operation in the chain | -| `before` | Head of insert/replace chains | N bytes immediately before `pos`, used for re-anchoring | -| `after` | Head of all chains | M bytes immediately after the affected range, used for re-anchoring | -| `deleted` | Head of delete chains, selection replace | Text that was removed | -| `inserted` | Every entry | Text that was added (empty string for pure deletion entries) | - -Fields absent from an entry are inherited from the chain head. - -### 6.3 Backspace Semantics - -Backspace interacts with the undo stack: - -1. **If the last stack entry is an `inserted` entry** and the character immediately before the cursor matches that entry's `inserted` text: **pop** that entry from the stack. No new operation is recorded. The character is removed from the file. - -2. **Otherwise** (stack empty, or last entry is a delete): record a **new chain head** with `deleted` set to the backspaced character, `pos` set to the byte offset of that character, and `after` set to the context after the cursor. - -3. **Subsequent backspaces** (after entering delete mode): each appends a `{deleted: ""}` entry to the current chain, inheriting `pos` and `after` from the head. - -This means backspacing over text the user just typed is "free" — it removes operations from the stack rather than adding new ones. Undo after backspace naturally restores the popped character. - -### 6.4 Undo Replay - -To undo, pop the top chain from the stack and replay in reverse: - -1. **Re-anchor**: search the file for the head's context (`before` + `after` for insert chains, `after` for delete chains). If found, update `pos` to the new location. If not found, skip the entire chain. - -2. **Apply in reverse**: for each entry from last to first: - - **Insert entry**: delete `inserted` text at the current position - - **Delete entry**: insert `deleted` text at the current position - - Advance/retreat position by the length of the text affected - -3. For delete chains, replay right-to-left (last entry first) so that each insert shifts subsequent text right without affecting earlier positions. - -### 6.5 Stack Limits - -| Parameter | Value | Rationale | -|---|---|---| -| Max operations | 1000 | Sufficient for ~1000 typing sessions or individual keystrokes | -| Context window | 50 bytes each side | Unique enough for prose/code, small enough for persistence | -| `deleted` field size | **1 MB inline** | Below: stored directly in the operation. Above: stored in a separate backup file (see below) | -| Total undo disk budget | 10 MB | Per `.pad/undo/` directory; oldest files evicted on excess | -| Redo | Not supported | Can be added later; keeps initial implementation minimal | - -**Large deletion handling**: When a single deletion exceeds 1 MB, the deleted text is written to `.pad/undo/_backup_.bin` and the operation records a `deleted_file` path instead of inline text. The backup file is cleaned up when the operation is popped (by undo or eviction). This preserves the "no practical limits" philosophy — even a 4 GB file deletion is undoable — while keeping the serialized undo stack small enough for fast persistence. - -```json -{ - "type": "delete_chain", - "pos": 0, - "after": "...", - "deleted_file": ".pad/undo/abc123_backup_001.bin", - "deleted_size": 4294967296, - "entries": [] -} -``` - -The `deleted_size` field is used for display (e.g., toast: "Undo: 4.0 GB restored") and for eviction decisions. - -### 6.6 Persistence - -The undo stack for the active file is serialized to `.pad/undo/.json` (see §8) and persisted on every edit, debounced separately from auto-save. On restore, each operation's context is verified; operations that cannot be anchored are silently dropped from the stack. - -The stack survives: -- Process kill by the OS -- Device restart -- File close and reopen -- App restart - -The stack does **not** survive: -- Explicit user action to clear it (if such a feature is added) -- External file changes that make anchoring impossible (text context no longer exists) -- Eviction due to disk budget exceeded (oldest undo files pruned first) - -## 7. Auto-save - -Every edit to a file is written to disk automatically. - -### Behavior - -- **Debounce**: 500–1000 ms after the last keystroke (configurable) -- **Atomic write**: write to a temporary file inside `.pad/tmp/`, then rename over the target. The `.pad/` directory is excluded from Syncthing sync (via `.stignore`), so temp files are invisible to the sync daemon and do not generate spurious sync events -- **No UI**: there is no save button, no unsaved indicator, no "do you want to save?" dialog - -### Interaction with Undo - -Auto-save writes the current file state. The undo stack is persisted separately. Undo works transparently across saves — saving does not "commit" or "clear" undo history. - -### Syncthing Compatibility - -The `.pad/` directory contains editor-internal data (state, indices, temp files) and is excluded from sync. Syncthing ignores it via a `.stignore` file placed in the configured directory: - -``` -.pad/ -``` - -This ensures: -- Temp files used for atomic writes are never synced -- State and index files remain local to the device -- Only the user's text files are synchronized across devices - -## 8. State Management - -The editor persists its state across multiple files inside `.pad/`. This multi-file approach avoids the overhead of a database dependency and keeps each file small enough for fast atomic writes. - -### Directory Layout - -``` -.pad/ - state.json # App state (active file, cursor, scroll, browser) - cursors.json # Per-file cursor map - undo/ - .json # Undo stack for a specific file (one per actively edited file) - indices/ - .bin # Line-offset index (see §5) - tmp/ - # Temp files for atomic writes -``` - -All `.pad/` contents are excluded from Syncthing sync (see §7). - -### App State (`state.json`) - -```json -{ - "active_file": "path/to/open/file.txt", - "cursor_pos": 1024, - "scroll_offset": 500, - "browser_scroll": 1200, - "browser_path": "/user/documents", - "browser_query": "" -} -``` - -| Field | Description | -|---|---| -| `active_file` | Path of the currently open file (relative to configured directory) | -| `cursor_pos` | Byte offset of the cursor in the active file | -| `scroll_offset` | Vertical scroll position (line or pixel offset) | -| `browser_scroll` | Scroll position in the directory browser | -| `browser_path` | Currently browsed directory path (relative to configured directory) | -| `browser_query` | Current search query in the browser | - -### Cursor Map (`cursors.json`) - -```json -{ - "path/to/file1.txt": 2048, - "path/to/file2.txt": 512 -} -``` - -Map of file path → last cursor position. Evicts oldest entries to stay under 1000 entries. - -### Undo Stacks (`undo/.json`) - -Each actively edited file gets its own undo stack file, named by the SHA-256 hash of the file path. This keeps undo data isolated per-file and avoids serializing all undo stacks into one large blob. - -```json -{ - "file": "path/to/open/file.txt", - "file_mtime": 1715000000, - "file_size": 50000, - "operations": [ - { - "type": "insert_chain", - "pos": 1024, - "before": "...", - "after": "...", - "entries": [ - {"inserted": "h"}, - {"inserted": "e"}, - {"inserted": "l"}, - {"inserted": "l"}, - {"inserted": "o"} - ] - } - ] -} -``` - -The `file_mtime` and `file_size` fields are used to detect if the on-disk file has changed since the undo stack was last used (external update or sync conflict). If they differ, the stack is discarded. - -Old undo stack files are evicted when disk usage exceeds a budget (e.g., 10 MB total), using LRU by last access time. - -### Why Not SQLite? - -SQLite (via `modernc.org/sqlite`) was considered but rejected for the initial implementation: - -- **Simplicity**: JSON files are human-readable, trivially debuggable, and require no schema migrations -- **Workload**: The editor performs sequential appends to undo stacks and point reads for state — JSON files handle this well -- **Atomicity**: Temp+rename gives us crash-safe writes without a transaction log -- **Dependency**: One fewer dependency in the Go module - -If profiling reveals that JSON serialization or file I/O is a bottleneck (unlikely for the expected workload), SQLite can be adopted later with minimal API changes. - -### Write Strategy - -- **Continuous**: state is written on every navigation, edit, or cursor movement -- **Debounced**: writes are batched with a short debounce (separate from auto-save debounce) -- **Atomic**: write to temp file in `.pad/tmp/`, then rename (same as auto-save) -- **Bounded**: `cursors.json` evicts oldest entries (max 1000). Undo stacks are capped at 1000 operations per file and evicted by total disk size. - -### Restore Flow - -1. App launches → read `state.json` -2. If `active_file` exists on disk → open it, load chunk around `cursor_pos`, place cursor -3. If `active_file` was deleted → show browser at `browser_scroll` position -4. If no state file exists → show browser at root -5. Load undo stack from `undo/.json`; verify file mtime/size match; drop stack if mismatched - -## 9. File System Monitor - -The editor watches the configured directory for external modifications (e.g., from a sync tool on another device). - -### Detection - -- Uses the platform's file system watch API (inotify on Android/Linux equivalent) -- Detects: file content changes, file creation, file deletion, file rename -- Watches the entire directory tree recursively - -### Response - -| Event | Action | -|---|---| -| File modified externally (editor not open) | Update browser listing (timestamp, size) | -| File modified externally (editor is open) | Trigger conflict resolution (§10) | -| File created | Appear in browser on next scroll/search | -| File deleted | Remove from browser; if open, close and show browser | -| File renamed | Update browser listing | - -## 10. Conflict Resolution - -When the file system monitor detects an external modification to the currently open file: - -### Detection - -Compare the file's last-modified timestamp and size against the editor's known state. If either differs, the file has changed externally. - -### Strategies - -| Strategy | Behavior | -|---|---| -| **Discard local** | Replace buffer with disk content, preserve cursor position if possible | -| **Keep local** | Ignore external change, continue editing (next sync may overwrite) | -| **Manual** | Show a brief prompt asking the user to choose | - -The default strategy is configurable. The prompt (if shown) is the only modal UI in the application. - -### Interaction with Undo - -When an external change is applied to the buffer: - -- The undo stack is **preserved** (not flushed) -- Operations are re-anchored at undo time using context search (§6.4) -- Operations that cannot be anchored are silently skipped -- New operations recorded after the external change use positions in the new coordinate space - -## 11. Performance Guarantees - -| Operation | Target | Mechanism | -|---|---|---| -| Open any file | < 100 ms | Chunked load: only the chunk around the cursor is read | -| Scroll | 60 fps | Virtual rendering: only visible lines + prefetch buffer | -| Type | < 16 ms per keystroke | In-memory chunk edit, async chunk flush | -| Search in directory | < 100 ms for 100k files | Lazy index built on first browse, cached on disk | -| Search in file | < 50 ms for 1M lines | In-memory scan of loaded chunks; line index for navigation | -| Jump to line | < 10 ms | Binary search on cached line-offset index | -| Undo | < 100 ms | Context search is O(n) in file size but bounded by context window | -| State restore | < 500 ms | Read state file, open file, load chunk, place cursor | -| Auto-save | < 100 ms | Debounced atomic write | - -### Memory - -| Component | Budget | -|---|---| -| Chunk cache | Configurable, e.g., 16–64 MB | -| Undo stack (in memory) | ~200 KB for 1000 ops with 50B context (excluding large `deleted` fields) | -| Undo stack (on disk) | 10 MB total budget across all files | -| State files | < 1 KB each (`state.json`, `cursors.json`) | -| Line index | ~4 bytes per line (stored on disk, loaded fully for active file) | - -## 12. Out of Scope - -- Syntax highlighting, themes, or formatting -- Multiple tabs or split views -- Find and replace (search is in scope; replace may be added later) -- File management (create, delete, rename, move) — handled by external tools -- Cloud sync — handled by external tools (Syncthing, Dropbox, etc.) -- Collaborative editing -- Redo (may be added later) -- Keyboard shortcut configuration -- Plugins or extensions