Pad/doc/SPEC.md

Pad — Text Editor Specification

1. Overview

Pad is a minimal, high-performance plain text editor for Android. It is configured with a single directory of text files and provides instant file access, automatic persistence, and seamless recovery from process death. The editor is designed for a directory synced across devices, with reactive awareness of external file modifications.

Design philosophy:

• No practical limits — files and directories of arbitrary size must work without degradation
• Minimal surface area — plain text only, no formatting, no file management UI
• Automatic everything — save, state persistence, sync awareness happen without user intervention
• Survive anything — process kill, battery death, device restart: the editor restores exactly where the user left off

2. Core Requirements

Requirement	Detail
Instant open	Files open immediately regardless of size; content loads on demand
Auto-save	Every edit is persisted to disk with a short debounce; no save button exists
Unlimited scale	No caps on file size, line count, or directory entry count
External change detection	The editor monitors the configured directory for modifications from sync tools
Cross-session undo	Full undo history survives process death, file close, and device restart
State restoration	On relaunch, the editor restores the exact file, cursor position, and scroll state
Per-file cursor memory	Reopening any previously visited file restores the last cursor position
Plain text only	No formatting, no syntax highlighting, no markdown rendering

3. Architecture

cmd/
  pad/                  # Mobile entry point (Gioui window + frame loop)

internal/
  browser/              # Directory listing, pagination, search, alphabetical index
  editor/               # Virtual scroll renderer, chunked edit buffer, input handling
  filesystem/           # Chunked file I/O, debounced writes, atomic operations
  watcher/              # File system monitoring, change detection, notification
  conflict/             # External change reconciliation, merge strategies
  state/                # App state persistence, per-file cursor map, restore logic
  undo/                 # Undo operation log, context anchoring, rebasing
  ui/                   # Element types, renderer, Gioui drawing

4. File Browser

The browser presents the configured directory's contents. It must handle directories with hundreds of thousands of entries.

Behavior

• Lazy loading — directory entries are loaded in pages, not all at once
• Alphabetical index — a tap-on-letter sidebar jumps to the corresponding section
• Search — incremental text filter over entry names
• Single tap to open — no long-press menus, no file management actions

Performance

• First paint: directory root visible before all entries are read
• Scroll: smooth at 60 fps with virtualized list (only visible rows rendered)
• Search: results filter in real time, no debounce on keystrokes

5. Text Editor

The editor renders plain text with virtual scrolling and chunked loading. The entire file is never loaded into memory.

Virtual Scrolling

Only lines visible on screen (plus a small prefetch buffer) are rendered. Scrolling triggers on-demand loading of adjacent chunks.

Chunked Buffer

• Files are divided into fixed-size chunks (configurable, e.g., 64 KB)
• Only chunks near the cursor or visible region reside in memory
• Chunks are evicted under memory pressure, reloaded from disk on demand
• Edits at chunk boundaries are handled by splitting/merging chunks

Line Index

The editor maintains a line-offset index (line number → byte offset in file) for every file it has opened. This index is:

• Built in a single streaming pass over the file (O(n) time, O(lines) space for the index array)
• Cached on disk inside .pad/indices/ (one file per indexed file, named by the hash of the original path)
• Invalidated when the file's modification time or size changes (detected on open)
• Updated incrementally during editing sessions (insert/delete shifts subsequent offsets in the in-memory copy)
• Evicted under disk pressure (LRU by last access time)

The index enables O(log n) jumps to any line number via binary search. Without it, finding line N in a large file requires scanning from the start — O(N) in bytes. For a 1 GB file with average 50-byte lines, that's ~20M lines and 20 MB of index data (4 bytes per line as uint32). This is acceptable on modern devices.

During an editing session, the in-memory index is updated with each edit. On file close, the updated index is written back. On next open, if the disk file matches the index's mtime/size stamp, the cached index is reused.

Input

• Standard text input: insert characters, backspace, delete
• Selection: tap-and-drag or tap-hold to select a range
• Paste: clipboard content inserted at cursor or replacing selection
• No keyboard shortcuts beyond what the OS input method provides

Word Wrap

Long lines are wrapped visually at word boundaries within the viewport width. This is a display-only feature — no newlines are inserted into the underlying text or written to disk. The logic layer computes wrapped display lines from the raw text, region width, and estimated character advance.

Word wrap is on by default. Users can toggle it off for horizontal scrolling instead.

Search

The editor has an in-editor search bar (appears below the header when activated). It contains a text field and up/down arrow buttons. Behavior:

1. As the user types, the logic layer finds all occurrences of the query in the loaded buffer
2. The cursor jumps to the first match immediately
3. Up/down arrows cycle through matches (up = previous, down = next)
4. Each keystroke that changes the query re-runs the search and jumps to the first match
5. The search bar displays Match N of M so the user knows their position

For large files, search runs over the currently loaded chunks. If a match exists in an unloaded region, the editor loads that chunk and scrolls to it.

6. Undo System

The undo system records reversible edit operations with context anchoring, persisted to disk for cross-session survival.

6.1 Operation Model

Edits are recorded as a stack of chained operations. A chain is a contiguous sequence of operations sharing a common anchor (position and context). A new chain begins whenever the cursor moves or a new selection is made.

6.2 Entry Formats

Insert chain (typing at a fixed cursor position):

Head:   {pos: 1024, before: "<N bytes before>", after: "<M bytes after>", inserted: "h"}
Tail:   {inserted: "e"}
Tail:   {inserted: "l"}
Tail:   {inserted: "l"}
Tail:   {inserted: "o"}

Entries without pos, before, or after inherit from the head. Position advances by the length of the previous entry's inserted text.

Delete chain (backspacing without hitting original text):

Head:   {pos: 1023, after: "<M bytes after>", deleted: "o"}
Tail:   {deleted: "l"}
Tail:   {deleted: "l"}

Entries without pos or after inherit from the head. Position for entry i is head.pos + i. The before field is omitted because it shifts with each deletion; after is the stable anchor.

Selection replacement (select text, type to replace):

Head:   {pos: 5000, before: "<N>", after: "<M>", deleted: "<selected text>", inserted: "x"}
Tail:   {inserted: "y"}
Tail:   {inserted: "z"}

The head records both the deleted selection and the first inserted character. Continuation typing appends to the chain.

Field semantics:

Field	Present on	Meaning
pos	Head only	Byte offset of the first operation in the chain
before	Head of insert/replace chains	N bytes immediately before pos, used for re-anchoring
after	Head of all chains	M bytes immediately after the affected range, used for re-anchoring
deleted	Head of delete chains, selection replace	Text that was removed
inserted	Every entry	Text that was added (empty string for pure deletion entries)

Fields absent from an entry are inherited from the chain head.

6.3 Backspace Semantics

Backspace interacts with the undo stack:

1. If the last stack entry is an inserted entry and the character immediately before the cursor matches that entry's inserted text: pop that entry from the stack. No new operation is recorded. The character is removed from the file.

2. Otherwise (stack empty, or last entry is a delete): record a new chain head with deleted set to the backspaced character, pos set to the byte offset of that character, and after set to the context after the cursor.

3. Subsequent backspaces (after entering delete mode): each appends a {deleted: "<char>"} entry to the current chain, inheriting pos and after from the head.

This means backspacing over text the user just typed is "free" — it removes operations from the stack rather than adding new ones. Undo after backspace naturally restores the popped character.

6.4 Undo Replay

To undo, pop the top chain from the stack and replay in reverse:

1. Re-anchor: search the file for the head's context (before + after for insert chains, after for delete chains). If found, update pos to the new location. If not found, skip the entire chain.

2. Apply in reverse: for each entry from last to first:

   • Insert entry: delete inserted text at the current position
   • Delete entry: insert deleted text at the current position
   • Advance/retreat position by the length of the text affected

3. For delete chains, replay right-to-left (last entry first) so that each insert shifts subsequent text right without affecting earlier positions.

6.5 Stack Limits

Parameter	Value	Rationale
Max operations	1000	Sufficient for ~1000 typing sessions or individual keystrokes
Context window	50 bytes each side	Unique enough for prose/code, small enough for persistence
deleted field size	1 MB inline	Below: stored directly in the operation. Above: stored in a separate backup file (see below)
Total undo disk budget	10 MB	Per .pad/undo/ directory; oldest files evicted on excess
Redo	Not supported	Can be added later; keeps initial implementation minimal

Large deletion handling: When a single deletion exceeds 1 MB, the deleted text is written to .pad/undo/<hash>_backup_<seq>.bin and the operation records a deleted_file path instead of inline text. The backup file is cleaned up when the operation is popped (by undo or eviction). This preserves the "no practical limits" philosophy — even a 4 GB file deletion is undoable — while keeping the serialized undo stack small enough for fast persistence.

{
  "type": "delete_chain",
  "pos": 0,
  "after": "...",
  "deleted_file": ".pad/undo/abc123_backup_001.bin",
  "deleted_size": 4294967296,
  "entries": []
}

The deleted_size field is used for display (e.g., toast: "Undo: 4.0 GB restored") and for eviction decisions.

6.6 Persistence

The undo stack for the active file is serialized to .pad/undo/<hash>.json (see §8) and persisted on every edit, debounced separately from auto-save. On restore, each operation's context is verified; operations that cannot be anchored are silently dropped from the stack.

The stack survives:

• Process kill by the OS
• Device restart
• File close and reopen
• App restart

The stack does not survive:

• Explicit user action to clear it (if such a feature is added)
• External file changes that make anchoring impossible (text context no longer exists)
• Eviction due to disk budget exceeded (oldest undo files pruned first)

7. Auto-save

Every edit to a file is written to disk automatically.

Behavior

• Debounce: 500–1000 ms after the last keystroke (configurable)
• Atomic write: write to a temporary file inside .pad/tmp/, then rename over the target. The .pad/ directory is excluded from Syncthing sync (via .stignore), so temp files are invisible to the sync daemon and do not generate spurious sync events
• No UI: there is no save button, no unsaved indicator, no "do you want to save?" dialog

Interaction with Undo

Auto-save writes the current file state. The undo stack is persisted separately. Undo works transparently across saves — saving does not "commit" or "clear" undo history.

Syncthing Compatibility

The .pad/ directory contains editor-internal data (state, indices, temp files) and is excluded from sync. Syncthing ignores it via a .stignore file placed in the configured directory:

.pad/

This ensures:

• Temp files used for atomic writes are never synced
• State and index files remain local to the device
• Only the user's text files are synchronized across devices

8. State Management

The editor persists its state across multiple files inside .pad/. This multi-file approach avoids the overhead of a database dependency and keeps each file small enough for fast atomic writes.

Directory Layout

.pad/
  state.json          # App state (active file, cursor, scroll, browser)
  cursors.json        # Per-file cursor map
  undo/
    <file_hash>.json  # Undo stack for a specific file (one per actively edited file)
  indices/
    <file_hash>.bin   # Line-offset index (see §5)
  tmp/
                      # Temp files for atomic writes

All .pad/ contents are excluded from Syncthing sync (see §7).

App State (state.json)

{
  "active_file": "path/to/open/file.txt",
  "cursor_pos": 1024,
  "scroll_offset": 500,
  "browser_scroll": 1200
}

Field	Description
active_file	Path of the currently open file (relative to configured directory)
cursor_pos	Byte offset of the cursor in the active file
scroll_offset	Vertical scroll position (line or pixel offset)
browser_scroll	Scroll position in the directory browser

Cursor Map (cursors.json)

{
  "path/to/file1.txt": 2048,
  "path/to/file2.txt": 512
}

Map of file path → last cursor position. Evicts oldest entries to stay under 1000 entries.

Undo Stacks (undo/<hash>.json)

Each actively edited file gets its own undo stack file, named by the SHA-256 hash of the file path. This keeps undo data isolated per-file and avoids serializing all undo stacks into one large blob.

{
  "file": "path/to/open/file.txt",
  "file_mtime": 1715000000,
  "file_size": 50000,
  "operations": [
    {
      "type": "insert_chain",
      "pos": 1024,
      "before": "...",
      "after": "...",
      "entries": [
        {"inserted": "h"},
        {"inserted": "e"},
        {"inserted": "l"},
        {"inserted": "l"},
        {"inserted": "o"}
      ]
    }
  ]
}

The file_mtime and file_size fields are used to detect if the on-disk file has changed since the undo stack was last used (external update or sync conflict). If they differ, the stack is discarded.

Old undo stack files are evicted when disk usage exceeds a budget (e.g., 10 MB total), using LRU by last access time.

Why Not SQLite?

SQLite (via modernc.org/sqlite) was considered but rejected for the initial implementation:

• Simplicity: JSON files are human-readable, trivially debuggable, and require no schema migrations
• Workload: The editor performs sequential appends to undo stacks and point reads for state — JSON files handle this well
• Atomicity: Temp+rename gives us crash-safe writes without a transaction log
• Dependency: One fewer dependency in the Go module

If profiling reveals that JSON serialization or file I/O is a bottleneck (unlikely for the expected workload), SQLite can be adopted later with minimal API changes.

Write Strategy

• Continuous: state is written on every navigation, edit, or cursor movement
• Debounced: writes are batched with a short debounce (separate from auto-save debounce)
• Atomic: write to temp file in .pad/tmp/, then rename (same as auto-save)
• Bounded: cursors.json evicts oldest entries (max 1000). Undo stacks are capped at 1000 operations per file and evicted by total disk size.

Restore Flow

1. App launches → read state.json
2. If active_file exists on disk → open it, load chunk around cursor_pos, place cursor
3. If active_file was deleted → show browser at browser_scroll position
4. If no state file exists → show browser at root
5. Load undo stack from undo/<hash>.json; verify file mtime/size match; drop stack if mismatched

9. File System Monitor

The editor watches the configured directory for external modifications (e.g., from a sync tool on another device).

Detection

• Uses the platform's file system watch API (inotify on Android/Linux equivalent)
• Detects: file content changes, file creation, file deletion, file rename
• Watches the entire directory tree recursively

Response

Event	Action
File modified externally (editor not open)	Update browser listing (timestamp, size)
File modified externally (editor is open)	Trigger conflict resolution (§10)
File created	Appear in browser on next scroll/search
File deleted	Remove from browser; if open, close and show browser
File renamed	Update browser listing

10. Conflict Resolution

When the file system monitor detects an external modification to the currently open file:

Detection

Compare the file's last-modified timestamp and size against the editor's known state. If either differs, the file has changed externally.

Strategies

Strategy	Behavior
Discard local	Replace buffer with disk content, preserve cursor position if possible
Keep local	Ignore external change, continue editing (next sync may overwrite)
Manual	Show a brief prompt asking the user to choose

The default strategy is configurable. The prompt (if shown) is the only modal UI in the application.

Interaction with Undo

When an external change is applied to the buffer:

• The undo stack is preserved (not flushed)
• Operations are re-anchored at undo time using context search (§6.4)
• Operations that cannot be anchored are silently skipped
• New operations recorded after the external change use positions in the new coordinate space

11. Performance Guarantees

Operation	Target	Mechanism
Open any file	< 100 ms	Chunked load: only the chunk around the cursor is read
Scroll	60 fps	Virtual rendering: only visible lines + prefetch buffer
Type	< 16 ms per keystroke	In-memory chunk edit, async chunk flush
Search in directory	< 100 ms for 100k files	Lazy index built on first browse, cached on disk
Search in file	< 50 ms for 1M lines	In-memory scan of loaded chunks; line index for navigation
Jump to line	< 10 ms	Binary search on cached line-offset index
Undo	< 100 ms	Context search is O(n) in file size but bounded by context window
State restore	< 500 ms	Read state file, open file, load chunk, place cursor
Auto-save	< 100 ms	Debounced atomic write

Memory

Component	Budget
Chunk cache	Configurable, e.g., 16–64 MB
Undo stack (in memory)	~200 KB for 1000 ops with 50B context (excluding large deleted fields)
Undo stack (on disk)	10 MB total budget across all files
State files	< 1 KB each (state.json, cursors.json)
Line index	~4 bytes per line (stored on disk, loaded fully for active file)

12. Out of Scope

• Syntax highlighting, themes, or formatting
• Multiple tabs or split views
• Find and replace (search is in scope; replace may be added later)
• File management (create, delete, rename, move) — handled by external tools
• Cloud sync — handled by external tools (Syncthing, Dropbox, etc.)
• Collaborative editing
• Redo (may be added later)
• Keyboard shortcut configuration
• Plugins or extensions