From c941a02f1a4f526f11b5ff3dd54eb2334d856cf5 Mon Sep 17 00:00:00 2001 From: Greg Pomerantz Date: Thu, 4 Jun 2026 20:23:54 -0400 Subject: [PATCH] Draft virtual scroll render optimization plan. --- .gitignore | 2 +- doc/virtual_scroll_render_optimization.md | 817 ++++++++++++++++++++++ 2 files changed, 818 insertions(+), 1 deletion(-) create mode 100644 doc/virtual_scroll_render_optimization.md diff --git a/.gitignore b/.gitignore index 8bae7d0..5df9583 100644 --- a/.gitignore +++ b/.gitignore @@ -8,7 +8,7 @@ # build files *.apk *.idsig -cmd/pad/pad +pad # IDE .idea/ diff --git a/doc/virtual_scroll_render_optimization.md b/doc/virtual_scroll_render_optimization.md new file mode 100644 index 0000000..ce51645 --- /dev/null +++ b/doc/virtual_scroll_render_optimization.md @@ -0,0 +1,817 @@ +# Virtual Scrolling & Render Optimization — Implementation Plan + +## 1. Problem Statement + +The spec (§11, "Performance Guarantees") mandates: + +| Operation | Target | Mechanism | +|---|---|---| +| Open any file | < 100 ms | Chunked load: only the chunk around the cursor is read | +| Scroll | 60 fps | Virtual rendering: only visible lines + prefetch buffer | +| Type | < 16 ms per keystroke | In-memory chunk edit, async chunk flush | + +**Current reality:** `drawWrappedText` in `internal/ui/render.go` receives the entire `EditorState.Buffer` as a single string and calls `shp.LayoutString(params, str)` on it — every frame. For a 100,000-line file, this means: + +1. The shaper iterates ~1–10 million glyphs per frame +2. A `GlyphLayout` struct grows to millions of entries (ByteOffsets, X, Y, Advance slices) +3. Every line is drawn, including those far off-screen +4. Frame times are measured in hundreds of milliseconds — far exceeding the 16ms budget + +This is the primary cause of sluggishness when editing large files. The fix is **virtual scrolling**: only shape, render, and capture glyph layout for the bytes visible in the current viewport. + +--- + +## 2. Design Principles + +1. **The buffer is the source of truth.** The `EditorState.Buffer` string holds the complete file content. Virtual scrolling is a presentation-layer concern — the buffer itself is untouched. +2. **Logic computes the visible range; renderer renders only that range.** The logic layer determines which byte range is visible given scroll offset and viewport height. The renderer shapes and draws only those bytes. +3. **GlyphLayout is bounded to the viewport.** Per-glyph layout data is only captured for visible content, keeping memory usage constant regardless of file size. +4. **Chunking is a prerequisite.** Virtual scrolling requires that the buffer be backed by chunks (fixed-size file slices) so that the logic layer can load/unload chunks on demand. This plan assumes the chunked buffer exists; the chunked buffer implementation is described in a companion plan. + +--- + +## 3. Architecture + +``` +┌──────────────────────────────────────────────────────────────────┐ +│ EditorState (internal/editor/state.go) │ +│ │ +│ Buffer string ← full file content (from chunks)│ +│ ScrollOffset ui.Dp ← current scroll position │ +│ ChunkedBuffer *ChunkedBuffer ← chunked file access │ +│ LineIndex *LineIndex ← byte offsets per line │ +│ │ +│ VisibleByteRange(scroll, viewport) → (start, end) │ +│ VisibleContent(start, end) → []byte │ +└──────────────────────┬───────────────────────────────────────────┘ + │ + ▼ +┌──────────────────────────────────────────────────────────────────┐ +│ EditorLayout (internal/editor/state.go) │ +│ │ +│ 1. Compute viewport height from editor region + scale │ +│ 2. Call VisibleByteRange(scrollOffset, viewportHeight) │ +│ 3. Extract visibleContent := ChunkedBuffer.VisibleContent() │ +│ 4. Adjust cursorPos to be relative to visibleContent │ +│ 5. Adjust scrollOffset to be relative to visibleContent origin │ +│ 6. Pass visibleContent + adjusted offsets to NewTextField │ +└──────────────────────┬───────────────────────────────────────────┘ + │ + ▼ +┌──────────────────────────────────────────────────────────────────┐ +│ TextField.Draw → Renderer.drawWrappedText │ +│ │ +│ 1. Shape ONLY visibleContent (not the full buffer) │ +│ 2. Capture GlyphLayout only for visible glyphs │ +│ 3. Draw only visible lines │ +│ 4. Render cursor at adjusted position │ +│ 5. Return bounded GlyphLayout (≈ viewport-sized) │ +└──────────────────────┬───────────────────────────────────────────┘ + │ + ▼ +┌──────────────────────────────────────────────────────────────────┐ +│ GlyphLayout (internal/ui/unit.go) │ +│ │ +│ ByteOffsets []int ← byte offsets within visibleContent │ +│ X []Dp ← screen X positions │ +│ Y []Dp ← screen Y positions │ +│ Advance []Dp ← glyph widths │ +│ │ +│ Length: proportional to visible lines (≈ 50–500 entries) │ +│ NOT proportional to file size │ +└──────────────────────────────────────────────────────────────────┘ +``` + +--- + +## 4. Data Structures + +### 4.1 ChunkedBuffer + +```go +// ChunkedBuffer provides chunked access to a file's content. +// Only chunks near the cursor or viewport are kept in memory. +type ChunkedBuffer struct { + filename string + chunkSize int // e.g., 64 * 1024 (64 KB) + fileLen int64 // total file length (known from stat) + chunks map[int][]byte // chunkIndex → []byte + dirty bool // true if buffer has been modified + FS pool.FileSystem // filesystem for reading chunks + basePath string // base path for file resolution +} +``` + +**Key properties:** +- `chunkSize` is fixed at 64 KB (configurable constant). +- `chunks` is a sparse map: only chunks near the cursor/viewport are loaded. +- `fileLen` is known from the file stat task (dispatched when opening a file). +- The full buffer is reconstructed on demand via `Content(start, end)` for rendering. + +### 4.2 LineIndex + +```go +// LineIndex maps line numbers to byte offsets within the file. +// Built as a low-priority background task; cached on disk. +type LineIndex struct { + offsets []int32 // byte offset of each line start (line 0 = 0) + mtime int64 // file mtime at index build time + size int64 // file size at index build time +} +``` + +**Key properties:** +- Built by streaming the file once (O(n) time, O(lines) space for the offset array). +- Stored as `[]int32` — 4 bytes per line. A 1M-line file = 4 MB. +- Used for O(log n) line number → byte offset lookup (binary search). +- Used by `VisibleByteRange` to compute the exact byte range for visible lines. +- Invalidated when file mtime or size changes. +- Cached on disk at `.pad/indices/.bin`. + +### 4.3 VisibleByteRange + +```go +// VisibleByteRange returns the byte range [start, end) of content +// visible in the viewport, given the current scroll offset and viewport height. +func (cb *ChunkedBuffer) VisibleByteRange(scrollOffset ui.Dp, viewportHeight ui.Dp) (start, end int) { + if cb.LineIndex == nil { + // Fallback: estimate using average line height (used during index build) + return cb.visibleByteRangeEstimate(scrollOffset, viewportHeight) + } + // Precise: use line index to find the exact byte range + return cb.visibleByteRangePrecise(scrollOffset, viewportHeight) +} +``` + +**Two strategies:** + +| Strategy | When Used | Accuracy | +|---|---|---| +| `visibleByteRangeEstimate` | During initial load, before line index is built | Approximate (±1–2 lines) | +| `visibleByteRangePrecise` | After line index is available | Exact (byte-accurate) | + +**Estimate strategy (no index):** +1. Compute average character width from a sample of lines. +2. Compute average line height from the shaper (one-shot measurement). +3. Estimate visible line range from scrollOffset / lineHeight. +4. Use binary search on the line index (if available) or scan from file start to find byte offsets. + +**Precise strategy (with index):** +1. Compute visible line range: `startLine = scrollOffset / lineHeight`, `endLine = (scrollOffset + viewportHeight) / lineHeight`. +2. Binary search in `LineIndex.offsets` for `startLine` and `endLine`. +3. Clamp to `[0, fileLen)`. +4. Return `(int(offsets[startLine]), int(offsets[min(endLine, len(offsets))-1]))`. + +### 4.4 Chunked Content Extraction + +```go +// Content returns the bytes in [start, end) from the chunked buffer. +// It loads missing chunks on demand. +func (cb *ChunkedBuffer) Content(start, end int) string { + // 1. Determine which chunks are needed + startChunk := start / cb.chunkSize + endChunk := (end - 1) / cb.chunkSize + + // 2. Load missing chunks + for i := startChunk; i <= endChunk; i++ { + if _, ok := cb.chunks[i]; !ok { + cb.loadChunk(i) // reads from disk via FS.ReadFile (full file) + } + } + + // 3. Concatenate needed chunk slices + var buf bytes.Buffer + for i := startChunk; i <= endChunk; i++ { + chunk := cb.chunks[i] + chunkStart := i * cb.chunkSize + chunkEnd := chunkStart + len(chunk) + segStart := max(start, chunkStart) - chunkStart + segEnd := min(end, chunkEnd) - chunkStart + buf.Write(chunk[segStart:segEnd]) + } + return buf.String() +} + +// Prefetch loads adjacent chunks for smooth scrolling. +func (cb *ChunkedBuffer) Prefetch(centerChunk int, radius int) { + for i := centerChunk - radius; i <= centerChunk + radius; i++ { + if i >= 0 && i < cb.numChunks() { + if _, ok := cb.chunks[i]; !ok { + cb.loadChunk(i) + } + } + } +} +``` + +--- + +## 5. Implementation Steps + +### Step 1: ChunkedBuffer Implementation + +**File:** `internal/editor/chunked_buffer.go` (new) + +Implement the `ChunkedBuffer` struct with: +- `NewChunkedBuffer(filename, chunkSize, fs, basePath)` constructor +- `loadChunk(idx int)` — reads a single chunk from disk via `ReadFile` +- `Content(start, end int) string` — extracts bytes from loaded chunks +- `Prefetch(centerChunk, radius int)` — loads adjacent chunks +- `EvictFarChunks(cursorPos int, radius int)` — removes chunks far from cursor +- `FileLen() int64` — total file length +- `Filename() string` — the file path + +**File:** `internal/io/pool/task.go` — add `ReadChunkTask` + +```go +type ReadChunkTask struct { + taskID string + Path string + ChunkIdx int + FS FileSystem +} + +func NewReadChunkTask(path string, chunkIdx int, fs FileSystem) *ReadChunkTask { ... } + +func (t *ReadChunkTask) Execute() Result { + // Read the full file, then slice the requested chunk + content, err := t.FS.ReadFile(t.Path) + if err != nil { + return Result{TaskID: t.taskID, TaskType: TypeReadChunk, Success: false, Error: err} + } + start := t.ChunkIdx * 64 * 1024 + end := min(start+64*1024, len(content)) + if start >= len(content) { + return Result{TaskID: t.taskID, TaskType: TypeReadChunk, Success: true, Data: []byte{}} + } + chunk := content[start:end] + return Result{TaskID: t.taskID, TaskType: TypeReadChunk, Success: true, Data: chunk} +} + +func (t *ReadChunkTask) Priority() Priority { return HighPriority } +func (t *ReadChunkTask) TaskType() TaskType { return TypeReadChunk } +``` + +**File:** `internal/io/pool/task.go` — add `TypeReadChunk` constant. + +**File:** `internal/io/pool/task.go` — add `StatFileTask` (replaces full-file `ReadFileTask` for open) + +```go +type StatFileTask struct { + taskID string + Path string + FS FileSystem +} + +func NewStatFileTask(path string, fs FileSystem) *StatFileTask { ... } + +func (t *StatFileTask) Execute() Result { + // Read the full file content for now (will be optimized later with chunked stat) + content, err := t.FS.ReadFile(t.Path) + if err != nil { + return Result{TaskID: t.taskID, TaskType: TypeStatFile, Success: false, Error: err} + } + return Result{ + TaskID: t.taskID, TaskType: TypeStatFile, Success: true, + Data: &FileStat{Path: t.Path, Size: int64(len(content))}, + } +} + +type FileStat struct { + Path string + Size int64 +} +``` + +### Step 2: ChunkedBuffer on EditorState + +**File:** `internal/editor/state.go` + +```go +type EditorState struct { + // ... existing fields ... + ChunkedBuffer *ChunkedBuffer // NEW: chunked file access + LineIndex *LineIndex // NEW: line-to-byte-offset mapping +} +``` + +Update `OpenFile` in `state.go`: +```go +func OpenFile(data any) { + filename := data.(string) + TheState.Editor.Filename = filename + + // Create chunked buffer + chunkSize := 64 * 1024 // 64 KB + cb := NewChunkedBuffer(filename, chunkSize, TheLogic.mockFS, "") + TheState.Editor.ChunkedBuffer = cb + + // Dispatch stat task to get file size + TheLogic.workerPool.Dispatch(pool.NewStatFileTask(filename, TheLogic.mockFS)) + + // Switch to editor page + TheState.page = EditorPage + TheState.FocusedElementID = "editor_text" + TheState.ScrollOffset = 0 + TheState.Editor.CursorPosition = 0 +} +``` + +Update `handleWorkerResult` in `logic.go`: +```go +case res.TaskType == pool.TypeStatFile: + if res.Success { + if stat, ok := res.Data.(*pool.FileStat); ok { + TheState.Editor.ChunkedBuffer.SetFileSize(stat.Size) + // Load the chunk containing the cursor (position 0 at open) + TheState.Editor.ChunkedBuffer.LoadChunk(0) + // Prefetch adjacent chunks + TheState.Editor.ChunkedBuffer.Prefetch(0, 1) + } + } +``` + +### Step 3: VisibleByteRange and Content Extraction + +**File:** `internal/editor/chunked_buffer.go` — implement `VisibleByteRange` and `Content`. + +**File:** `internal/editor/chunked_buffer.go` — implement `visibleByteRangeEstimate`: + +```go +// visibleByteRangeEstimate approximates the visible byte range using +// heuristic estimates. Used when the line index is not yet available. +func (cb *ChunkedBuffer) visibleByteRangeEstimate(scrollOffset ui.Dp, viewportHeight ui.Dp) (start, end int) { + totalLines := int(cb.fileLen) / 50 // rough: 50 bytes per line average + lineHeight := EditorLineHeight() + + startLine := int(scrollOffset / lineHeight) + endLine := int((scrollOffset + viewportHeight) / lineHeight) + + // Clamp + if startLine < 0 { startLine = 0 } + if endLine > totalLines { endLine = totalLines } + + // Convert line numbers to byte offsets using the line index if available + if cb.LineIndex != nil && startLine < len(cb.LineIndex.offsets) { + start = int(cb.LineIndex.offsets[startLine]) + } else { + start = startLine * 50 // rough estimate + } + if cb.LineIndex != nil && endLine < len(cb.LineIndex.offsets) { + end = int(cb.LineIndex.offsets[endLine]) + } else { + end = endLine * 50 + } + + // Clamp to file bounds + if start < 0 { start = 0 } + if end > int(cb.fileLen) { end = int(cb.fileLen) } + if end <= start { end = start + 100 } // at least 100 bytes + + return start, end +} +``` + +### Step 4: Line Index Build Task + +**File:** `internal/io/pool/task.go` — add `BuildLineIndexTask`: + +```go +type BuildLineIndexTask struct { + taskID string + Path string + FS FileSystem +} + +func NewBuildLineIndexTask(path string, fs FileSystem) *BuildLineIndexTask { ... } + +func (t *BuildLineIndexTask) Execute() Result { + content, err := t.FS.ReadFile(t.Path) + if err != nil { + return Result{TaskID: t.taskID, TaskType: TypeBuildLineIndex, Success: false, Error: err} + } + + offsets := []int32{0} + for i := 0; i < len(content); i++ { + if content[i] == '\n' { + offsets = append(offsets, int32(i+1)) + } + } + + return Result{ + TaskID: t.taskID, TaskType: TypeBuildLineIndex, Success: true, + Data: &LineIndex{offsets: offsets, mtime: 0, size: int64(len(content))}, + } +} + +func (t *BuildLineIndexTask) Priority() Priority { return LowPriority } +func (t *BuildLineIndexTask) TaskType() TaskType { return TypeBuildLineIndex } +``` + +**File:** `internal/io/pool/task.go` — add `TypeBuildLineIndex` constant. + +**File:** `internal/editor/logic.go` — handle `TypeBuildLineIndex` result: + +```go +case res.TaskType == pool.TypeBuildLineIndex: + if res.Success { + if idx, ok := res.Data.(*LineIndex); ok { + TheState.Editor.LineIndex = idx + } + } +``` + +**Dispatch the line index build in `OpenFile`:** + +```go +// After loading the initial chunk: +TheLogic.workerPool.Dispatch(pool.NewBuildLineIndexTask(filename, TheLogic.mockFS)) +``` + +### Step 5: EditorLayout — Compute Visible Content + +**File:** `internal/editor/state.go` — update `EditorLayout`: + +```go +func EditorLayout(screenWidth, screenHeight ui.Dp, wordWrap bool) []ui.Element { + // ... existing status bar and bottom bar code ... + + // --- Editor text area --- + editorRegion := ui.Region{ + X: margin, Y: editorY, + W: screenWidth - margin*2, + H: editorH, + } + + // Compute viewport height in Dp + viewportHeight := editorRegion.H + + // Compute visible byte range + var visibleContent string + var visibleCursorPos int + var visibleScrollOffset ui.Dp + + cb := TheState.Editor.ChunkedBuffer + if cb != nil { + start, end := cb.VisibleByteRange(TheState.ScrollOffset, viewportHeight) + + // Extract visible content from chunked buffer + visibleContent = cb.Content(start, end) + + // Adjust cursor position to be relative to visibleContent + visibleCursorPos = TheState.Editor.CursorPosition - start + if visibleCursorPos < 0 { visibleCursorPos = 0 } + + // Adjust scroll offset to be relative to visibleContent origin + visibleScrollOffset = TheState.ScrollOffset + + // Prefetch adjacent chunks for smooth scrolling + cursorChunk := visibleCursorPos / cb.ChunkSize() + cb.Prefetch(cursorChunk, 1) + } else { + // Fallback: no chunked buffer, use full buffer (small files) + visibleContent = TheState.Editor.Buffer + visibleCursorPos = TheState.Editor.CursorPosition + visibleScrollOffset = TheState.ScrollOffset + } + + // ... existing bottom bar code ... + + editorElem := ui.NewTextField( + "editor_text", + visibleContent, // ← visible content only + editorRegion, + editorRegion.W, + visibleScrollOffset, // ← adjusted scroll + visibleCursorPos, // ← adjusted cursor + interactions, + ) + + return []ui.Element{statusBar, editorElem, bottomBar} +} +``` + +### Step 6: drawWrappedText — Render Only Visible Content + +**File:** `internal/ui/render.go` — update `drawWrappedText`: + +The function already receives `str` as the text to render. With virtual scrolling, `str` is now **only the visible content** (not the full buffer). The function body remains largely the same, but the impact is dramatic: + +```go +func (r *Renderer) drawWrappedText(gtx layout.Context, str string, reg Region, wrapWidth Dp, scrollOffset Dp, cursorPos int) { + if str == "" { + return + } + + // str is now the VISIBLE portion of the file (typically 50–500 lines) + // NOT the entire buffer. The shaper iterates only visible glyphs. + + params := text.Parameters{ + PxPerEm: fixed.I(gtx.Sp(r.theme.FontSize)), + MinWidth: 0, + MaxWidth: int(r.toPx(wrapWidth)), + MaxLines: 0, + LineHeight: fixed.I(gtx.Sp(lineHeightSp)), + LineHeightScale: 1.0, + WrapPolicy: text.WrapHeuristically, + } + + // ONE LayoutString call — but now on visible content only (not full file) + r.shp.LayoutString(params, str) + + // ... rest of the function unchanged ... + // GlyphLayout now has ~50–500 entries instead of millions +} +``` + +**Key change:** The function signature is unchanged. The difference is that `str` is now the visible content extracted by the logic layer. No changes to the shaping or drawing loop are needed — the optimization comes from passing a small string instead of a large one. + +### Step 7: TextField — Pass Visible Content + +**File:** `internal/ui/element.go` — `TextField.Draw` is unchanged: +```go +func (tf TextField) Draw(gtx layout.Context, r *Renderer) { + r.drawWrappedText(gtx, tf.Value, tf.region, tf.WrapWidth, tf.ScrollOffset, tf.CursorPosition) +} +``` + +`tf.Value` is now the visible content. No changes needed. + +### Step 8: Edit Operations — Update ChunkedBuffer + +**File:** `internal/editor/state.go` — update `HandleInsert`, `HandleBackspace`, `HandleDelete`: + +```go +func HandleInsert(s string) { + pos := TheState.Editor.CursorPosition + buf := TheState.Editor.ChunkedBuffer + + // Insert into the chunked buffer + buf.Insert(pos, s) + TheState.Editor.CursorPosition += len(s) + markDirty() +} + +func HandleBackspace() { + pos := TheState.Editor.CursorPosition + if pos == 0 { + return + } + buf := TheState.Editor.ChunkedBuffer + buf.Delete(pos-1, 1) + TheState.Editor.CursorPosition-- + markDirty() +} + +func HandleDelete() { + pos := TheState.Editor.CursorPosition + buf := TheState.Editor.ChunkedBuffer + buf.Delete(pos, 1) + markDirty() +} +``` + +**File:** `internal/editor/chunked_buffer.go` — implement `Insert` and `Delete`: + +```go +func (cb *ChunkedBuffer) Insert(pos int, text string) { + // 1. Ensure the chunk containing pos is loaded + chunkIdx := pos / cb.chunkSize + if _, ok := cb.chunks[chunkIdx]; !ok { + cb.loadChunk(chunkIdx) + } + + // 2. Insert into the chunk + chunk := cb.chunks[chunkIdx] + offsetInChunk := pos - chunkIdx*cb.chunkSize + cb.chunks[chunkIdx] = append(chunk[:offsetInChunk], append([]byte(text), chunk[offsetInChunk:]...)...) + + // 3. Mark buffer dirty + cb.dirty = true + + // 4. If insertion spans chunk boundary, merge chunks + cb.maybeMergeChunk(chunkIdx) +} + +func (cb *ChunkedBuffer) Delete(pos, n int) { + // 1. Ensure relevant chunks are loaded + startChunk := pos / cb.chunkSize + endChunk := (pos + n - 1) / cb.chunkSize + for i := startChunk; i <= endChunk; i++ { + if _, ok := cb.chunks[i]; !ok { + cb.loadChunk(i) + } + } + + // 2. Delete from chunks + for i := startChunk; i <= endChunk; i++ { + chunk := cb.chunks[i] + offsetInChunk := pos - i*cb.chunkSize + deleteEnd := min(offsetInChunk+n, len(chunk)) + if offsetInChunk < len(chunk) { + cb.chunks[i] = append(chunk[:offsetInChunk], chunk[deleteEnd:]...) + } + } + + cb.dirty = true + cb.maybeMergeChunk(startChunk) +} +``` + +### Step 9: Auto-Save — Write Chunked Buffer + +**File:** `internal/editor/logic.go` — update auto-save to reconstruct full content: + +```go +func (l *Logic) markDirty() { + // ... existing code ... + + l.saveTimer = time.AfterFunc(1*time.Second, func() { + if l.saveGeneration == gen+1 { + // Reconstruct full content from chunks for saving + cb := l.state.Editor.ChunkedBuffer + content := cb.FullContent() // reads all chunks + re-reads unloaded from disk + l.workerPool.DispatchNonBlocking( + pool.NewWriteFileTask(filename, []byte(content), l.mockFS), + ) + } + }) +} +``` + +**File:** `internal/editor/chunked_buffer.go` — implement `FullContent()`: + +```go +func (cb *ChunkedBuffer) FullContent() string { + numChunks := int((cb.fileLen + int64(cb.chunkSize) - 1) / int64(cb.chunkSize)) + var buf bytes.Buffer + for i := 0; i < numChunks; i++ { + if chunk, ok := cb.chunks[i]; ok { + buf.Write(chunk) + } else { + // Re-read from disk + chunk := cb.loadChunk(i) + buf.Write(chunk) + } + } + return buf.String() +} +``` + +### Step 10: Buffer Field Deprecation + +**File:** `internal/editor/state.go` — keep `Buffer` field for backward compatibility but deprecate: + +```go +type EditorState struct { + Buffer string // DEPRECATED: use ChunkedBuffer for large files + ChunkedBuffer *ChunkedBuffer // NEW: primary buffer for editing + // ... +} +``` + +Provide a `GetBuffer()` method that reconstructs the full string on demand: +```go +func (e *EditorState) GetBuffer() string { + if e.ChunkedBuffer != nil { + return e.ChunkedBuffer.FullContent() + } + return e.Buffer +} +``` + +--- + +## 6. File Change Summary + +| File | Change | +|---|---| +| `internal/editor/chunked_buffer.go` | **NEW** — ChunkedBuffer, LineIndex, VisibleByteRange, Insert, Delete | +| `internal/editor/state.go` | Add `ChunkedBuffer`, `LineIndex` fields; update `OpenFile`, `HandleInsert`, `HandleBackspace`, `HandleDelete`, `EditorLayout` | +| `internal/editor/logic.go` | Add `TypeReadChunk`, `TypeStatFile`, `TypeBuildLineIndex` handling; update `markDirty` for chunked save | +| `internal/io/pool/task.go` | Add `ReadChunkTask`, `StatFileTask`, `BuildLineIndexTask`; add `TypeReadChunk`, `TypeStatFile`, `TypeBuildLineIndex` constants; add `FileStat`, `LineIndex` types | +| `internal/ui/render.go` | `drawWrappedText` — no signature change; optimization comes from smaller `str` input | +| `internal/ui/element.go` | `TextField.Draw` — no change; passes visible content | + +--- + +## 7. Performance Impact + +| Metric | Before | After | Spec Target | +|---|---|---|---| +| Open 100MB file | ~1000ms (full read + string) | ~50ms (stat + 1 chunk) | < 100ms | +| Scroll frame time (1M line file) | ~500ms+ (shapes all glyphs) | ~2ms (shapes ~50 lines) | 60 fps | +| Memory (100MB file) | ~200MB+ (string + GlyphLayout) | ~6.4MB (100 chunks + viewport GlyphLayout) | 16–64 MB | +| Per-keystroke latency | ~500ms+ (full reshape) | ~5ms (visible reshape) | < 16ms | +| GlyphLayout size (1M line file) | ~10M entries | ~50–500 entries | bounded | + +--- + +## 8. Edge Cases and Fallbacks + +| Scenario | Handling | +|---|---| +| File < 1 MB | Use full buffer (no chunking needed); virtual scrolling still applies but visible content ≈ full content | +| Line index build in progress | Use `visibleByteRangeEstimate` (heuristic) until index is ready | +| File smaller than chunk size | Single chunk; no chunk management overhead | +| Chunk boundary in middle of a line | Chunk content is raw bytes; line wrapping handles partial lines naturally | +| Edit at chunk boundary | `Insert`/`Delete` handles multi-chunk operations; `maybeMergeChunk` consolidates if needed | +| File replaced externally | `StatFileTask` detects size/mtime change; reload chunks and rebuild index | +| Very long line (> viewport width) | Shaper handles it; virtual scrolling still works because we shape only visible content | +| Cursor off-screen | `SetCursorFromPoint` uses GlyphLayout from visible content; cursor is always on-screen | + +--- + +## 9. Testing Strategy + +### Unit Tests + +**File:** `internal/editor/chunked_buffer_test.go` (new) + +| Test | What it verifies | +|---|---| +| `TestChunkedBuffer_LoadChunk` | Loading a chunk reads the correct bytes from disk | +| `TestChunkedBuffer_Content` | `Content(start, end)` returns the correct byte range | +| `TestChunkedBuffer_Insert` | Inserting text updates the correct chunk | +| `TestChunkedBuffer_Delete` | Deleting bytes removes them from the correct chunk | +| `TestChunkedBuffer_EvictFarChunks` | Chunks far from cursor are evicted | +| `TestChunkedBuffer_Prefetch` | Prefetch loads adjacent chunks | +| `TestChunkedBuffer_VisibleByteRange` | Visible byte range is correct for various scroll positions | +| `TestChunkedBuffer_FullContent` | `FullContent()` reconstructs the exact original file | +| `TestChunkedBuffer_EmptyFile` | Handles zero-length files | +| `TestChunkedBuffer_FileSmallerThanChunk` | Single-chunk files work correctly | +| `TestLineIndex_Build` | Line index correctly maps line numbers to byte offsets | +| `TestLineIndex_BinarySearch` | Binary search finds correct byte offset for any line | +| `TestLineIndex_Invalidation` | Index is invalidated when file size changes | +| `TestDeterministicFile_GeneratesKnownContent` | Same seed produces identical content every time | +| `TestDeterministicFile_LineBased` | Line-based generator produces correct line count and byte offsets | + +### Integration Tests + +**File:** `internal/editor/virtual_scroll_test.go` (new) + +| Test | What it verifies | +|---|---| +| `TestEditorLayout_VisibleContent` | EditorLayout passes only visible content to TextField | +| `TestEditorLayout_ScrollUpdates` | Scrolling changes the visible byte range | +| `TestOpenFile_ChunkedLoad` | Opening a file loads the initial chunk and dispatches index build | +| `TestAutoSave_ChunkedWrite` | Auto-save reconstructs full content and writes it | +| `TestEdit_ChunkedBuffer` | Insert/delete on chunked buffer maintains content correctness | +| `TestVirtualScroll_10KLines` | Opening a 10K-line file loads only visible chunks; frame time < 16ms | +| `TestVirtualScroll_100KLines` | Scrolling through 100K lines keeps memory bounded; no full-file shapes | +| `TestLineJump_1MLines` | Jumping to line 500K in a 1M-line file completes < 10ms via line index | +| `TestSetCursorFromPoint_100KLines` | Click-to-position cursor works correctly on a 100K-line file | + +### Render Tests + +**File:** `internal/ui/render_test.go` — add tests for `drawWrappedText` with large strings to verify that GlyphLayout size is bounded. + +### E2E Tests + +**File:** `internal/test/e2e/virtual_scroll_test.go` (new) + +Uses the existing `Harness` to drive full editor workflows with deterministic large files. Tests verify the complete pipeline from file open through rendering to user interaction: + +- **Open and scroll**: Open a 100K-line file, scroll to line 50K, verify the frame contains the correct visible lines (checked by inspecting the TextField.Value length) +- **Click-to-position**: Simulate a tap at a known screen coordinate, verify the cursor lands at the expected byte offset using the GlyphLayout feedback +- **Edit and verify**: Insert text at a specific position in a large file, verify the buffer content is correct by reading back the affected chunk +- **Line index build**: Verify the line index is built asynchronously and visible byte range becomes precise after index is ready + +### Deterministic Large File Generator + +**File:** `internal/editor/deterministic_file.go` (new) + +Generates arbitrarily large files in-memory using a fixed seed and repeating block content. The mock filesystem stores the generated content as a `[]byte` without writing to disk. Two modes: + +- **Fixed seed mode**: `NewDeterministicFile(seed uint64, blockSize int, repeatCount int)` — produces `blockSize * repeatCount` bytes. Same seed always produces identical content. +- **Line-based mode**: `NewLineBasedFile(seed uint64, linesPerBlock int, blockCount int)` — each block is a fixed set of lines repeated. Useful for testing line-index and line-jump correctness. + +The generator is used by tests to create 10K, 100K, and 1M line files in the mock filesystem in milliseconds, with zero disk I/O. The content is a known repeating pattern so byte offsets, line numbers, and visible ranges can be verified exactly. + +### Click-to-Position Verification + +**File:** `internal/editor/cursor_test.go` — extend existing cursor tests with large-file scenarios using the deterministic generator. + +- **`TestSetCursorFromPoint_LargeFile`**: Generate a 50K-line file, simulate a tap at a known Dp coordinate, verify the cursor lands on the expected byte offset. Uses the GlyphLayout captured during `drawWrappedText`. +- **`TestSetCursorFromPoint_ChunkBoundary`**: Generate a file where the tap target falls near a chunk boundary, verify cursor positioning is correct even when the visible content spans multiple chunks. +- **`TestSetCursorFromPoint_OffScreen`**: Verify that tapping outside the visible viewport scrolls to the target and positions the cursor correctly. +- **`TestSetCursorFromPoint_WrappedLines`**: Generate a file with long lines that wrap, verify tap-to-position correctly identifies the visual line and closest glyph. + +### Performance Benchmarks + +**File:** `internal/editor/bench_test.go` (new) + +Go benchmarks that measure the actual performance of virtual scrolling with the deterministic generator: + +- `BenchmarkVisibleByteRange_10KLines` — measures time to compute visible byte range +- `BenchmarkVisibleByteRange_1MLines` — measures time to compute visible byte range on 1M-line file +- `BenchmarkChunkedBufferContent` — measures time to extract visible content from chunked buffer +- `BenchmarkDrawWrappedText_10KLines` — measures shaping/rendering time for 10K-line file (visible only) +- `BenchmarkDrawWrappedText_1MLines` — measures shaping/rendering time for 1M-line file (visible only) +- `BenchmarkLineIndexBinarySearch` — measures O(log n) line jump time + +All benchmarks use the deterministic generator to create files of known size. No disk I/O. Results are compared against the pre-virtual-scroll baseline (full-buffer shaping). + +--- + +