# Virtual Scrolling & Render Optimization — Implementation Plan ## 1. Problem Statement The spec (§11, "Performance Guarantees") mandates: | Operation | Target | Mechanism | |---|---|---| | Open any file | < 100 ms | Chunked load: only the chunk around the cursor is read | | Scroll | 60 fps | Virtual rendering: only visible lines + prefetch buffer | | Type | < 16 ms per keystroke | In-memory chunk edit, async chunk flush | **Current reality:** `drawWrappedText` in `internal/ui/render.go` receives the entire `EditorState.Buffer` as a single string and calls `shp.LayoutString(params, str)` on it — every frame. For a 100,000-line file, this means: 1. The shaper iterates ~1–10 million glyphs per frame 2. A `GlyphLayout` struct grows to millions of entries (ByteOffsets, X, Y, Advance slices) 3. Every line is drawn, including those far off-screen 4. Frame times are measured in hundreds of milliseconds — far exceeding the 16ms budget This is the primary cause of sluggishness when editing large files. The fix is **virtual scrolling**: only shape, render, and capture glyph layout for the bytes visible in the current viewport. --- ## 2. Design Principles 1. **The buffer is the source of truth.** The `EditorState.Buffer` string holds the complete file content. Virtual scrolling is a presentation-layer concern — the buffer itself is untouched. 2. **Logic computes the visible range; renderer renders only that range.** The logic layer determines which byte range is visible given scroll offset and viewport height. The renderer shapes and draws only those bytes. 3. **GlyphLayout is bounded to the viewport.** Per-glyph layout data is only captured for visible content, keeping memory usage constant regardless of file size. 4. **Chunking is a prerequisite.** Virtual scrolling requires that the buffer be backed by chunks (fixed-size file slices) so that the logic layer can load/unload chunks on demand. This plan assumes the chunked buffer exists; the chunked buffer implementation is described in a companion plan. --- ## 3. Architecture ``` ┌──────────────────────────────────────────────────────────────────┐ │ EditorState (internal/editor/state.go) │ │ │ │ Buffer string ← full file content (from chunks)│ │ ScrollOffset ui.Dp ← current scroll position │ │ ChunkedBuffer *ChunkedBuffer ← chunked file access │ │ LineIndex *LineIndex ← byte offsets per line │ │ │ │ VisibleByteRange(scroll, viewport) → (start, end) │ │ VisibleContent(start, end) → []byte │ └──────────────────────┬───────────────────────────────────────────┘ │ ▼ ┌──────────────────────────────────────────────────────────────────┐ │ EditorLayout (internal/editor/state.go) │ │ │ │ 1. Compute viewport height from editor region + scale │ │ 2. Call VisibleByteRange(scrollOffset, viewportHeight) │ │ 3. Extract visibleContent := ChunkedBuffer.VisibleContent() │ │ 4. Adjust cursorPos to be relative to visibleContent │ │ 5. Adjust scrollOffset to be relative to visibleContent origin │ │ 6. Pass visibleContent + adjusted offsets to NewTextField │ └──────────────────────┬───────────────────────────────────────────┘ │ ▼ ┌──────────────────────────────────────────────────────────────────┐ │ TextField.Draw → Renderer.drawWrappedText │ │ │ │ 1. Shape ONLY visibleContent (not the full buffer) │ │ 2. Capture GlyphLayout only for visible glyphs │ │ 3. Draw only visible lines │ │ 4. Render cursor at adjusted position │ │ 5. Return bounded GlyphLayout (≈ viewport-sized) │ └──────────────────────┬───────────────────────────────────────────┘ │ ▼ ┌──────────────────────────────────────────────────────────────────┐ │ GlyphLayout (internal/ui/unit.go) │ │ │ │ ByteOffsets []int ← byte offsets within visibleContent │ │ X []Dp ← screen X positions │ │ Y []Dp ← screen Y positions │ │ Advance []Dp ← glyph widths │ │ │ │ Length: proportional to visible lines (≈ 50–500 entries) │ │ NOT proportional to file size │ └──────────────────────────────────────────────────────────────────┘ ``` --- ## 4. Data Structures ### 4.1 ChunkedBuffer ```go // ChunkedBuffer provides chunked access to a file's content. // Only chunks near the cursor or viewport are kept in memory. type ChunkedBuffer struct { filename string chunkSize int // e.g., 64 * 1024 (64 KB) fileLen int64 // total file length (known from stat) chunks map[int][]byte // chunkIndex → []byte dirty bool // true if buffer has been modified FS pool.FileSystem // filesystem for reading chunks basePath string // base path for file resolution } ``` **Key properties:** - `chunkSize` is fixed at 64 KB (configurable constant). - `chunks` is a sparse map: only chunks near the cursor/viewport are loaded. - `fileLen` is known from the file stat task (dispatched when opening a file). - The full buffer is reconstructed on demand via `Content(start, end)` for rendering. ### 4.2 LineIndex ```go // LineIndex maps line numbers to byte offsets within the file. // Built as a low-priority background task; cached on disk. type LineIndex struct { offsets []int32 // byte offset of each line start (line 0 = 0) mtime int64 // file mtime at index build time size int64 // file size at index build time } ``` **Key properties:** - Built by streaming the file once (O(n) time, O(lines) space for the offset array). - Stored as `[]int32` — 4 bytes per line. A 1M-line file = 4 MB. - Used for O(log n) line number → byte offset lookup (binary search). - Used by `VisibleByteRange` to compute the exact byte range for visible lines. - Invalidated when file mtime or size changes. - Cached on disk at `.pad/indices/.bin`. ### 4.3 VisibleByteRange ```go // VisibleByteRange returns the byte range [start, end) of content // visible in the viewport, given the current scroll offset and viewport height. func (cb *ChunkedBuffer) VisibleByteRange(scrollOffset ui.Dp, viewportHeight ui.Dp) (start, end int) { if cb.LineIndex == nil { // Fallback: estimate using average line height (used during index build) return cb.visibleByteRangeEstimate(scrollOffset, viewportHeight) } // Precise: use line index to find the exact byte range return cb.visibleByteRangePrecise(scrollOffset, viewportHeight) } ``` **Two strategies:** | Strategy | When Used | Accuracy | |---|---|---| | `visibleByteRangeEstimate` | During initial load, before line index is built | Approximate (±1–2 lines) | | `visibleByteRangePrecise` | After line index is available | Exact (byte-accurate) | **Estimate strategy (no index):** 1. Compute average character width from a sample of lines. 2. Compute average line height from the shaper (one-shot measurement). 3. Estimate visible line range from scrollOffset / lineHeight. 4. Use binary search on the line index (if available) or scan from file start to find byte offsets. **Precise strategy (with index):** 1. Compute visible line range: `startLine = scrollOffset / lineHeight`, `endLine = (scrollOffset + viewportHeight) / lineHeight`. 2. Binary search in `LineIndex.offsets` for `startLine` and `endLine`. 3. Clamp to `[0, fileLen)`. 4. Return `(int(offsets[startLine]), int(offsets[min(endLine, len(offsets))-1]))`. ### 4.4 Chunked Content Extraction ```go // Content returns the bytes in [start, end) from the chunked buffer. // It loads missing chunks on demand. func (cb *ChunkedBuffer) Content(start, end int) string { // 1. Determine which chunks are needed startChunk := start / cb.chunkSize endChunk := (end - 1) / cb.chunkSize // 2. Load missing chunks for i := startChunk; i <= endChunk; i++ { if _, ok := cb.chunks[i]; !ok { cb.loadChunk(i) // reads from disk via FS.ReadFile (full file) } } // 3. Concatenate needed chunk slices var buf bytes.Buffer for i := startChunk; i <= endChunk; i++ { chunk := cb.chunks[i] chunkStart := i * cb.chunkSize chunkEnd := chunkStart + len(chunk) segStart := max(start, chunkStart) - chunkStart segEnd := min(end, chunkEnd) - chunkStart buf.Write(chunk[segStart:segEnd]) } return buf.String() } // Prefetch loads adjacent chunks for smooth scrolling. func (cb *ChunkedBuffer) Prefetch(centerChunk int, radius int) { for i := centerChunk - radius; i <= centerChunk + radius; i++ { if i >= 0 && i < cb.numChunks() { if _, ok := cb.chunks[i]; !ok { cb.loadChunk(i) } } } } ``` --- ## 5. Implementation Steps ### Step 1: ChunkedBuffer Implementation **File:** `internal/editor/chunked_buffer.go` (new) Implement the `ChunkedBuffer` struct with: - `NewChunkedBuffer(filename, chunkSize, fs, basePath)` constructor - `loadChunk(idx int)` — reads a single chunk from disk via `ReadFile` - `Content(start, end int) string` — extracts bytes from loaded chunks - `Prefetch(centerChunk, radius int)` — loads adjacent chunks - `EvictFarChunks(cursorPos int, radius int)` — removes chunks far from cursor - `FileLen() int64` — total file length - `Filename() string` — the file path **File:** `internal/io/pool/task.go` — add `ReadChunkTask` ```go type ReadChunkTask struct { taskID string Path string ChunkIdx int FS FileSystem } func NewReadChunkTask(path string, chunkIdx int, fs FileSystem) *ReadChunkTask { ... } func (t *ReadChunkTask) Execute() Result { // Read the full file, then slice the requested chunk content, err := t.FS.ReadFile(t.Path) if err != nil { return Result{TaskID: t.taskID, TaskType: TypeReadChunk, Success: false, Error: err} } start := t.ChunkIdx * 64 * 1024 end := min(start+64*1024, len(content)) if start >= len(content) { return Result{TaskID: t.taskID, TaskType: TypeReadChunk, Success: true, Data: []byte{}} } chunk := content[start:end] return Result{TaskID: t.taskID, TaskType: TypeReadChunk, Success: true, Data: chunk} } func (t *ReadChunkTask) Priority() Priority { return HighPriority } func (t *ReadChunkTask) TaskType() TaskType { return TypeReadChunk } ``` **File:** `internal/io/pool/task.go` — add `TypeReadChunk` constant. **File:** `internal/io/pool/task.go` — add `StatFileTask` (replaces full-file `ReadFileTask` for open) ```go type StatFileTask struct { taskID string Path string FS FileSystem } func NewStatFileTask(path string, fs FileSystem) *StatFileTask { ... } func (t *StatFileTask) Execute() Result { // Read the full file content for now (will be optimized later with chunked stat) content, err := t.FS.ReadFile(t.Path) if err != nil { return Result{TaskID: t.taskID, TaskType: TypeStatFile, Success: false, Error: err} } return Result{ TaskID: t.taskID, TaskType: TypeStatFile, Success: true, Data: &FileStat{Path: t.Path, Size: int64(len(content))}, } } type FileStat struct { Path string Size int64 } ``` ### Step 2: ChunkedBuffer on EditorState **File:** `internal/editor/state.go` ```go type EditorState struct { // ... existing fields ... ChunkedBuffer *ChunkedBuffer // NEW: chunked file access LineIndex *LineIndex // NEW: line-to-byte-offset mapping } ``` Update `OpenFile` in `state.go`: ```go func OpenFile(data any) { filename := data.(string) TheState.Editor.Filename = filename // Create chunked buffer chunkSize := 64 * 1024 // 64 KB cb := NewChunkedBuffer(filename, chunkSize, TheLogic.mockFS, "") TheState.Editor.ChunkedBuffer = cb // Dispatch stat task to get file size TheLogic.workerPool.Dispatch(pool.NewStatFileTask(filename, TheLogic.mockFS)) // Switch to editor page TheState.page = EditorPage TheState.FocusedElementID = "editor_text" TheState.ScrollOffset = 0 TheState.Editor.CursorPosition = 0 } ``` Update `handleWorkerResult` in `logic.go`: ```go case res.TaskType == pool.TypeStatFile: if res.Success { if stat, ok := res.Data.(*pool.FileStat); ok { TheState.Editor.ChunkedBuffer.SetFileSize(stat.Size) // Load the chunk containing the cursor (position 0 at open) TheState.Editor.ChunkedBuffer.LoadChunk(0) // Prefetch adjacent chunks TheState.Editor.ChunkedBuffer.Prefetch(0, 1) } } ``` ### Step 3: VisibleByteRange and Content Extraction **File:** `internal/editor/chunked_buffer.go` — implement `VisibleByteRange` and `Content`. **File:** `internal/editor/chunked_buffer.go` — implement `visibleByteRangeEstimate`: ```go // visibleByteRangeEstimate approximates the visible byte range using // heuristic estimates. Used when the line index is not yet available. func (cb *ChunkedBuffer) visibleByteRangeEstimate(scrollOffset ui.Dp, viewportHeight ui.Dp) (start, end int) { totalLines := int(cb.fileLen) / 50 // rough: 50 bytes per line average lineHeight := EditorLineHeight() startLine := int(scrollOffset / lineHeight) endLine := int((scrollOffset + viewportHeight) / lineHeight) // Clamp if startLine < 0 { startLine = 0 } if endLine > totalLines { endLine = totalLines } // Convert line numbers to byte offsets using the line index if available if cb.LineIndex != nil && startLine < len(cb.LineIndex.offsets) { start = int(cb.LineIndex.offsets[startLine]) } else { start = startLine * 50 // rough estimate } if cb.LineIndex != nil && endLine < len(cb.LineIndex.offsets) { end = int(cb.LineIndex.offsets[endLine]) } else { end = endLine * 50 } // Clamp to file bounds if start < 0 { start = 0 } if end > int(cb.fileLen) { end = int(cb.fileLen) } if end <= start { end = start + 100 } // at least 100 bytes return start, end } ``` ### Step 4: Line Index Build Task **File:** `internal/io/pool/task.go` — add `BuildLineIndexTask`: ```go type BuildLineIndexTask struct { taskID string Path string FS FileSystem } func NewBuildLineIndexTask(path string, fs FileSystem) *BuildLineIndexTask { ... } func (t *BuildLineIndexTask) Execute() Result { content, err := t.FS.ReadFile(t.Path) if err != nil { return Result{TaskID: t.taskID, TaskType: TypeBuildLineIndex, Success: false, Error: err} } offsets := []int32{0} for i := 0; i < len(content); i++ { if content[i] == '\n' { offsets = append(offsets, int32(i+1)) } } return Result{ TaskID: t.taskID, TaskType: TypeBuildLineIndex, Success: true, Data: &LineIndex{offsets: offsets, mtime: 0, size: int64(len(content))}, } } func (t *BuildLineIndexTask) Priority() Priority { return LowPriority } func (t *BuildLineIndexTask) TaskType() TaskType { return TypeBuildLineIndex } ``` **File:** `internal/io/pool/task.go` — add `TypeBuildLineIndex` constant. **File:** `internal/editor/logic.go` — handle `TypeBuildLineIndex` result: ```go case res.TaskType == pool.TypeBuildLineIndex: if res.Success { if idx, ok := res.Data.(*LineIndex); ok { TheState.Editor.LineIndex = idx } } ``` **Dispatch the line index build in `OpenFile`:** ```go // After loading the initial chunk: TheLogic.workerPool.Dispatch(pool.NewBuildLineIndexTask(filename, TheLogic.mockFS)) ``` ### Step 5: EditorLayout — Compute Visible Content **File:** `internal/editor/state.go` — update `EditorLayout`: ```go func EditorLayout(screenWidth, screenHeight ui.Dp, wordWrap bool) []ui.Element { // ... existing status bar and bottom bar code ... // --- Editor text area --- editorRegion := ui.Region{ X: margin, Y: editorY, W: screenWidth - margin*2, H: editorH, } // Compute viewport height in Dp viewportHeight := editorRegion.H // Compute visible byte range var visibleContent string var visibleCursorPos int var visibleScrollOffset ui.Dp cb := TheState.Editor.ChunkedBuffer if cb != nil { start, end := cb.VisibleByteRange(TheState.ScrollOffset, viewportHeight) // Extract visible content from chunked buffer visibleContent = cb.Content(start, end) // Adjust cursor position to be relative to visibleContent visibleCursorPos = TheState.Editor.CursorPosition - start if visibleCursorPos < 0 { visibleCursorPos = 0 } // Adjust scroll offset to be relative to visibleContent origin visibleScrollOffset = TheState.ScrollOffset // Prefetch adjacent chunks for smooth scrolling cursorChunk := visibleCursorPos / cb.ChunkSize() cb.Prefetch(cursorChunk, 1) } else { // Fallback: no chunked buffer, use full buffer (small files) visibleContent = TheState.Editor.Buffer visibleCursorPos = TheState.Editor.CursorPosition visibleScrollOffset = TheState.ScrollOffset } // ... existing bottom bar code ... editorElem := ui.NewTextField( "editor_text", visibleContent, // ← visible content only editorRegion, editorRegion.W, visibleScrollOffset, // ← adjusted scroll visibleCursorPos, // ← adjusted cursor interactions, ) return []ui.Element{statusBar, editorElem, bottomBar} } ``` ### Step 6: drawWrappedText — Render Only Visible Content **File:** `internal/ui/render.go` — update `drawWrappedText`: The function already receives `str` as the text to render. With virtual scrolling, `str` is now **only the visible content** (not the full buffer). The function body remains largely the same, but the impact is dramatic: ```go func (r *Renderer) drawWrappedText(gtx layout.Context, str string, reg Region, wrapWidth Dp, scrollOffset Dp, cursorPos int) { if str == "" { return } // str is now the VISIBLE portion of the file (typically 50–500 lines) // NOT the entire buffer. The shaper iterates only visible glyphs. params := text.Parameters{ PxPerEm: fixed.I(gtx.Sp(r.theme.FontSize)), MinWidth: 0, MaxWidth: int(r.toPx(wrapWidth)), MaxLines: 0, LineHeight: fixed.I(gtx.Sp(lineHeightSp)), LineHeightScale: 1.0, WrapPolicy: text.WrapHeuristically, } // ONE LayoutString call — but now on visible content only (not full file) r.shp.LayoutString(params, str) // ... rest of the function unchanged ... // GlyphLayout now has ~50–500 entries instead of millions } ``` **Key change:** The function signature is unchanged. The difference is that `str` is now the visible content extracted by the logic layer. No changes to the shaping or drawing loop are needed — the optimization comes from passing a small string instead of a large one. ### Step 7: TextField — Pass Visible Content **File:** `internal/ui/element.go` — `TextField.Draw` is unchanged: ```go func (tf TextField) Draw(gtx layout.Context, r *Renderer) { r.drawWrappedText(gtx, tf.Value, tf.region, tf.WrapWidth, tf.ScrollOffset, tf.CursorPosition) } ``` `tf.Value` is now the visible content. No changes needed. ### Step 8: Edit Operations — Update ChunkedBuffer **File:** `internal/editor/state.go` — update `HandleInsert`, `HandleBackspace`, `HandleDelete`: ```go func HandleInsert(s string) { pos := TheState.Editor.CursorPosition buf := TheState.Editor.ChunkedBuffer // Insert into the chunked buffer buf.Insert(pos, s) TheState.Editor.CursorPosition += len(s) markDirty() } func HandleBackspace() { pos := TheState.Editor.CursorPosition if pos == 0 { return } buf := TheState.Editor.ChunkedBuffer buf.Delete(pos-1, 1) TheState.Editor.CursorPosition-- markDirty() } func HandleDelete() { pos := TheState.Editor.CursorPosition buf := TheState.Editor.ChunkedBuffer buf.Delete(pos, 1) markDirty() } ``` **File:** `internal/editor/chunked_buffer.go` — implement `Insert` and `Delete`: ```go func (cb *ChunkedBuffer) Insert(pos int, text string) { // 1. Ensure the chunk containing pos is loaded chunkIdx := pos / cb.chunkSize if _, ok := cb.chunks[chunkIdx]; !ok { cb.loadChunk(chunkIdx) } // 2. Insert into the chunk chunk := cb.chunks[chunkIdx] offsetInChunk := pos - chunkIdx*cb.chunkSize cb.chunks[chunkIdx] = append(chunk[:offsetInChunk], append([]byte(text), chunk[offsetInChunk:]...)...) // 3. Mark buffer dirty cb.dirty = true // 4. If insertion spans chunk boundary, merge chunks cb.maybeMergeChunk(chunkIdx) } func (cb *ChunkedBuffer) Delete(pos, n int) { // 1. Ensure relevant chunks are loaded startChunk := pos / cb.chunkSize endChunk := (pos + n - 1) / cb.chunkSize for i := startChunk; i <= endChunk; i++ { if _, ok := cb.chunks[i]; !ok { cb.loadChunk(i) } } // 2. Delete from chunks for i := startChunk; i <= endChunk; i++ { chunk := cb.chunks[i] offsetInChunk := pos - i*cb.chunkSize deleteEnd := min(offsetInChunk+n, len(chunk)) if offsetInChunk < len(chunk) { cb.chunks[i] = append(chunk[:offsetInChunk], chunk[deleteEnd:]...) } } cb.dirty = true cb.maybeMergeChunk(startChunk) } ``` ### Step 9: Auto-Save — Write Chunked Buffer **File:** `internal/editor/logic.go` — update auto-save to reconstruct full content: ```go func (l *Logic) markDirty() { // ... existing code ... l.saveTimer = time.AfterFunc(1*time.Second, func() { if l.saveGeneration == gen+1 { // Reconstruct full content from chunks for saving cb := l.state.Editor.ChunkedBuffer content := cb.FullContent() // reads all chunks + re-reads unloaded from disk l.workerPool.DispatchNonBlocking( pool.NewWriteFileTask(filename, []byte(content), l.mockFS), ) } }) } ``` **File:** `internal/editor/chunked_buffer.go` — implement `FullContent()`: ```go func (cb *ChunkedBuffer) FullContent() string { numChunks := int((cb.fileLen + int64(cb.chunkSize) - 1) / int64(cb.chunkSize)) var buf bytes.Buffer for i := 0; i < numChunks; i++ { if chunk, ok := cb.chunks[i]; ok { buf.Write(chunk) } else { // Re-read from disk chunk := cb.loadChunk(i) buf.Write(chunk) } } return buf.String() } ``` ### Step 10: Buffer Field Deprecation **File:** `internal/editor/state.go` — keep `Buffer` field for backward compatibility but deprecate: ```go type EditorState struct { Buffer string // DEPRECATED: use ChunkedBuffer for large files ChunkedBuffer *ChunkedBuffer // NEW: primary buffer for editing // ... } ``` Provide a `GetBuffer()` method that reconstructs the full string on demand: ```go func (e *EditorState) GetBuffer() string { if e.ChunkedBuffer != nil { return e.ChunkedBuffer.FullContent() } return e.Buffer } ``` --- ## 6. File Change Summary | File | Change | |---|---| | `internal/editor/chunked_buffer.go` | **NEW** — ChunkedBuffer, LineIndex, VisibleByteRange, Insert, Delete | | `internal/editor/state.go` | Add `ChunkedBuffer`, `LineIndex` fields; update `OpenFile`, `HandleInsert`, `HandleBackspace`, `HandleDelete`, `EditorLayout` | | `internal/editor/logic.go` | Add `TypeReadChunk`, `TypeStatFile`, `TypeBuildLineIndex` handling; update `markDirty` for chunked save | | `internal/io/pool/task.go` | Add `ReadChunkTask`, `StatFileTask`, `BuildLineIndexTask`; add `TypeReadChunk`, `TypeStatFile`, `TypeBuildLineIndex` constants; add `FileStat`, `LineIndex` types | | `internal/ui/render.go` | `drawWrappedText` — no signature change; optimization comes from smaller `str` input | | `internal/ui/element.go` | `TextField.Draw` — no change; passes visible content | --- ## 7. Performance Impact | Metric | Before | After | Spec Target | |---|---|---|---| | Open 100MB file | ~1000ms (full read + string) | ~50ms (stat + 1 chunk) | < 100ms | | Scroll frame time (1M line file) | ~500ms+ (shapes all glyphs) | ~2ms (shapes ~50 lines) | 60 fps | | Memory (100MB file) | ~200MB+ (string + GlyphLayout) | ~6.4MB (100 chunks + viewport GlyphLayout) | 16–64 MB | | Per-keystroke latency | ~500ms+ (full reshape) | ~5ms (visible reshape) | < 16ms | | GlyphLayout size (1M line file) | ~10M entries | ~50–500 entries | bounded | --- ## 8. Edge Cases and Fallbacks | Scenario | Handling | |---|---| | File < 1 MB | Use full buffer (no chunking needed); virtual scrolling still applies but visible content ≈ full content | | Line index build in progress | Use `visibleByteRangeEstimate` (heuristic) until index is ready | | File smaller than chunk size | Single chunk; no chunk management overhead | | Chunk boundary in middle of a line | Chunk content is raw bytes; line wrapping handles partial lines naturally | | Edit at chunk boundary | `Insert`/`Delete` handles multi-chunk operations; `maybeMergeChunk` consolidates if needed | | File replaced externally | `StatFileTask` detects size/mtime change; reload chunks and rebuild index | | Very long line (> viewport width) | Shaper handles it; virtual scrolling still works because we shape only visible content | | Cursor off-screen | `SetCursorFromPoint` uses GlyphLayout from visible content; cursor is always on-screen | --- ## 9. Testing Strategy ### Unit Tests **File:** `internal/editor/chunked_buffer_test.go` (new) | Test | What it verifies | |---|---| | `TestChunkedBuffer_LoadChunk` | Loading a chunk reads the correct bytes from disk | | `TestChunkedBuffer_Content` | `Content(start, end)` returns the correct byte range | | `TestChunkedBuffer_Insert` | Inserting text updates the correct chunk | | `TestChunkedBuffer_Delete` | Deleting bytes removes them from the correct chunk | | `TestChunkedBuffer_EvictFarChunks` | Chunks far from cursor are evicted | | `TestChunkedBuffer_Prefetch` | Prefetch loads adjacent chunks | | `TestChunkedBuffer_VisibleByteRange` | Visible byte range is correct for various scroll positions | | `TestChunkedBuffer_FullContent` | `FullContent()` reconstructs the exact original file | | `TestChunkedBuffer_EmptyFile` | Handles zero-length files | | `TestChunkedBuffer_FileSmallerThanChunk` | Single-chunk files work correctly | | `TestLineIndex_Build` | Line index correctly maps line numbers to byte offsets | | `TestLineIndex_BinarySearch` | Binary search finds correct byte offset for any line | | `TestLineIndex_Invalidation` | Index is invalidated when file size changes | | `TestDeterministicFile_GeneratesKnownContent` | Same seed produces identical content every time | | `TestDeterministicFile_LineBased` | Line-based generator produces correct line count and byte offsets | ### Integration Tests **File:** `internal/editor/virtual_scroll_test.go` (new) | Test | What it verifies | |---|---| | `TestEditorLayout_VisibleContent` | EditorLayout passes only visible content to TextField | | `TestEditorLayout_ScrollUpdates` | Scrolling changes the visible byte range | | `TestOpenFile_ChunkedLoad` | Opening a file loads the initial chunk and dispatches index build | | `TestAutoSave_ChunkedWrite` | Auto-save reconstructs full content and writes it | | `TestEdit_ChunkedBuffer` | Insert/delete on chunked buffer maintains content correctness | | `TestVirtualScroll_10KLines` | Opening a 10K-line file loads only visible chunks; frame time < 16ms | | `TestVirtualScroll_100KLines` | Scrolling through 100K lines keeps memory bounded; no full-file shapes | | `TestLineJump_1MLines` | Jumping to line 500K in a 1M-line file completes < 10ms via line index | | `TestSetCursorFromPoint_100KLines` | Click-to-position cursor works correctly on a 100K-line file | ### Render Tests **File:** `internal/ui/render_test.go` — add tests for `drawWrappedText` with large strings to verify that GlyphLayout size is bounded. ### E2E Tests **File:** `internal/test/e2e/virtual_scroll_test.go` (new) Uses the existing `Harness` to drive full editor workflows with deterministic large files. Tests verify the complete pipeline from file open through rendering to user interaction: - **Open and scroll**: Open a 100K-line file, scroll to line 50K, verify the frame contains the correct visible lines (checked by inspecting the TextField.Value length) - **Click-to-position**: Simulate a tap at a known screen coordinate, verify the cursor lands at the expected byte offset using the GlyphLayout feedback - **Edit and verify**: Insert text at a specific position in a large file, verify the buffer content is correct by reading back the affected chunk - **Line index build**: Verify the line index is built asynchronously and visible byte range becomes precise after index is ready ### Deterministic Large File Generator **File:** `internal/editor/deterministic_file.go` (new) Generates arbitrarily large files in-memory using a fixed seed and repeating block content. The mock filesystem stores the generated content as a `[]byte` without writing to disk. Two modes: - **Fixed seed mode**: `NewDeterministicFile(seed uint64, blockSize int, repeatCount int)` — produces `blockSize * repeatCount` bytes. Same seed always produces identical content. - **Line-based mode**: `NewLineBasedFile(seed uint64, linesPerBlock int, blockCount int)` — each block is a fixed set of lines repeated. Useful for testing line-index and line-jump correctness. The generator is used by tests to create 10K, 100K, and 1M line files in the mock filesystem in milliseconds, with zero disk I/O. The content is a known repeating pattern so byte offsets, line numbers, and visible ranges can be verified exactly. ### Click-to-Position Verification **File:** `internal/editor/cursor_test.go` — extend existing cursor tests with large-file scenarios using the deterministic generator. - **`TestSetCursorFromPoint_LargeFile`**: Generate a 50K-line file, simulate a tap at a known Dp coordinate, verify the cursor lands on the expected byte offset. Uses the GlyphLayout captured during `drawWrappedText`. - **`TestSetCursorFromPoint_ChunkBoundary`**: Generate a file where the tap target falls near a chunk boundary, verify cursor positioning is correct even when the visible content spans multiple chunks. - **`TestSetCursorFromPoint_OffScreen`**: Verify that tapping outside the visible viewport scrolls to the target and positions the cursor correctly. - **`TestSetCursorFromPoint_WrappedLines`**: Generate a file with long lines that wrap, verify tap-to-position correctly identifies the visual line and closest glyph. ### Performance Benchmarks **File:** `internal/editor/bench_test.go` (new) Go benchmarks that measure the actual performance of virtual scrolling with the deterministic generator: - `BenchmarkVisibleByteRange_10KLines` — measures time to compute visible byte range - `BenchmarkVisibleByteRange_1MLines` — measures time to compute visible byte range on 1M-line file - `BenchmarkChunkedBufferContent` — measures time to extract visible content from chunked buffer - `BenchmarkDrawWrappedText_10KLines` — measures shaping/rendering time for 10K-line file (visible only) - `BenchmarkDrawWrappedText_1MLines` — measures shaping/rendering time for 1M-line file (visible only) - `BenchmarkLineIndexBinarySearch` — measures O(log n) line jump time All benchmarks use the deterministic generator to create files of known size. No disk I/O. Results are compared against the pre-virtual-scroll baseline (full-buffer shaping). ---