gmp/Pad

Greg Pomerantz c941a02f1a Draft virtual scroll render optimization plan.

2026-06-04 20:23:54 -04:00

33 KiB

Raw Blame History

Virtual Scrolling & Render Optimization — Implementation Plan

1. Problem Statement

The spec (§11, "Performance Guarantees") mandates:

Operation	Target	Mechanism
Open any file	< 100 ms	Chunked load: only the chunk around the cursor is read
Scroll	60 fps	Virtual rendering: only visible lines + prefetch buffer
Type	< 16 ms per keystroke	In-memory chunk edit, async chunk flush

Current reality: drawWrappedText in internal/ui/render.go receives the entire EditorState.Buffer as a single string and calls shp.LayoutString(params, str) on it — every frame. For a 100,000-line file, this means:

The shaper iterates ~1–10 million glyphs per frame
A GlyphLayout struct grows to millions of entries (ByteOffsets, X, Y, Advance slices)
Every line is drawn, including those far off-screen
Frame times are measured in hundreds of milliseconds — far exceeding the 16ms budget

This is the primary cause of sluggishness when editing large files. The fix is virtual scrolling: only shape, render, and capture glyph layout for the bytes visible in the current viewport.

2. Design Principles

The buffer is the source of truth. The EditorState.Buffer string holds the complete file content. Virtual scrolling is a presentation-layer concern — the buffer itself is untouched.
Logic computes the visible range; renderer renders only that range. The logic layer determines which byte range is visible given scroll offset and viewport height. The renderer shapes and draws only those bytes.
GlyphLayout is bounded to the viewport. Per-glyph layout data is only captured for visible content, keeping memory usage constant regardless of file size.
Chunking is a prerequisite. Virtual scrolling requires that the buffer be backed by chunks (fixed-size file slices) so that the logic layer can load/unload chunks on demand. This plan assumes the chunked buffer exists; the chunked buffer implementation is described in a companion plan.

3. Architecture

┌──────────────────────────────────────────────────────────────────┐
│  EditorState (internal/editor/state.go)                          │
│                                                                  │
│  Buffer          string          ← full file content (from chunks)│
│  ScrollOffset    ui.Dp           ← current scroll position       │
│  ChunkedBuffer   *ChunkedBuffer  ← chunked file access           │
│  LineIndex       *LineIndex      ← byte offsets per line         │
│                                                                  │
│  VisibleByteRange(scroll, viewport) → (start, end)              │
│  VisibleContent(start, end) → []byte                             │
└──────────────────────┬───────────────────────────────────────────┘
                       │
                       ▼
┌──────────────────────────────────────────────────────────────────┐
│  EditorLayout (internal/editor/state.go)                         │
│                                                                  │
│  1. Compute viewport height from editor region + scale           │
│  2. Call VisibleByteRange(scrollOffset, viewportHeight)          │
│  3. Extract visibleContent := ChunkedBuffer.VisibleContent()     │
│  4. Adjust cursorPos to be relative to visibleContent            │
│  5. Adjust scrollOffset to be relative to visibleContent origin  │
│  6. Pass visibleContent + adjusted offsets to NewTextField       │
└──────────────────────┬───────────────────────────────────────────┘
                       │
                       ▼
┌──────────────────────────────────────────────────────────────────┐
│  TextField.Draw → Renderer.drawWrappedText                       │
│                                                                  │
│  1. Shape ONLY visibleContent (not the full buffer)              │
│  2. Capture GlyphLayout only for visible glyphs                  │
│  3. Draw only visible lines                                      │
│  4. Render cursor at adjusted position                           │
│  5. Return bounded GlyphLayout (≈ viewport-sized)                │
└──────────────────────┬───────────────────────────────────────────┘
                       │
                       ▼
┌──────────────────────────────────────────────────────────────────┐
│  GlyphLayout (internal/ui/unit.go)                               │
│                                                                  │
│  ByteOffsets []int   ← byte offsets within visibleContent        │
│  X []Dp              ← screen X positions                        │
│  Y []Dp              ← screen Y positions                        │
│  Advance []Dp        ← glyph widths                              │
│                                                                  │
│  Length: proportional to visible lines (≈ 50–500 entries)        │
│  NOT proportional to file size                                   │
└──────────────────────────────────────────────────────────────────┘

4. Data Structures

4.1 ChunkedBuffer

// ChunkedBuffer provides chunked access to a file's content.
// Only chunks near the cursor or viewport are kept in memory.
type ChunkedBuffer struct {
    filename   string
    chunkSize  int       // e.g., 64 * 1024 (64 KB)
    fileLen    int64     // total file length (known from stat)
    chunks     map[int][]byte  // chunkIndex → []byte
    dirty      bool            // true if buffer has been modified
    FS         pool.FileSystem // filesystem for reading chunks
    basePath   string          // base path for file resolution
}

Key properties:

chunkSize is fixed at 64 KB (configurable constant).
chunks is a sparse map: only chunks near the cursor/viewport are loaded.
fileLen is known from the file stat task (dispatched when opening a file).
The full buffer is reconstructed on demand via Content(start, end) for rendering.

4.2 LineIndex

// LineIndex maps line numbers to byte offsets within the file.
// Built as a low-priority background task; cached on disk.
type LineIndex struct {
    offsets []int32   // byte offset of each line start (line 0 = 0)
    mtime   int64     // file mtime at index build time
    size    int64     // file size at index build time
}

Key properties:

Built by streaming the file once (O(n) time, O(lines) space for the offset array).
Stored as []int32 — 4 bytes per line. A 1M-line file = 4 MB.
Used for O(log n) line number → byte offset lookup (binary search).
Used by VisibleByteRange to compute the exact byte range for visible lines.
Invalidated when file mtime or size changes.
Cached on disk at .pad/indices/<sha256(path)>.bin.

4.3 VisibleByteRange

// VisibleByteRange returns the byte range [start, end) of content
// visible in the viewport, given the current scroll offset and viewport height.
func (cb *ChunkedBuffer) VisibleByteRange(scrollOffset ui.Dp, viewportHeight ui.Dp) (start, end int) {
    if cb.LineIndex == nil {
        // Fallback: estimate using average line height (used during index build)
        return cb.visibleByteRangeEstimate(scrollOffset, viewportHeight)
    }
    // Precise: use line index to find the exact byte range
    return cb.visibleByteRangePrecise(scrollOffset, viewportHeight)
}

Two strategies:

Strategy	When Used	Accuracy
`visibleByteRangeEstimate`	During initial load, before line index is built	Approximate (±1–2 lines)
`visibleByteRangePrecise`	After line index is available	Exact (byte-accurate)

Estimate strategy (no index):

Compute average character width from a sample of lines.
Compute average line height from the shaper (one-shot measurement).
Estimate visible line range from scrollOffset / lineHeight.
Use binary search on the line index (if available) or scan from file start to find byte offsets.

Precise strategy (with index):

Compute visible line range: startLine = scrollOffset / lineHeight, endLine = (scrollOffset + viewportHeight) / lineHeight.
Binary search in LineIndex.offsets for startLine and endLine.
Clamp to [0, fileLen).
Return (int(offsets[startLine]), int(offsets[min(endLine, len(offsets))-1])).

4.4 Chunked Content Extraction

// Content returns the bytes in [start, end) from the chunked buffer.
// It loads missing chunks on demand.
func (cb *ChunkedBuffer) Content(start, end int) string {
    // 1. Determine which chunks are needed
    startChunk := start / cb.chunkSize
    endChunk := (end - 1) / cb.chunkSize

    // 2. Load missing chunks
    for i := startChunk; i <= endChunk; i++ {
        if _, ok := cb.chunks[i]; !ok {
            cb.loadChunk(i)  // reads from disk via FS.ReadFile (full file)
        }
    }

    // 3. Concatenate needed chunk slices
    var buf bytes.Buffer
    for i := startChunk; i <= endChunk; i++ {
        chunk := cb.chunks[i]
        chunkStart := i * cb.chunkSize
        chunkEnd := chunkStart + len(chunk)
        segStart := max(start, chunkStart) - chunkStart
        segEnd := min(end, chunkEnd) - chunkStart
        buf.Write(chunk[segStart:segEnd])
    }
    return buf.String()
}

// Prefetch loads adjacent chunks for smooth scrolling.
func (cb *ChunkedBuffer) Prefetch(centerChunk int, radius int) {
    for i := centerChunk - radius; i <= centerChunk + radius; i++ {
        if i >= 0 && i < cb.numChunks() {
            if _, ok := cb.chunks[i]; !ok {
                cb.loadChunk(i)
            }
        }
    }
}

5. Implementation Steps

Step 1: ChunkedBuffer Implementation

File: internal/editor/chunked_buffer.go (new)

Implement the ChunkedBuffer struct with:

NewChunkedBuffer(filename, chunkSize, fs, basePath) constructor
loadChunk(idx int) — reads a single chunk from disk via ReadFile
Content(start, end int) string — extracts bytes from loaded chunks
Prefetch(centerChunk, radius int) — loads adjacent chunks
EvictFarChunks(cursorPos int, radius int) — removes chunks far from cursor
FileLen() int64 — total file length
Filename() string — the file path

File: internal/io/pool/task.go — add ReadChunkTask

type ReadChunkTask struct {
    taskID   string
    Path     string
    ChunkIdx int
    FS       FileSystem
}

func NewReadChunkTask(path string, chunkIdx int, fs FileSystem) *ReadChunkTask { ... }

func (t *ReadChunkTask) Execute() Result {
    // Read the full file, then slice the requested chunk
    content, err := t.FS.ReadFile(t.Path)
    if err != nil {
        return Result{TaskID: t.taskID, TaskType: TypeReadChunk, Success: false, Error: err}
    }
    start := t.ChunkIdx * 64 * 1024
    end := min(start+64*1024, len(content))
    if start >= len(content) {
        return Result{TaskID: t.taskID, TaskType: TypeReadChunk, Success: true, Data: []byte{}}
    }
    chunk := content[start:end]
    return Result{TaskID: t.taskID, TaskType: TypeReadChunk, Success: true, Data: chunk}
}

func (t *ReadChunkTask) Priority() Priority { return HighPriority }
func (t *ReadChunkTask) TaskType() TaskType { return TypeReadChunk }

File: internal/io/pool/task.go — add TypeReadChunk constant.

File: internal/io/pool/task.go — add StatFileTask (replaces full-file ReadFileTask for open)

type StatFileTask struct {
    taskID string
    Path   string
    FS     FileSystem
}

func NewStatFileTask(path string, fs FileSystem) *StatFileTask { ... }

func (t *StatFileTask) Execute() Result {
    // Read the full file content for now (will be optimized later with chunked stat)
    content, err := t.FS.ReadFile(t.Path)
    if err != nil {
        return Result{TaskID: t.taskID, TaskType: TypeStatFile, Success: false, Error: err}
    }
    return Result{
        TaskID: t.taskID, TaskType: TypeStatFile, Success: true,
        Data: &FileStat{Path: t.Path, Size: int64(len(content))},
    }
}

type FileStat struct {
    Path string
    Size int64
}

Step 2: ChunkedBuffer on EditorState

File: internal/editor/state.go

type EditorState struct {
    // ... existing fields ...
    ChunkedBuffer *ChunkedBuffer  // NEW: chunked file access
    LineIndex     *LineIndex      // NEW: line-to-byte-offset mapping
}

Update OpenFile in state.go:

func OpenFile(data any) {
    filename := data.(string)
    TheState.Editor.Filename = filename
    
    // Create chunked buffer
    chunkSize := 64 * 1024 // 64 KB
    cb := NewChunkedBuffer(filename, chunkSize, TheLogic.mockFS, "")
    TheState.Editor.ChunkedBuffer = cb
    
    // Dispatch stat task to get file size
    TheLogic.workerPool.Dispatch(pool.NewStatFileTask(filename, TheLogic.mockFS))
    
    // Switch to editor page
    TheState.page = EditorPage
    TheState.FocusedElementID = "editor_text"
    TheState.ScrollOffset = 0
    TheState.Editor.CursorPosition = 0
}

Update handleWorkerResult in logic.go:

case res.TaskType == pool.TypeStatFile:
    if res.Success {
        if stat, ok := res.Data.(*pool.FileStat); ok {
            TheState.Editor.ChunkedBuffer.SetFileSize(stat.Size)
            // Load the chunk containing the cursor (position 0 at open)
            TheState.Editor.ChunkedBuffer.LoadChunk(0)
            // Prefetch adjacent chunks
            TheState.Editor.ChunkedBuffer.Prefetch(0, 1)
        }
    }

Step 3: VisibleByteRange and Content Extraction

File: internal/editor/chunked_buffer.go — implement VisibleByteRange and Content.

File: internal/editor/chunked_buffer.go — implement visibleByteRangeEstimate:

// visibleByteRangeEstimate approximates the visible byte range using
// heuristic estimates. Used when the line index is not yet available.
func (cb *ChunkedBuffer) visibleByteRangeEstimate(scrollOffset ui.Dp, viewportHeight ui.Dp) (start, end int) {
    totalLines := int(cb.fileLen) / 50 // rough: 50 bytes per line average
    lineHeight := EditorLineHeight()
    
    startLine := int(scrollOffset / lineHeight)
    endLine := int((scrollOffset + viewportHeight) / lineHeight)
    
    // Clamp
    if startLine < 0 { startLine = 0 }
    if endLine > totalLines { endLine = totalLines }
    
    // Convert line numbers to byte offsets using the line index if available
    if cb.LineIndex != nil && startLine < len(cb.LineIndex.offsets) {
        start = int(cb.LineIndex.offsets[startLine])
    } else {
        start = startLine * 50 // rough estimate
    }
    if cb.LineIndex != nil && endLine < len(cb.LineIndex.offsets) {
        end = int(cb.LineIndex.offsets[endLine])
    } else {
        end = endLine * 50
    }
    
    // Clamp to file bounds
    if start < 0 { start = 0 }
    if end > int(cb.fileLen) { end = int(cb.fileLen) }
    if end <= start { end = start + 100 } // at least 100 bytes
    
    return start, end
}

Step 4: Line Index Build Task

File: internal/io/pool/task.go — add BuildLineIndexTask:

type BuildLineIndexTask struct {
    taskID string
    Path   string
    FS     FileSystem
}

func NewBuildLineIndexTask(path string, fs FileSystem) *BuildLineIndexTask { ... }

func (t *BuildLineIndexTask) Execute() Result {
    content, err := t.FS.ReadFile(t.Path)
    if err != nil {
        return Result{TaskID: t.taskID, TaskType: TypeBuildLineIndex, Success: false, Error: err}
    }
    
    offsets := []int32{0}
    for i := 0; i < len(content); i++ {
        if content[i] == '\n' {
            offsets = append(offsets, int32(i+1))
        }
    }
    
    return Result{
        TaskID: t.taskID, TaskType: TypeBuildLineIndex, Success: true,
        Data: &LineIndex{offsets: offsets, mtime: 0, size: int64(len(content))},
    }
}

func (t *BuildLineIndexTask) Priority() Priority { return LowPriority }
func (t *BuildLineIndexTask) TaskType() TaskType { return TypeBuildLineIndex }

File: internal/io/pool/task.go — add TypeBuildLineIndex constant.

File: internal/editor/logic.go — handle TypeBuildLineIndex result:

case res.TaskType == pool.TypeBuildLineIndex:
    if res.Success {
        if idx, ok := res.Data.(*LineIndex); ok {
            TheState.Editor.LineIndex = idx
        }
    }

Dispatch the line index build in OpenFile:

// After loading the initial chunk:
TheLogic.workerPool.Dispatch(pool.NewBuildLineIndexTask(filename, TheLogic.mockFS))

Step 5: EditorLayout — Compute Visible Content

File: internal/editor/state.go — update EditorLayout:

func EditorLayout(screenWidth, screenHeight ui.Dp, wordWrap bool) []ui.Element {
    // ... existing status bar and bottom bar code ...
    
    // --- Editor text area ---
    editorRegion := ui.Region{
        X: margin, Y: editorY,
        W: screenWidth - margin*2,
        H: editorH,
    }
    
    // Compute viewport height in Dp
    viewportHeight := editorRegion.H
    
    // Compute visible byte range
    var visibleContent string
    var visibleCursorPos int
    var visibleScrollOffset ui.Dp
    
    cb := TheState.Editor.ChunkedBuffer
    if cb != nil {
        start, end := cb.VisibleByteRange(TheState.ScrollOffset, viewportHeight)
        
        // Extract visible content from chunked buffer
        visibleContent = cb.Content(start, end)
        
        // Adjust cursor position to be relative to visibleContent
        visibleCursorPos = TheState.Editor.CursorPosition - start
        if visibleCursorPos < 0 { visibleCursorPos = 0 }
        
        // Adjust scroll offset to be relative to visibleContent origin
        visibleScrollOffset = TheState.ScrollOffset
        
        // Prefetch adjacent chunks for smooth scrolling
        cursorChunk := visibleCursorPos / cb.ChunkSize()
        cb.Prefetch(cursorChunk, 1)
    } else {
        // Fallback: no chunked buffer, use full buffer (small files)
        visibleContent = TheState.Editor.Buffer
        visibleCursorPos = TheState.Editor.CursorPosition
        visibleScrollOffset = TheState.ScrollOffset
    }
    
    // ... existing bottom bar code ...
    
    editorElem := ui.NewTextField(
        "editor_text",
        visibleContent,                    // ← visible content only
        editorRegion,
        editorRegion.W,
        visibleScrollOffset,               // ← adjusted scroll
        visibleCursorPos,                  // ← adjusted cursor
        interactions,
    )
    
    return []ui.Element{statusBar, editorElem, bottomBar}
}

Step 6: drawWrappedText — Render Only Visible Content

File: internal/ui/render.go — update drawWrappedText:

The function already receives str as the text to render. With virtual scrolling, str is now only the visible content (not the full buffer). The function body remains largely the same, but the impact is dramatic:

func (r *Renderer) drawWrappedText(gtx layout.Context, str string, reg Region, wrapWidth Dp, scrollOffset Dp, cursorPos int) {
    if str == "" {
        return
    }

    // str is now the VISIBLE portion of the file (typically 50–500 lines)
    // NOT the entire buffer. The shaper iterates only visible glyphs.

    params := text.Parameters{
        PxPerEm:         fixed.I(gtx.Sp(r.theme.FontSize)),
        MinWidth:        0,
        MaxWidth:        int(r.toPx(wrapWidth)),
        MaxLines:        0,
        LineHeight:      fixed.I(gtx.Sp(lineHeightSp)),
        LineHeightScale: 1.0,
        WrapPolicy:      text.WrapHeuristically,
    }
    
    // ONE LayoutString call — but now on visible content only (not full file)
    r.shp.LayoutString(params, str)
    
    // ... rest of the function unchanged ...
    // GlyphLayout now has ~50–500 entries instead of millions
}

Key change: The function signature is unchanged. The difference is that str is now the visible content extracted by the logic layer. No changes to the shaping or drawing loop are needed — the optimization comes from passing a small string instead of a large one.

Step 7: TextField — Pass Visible Content

File: internal/ui/element.go — TextField.Draw is unchanged:

func (tf TextField) Draw(gtx layout.Context, r *Renderer) {
    r.drawWrappedText(gtx, tf.Value, tf.region, tf.WrapWidth, tf.ScrollOffset, tf.CursorPosition)
}

tf.Value is now the visible content. No changes needed.

Step 8: Edit Operations — Update ChunkedBuffer

File: internal/editor/state.go — update HandleInsert, HandleBackspace, HandleDelete:

func HandleInsert(s string) {
    pos := TheState.Editor.CursorPosition
    buf := TheState.Editor.ChunkedBuffer
    
    // Insert into the chunked buffer
    buf.Insert(pos, s)
    TheState.Editor.CursorPosition += len(s)
    markDirty()
}

func HandleBackspace() {
    pos := TheState.Editor.CursorPosition
    if pos == 0 {
        return
    }
    buf := TheState.Editor.ChunkedBuffer
    buf.Delete(pos-1, 1)
    TheState.Editor.CursorPosition--
    markDirty()
}

func HandleDelete() {
    pos := TheState.Editor.CursorPosition
    buf := TheState.Editor.ChunkedBuffer
    buf.Delete(pos, 1)
    markDirty()
}

File: internal/editor/chunked_buffer.go — implement Insert and Delete:

func (cb *ChunkedBuffer) Insert(pos int, text string) {
    // 1. Ensure the chunk containing pos is loaded
    chunkIdx := pos / cb.chunkSize
    if _, ok := cb.chunks[chunkIdx]; !ok {
        cb.loadChunk(chunkIdx)
    }
    
    // 2. Insert into the chunk
    chunk := cb.chunks[chunkIdx]
    offsetInChunk := pos - chunkIdx*cb.chunkSize
    cb.chunks[chunkIdx] = append(chunk[:offsetInChunk], append([]byte(text), chunk[offsetInChunk:]...)...)
    
    // 3. Mark buffer dirty
    cb.dirty = true
    
    // 4. If insertion spans chunk boundary, merge chunks
    cb.maybeMergeChunk(chunkIdx)
}

func (cb *ChunkedBuffer) Delete(pos, n int) {
    // 1. Ensure relevant chunks are loaded
    startChunk := pos / cb.chunkSize
    endChunk := (pos + n - 1) / cb.chunkSize
    for i := startChunk; i <= endChunk; i++ {
        if _, ok := cb.chunks[i]; !ok {
            cb.loadChunk(i)
        }
    }
    
    // 2. Delete from chunks
    for i := startChunk; i <= endChunk; i++ {
        chunk := cb.chunks[i]
        offsetInChunk := pos - i*cb.chunkSize
        deleteEnd := min(offsetInChunk+n, len(chunk))
        if offsetInChunk < len(chunk) {
            cb.chunks[i] = append(chunk[:offsetInChunk], chunk[deleteEnd:]...)
        }
    }
    
    cb.dirty = true
    cb.maybeMergeChunk(startChunk)
}

Step 9: Auto-Save — Write Chunked Buffer

File: internal/editor/logic.go — update auto-save to reconstruct full content:

func (l *Logic) markDirty() {
    // ... existing code ...
    
    l.saveTimer = time.AfterFunc(1*time.Second, func() {
        if l.saveGeneration == gen+1 {
            // Reconstruct full content from chunks for saving
            cb := l.state.Editor.ChunkedBuffer
            content := cb.FullContent()  // reads all chunks + re-reads unloaded from disk
            l.workerPool.DispatchNonBlocking(
                pool.NewWriteFileTask(filename, []byte(content), l.mockFS),
            )
        }
    })
}

File: internal/editor/chunked_buffer.go — implement FullContent():

func (cb *ChunkedBuffer) FullContent() string {
    numChunks := int((cb.fileLen + int64(cb.chunkSize) - 1) / int64(cb.chunkSize))
    var buf bytes.Buffer
    for i := 0; i < numChunks; i++ {
        if chunk, ok := cb.chunks[i]; ok {
            buf.Write(chunk)
        } else {
            // Re-read from disk
            chunk := cb.loadChunk(i)
            buf.Write(chunk)
        }
    }
    return buf.String()
}

Step 10: Buffer Field Deprecation

File: internal/editor/state.go — keep Buffer field for backward compatibility but deprecate:

type EditorState struct {
    Buffer           string          // DEPRECATED: use ChunkedBuffer for large files
    ChunkedBuffer    *ChunkedBuffer  // NEW: primary buffer for editing
    // ...
}

Provide a GetBuffer() method that reconstructs the full string on demand:

func (e *EditorState) GetBuffer() string {
    if e.ChunkedBuffer != nil {
        return e.ChunkedBuffer.FullContent()
    }
    return e.Buffer
}

6. File Change Summary

File	Change
`internal/editor/chunked_buffer.go`	NEW — ChunkedBuffer, LineIndex, VisibleByteRange, Insert, Delete
`internal/editor/state.go`	Add `ChunkedBuffer`, `LineIndex` fields; update `OpenFile`, `HandleInsert`, `HandleBackspace`, `HandleDelete`, `EditorLayout`
`internal/editor/logic.go`	Add `TypeReadChunk`, `TypeStatFile`, `TypeBuildLineIndex` handling; update `markDirty` for chunked save
`internal/io/pool/task.go`	Add `ReadChunkTask`, `StatFileTask`, `BuildLineIndexTask`; add `TypeReadChunk`, `TypeStatFile`, `TypeBuildLineIndex` constants; add `FileStat`, `LineIndex` types
`internal/ui/render.go`	`drawWrappedText` — no signature change; optimization comes from smaller `str` input
`internal/ui/element.go`	`TextField.Draw` — no change; passes visible content

7. Performance Impact

Metric	Before	After	Spec Target
Open 100MB file	~1000ms (full read + string)	~50ms (stat + 1 chunk)	< 100ms
Scroll frame time (1M line file)	~500ms+ (shapes all glyphs)	~2ms (shapes ~50 lines)	60 fps
Memory (100MB file)	~200MB+ (string + GlyphLayout)	~6.4MB (100 chunks + viewport GlyphLayout)	16–64 MB
Per-keystroke latency	~500ms+ (full reshape)	~5ms (visible reshape)	< 16ms
GlyphLayout size (1M line file)	~10M entries	~50–500 entries	bounded

8. Edge Cases and Fallbacks

Scenario	Handling
File < 1 MB	Use full buffer (no chunking needed); virtual scrolling still applies but visible content ≈ full content
Line index build in progress	Use `visibleByteRangeEstimate` (heuristic) until index is ready
File smaller than chunk size	Single chunk; no chunk management overhead
Chunk boundary in middle of a line	Chunk content is raw bytes; line wrapping handles partial lines naturally
Edit at chunk boundary	`Insert`/`Delete` handles multi-chunk operations; `maybeMergeChunk` consolidates if needed
File replaced externally	`StatFileTask` detects size/mtime change; reload chunks and rebuild index
Very long line (> viewport width)	Shaper handles it; virtual scrolling still works because we shape only visible content
Cursor off-screen	`SetCursorFromPoint` uses GlyphLayout from visible content; cursor is always on-screen

9. Testing Strategy

Unit Tests

File: internal/editor/chunked_buffer_test.go (new)

Test	What it verifies
`TestChunkedBuffer_LoadChunk`	Loading a chunk reads the correct bytes from disk
`TestChunkedBuffer_Content`	`Content(start, end)` returns the correct byte range
`TestChunkedBuffer_Insert`	Inserting text updates the correct chunk
`TestChunkedBuffer_Delete`	Deleting bytes removes them from the correct chunk
`TestChunkedBuffer_EvictFarChunks`	Chunks far from cursor are evicted
`TestChunkedBuffer_Prefetch`	Prefetch loads adjacent chunks
`TestChunkedBuffer_VisibleByteRange`	Visible byte range is correct for various scroll positions
`TestChunkedBuffer_FullContent`	`FullContent()` reconstructs the exact original file
`TestChunkedBuffer_EmptyFile`	Handles zero-length files
`TestChunkedBuffer_FileSmallerThanChunk`	Single-chunk files work correctly
`TestLineIndex_Build`	Line index correctly maps line numbers to byte offsets
`TestLineIndex_BinarySearch`	Binary search finds correct byte offset for any line
`TestLineIndex_Invalidation`	Index is invalidated when file size changes
`TestDeterministicFile_GeneratesKnownContent`	Same seed produces identical content every time
`TestDeterministicFile_LineBased`	Line-based generator produces correct line count and byte offsets

Integration Tests

File: internal/editor/virtual_scroll_test.go (new)

Test	What it verifies
`TestEditorLayout_VisibleContent`	EditorLayout passes only visible content to TextField
`TestEditorLayout_ScrollUpdates`	Scrolling changes the visible byte range
`TestOpenFile_ChunkedLoad`	Opening a file loads the initial chunk and dispatches index build
`TestAutoSave_ChunkedWrite`	Auto-save reconstructs full content and writes it
`TestEdit_ChunkedBuffer`	Insert/delete on chunked buffer maintains content correctness
`TestVirtualScroll_10KLines`	Opening a 10K-line file loads only visible chunks; frame time < 16ms
`TestVirtualScroll_100KLines`	Scrolling through 100K lines keeps memory bounded; no full-file shapes
`TestLineJump_1MLines`	Jumping to line 500K in a 1M-line file completes < 10ms via line index
`TestSetCursorFromPoint_100KLines`	Click-to-position cursor works correctly on a 100K-line file

Render Tests

File: internal/ui/render_test.go — add tests for drawWrappedText with large strings to verify that GlyphLayout size is bounded.

E2E Tests

File: internal/test/e2e/virtual_scroll_test.go (new)

Uses the existing Harness to drive full editor workflows with deterministic large files. Tests verify the complete pipeline from file open through rendering to user interaction:

Open and scroll: Open a 100K-line file, scroll to line 50K, verify the frame contains the correct visible lines (checked by inspecting the TextField.Value length)
Click-to-position: Simulate a tap at a known screen coordinate, verify the cursor lands at the expected byte offset using the GlyphLayout feedback
Edit and verify: Insert text at a specific position in a large file, verify the buffer content is correct by reading back the affected chunk
Line index build: Verify the line index is built asynchronously and visible byte range becomes precise after index is ready

Deterministic Large File Generator

File: internal/editor/deterministic_file.go (new)

Generates arbitrarily large files in-memory using a fixed seed and repeating block content. The mock filesystem stores the generated content as a []byte without writing to disk. Two modes:

Fixed seed mode: NewDeterministicFile(seed uint64, blockSize int, repeatCount int) — produces blockSize * repeatCount bytes. Same seed always produces identical content.
Line-based mode: NewLineBasedFile(seed uint64, linesPerBlock int, blockCount int) — each block is a fixed set of lines repeated. Useful for testing line-index and line-jump correctness.

The generator is used by tests to create 10K, 100K, and 1M line files in the mock filesystem in milliseconds, with zero disk I/O. The content is a known repeating pattern so byte offsets, line numbers, and visible ranges can be verified exactly.

Click-to-Position Verification

File: internal/editor/cursor_test.go — extend existing cursor tests with large-file scenarios using the deterministic generator.

TestSetCursorFromPoint_LargeFile: Generate a 50K-line file, simulate a tap at a known Dp coordinate, verify the cursor lands on the expected byte offset. Uses the GlyphLayout captured during drawWrappedText.
TestSetCursorFromPoint_ChunkBoundary: Generate a file where the tap target falls near a chunk boundary, verify cursor positioning is correct even when the visible content spans multiple chunks.
TestSetCursorFromPoint_OffScreen: Verify that tapping outside the visible viewport scrolls to the target and positions the cursor correctly.
TestSetCursorFromPoint_WrappedLines: Generate a file with long lines that wrap, verify tap-to-position correctly identifies the visual line and closest glyph.

Performance Benchmarks

File: internal/editor/bench_test.go (new)

Go benchmarks that measure the actual performance of virtual scrolling with the deterministic generator:

BenchmarkVisibleByteRange_10KLines — measures time to compute visible byte range
BenchmarkVisibleByteRange_1MLines — measures time to compute visible byte range on 1M-line file
BenchmarkChunkedBufferContent — measures time to extract visible content from chunked buffer
BenchmarkDrawWrappedText_10KLines — measures shaping/rendering time for 10K-line file (visible only)
BenchmarkDrawWrappedText_1MLines — measures shaping/rendering time for 1M-line file (visible only)
BenchmarkLineIndexBinarySearch — measures O(log n) line jump time

All benchmarks use the deterministic generator to create files of known size. No disk I/O. Results are compared against the pre-virtual-scroll baseline (full-buffer shaping).

33 KiB Raw Blame History Unescape Escape