gmp/Pad
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Draft virtual scroll render optimization plan.

Browse Source
This commit is contained in:
Greg Pomerantz 2026-06-04 20:23:54 -04:00
parent 93a5f879f5
commit c941a02f1a
2 changed files with 818 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.gitignore vendored
View File

@ -8,7 +8,7 @@
# build files # build files
*.apk *.apk
*.idsig *.idsig
cmd/pad/pad pad
# IDE # IDE
.idea/ .idea/

817
doc/virtual_scroll_render_optimization.md Normal file
View File

@ -0,0 +1,817 @@
# Virtual Scrolling & Render Optimization — Implementation Plan
## 1. Problem Statement
The spec (§11, "Performance Guarantees") mandates:
| Operation | Target | Mechanism |
|---|---|---|
| Open any file | < 100 ms | Chunked load: only the chunk around the cursor is read |
| Scroll | 60 fps | Virtual rendering: only visible lines + prefetch buffer |
| Type | < 16 ms per keystroke | In-memory chunk edit, async chunk flush |
**Current reality:** `drawWrappedText` in `internal/ui/render.go` receives the entire `EditorState.Buffer` as a single string and calls `shp.LayoutString(params, str)` on it — every frame. For a 100,000-line file, this means:
1. The shaper iterates ~1–10 million glyphs per frame
2. A `GlyphLayout` struct grows to millions of entries (ByteOffsets, X, Y, Advance slices)
3. Every line is drawn, including those far off-screen
4. Frame times are measured in hundreds of milliseconds — far exceeding the 16ms budget
This is the primary cause of sluggishness when editing large files. The fix is **virtual scrolling**: only shape, render, and capture glyph layout for the bytes visible in the current viewport.
---
## 2. Design Principles
1. **The buffer is the source of truth.** The `EditorState.Buffer` string holds the complete file content. Virtual scrolling is a presentation-layer concern — the buffer itself is untouched.
2. **Logic computes the visible range; renderer renders only that range.** The logic layer determines which byte range is visible given scroll offset and viewport height. The renderer shapes and draws only those bytes.
3. **GlyphLayout is bounded to the viewport.** Per-glyph layout data is only captured for visible content, keeping memory usage constant regardless of file size.
4. **Chunking is a prerequisite.** Virtual scrolling requires that the buffer be backed by chunks (fixed-size file slices) so that the logic layer can load/unload chunks on demand. This plan assumes the chunked buffer exists; the chunked buffer implementation is described in a companion plan.
---
## 3. Architecture
```
┌──────────────────────────────────────────────────────────────────┐
│ EditorState (internal/editor/state.go) │
│ │
│ Buffer string ← full file content (from chunks)│
│ ScrollOffset ui.Dp ← current scroll position │
│ ChunkedBuffer *ChunkedBuffer ← chunked file access │
│ LineIndex *LineIndex ← byte offsets per line │
│ │
│ VisibleByteRange(scroll, viewport) → (start, end) │
│ VisibleContent(start, end) → []byte │
└──────────────────────┬───────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────────┐
│ EditorLayout (internal/editor/state.go) │
│ │
│ 1. Compute viewport height from editor region + scale │
│ 2. Call VisibleByteRange(scrollOffset, viewportHeight) │
│ 3. Extract visibleContent := ChunkedBuffer.VisibleContent() │
│ 4. Adjust cursorPos to be relative to visibleContent │
│ 5. Adjust scrollOffset to be relative to visibleContent origin │
│ 6. Pass visibleContent + adjusted offsets to NewTextField │
└──────────────────────┬───────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────────┐
│ TextField.Draw → Renderer.drawWrappedText │
│ │
│ 1. Shape ONLY visibleContent (not the full buffer) │
│ 2. Capture GlyphLayout only for visible glyphs │
│ 3. Draw only visible lines │
│ 4. Render cursor at adjusted position │
│ 5. Return bounded GlyphLayout (≈ viewport-sized) │
└──────────────────────┬───────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────────┐
│ GlyphLayout (internal/ui/unit.go) │
│ │
│ ByteOffsets []int ← byte offsets within visibleContent │
│ X []Dp ← screen X positions │
│ Y []Dp ← screen Y positions │
│ Advance []Dp ← glyph widths │
│ │
│ Length: proportional to visible lines (≈ 50–500 entries) │
│ NOT proportional to file size │
└──────────────────────────────────────────────────────────────────┘
```
---
## 4. Data Structures
### 4.1 ChunkedBuffer
```go
// ChunkedBuffer provides chunked access to a file's content.
// Only chunks near the cursor or viewport are kept in memory.
type ChunkedBuffer struct {
filename string
chunkSize int // e.g., 64 * 1024 (64 KB)
fileLen int64 // total file length (known from stat)
chunks map[int][]byte // chunkIndex → []byte
dirty bool // true if buffer has been modified
FS pool.FileSystem // filesystem for reading chunks
basePath string // base path for file resolution
}
```
**Key properties:**
- `chunkSize` is fixed at 64 KB (configurable constant).
- `chunks` is a sparse map: only chunks near the cursor/viewport are loaded.
- `fileLen` is known from the file stat task (dispatched when opening a file).
- The full buffer is reconstructed on demand via `Content(start, end)` for rendering.
### 4.2 LineIndex
```go
// LineIndex maps line numbers to byte offsets within the file.
// Built as a low-priority background task; cached on disk.
type LineIndex struct {
offsets []int32 // byte offset of each line start (line 0 = 0)
mtime int64 // file mtime at index build time
size int64 // file size at index build time
}
```
**Key properties:**
- Built by streaming the file once (O(n) time, O(lines) space for the offset array).
- Stored as `[]int32` — 4 bytes per line. A 1M-line file = 4 MB.
- Used for O(log n) line number → byte offset lookup (binary search).
- Used by `VisibleByteRange` to compute the exact byte range for visible lines.
- Invalidated when file mtime or size changes.
- Cached on disk at `.pad/indices/<sha256(path)>.bin`.
### 4.3 VisibleByteRange
```go
// VisibleByteRange returns the byte range [start, end) of content
// visible in the viewport, given the current scroll offset and viewport height.
func (cb *ChunkedBuffer) VisibleByteRange(scrollOffset ui.Dp, viewportHeight ui.Dp) (start, end int) {
if cb.LineIndex == nil {
// Fallback: estimate using average line height (used during index build)
return cb.visibleByteRangeEstimate(scrollOffset, viewportHeight)
}
// Precise: use line index to find the exact byte range
return cb.visibleByteRangePrecise(scrollOffset, viewportHeight)
}
```
**Two strategies:**
| Strategy | When Used | Accuracy |
|---|---|---|
| `visibleByteRangeEstimate` | During initial load, before line index is built | Approximate (±1–2 lines) |
| `visibleByteRangePrecise` | After line index is available | Exact (byte-accurate) |
**Estimate strategy (no index):**
1. Compute average character width from a sample of lines.
2. Compute average line height from the shaper (one-shot measurement).
3. Estimate visible line range from scrollOffset / lineHeight.
4. Use binary search on the line index (if available) or scan from file start to find byte offsets.
**Precise strategy (with index):**
1. Compute visible line range: `startLine = scrollOffset / lineHeight`, `endLine = (scrollOffset + viewportHeight) / lineHeight`.
2. Binary search in `LineIndex.offsets` for `startLine` and `endLine`.
3. Clamp to `[0, fileLen)`.
4. Return `(int(offsets[startLine]), int(offsets[min(endLine, len(offsets))-1]))`.
### 4.4 Chunked Content Extraction
```go
// Content returns the bytes in [start, end) from the chunked buffer.
// It loads missing chunks on demand.
func (cb *ChunkedBuffer) Content(start, end int) string {
// 1. Determine which chunks are needed
startChunk := start / cb.chunkSize
endChunk := (end - 1) / cb.chunkSize
// 2. Load missing chunks
for i := startChunk; i <= endChunk; i++ {
if _, ok := cb.chunks[i]; !ok {
cb.loadChunk(i) // reads from disk via FS.ReadFile (full file)
}
}
// 3. Concatenate needed chunk slices
var buf bytes.Buffer
for i := startChunk; i <= endChunk; i++ {
chunk := cb.chunks[i]
chunkStart := i * cb.chunkSize
chunkEnd := chunkStart + len(chunk)
segStart := max(start, chunkStart) - chunkStart
segEnd := min(end, chunkEnd) - chunkStart
buf.Write(chunk[segStart:segEnd])
}
return buf.String()
}
// Prefetch loads adjacent chunks for smooth scrolling.
func (cb *ChunkedBuffer) Prefetch(centerChunk int, radius int) {
for i := centerChunk - radius; i <= centerChunk + radius; i++ {
if i >= 0 && i < cb.numChunks() {
if _, ok := cb.chunks[i]; !ok {
cb.loadChunk(i)
}
}
}
}
```
---
## 5. Implementation Steps
### Step 1: ChunkedBuffer Implementation
**File:** `internal/editor/chunked_buffer.go` (new)
Implement the `ChunkedBuffer` struct with:
- `NewChunkedBuffer(filename, chunkSize, fs, basePath)` constructor
- `loadChunk(idx int)` — reads a single chunk from disk via `ReadFile`
- `Content(start, end int) string` — extracts bytes from loaded chunks
- `Prefetch(centerChunk, radius int)` — loads adjacent chunks
- `EvictFarChunks(cursorPos int, radius int)` — removes chunks far from cursor
- `FileLen() int64` — total file length
- `Filename() string` — the file path
**File:** `internal/io/pool/task.go` — add `ReadChunkTask`
```go
type ReadChunkTask struct {
taskID string
Path string
ChunkIdx int
FS FileSystem
}
func NewReadChunkTask(path string, chunkIdx int, fs FileSystem) *ReadChunkTask { ... }
func (t *ReadChunkTask) Execute() Result {
// Read the full file, then slice the requested chunk
content, err := t.FS.ReadFile(t.Path)
if err != nil {
return Result{TaskID: t.taskID, TaskType: TypeReadChunk, Success: false, Error: err}
}
start := t.ChunkIdx * 64 * 1024
end := min(start+64*1024, len(content))
if start >= len(content) {
return Result{TaskID: t.taskID, TaskType: TypeReadChunk, Success: true, Data: []byte{}}
}
chunk := content[start:end]
return Result{TaskID: t.taskID, TaskType: TypeReadChunk, Success: true, Data: chunk}
}
func (t *ReadChunkTask) Priority() Priority { return HighPriority }
func (t *ReadChunkTask) TaskType() TaskType { return TypeReadChunk }
```
**File:** `internal/io/pool/task.go` — add `TypeReadChunk` constant.
**File:** `internal/io/pool/task.go` — add `StatFileTask` (replaces full-file `ReadFileTask` for open)
```go
type StatFileTask struct {
taskID string
Path string
FS FileSystem
}
func NewStatFileTask(path string, fs FileSystem) *StatFileTask { ... }
func (t *StatFileTask) Execute() Result {
// Read the full file content for now (will be optimized later with chunked stat)
content, err := t.FS.ReadFile(t.Path)
if err != nil {
return Result{TaskID: t.taskID, TaskType: TypeStatFile, Success: false, Error: err}
}
return Result{
TaskID: t.taskID, TaskType: TypeStatFile, Success: true,
Data: &FileStat{Path: t.Path, Size: int64(len(content))},
}
}
type FileStat struct {
Path string
Size int64
}
```
### Step 2: ChunkedBuffer on EditorState
**File:** `internal/editor/state.go`
```go
type EditorState struct {
// ... existing fields ...
ChunkedBuffer *ChunkedBuffer // NEW: chunked file access
LineIndex *LineIndex // NEW: line-to-byte-offset mapping
}
```
Update `OpenFile` in `state.go`:
```go
func OpenFile(data any) {
filename := data.(string)
TheState.Editor.Filename = filename
// Create chunked buffer
chunkSize := 64 * 1024 // 64 KB
cb := NewChunkedBuffer(filename, chunkSize, TheLogic.mockFS, "")
TheState.Editor.ChunkedBuffer = cb
// Dispatch stat task to get file size
TheLogic.workerPool.Dispatch(pool.NewStatFileTask(filename, TheLogic.mockFS))
// Switch to editor page
TheState.page = EditorPage
TheState.FocusedElementID = "editor_text"
TheState.ScrollOffset = 0
TheState.Editor.CursorPosition = 0
}
```
Update `handleWorkerResult` in `logic.go`:
```go
case res.TaskType == pool.TypeStatFile:
if res.Success {
if stat, ok := res.Data.(*pool.FileStat); ok {
TheState.Editor.ChunkedBuffer.SetFileSize(stat.Size)
// Load the chunk containing the cursor (position 0 at open)
TheState.Editor.ChunkedBuffer.LoadChunk(0)
// Prefetch adjacent chunks
TheState.Editor.ChunkedBuffer.Prefetch(0, 1)
}
}
```
### Step 3: VisibleByteRange and Content Extraction
**File:** `internal/editor/chunked_buffer.go` — implement `VisibleByteRange` and `Content`.
**File:** `internal/editor/chunked_buffer.go` — implement `visibleByteRangeEstimate`:
```go
// visibleByteRangeEstimate approximates the visible byte range using
// heuristic estimates. Used when the line index is not yet available.
func (cb *ChunkedBuffer) visibleByteRangeEstimate(scrollOffset ui.Dp, viewportHeight ui.Dp) (start, end int) {
totalLines := int(cb.fileLen) / 50 // rough: 50 bytes per line average
lineHeight := EditorLineHeight()
startLine := int(scrollOffset / lineHeight)
endLine := int((scrollOffset + viewportHeight) / lineHeight)
// Clamp
if startLine < 0 { startLine = 0 }
if endLine > totalLines { endLine = totalLines }
// Convert line numbers to byte offsets using the line index if available
if cb.LineIndex != nil && startLine < len(cb.LineIndex.offsets) {
start = int(cb.LineIndex.offsets[startLine])
} else {
start = startLine * 50 // rough estimate
}
if cb.LineIndex != nil && endLine < len(cb.LineIndex.offsets) {
end = int(cb.LineIndex.offsets[endLine])
} else {
end = endLine * 50
}
// Clamp to file bounds
if start < 0 { start = 0 }
if end > int(cb.fileLen) { end = int(cb.fileLen) }
if end <= start { end = start + 100 } // at least 100 bytes
return start, end
}
```
### Step 4: Line Index Build Task
**File:** `internal/io/pool/task.go` — add `BuildLineIndexTask`:
```go
type BuildLineIndexTask struct {
taskID string
Path string
FS FileSystem
}
func NewBuildLineIndexTask(path string, fs FileSystem) *BuildLineIndexTask { ... }
func (t *BuildLineIndexTask) Execute() Result {
content, err := t.FS.ReadFile(t.Path)
if err != nil {
return Result{TaskID: t.taskID, TaskType: TypeBuildLineIndex, Success: false, Error: err}
}
offsets := []int32{0}
for i := 0; i < len(content); i++ {
if content[i] == '\n' {
offsets = append(offsets, int32(i+1))
}
}
return Result{
TaskID: t.taskID, TaskType: TypeBuildLineIndex, Success: true,
Data: &LineIndex{offsets: offsets, mtime: 0, size: int64(len(content))},
}
}
func (t *BuildLineIndexTask) Priority() Priority { return LowPriority }
func (t *BuildLineIndexTask) TaskType() TaskType { return TypeBuildLineIndex }
```
**File:** `internal/io/pool/task.go` — add `TypeBuildLineIndex` constant.
**File:** `internal/editor/logic.go` — handle `TypeBuildLineIndex` result:
```go
case res.TaskType == pool.TypeBuildLineIndex:
if res.Success {
if idx, ok := res.Data.(*LineIndex); ok {
TheState.Editor.LineIndex = idx
}
}
```
**Dispatch the line index build in `OpenFile`:**
```go
// After loading the initial chunk:
TheLogic.workerPool.Dispatch(pool.NewBuildLineIndexTask(filename, TheLogic.mockFS))
```
### Step 5: EditorLayout — Compute Visible Content
**File:** `internal/editor/state.go` — update `EditorLayout`:
```go
func EditorLayout(screenWidth, screenHeight ui.Dp, wordWrap bool) []ui.Element {
// ... existing status bar and bottom bar code ...
// --- Editor text area ---
editorRegion := ui.Region{
X: margin, Y: editorY,
W: screenWidth - margin*2,
H: editorH,
}
// Compute viewport height in Dp
viewportHeight := editorRegion.H
// Compute visible byte range
var visibleContent string
var visibleCursorPos int
var visibleScrollOffset ui.Dp
cb := TheState.Editor.ChunkedBuffer
if cb != nil {
start, end := cb.VisibleByteRange(TheState.ScrollOffset, viewportHeight)
// Extract visible content from chunked buffer
visibleContent = cb.Content(start, end)
// Adjust cursor position to be relative to visibleContent
visibleCursorPos = TheState.Editor.CursorPosition - start
if visibleCursorPos < 0 { visibleCursorPos = 0 }
// Adjust scroll offset to be relative to visibleContent origin
visibleScrollOffset = TheState.ScrollOffset
// Prefetch adjacent chunks for smooth scrolling
cursorChunk := visibleCursorPos / cb.ChunkSize()
cb.Prefetch(cursorChunk, 1)
} else {
// Fallback: no chunked buffer, use full buffer (small files)
visibleContent = TheState.Editor.Buffer
visibleCursorPos = TheState.Editor.CursorPosition
visibleScrollOffset = TheState.ScrollOffset
}
// ... existing bottom bar code ...
editorElem := ui.NewTextField(
"editor_text",
visibleContent, // ← visible content only
editorRegion,
editorRegion.W,
visibleScrollOffset, // ← adjusted scroll
visibleCursorPos, // ← adjusted cursor
interactions,
)
return []ui.Element{statusBar, editorElem, bottomBar}
}
```
### Step 6: drawWrappedText — Render Only Visible Content
**File:** `internal/ui/render.go` — update `drawWrappedText`:
The function already receives `str` as the text to render. With virtual scrolling, `str` is now **only the visible content** (not the full buffer). The function body remains largely the same, but the impact is dramatic:
```go
func (r *Renderer) drawWrappedText(gtx layout.Context, str string, reg Region, wrapWidth Dp, scrollOffset Dp, cursorPos int) {
if str == "" {
return
}
// str is now the VISIBLE portion of the file (typically 50–500 lines)
// NOT the entire buffer. The shaper iterates only visible glyphs.
params := text.Parameters{
PxPerEm: fixed.I(gtx.Sp(r.theme.FontSize)),
MinWidth: 0,
MaxWidth: int(r.toPx(wrapWidth)),
MaxLines: 0,
LineHeight: fixed.I(gtx.Sp(lineHeightSp)),
LineHeightScale: 1.0,
WrapPolicy: text.WrapHeuristically,
}
// ONE LayoutString call — but now on visible content only (not full file)
r.shp.LayoutString(params, str)
// ... rest of the function unchanged ...
// GlyphLayout now has ~50–500 entries instead of millions
}
```
**Key change:** The function signature is unchanged. The difference is that `str` is now the visible content extracted by the logic layer. No changes to the shaping or drawing loop are needed — the optimization comes from passing a small string instead of a large one.
### Step 7: TextField — Pass Visible Content
**File:** `internal/ui/element.go` — `TextField.Draw` is unchanged:
```go
func (tf TextField) Draw(gtx layout.Context, r *Renderer) {
r.drawWrappedText(gtx, tf.Value, tf.region, tf.WrapWidth, tf.ScrollOffset, tf.CursorPosition)
}
```
`tf.Value` is now the visible content. No changes needed.
### Step 8: Edit Operations — Update ChunkedBuffer
**File:** `internal/editor/state.go` — update `HandleInsert`, `HandleBackspace`, `HandleDelete`:
```go
func HandleInsert(s string) {
pos := TheState.Editor.CursorPosition
buf := TheState.Editor.ChunkedBuffer
// Insert into the chunked buffer
buf.Insert(pos, s)
TheState.Editor.CursorPosition += len(s)
markDirty()
}
func HandleBackspace() {
pos := TheState.Editor.CursorPosition
if pos == 0 {
return
}
buf := TheState.Editor.ChunkedBuffer
buf.Delete(pos-1, 1)
TheState.Editor.CursorPosition--
markDirty()
}
func HandleDelete() {
pos := TheState.Editor.CursorPosition
buf := TheState.Editor.ChunkedBuffer
buf.Delete(pos, 1)
markDirty()
}
```
**File:** `internal/editor/chunked_buffer.go` — implement `Insert` and `Delete`:
```go
func (cb *ChunkedBuffer) Insert(pos int, text string) {
// 1. Ensure the chunk containing pos is loaded
chunkIdx := pos / cb.chunkSize
if _, ok := cb.chunks[chunkIdx]; !ok {
cb.loadChunk(chunkIdx)
}
// 2. Insert into the chunk
chunk := cb.chunks[chunkIdx]
offsetInChunk := pos - chunkIdx*cb.chunkSize
cb.chunks[chunkIdx] = append(chunk[:offsetInChunk], append([]byte(text), chunk[offsetInChunk:]...)...)
// 3. Mark buffer dirty
cb.dirty = true
// 4. If insertion spans chunk boundary, merge chunks
cb.maybeMergeChunk(chunkIdx)
}
func (cb *ChunkedBuffer) Delete(pos, n int) {
// 1. Ensure relevant chunks are loaded
startChunk := pos / cb.chunkSize
endChunk := (pos + n - 1) / cb.chunkSize
for i := startChunk; i <= endChunk; i++ {
if _, ok := cb.chunks[i]; !ok {
cb.loadChunk(i)
}
}
// 2. Delete from chunks
for i := startChunk; i <= endChunk; i++ {
chunk := cb.chunks[i]
offsetInChunk := pos - i*cb.chunkSize
deleteEnd := min(offsetInChunk+n, len(chunk))
if offsetInChunk < len(chunk) {
cb.chunks[i] = append(chunk[:offsetInChunk], chunk[deleteEnd:]...)
}
}
cb.dirty = true
cb.maybeMergeChunk(startChunk)
}
```
### Step 9: Auto-Save — Write Chunked Buffer
**File:** `internal/editor/logic.go` — update auto-save to reconstruct full content:
```go
func (l *Logic) markDirty() {
// ... existing code ...
l.saveTimer = time.AfterFunc(1*time.Second, func() {
if l.saveGeneration == gen+1 {
// Reconstruct full content from chunks for saving
cb := l.state.Editor.ChunkedBuffer
content := cb.FullContent() // reads all chunks + re-reads unloaded from disk
l.workerPool.DispatchNonBlocking(
pool.NewWriteFileTask(filename, []byte(content), l.mockFS),
)
}
})
}
```
**File:** `internal/editor/chunked_buffer.go` — implement `FullContent()`:
```go
func (cb *ChunkedBuffer) FullContent() string {
numChunks := int((cb.fileLen + int64(cb.chunkSize) - 1) / int64(cb.chunkSize))
var buf bytes.Buffer
for i := 0; i < numChunks; i++ {
if chunk, ok := cb.chunks[i]; ok {
buf.Write(chunk)
} else {
// Re-read from disk
chunk := cb.loadChunk(i)
buf.Write(chunk)
}
}
return buf.String()
}
```
### Step 10: Buffer Field Deprecation
**File:** `internal/editor/state.go` — keep `Buffer` field for backward compatibility but deprecate:
```go
type EditorState struct {
Buffer string // DEPRECATED: use ChunkedBuffer for large files
ChunkedBuffer *ChunkedBuffer // NEW: primary buffer for editing
// ...
}
```
Provide a `GetBuffer()` method that reconstructs the full string on demand:
```go
func (e *EditorState) GetBuffer() string {
if e.ChunkedBuffer != nil {
return e.ChunkedBuffer.FullContent()
}
return e.Buffer
}
```
---
## 6. File Change Summary
| File | Change |
|---|---|
| `internal/editor/chunked_buffer.go` | **NEW** — ChunkedBuffer, LineIndex, VisibleByteRange, Insert, Delete |
| `internal/editor/state.go` | Add `ChunkedBuffer`, `LineIndex` fields; update `OpenFile`, `HandleInsert`, `HandleBackspace`, `HandleDelete`, `EditorLayout` |
| `internal/editor/logic.go` | Add `TypeReadChunk`, `TypeStatFile`, `TypeBuildLineIndex` handling; update `markDirty` for chunked save |
| `internal/io/pool/task.go` | Add `ReadChunkTask`, `StatFileTask`, `BuildLineIndexTask`; add `TypeReadChunk`, `TypeStatFile`, `TypeBuildLineIndex` constants; add `FileStat`, `LineIndex` types |
| `internal/ui/render.go` | `drawWrappedText` — no signature change; optimization comes from smaller `str` input |
| `internal/ui/element.go` | `TextField.Draw` — no change; passes visible content |
---
## 7. Performance Impact
| Metric | Before | After | Spec Target |
|---|---|---|---|
| Open 100MB file | ~1000ms (full read + string) | ~50ms (stat + 1 chunk) | < 100ms |
| Scroll frame time (1M line file) | ~500ms+ (shapes all glyphs) | ~2ms (shapes ~50 lines) | 60 fps |
| Memory (100MB file) | ~200MB+ (string + GlyphLayout) | ~6.4MB (100 chunks + viewport GlyphLayout) | 16–64 MB |
| Per-keystroke latency | ~500ms+ (full reshape) | ~5ms (visible reshape) | < 16ms |
| GlyphLayout size (1M line file) | ~10M entries | ~50–500 entries | bounded |
---
## 8. Edge Cases and Fallbacks
| Scenario | Handling |
|---|---|
| File < 1 MB | Use full buffer (no chunking needed); virtual scrolling still applies but visible content ≈ full content |
| Line index build in progress | Use `visibleByteRangeEstimate` (heuristic) until index is ready |
| File smaller than chunk size | Single chunk; no chunk management overhead |
| Chunk boundary in middle of a line | Chunk content is raw bytes; line wrapping handles partial lines naturally |
| Edit at chunk boundary | `Insert`/`Delete` handles multi-chunk operations; `maybeMergeChunk` consolidates if needed |
| File replaced externally | `StatFileTask` detects size/mtime change; reload chunks and rebuild index |
| Very long line (> viewport width) | Shaper handles it; virtual scrolling still works because we shape only visible content |
| Cursor off-screen | `SetCursorFromPoint` uses GlyphLayout from visible content; cursor is always on-screen |
---
## 9. Testing Strategy
### Unit Tests
**File:** `internal/editor/chunked_buffer_test.go` (new)
| Test | What it verifies |
|---|---|
| `TestChunkedBuffer_LoadChunk` | Loading a chunk reads the correct bytes from disk |
| `TestChunkedBuffer_Content` | `Content(start, end)` returns the correct byte range |
| `TestChunkedBuffer_Insert` | Inserting text updates the correct chunk |
| `TestChunkedBuffer_Delete` | Deleting bytes removes them from the correct chunk |
| `TestChunkedBuffer_EvictFarChunks` | Chunks far from cursor are evicted |
| `TestChunkedBuffer_Prefetch` | Prefetch loads adjacent chunks |
| `TestChunkedBuffer_VisibleByteRange` | Visible byte range is correct for various scroll positions |
| `TestChunkedBuffer_FullContent` | `FullContent()` reconstructs the exact original file |
| `TestChunkedBuffer_EmptyFile` | Handles zero-length files |
| `TestChunkedBuffer_FileSmallerThanChunk` | Single-chunk files work correctly |
| `TestLineIndex_Build` | Line index correctly maps line numbers to byte offsets |
| `TestLineIndex_BinarySearch` | Binary search finds correct byte offset for any line |
| `TestLineIndex_Invalidation` | Index is invalidated when file size changes |
| `TestDeterministicFile_GeneratesKnownContent` | Same seed produces identical content every time |
| `TestDeterministicFile_LineBased` | Line-based generator produces correct line count and byte offsets |
### Integration Tests
**File:** `internal/editor/virtual_scroll_test.go` (new)
| Test | What it verifies |
|---|---|
| `TestEditorLayout_VisibleContent` | EditorLayout passes only visible content to TextField |
| `TestEditorLayout_ScrollUpdates` | Scrolling changes the visible byte range |
| `TestOpenFile_ChunkedLoad` | Opening a file loads the initial chunk and dispatches index build |
| `TestAutoSave_ChunkedWrite` | Auto-save reconstructs full content and writes it |
| `TestEdit_ChunkedBuffer` | Insert/delete on chunked buffer maintains content correctness |
| `TestVirtualScroll_10KLines` | Opening a 10K-line file loads only visible chunks; frame time < 16ms |
| `TestVirtualScroll_100KLines` | Scrolling through 100K lines keeps memory bounded; no full-file shapes |
| `TestLineJump_1MLines` | Jumping to line 500K in a 1M-line file completes < 10ms via line index |
| `TestSetCursorFromPoint_100KLines` | Click-to-position cursor works correctly on a 100K-line file |
### Render Tests
**File:** `internal/ui/render_test.go` — add tests for `drawWrappedText` with large strings to verify that GlyphLayout size is bounded.
### E2E Tests
**File:** `internal/test/e2e/virtual_scroll_test.go` (new)
Uses the existing `Harness` to drive full editor workflows with deterministic large files. Tests verify the complete pipeline from file open through rendering to user interaction:
- **Open and scroll**: Open a 100K-line file, scroll to line 50K, verify the frame contains the correct visible lines (checked by inspecting the TextField.Value length)
- **Click-to-position**: Simulate a tap at a known screen coordinate, verify the cursor lands at the expected byte offset using the GlyphLayout feedback
- **Edit and verify**: Insert text at a specific position in a large file, verify the buffer content is correct by reading back the affected chunk
- **Line index build**: Verify the line index is built asynchronously and visible byte range becomes precise after index is ready
### Deterministic Large File Generator
**File:** `internal/editor/deterministic_file.go` (new)
Generates arbitrarily large files in-memory using a fixed seed and repeating block content. The mock filesystem stores the generated content as a `[]byte` without writing to disk. Two modes:
- **Fixed seed mode**: `NewDeterministicFile(seed uint64, blockSize int, repeatCount int)` — produces `blockSize * repeatCount` bytes. Same seed always produces identical content.
- **Line-based mode**: `NewLineBasedFile(seed uint64, linesPerBlock int, blockCount int)` — each block is a fixed set of lines repeated. Useful for testing line-index and line-jump correctness.
The generator is used by tests to create 10K, 100K, and 1M line files in the mock filesystem in milliseconds, with zero disk I/O. The content is a known repeating pattern so byte offsets, line numbers, and visible ranges can be verified exactly.
### Click-to-Position Verification
**File:** `internal/editor/cursor_test.go` — extend existing cursor tests with large-file scenarios using the deterministic generator.
- **`TestSetCursorFromPoint_LargeFile`**: Generate a 50K-line file, simulate a tap at a known Dp coordinate, verify the cursor lands on the expected byte offset. Uses the GlyphLayout captured during `drawWrappedText`.
- **`TestSetCursorFromPoint_ChunkBoundary`**: Generate a file where the tap target falls near a chunk boundary, verify cursor positioning is correct even when the visible content spans multiple chunks.
- **`TestSetCursorFromPoint_OffScreen`**: Verify that tapping outside the visible viewport scrolls to the target and positions the cursor correctly.
- **`TestSetCursorFromPoint_WrappedLines`**: Generate a file with long lines that wrap, verify tap-to-position correctly identifies the visual line and closest glyph.
### Performance Benchmarks
**File:** `internal/editor/bench_test.go` (new)
Go benchmarks that measure the actual performance of virtual scrolling with the deterministic generator:
- `BenchmarkVisibleByteRange_10KLines` — measures time to compute visible byte range
- `BenchmarkVisibleByteRange_1MLines` — measures time to compute visible byte range on 1M-line file
- `BenchmarkChunkedBufferContent` — measures time to extract visible content from chunked buffer
- `BenchmarkDrawWrappedText_10KLines` — measures shaping/rendering time for 10K-line file (visible only)
- `BenchmarkDrawWrappedText_1MLines` — measures shaping/rendering time for 1M-line file (visible only)
- `BenchmarkLineIndexBinarySearch` — measures O(log n) line jump time
All benchmarks use the deterministic generator to create files of known size. No disk I/O. Results are compared against the pre-virtual-scroll baseline (full-buffer shaping).
---