gmp/Pad
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

editor: fix whole-file shaper leak + LineIndex mismatch (Phase 3)

Browse Source 
Root cause of the ~1GB 'Unknown' memory on large files: with word wrap
on (the default), VisibleByteRange used the previously-shaped
GlyphLayout.VisualLineStarts to bound the visible byte range. That layout
only covers the visible window (~50 lines), not the whole document, so
whenever the viewport's line count exceeded the window's visual-line count
the range fell back to end=fileLen. The text shaper then laid out the ENTIRE
file each frame; its internal line/glyph buffers grow to the largest layout
ever shaped and are never released (document.reset() keeps the backing cap),
so memory ballooned to the file size (~1.1 GB for 10 MB) and OOM-killed the
process under scroll.

Fix: always derive the visible range from the real-line LineIndex (or the
heuristic estimate before it is ready). Word wrap needs no separate path:
each real line yields >= 1 visual line, so shaping viewportHeight/lineHeight
real lines always fills the viewport, and the range can never collapse to the
whole file.

Also fixed the pre-existing LineIndex storage mismatch: EditorLayout read
TheState.Editor.LineIndex (never set; SetLineIndex had zero callers) for
maxScroll and scrollByteOffset, so it always used the huge pre-index estimate.
It now reads cb.LineIndex, the single source of truth; the dead
EditorState.LineIndex field and SetLineIndex are removed.

On-device (emulator/SwiftShader), a 10 MB file now uses ~150 MB PSS / ~230 MB
RSS at steady state and stays flat under scroll (was 1.13 GB PSS / 1.5 GB RSS,
growing, OOM-killed). go test -race ./... green.
This commit is contained in:
Greg Pomerantz 2026-08-16 12:16:52 -04:00
parent 3460ef3993
commit c79c142397
2 changed files with 75 additions and 96 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

63
internal/editor/chunked_buffer.go
View File

@ -14,9 +14,16 @@ const (
// MaxEditableFileSize is the largest file the editor will open for editing.
// In-range files are loaded fully into memory (see Phase 3): the chunked
// buffer keeps the raw bytes (~file size) resident and the renderer
// virtualizes the glyph layout to the visible window. Files above this are
// rejected with a "too large to edit" state (the browser can still list
// them). Set from on-device measurement (see doc/development_plan.md §5).
// virtualizes the glyph layout to the visible window, so memory scales
// roughly linearly with file size and stays bounded (no leak). Files above
// this are rejected with a "too large to edit" state (the browser can still
// list them).
//
// On-device measurement (Android emulator, SwiftShader): a 10 MB file uses
// ~150 MB PSS / ~230 MB RSS at steady state and stays flat under scroll
// (previously the shaper was handed the whole file each frame, ballooning
// to ~1.1 GB and OOM-killing the process). 50 MB extrapolates to a few
// hundred MB, comfortable on a modern phone.
MaxEditableFileSize = 50 * 1024 * 1024 // 50 MB
)
@ -390,41 +397,21 @@ func (cb *ChunkedBuffer) Delete(pos, n int) {
// viewport, not the file. It uses actual (prefix-sum) chunk offsets so the
// range stays correct after length-changing edits.
func (cb *ChunkedBuffer) VisibleByteRange(scrollOffset ui.Dp, byteOffset int, viewportHeight ui.Dp, lineHeight ui.Dp, wordWrap bool, layout ui.GlyphLayout, visualIndex *types.VisualLineIndex) (start, end, startLine int) {
// If wrapping and we have visual line data, use it for precise calculation
if wordWrap && len(layout.VisualLineStarts) > 0 {
// Calculate which visual line should be at the top based on scroll offset
visualLine := int(scrollOffset / lineHeight)
if visualLine >= len(layout.VisualLineStarts) {
visualLine = len(layout.VisualLineStarts) - 1
}
start = layout.VisualLineStarts[visualLine] + byteOffset
// Calculate end: enough content to fill viewport + buffer
linesInViewport := int(viewportHeight/lineHeight) + 2
endLine := visualLine + linesInViewport
if endLine >= len(layout.VisualLineStarts) {
end = int(cb.fileLen)
} else {
end = layout.VisualLineStarts[endLine]
}
// Clamp to file bounds
if end > int(cb.fileLen) {
end = int(cb.fileLen)
}
if start >= end {
end = start + 1000 // minimum buffer
if end > int(cb.fileLen) {
end = int(cb.fileLen)
}
}
return start, end, visualLine
}
// Fallback to LineIndex (logical lines) or estimate if:
// 1. Not word wrapping
// 2. No visual index available
// The visible range is always derived from the real-line LineIndex (or a
// heuristic estimate before the index is ready). The previously-shaped
// GlyphLayout (layout.VisualLineStarts) only covers the visible window, NOT
// the whole document. Using it to bound the range made the end fall back to
// the entire file whenever the viewport's line count exceeded the window's
// visual-line count, which forced the text shaper to lay out the whole
// document and ballooned memory to the file size (the shaper's internal
// line/glyph buffers grow to the largest layout ever shaped and are never
// released). That is the root cause of the ~1GB "Unknown" memory on large
// files.
//
// Word wrap does not require a separate path: each real line produces at
// least one visual line, so shaping viewportHeight/lineHeight real lines
// always yields at least as many visual lines as fit in the viewport. The
// extra wrapped lines are simply clipped by the renderer.
if cb.LineIndex == nil {
start, end = cb.visibleByteRangeEstimate(scrollOffset, viewportHeight)
return start, end, 0

28
internal/editor/state.go
View File

@ -8,10 +8,9 @@ import (
"time"
"unicode/utf8"
"pad/internal/browser"
"pad/internal/io/pool/types"
"pad/internal/ui"
"gioui.org/io/key"
"pad/internal/browser"
"pad/internal/ui"
)
// EditorFontSize is the font size used for editor text.
@ -61,7 +60,6 @@ const (
type EditorState struct {
Buffer string // DEPRECATED: use ChunkedBuffer for large files
ChunkedBuffer *ChunkedBuffer // NEW: chunked file access for virtual scrolling
LineIndex *types.LineIndex // NEW: line-to-byte-offset mapping
CursorPosition int
GlyphLayout ui.GlyphLayout
SelectionStart int
@ -174,8 +172,6 @@ func NewState() *State {
}
}
func (s *State) SetScale(scale float32) {
s.scale = scale
}
@ -192,7 +188,6 @@ func (s *State) layout(bm *browser.BrowserManager) []ui.Element {
dpW := ui.ToDp(ui.Px(s.PixelWidth), s.scale)
dpH := ui.ToDp(ui.Px(s.PixelHeight), s.scale)
// Calculate VisibleCount before laying out the browser page.
// This ensures the browser shows entries based on the current viewport.
if s.PixelHeight > 0 && s.scale > 0 {
@ -309,11 +304,6 @@ func SetChunkedBuffer(cb *ChunkedBuffer) {
TheState.Editor.ChunkedBuffer = cb
}
// SetLineIndex sets the line index for the current editor state.
func SetLineIndex(li *types.LineIndex) {
TheState.Editor.LineIndex = li
}
// ToggleSortOrder cycles the browser sort mode through four modes.
func ToggleSortOrder(data any) {
// Cycle through the 4 sort modes
@ -842,7 +832,7 @@ func EditorLayout(screenWidth, screenHeight ui.Dp, wordWrap bool) []ui.Element {
// Add bottom padding (half line height) so last line isn't flush with the bottom bar.
var maxScroll ui.Dp
if cb := TheState.Editor.ChunkedBuffer; cb != nil {
if li := TheState.Editor.LineIndex; li != nil {
if li := cb.LineIndex; li != nil {
totalLines := li.LineCount()
maxScroll = ui.Dp(totalLines)*EditorLineHeight() - editorRegion.H + EditorLineHeight()/2
} else {
@ -915,12 +905,12 @@ func EditorLayout(screenWidth, screenHeight ui.Dp, wordWrap bool) []ui.Element {
}
// Adjust scroll offset to be relative to visibleContent origin
visibleScrollOffset = ui.Dp(math.Mod(float64(TheState.ScrollOffset),float64(lineHeight)))
visibleScrollOffset = ui.Dp(math.Mod(float64(TheState.ScrollOffset), float64(lineHeight)))
// Map scroll offset to a chunk index
// Estimate line-to-byte conversion if LineIndex is missing
var scrollByteOffset int
if li := TheState.Editor.LineIndex; li != nil {
if li := cb.LineIndex; li != nil {
// Precise
lineHeight := EditorLineHeight()
startLine := int(TheState.ScrollOffset / lineHeight)
@ -931,7 +921,7 @@ func EditorLayout(screenWidth, screenHeight ui.Dp, wordWrap bool) []ui.Element {
}
} else {
// Estimate
scrollByteOffset = int(TheState.ScrollOffset / EditorLineHeight()) * 50
scrollByteOffset = int(TheState.ScrollOffset/EditorLineHeight()) * 50
}
scrollChunk := scrollByteOffset / cb.ChunkSize()
@ -1033,8 +1023,10 @@ func SetCursorFromPoint(x, y float64) {
groups = []lineGroup{} // Reset groups
for i, yVal := range layout.Y {
yFloat := float64(yVal)
lineIdx := int((yFloat - minY) / lineHeight + 0.5) // round to nearest line
if lineIdx < 0 { lineIdx = 0 }
lineIdx := int((yFloat-minY)/lineHeight + 0.5) // round to nearest line
if lineIdx < 0 {
lineIdx = 0
}
// Ensure enough groups
for len(groups) <= lineIdx {