gallery/HANDOFF.md

Gallery — development hand-off

A high-performance image gallery for Android (primary), Linux and macOS, written in Rust with the Slint UI toolkit. This document is the starting point for development: it captures every decision, gotcha, and measurement from the framework evaluation that preceded it, so no learning has to be rediscovered.

Status: M0 done — this scaffold builds and runs on the emulator (./scripts/build.sh debug), including the permission-aware Android init and the lifecycle-event hook.

---

1. Goals

• Browse and view a local photo collection (thousands of images) that stays smooth: instant grid, no decode jank, cached thumbnails.
• Full-screen viewer with pinch-zoom/pan.
• Correct EXIF orientation, HEIC support (Android camera default).
• Same codebase on macOS + Linux (desktop cargo run) and Android (cargo apk2).
• Quality of the platform experience matters: safe-area insets, IME-free UI, native-feeling widgets, animations — first-class support preferred over manual patching.

Non-goals (for now): editing, cloud sync, video, widgets/live-updates.

2. Why Slint (decision summary)

Full evaluation in /mnt/Backup/prog/android/ui/REPORT.md (same tree: ~/prog/android/ui/REPORT.md), raw notes in NOTES.md there. Headline:

	Slint 1.18.1	eframe/egui 0.37.0
Virtualized list (the gallery's lifeline)	✅ ListView std-widget, only visible rows instantiated	❌ ScrollArea::show lays out all children; show_rows = manual virtualization API
Async image → UI	swap slint::Image into a property; only that tile redraws (retained mode)	ctx.load_texture + custom loader; visible set rebuilt every frame
Lifecycle hook for permissions	✅ init_with_event_listener (Start/Resume/Pause/Stop/SaveState/...)	❌ none in App trait (winit swallows it)
Android safe-area	✅ safe-area-insets built in	manual 32px-ish panel workaround (winit#3910)
Animation	✅ declarative built-in engine	hand-rolled
IME text entry	✅	✅
APK size (release aarch64)	8.57 MB	2.87 MB
Clean build time (x86_64, this machine)	195 s / 270 crates	76 s / 137 crates
Desktop Linux system deps	libfontconfig1 + pkg-config	none
License	royalty-free for proprietary mobile/desktop (embedded excluded) / GPLv3 / commercial	MIT OR Apache-2.0
Min Android API	26	24

For a stateful, form-and-grid app, the declarative model was the right fit (see the discussion about pad's immediate-mode frame friction); egui wins for games/dashboards and on APK size. Known Slint caveat: an open upstream "abort on pointer scroll" bug in the Android backend (todo!() in the pointer-event path) — verify it's fixed in 1.19 before relying on heavy scrolling; if it fires, the panic is obvious in logcat.

3. Environment, build & test loop

Everything below is verified working on this machine (2026-10-27).

export ANDROID_HOME=$HOME/android-sdk
export ANDROID_NDK_ROOT=$HOME/android-sdk/ndk/27.0.12077973
export PATH="$HOME/.cargo/bin:$ANDROID_HOME/platform-tools:$PATH"

# Rust 1.99.1 via rustup; tools: cargo-apk2 1.5.0, cargo-ndk 4.1.2
./scripts/build.sh debug      # debug x86_64 APK + adb install + launch
./scripts/build.sh            # release x86_64 (emulator)
./scripts/build.sh aarch64    # release aarch64 (device)
cargo run                     # desktop (Linux needs libfontconfig1+pkg-config)

• Emulator: emulator -avd pad_avd -no-window -no-audio -gpu off -no-snapshot (google_apis x86_64, Android 15). -gpu off -no-snapshot matters — swiftshader_indirect + snapshot crashed the emulator once during the eval. If it won't start, the host key lost its wireless pairing — see the pairing procedure in ~/.pi/agent/AGENTS.md (phone) / emulator docs (this AVD).
• APK lands at target/<debug|release>/apk/gallery.apk.
• Release signing uses the debug keystore (CARGO_APK_RELEASE_KEYSTORE=$HOME/.android/debug.keystore, CARGO_APK_RELEASE_KEYSTORE_PASSWORD=android) — fine for personal use; for Play, generate a real keystore and point the env vars at it.
• cargo apk2 quirk: --target (not -t). --lib builds the cdylib.
• Grant "All files access" to the app on the emulator after install: adb shell am start -a android.settings.MANAGE_ALL_FILES_ACCESS_PERMISSION (or Settings UI). adb install -r keeps the grant.
• Perf measurement: adb shell perfetto / adb shell dumpsys gfxinfo wow.gmp.gallery, systrace-style capture via Perfetto trace shell. Cold start: adb shell am start -W.

4. Architecture

The UI framework is only the display layer. The performance lives in the thumbnail engine, and it is framework-agnostic:

┌─ metadata ─────────────────────────────────────────────────────────┐
│ sqlite (rusqlite, bundled): path, mtime, size, w, h, orientation,  │
│ thumb_path, date_taken            — one background scan pass       │
└──────────────┬─────────────────────────────────────────────────────┘
               │ model → Slint (list of visible-range items)
┌─ thumbnail pool (tokio-free: N std::threads, N ≈ cores/2) ─────────┐
│ per job: token cancellation (AtomicU64 gen counter) + priority     │
│  1. disk cache hit?  .thumbs/{sha1(path|mtime|size)}.jpg  → done   │
│  2. decode AT TARGET SIZE (never full-size for the grid):          │
│       JPEG: zune-jpeg "turbo" (decode+downscale in ONE pass,       │
│             pure Rust, actively maintained) or turbojpeg (SIMD)    │
│       HEIC: imagepipe (0.5, 2026-08) — VERIFY Android NDK build    │
│       PNG:  image/png                                              │
│  3. apply EXIF orientation (once, here — never per frame)          │
│  4. re-encode small JPEG → disk cache                              │
│  5. channel → UI: (id, slint::Image)                               │
│ prefetch: decode visible ± 1 row; cancel items scrolled out        │
└────────────────────────────────────────────────────────────────────┘
┌─ UI (Slint) ───────────────────────────────────────────────────────┐
│ grid screen:   ListView (vertical) → row of N square Image cells   │
│ viewer screen: full-size async decode (once, cached) + manual      │
│                two-finger pinch transform (no built-in in 1.18)    │
└────────────────────────────────────────────────────────────────────┘

Planned module layout:

src/
  lib.rs        # android_main, ANDROID_APP stash, platform init   [exists]
  main.rs       # desktop entry                                     [exists]
  main.slint    # screens (split into ui/ components at M3)        [exists]
  db.rs         # sqlite schema + scan (M1)
  scan.rs       # directory walk, image detection, metadata read    (M1)
  thumb.rs      # pool, decode@size, disk cache, cancellation      (M2)
  images.rs     # decode dispatch (zune-jpeg / imagepipe / png)    (M2)
  ui.rs         # Rust↔Slint glue: Model provider, thumb→Image     (M3)
  viewer.rs     # full-size load + transform state                  (M4)
  perm.rs       # storage permission check/request (M5)

Crate choices (all verified current on crates.io, 2026-10):

crate	version	why
slint / slint-build	1.18.1	pinned; upgrade to 1.19 after the pointer-scroll fix lands
rusqlite	latest, bundled feature	SQLite without system deps; cross-compiles to Android
zune-jpeg	0.5.x (upd 2026-10)	pure-Rust JPEG, turbo mode = decode+downscale one pass. Preferred over turbojpeg for Android: no C dep to cross-build
image	0.25.x	PNG + header/metadata (ImageReader::into_dimensions, EXIF orientation)
imagepipe	0.5.x (upd 2026-08)	HEIC/HEIF via Apple ImagePipe — open question: Android NDK support (see §10)
slint::Image::from_rgba8_premultiplied	—	how decoded thumbs enter the UI

Decode sizes: grid tiles render at ~cell×2 (Retina) — e.g. 500 px side. Full-screen: full decode once (12 MP ≈ 30–80 ms off-thread, fine), cached in the in-memory LRU until memory pressure.

The five failure modes the current (unknown) app likely has — this design eliminates all five: (1) full-size decode for grid cells, (2) no disk thumbnail cache, (3) no list virtualization, (4) decode on UI thread without cancellation, (5) EXIF rotation applied per frame.

5. UI plan (Slint patterns)

• Grid: ListView is the only virtualized std-widget (confirmed in source docs: "Only visible rows are instantiated"). Grid = vertical ListView, each row a horizontal layout of square Image cells (columns ≈ floor(window-width / cell); recompute on resize). Cells bind image <=> model.image — when the pool delivers a thumbnail, only that cell invalidates (retained mode; no repaint calls to remember).
• Placeholder→image: cell shows a Rectangle (shimmer optional, M6) until its image is set.
• Image cache: Slint's built-in path-based LRU is 5 MB (CLruCache in internal/core/graphics/image/cache.rs) — too small and the wrong layer for us; our cache lives in Rust (disk + compressed-bytes LRU). No mipmaps in the Skia renderer — irrelevant since we pre-downsample.
• Viewer: no pinch helper in Slint 1.18 (TouchArea has no pinch callback; grepped the stdlib). Manual two-finger math on TouchArea (distance/centroid → scale/translation, ~50 lines; crib slint-src/examples/native-gestures — source checkout is at ~/prog/android/ui/slint-src, tag v1.18.1). Double-tap-to-zoom is a nice extra (M6).
• Screens: single Window with a stacked layout (grid + viewer overlay) or a Navigator-style state property; keep it simple until M4.
• Safe area: always pad with root.safe-area-insets.{top,bottom} (done in the scaffold).
• Animation: use built-in animate statements (syntax: animate <prop> { duration: ...; easing: ...; } — it is a statement, not an element). Slide/fade for the viewer is a one-liner.

6. Slint 1.18 API gotchas (all hit firsthand in the eval)

1. animate is a statement (animate width { duration: 300ms; }), not an Animation {} element.
2. Text input widget is LineEdit, not TextField.
3. Markdown interpolation: text: "Count: \{count}" (backslash-brace).
4. length is the unit type for lengths (property <length>, not logical<length> in the 1.18 Rust API).
5. No Rust macros/closures inside .slint callbacks — callbacks call global functions or invoke Rust via @global / imported callbacks.
6. 1.18 ↔ 1.19 docs drift — when reading docs online, check which version; the local source at ~/prog/android/ui/slint-src (tag v1.18.1) is ground truth for the pinned version. git grep beats the docs.
7. slint::android::init_with_event_listener(app, |event: &PollEvent| ...) re-exports the whole android-activity crate as slint::android::android_activity; AndroidApp is also re-exported at slint::android::AndroidApp.
8. slint::Image runtime construction: Image::from_rgba8_premultiplied (also from_array); assign to an image property to update a cell.
9. Desktop Linux: needs libfontconfig1 + pkg-config (pure Rust on macOS).
10. Retained-mode redraw: changing a property re-renders only affected elements — design bindings so thumbnail delivery touches one property.

7. Permissions & platform integration (Android)

Declared in Cargo.toml (verified schema in cargo-apk2 1.5 / ndk-build2: uses_permission is top-level under [package.metadata.android], and Permission supports only name + max_sdk_version): READ_MEDIA_IMAGES, READ_EXTERNAL_STORAGE (maxSdk 32), MANAGE_EXTERNAL_STORAGE.

Runtime flow (M5), using the ANDROID_APP clone stashed in src/lib.rs:

1. MANAGE_EXTERNAL_STORAGE (primary path, same as passgo): no runtime dialog exists — check Environment.isExternalStorageManager() via JNI, if false launch Settings.ACTION_MANAGE_ALL_FILES_ACCESS_PERMISSION via ANDROID_APP.get().run_on_java_main_thread(...) (attach JNIEnv from vm_as_ptr(); Activity from activity_as_ptr()), then re-check on MainEvent::Resume — the hook is already wired in src/lib.rs (TODO(M5) marker).
2. READ_MEDIA_IMAGES / READ_EXTERNAL_STORAGE: requestPermissions via the same JNI route; the bare NativeActivity does not forward onRequestPermissionsResult to native, so either (a) poll checkSelfPermission after requesting (good enough), or (b) ship a custom NativeActivity subclass via cargo-apk2 java_sources = "android/java" (manifest activity name points at it) — the proper callback, reusable pattern from the eval.
3. Facts that constrain design (verified in android-activity 0.6.1 source): no NewIntent event; poll_events may only run on the android_main thread (Slint owns it — don't add a second consumer); AndroidApp is Clone and a clone works from any thread (run_on_java_main_thread, vm_as_ptr, activity_as_ptr).
4. Source discovery for photos: start with a plain directory scan of /storage/emulated/0/Pictures + DCIM (works with all-files access, emulator-friendly, and it's what makes the app testable without MediaStore). MediaStore integration (API-aware, scoped to READ_MEDIA_IMAGES) is a later refinement if directory scanning proves insufficient.

8. Performance targets & measurement

Targets (emulator is NOT representative for absolute decode times — validate on a real device with the aarch64 build):

• Grid: 60 fps scrolling at 5k+ items; zero full-size decodes on the hot path.
• First frame: grid of placeholders < 100 ms after launch (metadata from sqlite, images async).
• Thumbnail decode (grid size): < 5 ms off-thread per JPEG (measure with std::time::Instant logs; zune-jpeg turbo should be well under).
• Steady-state memory: in-memory thumb cache bounded (~64–128 MB compressed bytes on a phone) + Slint's texture memory for visible tiles only.
• Cold start (emulator, release): compare against the eval baseline (Slint sample ≈ 1–2 s; gallery must not regress > 0.5 s).

Test data (M2): generate a synthetic set on the device — a small Rust bin or Python script that writes N=5,000 JPEGs of varied sizes/orientations into /storage/emulated/0/Pictures/gallery-test/ (include a few HEICs if imagepipe works; include rotated EXIF). Reuse it for every perf run.

Measure with: adb shell am start -W (cold start), dumpsys gfxinfo wow.gmp.gallery framestats, Perfetto trace (GPU + main + pool threads), and logcat pool timing logs behind a debug_timing flag.

9. Milestones

• M0 — scaffold (DONE): builds (x86_64 + aarch64 toolchain), installs, runs on emulator; permission-aware init + Resume hook; build script; manifest with permissions; this document.
• M1 — metadata: db.rs + scan.rs; scan Pictures+DCIM on first launch (background thread, progress in UI); sqlite table; list of items as a Slint Model fed to a placeholder ListView (paths as text, no images). Accept: 5k files scanned < 2 s; relaunch reuses DB; mtime-changed files refreshed.
• M2 — thumbnail engine: thumb.rs + images.rs; disk cache; cancellation; pool; synthetic 5k test set; timing logs. Accept: first grid fill < 2 s; re-launch fills instantly from disk cache; fast scroll causes no backlog (watch pending-queue length in logs).
• M3 — grid UI: ListView grid, cell sizing, placeholder shimmer, pull-to-refresh optional; Rust Model ↔ Slint; thumbnail delivery → cell image swap. Accept: 60 fps scroll on 5k items (emulator: no dropped- frame spikes > 16 ms in gfxinfo).
• M4 — viewer: tap cell → full decode + overlay; pinch/pan (manual math), double-tap zoom; back gesture/button; EXIF orientation correct.
• M5 — permissions: implement perm.rs per §7; launch flow when un-granted (settings card → open Settings → recheck on Resume); test the exact passgo-style flow on a fresh install.
• M6 — polish: dark theme, animations (slide/fade), search/filter (name/date), sort; double-tap zoom; battery-friendly (pool idles, no busy repaint — Slint is retained, verify with gfxinfo "total frames").
• M7 — perf & release: real-device (aarch64) pass; Perfetto; memory bounds; release APK size check (expect ~8–9 MB like the sample); consider the 1.19 upgrade (verify pointer-scroll fix first).

10. Open questions & risks

1. HEIC on Android: imagepipe is Apple's ImagePipe — its Rust crate may only ship Apple prebuilts. First task of M2: verify an NDK build (cargo apk2 build with imagepipe). Fallbacks: libheif-rs/libheif (C, cross-buildable), or skip HEIC initially and document JPEG/PNG only (the emulator test set is JPEG anyway; real device photos from this ecosystem may already be JPEG — check passgo's existing files).
2. Slint pointer-scroll todo!() panic (open issue in the Android backend at eval time): could fire on trackpad/touch scroll. Test scrolling on M3; if it fires, patch locally (the fix is small) or upgrade to 1.19 when available.
3. Max texture size: full-screen decode of 12 MP+ images (e.g. 8192 px) may exceed GPU max texture size on some devices — verify on M4; fall back to a 2×-max downscale for display.
4. Slint license: royalty-free tier covers proprietary mobile/desktop/ web; embedded devices excluded — irrelevant for us, but the repo README should state the chosen tier (royalty-free) once the app ships.
5. Docs drift: trust the pinned source (~/prog/android/ui/slint-src @ v1.18.1) over online docs; bump pin + re-verify after any Slint upgrade.
6. rusqlite bundled compiles SQLite from source in every target (adds ~30–60 s to clean builds) — acceptable; revisit if it hurts.

11. Pointers

• Framework eval (full report + raw notes + both sample apps + screenshots): ~/prog/android/ui/ — REPORT.md, NOTES.md, slint_app/, eframe_app/, slint-src/ (v1.18.1), egui-src/ (0.37.0).
• Slint docs (version-sensitive!): https://slint.dev/docs — Android guide: docs/guide/platforms/mobile/android.mdx in the source tree.
• examples/native-gestures, demos/ in slint-src — reference code for gestures, lists, images.
• passgo (existing Go app using the same all-files-access pattern): ~/prog/go/passgo — its scripts/build.sh and manifest handling are a proven reference for the permission UX flow.
• Emulator AVD: pad_avd (google_apis x86_64, API 35); start command in §3.
• Gitea: push target git+ssh://gitea@git.wow.st/gmp/gallery.git (repo must exist on the server first, like the other gmp/* repos).