gallery/HANDOFF.md

# Gallery — development hand-off

A high-performance image gallery for Android (primary), Linux and macOS,
written in Rust with the **Slint** UI toolkit. This document is the starting
point for development: it captures every decision, gotcha, and measurement
from the framework evaluation that preceded it, so no learning has to be
rediscovered.

**Status: M0 done** — this scaffold builds and runs on the emulator
(`./scripts/build.sh debug`), including the permission-aware Android init and
the lifecycle-event hook.

---

## 1. Goals

- Browse and view a local photo collection (thousands of images) that stays
  smooth: instant grid, no decode jank, cached thumbnails.
- Full-screen viewer with pinch-zoom/pan.
- Correct EXIF orientation, HEIC support (Android camera default).
- Same codebase on macOS + Linux (desktop `cargo run`) and Android
  (`cargo apk2`).
- Quality of the *platform experience* matters: safe-area insets, IME-free
  UI, native-feeling widgets, animations — first-class support preferred over
  manual patching.

Non-goals (for now): editing, cloud sync, video, widgets/live-updates.

## 2. Why Slint (decision summary)

Full evaluation in `/mnt/Backup/prog/android/ui/REPORT.md` (same tree:
`~/prog/android/ui/REPORT.md`), raw notes in `NOTES.md` there. Headline:

| | Slint 1.18.1 | eframe/egui 0.37.0 |
|---|---|---|
| Virtualized list (the gallery's lifeline) | ✅ `ListView` std-widget, only visible rows instantiated | ❌ `ScrollArea::show` lays out all children; `show_rows` = manual virtualization API |
| Async image → UI | swap `slint::Image` into a property; only that tile redraws (retained mode) | `ctx.load_texture` + custom loader; visible set rebuilt every frame |
| Lifecycle hook for permissions | ✅ `init_with_event_listener` (Start/Resume/Pause/Stop/SaveState/...) | ❌ none in `App` trait (winit swallows it) |
| Android safe-area | ✅ `safe-area-insets` built in | manual 32px-ish panel workaround (winit#3910) |
| Animation | ✅ declarative built-in engine | hand-rolled |
| IME text entry | ✅ | ✅ |
| APK size (release aarch64) | 8.57 MB | 2.87 MB |
| Clean build time (x86_64, this machine) | 195 s / 270 crates | 76 s / 137 crates |
| Desktop Linux system deps | `libfontconfig1` + `pkg-config` | none |
| License | royalty-free for proprietary mobile/desktop (embedded excluded) / GPLv3 / commercial | MIT OR Apache-2.0 |
| Min Android API | 26 | 24 |

For a stateful, form-and-grid app, the declarative model was the right fit
(see the discussion about `pad`'s immediate-mode frame friction); egui wins
for games/dashboards and on APK size. Known Slint caveat: an open upstream
"abort on pointer scroll" bug in the Android backend (`todo!()` in the
pointer-event path) — verify it's fixed in 1.19 before relying on heavy
scrolling; if it fires, the panic is obvious in logcat.

## 3. Environment, build & test loop

Everything below is verified working on this machine (2026-10-27).

```bash
export ANDROID_HOME=$HOME/android-sdk
export ANDROID_NDK_ROOT=$HOME/android-sdk/ndk/27.0.12077973
export PATH="$HOME/.cargo/bin:$ANDROID_HOME/platform-tools:$PATH"

# Rust 1.99.1 via rustup; tools: cargo-apk2 1.5.0, cargo-ndk 4.1.2
./scripts/build.sh debug      # debug x86_64 APK + adb install + launch
./scripts/build.sh            # release x86_64 (emulator)
./scripts/build.sh aarch64    # release aarch64 (device)
cargo run                     # desktop (Linux needs libfontconfig1+pkg-config)
```

- Emulator: `emulator -avd pad_avd -no-window -no-audio -gpu off -no-snapshot`
  (google_apis x86_64, Android 15). **`-gpu off -no-snapshot` matters** —
  swiftshader_indirect + snapshot crashed the emulator once during the eval.
  If it won't start, the host key lost its wireless pairing — see the pairing
  procedure in `~/.pi/agent/AGENTS.md` (phone) / emulator docs (this AVD).
- APK lands at `target/<debug|release>/apk/gallery.apk`.
- Release signing uses the debug keystore
  (`CARGO_APK_RELEASE_KEYSTORE=$HOME/.android/debug.keystore`,
  `CARGO_APK_RELEASE_KEYSTORE_PASSWORD=android`) — fine for personal use;
  for Play, generate a real keystore and point the env vars at it.
- `cargo apk2` quirk: `--target` (not `-t`). `--lib` builds the cdylib.
- Grant "All files access" to the app on the emulator after install:
  `adb shell am start -a android.settings.MANAGE_ALL_FILES_ACCESS_PERMISSION`
  (or Settings UI). `adb install -r` keeps the grant.
- Perf measurement: `adb shell perfetto` / `adb shell dumpsys gfxinfo wow.gmp.gallery`,
  `systrace`-style capture via Perfetto trace shell. Cold start:
  `adb shell am start -W`.

## 4. Architecture

The UI framework is only the display layer. The performance lives in the
thumbnail engine, and it is framework-agnostic:

```
┌─ metadata ─────────────────────────────────────────────────────────┐
│ sqlite (rusqlite, bundled): path, mtime, size, w, h, orientation,  │
│ thumb_path, date_taken            — one background scan pass       │
└──────────────┬─────────────────────────────────────────────────────┘
               │ model → Slint (list of visible-range items)
┌─ thumbnail pool (tokio-free: N std::threads, N ≈ cores/2) ─────────┐
│ per job: token cancellation (AtomicU64 gen counter) + priority     │
│  1. disk cache hit?  .thumbs/{sha1(path|mtime|size)}.jpg  → done   │
│  2. decode AT TARGET SIZE (never full-size for the grid):          │
│       JPEG: zune-jpeg "turbo" (decode+downscale in ONE pass,       │
│             pure Rust, actively maintained) or turbojpeg (SIMD)    │
│       HEIC: imagepipe (0.5, 2026-08) — VERIFY Android NDK build    │
│       PNG:  image/png                                              │
│  3. apply EXIF orientation (once, here — never per frame)          │
│  4. re-encode small JPEG → disk cache                              │
│  5. channel → UI: (id, slint::Image)                               │
│ prefetch: decode visible ± 1 row; cancel items scrolled out        │
└────────────────────────────────────────────────────────────────────┘
┌─ UI (Slint) ───────────────────────────────────────────────────────┐
│ grid screen:   ListView (vertical) → row of N square Image cells   │
│ viewer screen: full-size async decode (once, cached) + manual      │
│                two-finger pinch transform (no built-in in 1.18)    │
└────────────────────────────────────────────────────────────────────┘
```

Planned module layout:

```
src/
  lib.rs        # android_main, ANDROID_APP stash, platform init   [exists]
  main.rs       # desktop entry                                     [exists]
  main.slint    # screens (split into ui/ components at M3)        [exists]
  db.rs         # sqlite schema + scan (M1)
  scan.rs       # directory walk, image detection, metadata read    (M1)
  thumb.rs      # pool, decode@size, disk cache, cancellation      (M2)
  images.rs     # decode dispatch (zune-jpeg / imagepipe / png)    (M2)
  ui.rs         # Rust↔Slint glue: Model provider, thumb→Image     (M3)
  viewer.rs     # full-size load + transform state                  (M4)
  perm.rs       # storage permission check/request (M5)
```

Crate choices (all verified current on crates.io, 2026-10):

| crate | version | why |
|---|---|---|
| `slint` / `slint-build` | 1.18.1 | pinned; upgrade to 1.19 after the pointer-scroll fix lands |
| `rusqlite` | latest, `bundled` feature | SQLite without system deps; cross-compiles to Android |
| `zune-jpeg` | 0.5.x (upd 2026-10) | pure-Rust JPEG, **turbo mode** = decode+downscale one pass. Preferred over `turbojpeg` for Android: no C dep to cross-build |
| `image` | 0.25.x | PNG + header/metadata (`ImageReader::into_dimensions`, EXIF orientation) |
| `imagepipe` | 0.5.x (upd 2026-08) | HEIC/HEIF via Apple ImagePipe — **open question: Android NDK support** (see §10) |
| `slint::Image::from_rgba8_premultiplied` | — | how decoded thumbs enter the UI |

Decode sizes: grid tiles render at ~cell×2 (Retina) — e.g. 500 px side.
Full-screen: full decode once (12 MP ≈ 30–80 ms off-thread, fine), cached in
the in-memory LRU until memory pressure.

**The five failure modes the current (unknown) app likely has** — this design
eliminates all five: (1) full-size decode for grid cells, (2) no disk
thumbnail cache, (3) no list virtualization, (4) decode on UI thread without
cancellation, (5) EXIF rotation applied per frame.

## 5. UI plan (Slint patterns)

- **Grid**: `ListView` is the only virtualized std-widget (confirmed in
  source docs: "Only visible rows are instantiated"). Grid = vertical
  `ListView`, each row a horizontal layout of square `Image` cells
  (columns ≈ `floor(window-width / cell)`; recompute on resize).
  Cells bind `image <=> model.image` — when the pool delivers a thumbnail,
  only that cell invalidates (retained mode; no repaint calls to remember).
- **Placeholder→image**: cell shows a `Rectangle` (shimmer optional, M6)
  until its `image` is set.
- **Image cache**: Slint's built-in path-based LRU is 5 MB (`CLruCache` in
  `internal/core/graphics/image/cache.rs`) — too small and the wrong layer
  for us; our cache lives in Rust (disk + compressed-bytes LRU).
  No mipmaps in the Skia renderer — irrelevant since we pre-downsample.
- **Viewer**: no pinch helper in Slint 1.18 (`TouchArea` has no pinch
  callback; grepped the stdlib). Manual two-finger math on `TouchArea`
  (distance/centroid → scale/translation, ~50 lines; crib
  `slint-src/examples/native-gestures` — source checkout is at
  `~/prog/android/ui/slint-src`, tag v1.18.1). Double-tap-to-zoom is a nice
  extra (M6).
- **Screens**: single `Window` with a stacked layout (grid + viewer overlay)
  or a `Navigator`-style state property; keep it simple until M4.
- **Safe area**: always pad with `root.safe-area-insets.{top,bottom}` (done
  in the scaffold).
- **Animation**: use built-in `animate` statements (syntax: `animate <prop>
  { duration: ...; easing: ...; }` — it is a *statement*, not an element).
  Slide/fade for the viewer is a one-liner.

## 6. Slint 1.18 API gotchas (all hit firsthand in the eval)

1. `animate` is a statement (`animate width { duration: 300ms; }`), not an
   `Animation {}` element.
2. Text input widget is `LineEdit`, not `TextField`.
3. Markdown interpolation: `text: "Count: \{count}"` (backslash-brace).
4. `length` is the unit type for lengths (`property <length>`, not
   `logical<length>` in the 1.18 Rust API).
5. No Rust macros/closures inside `.slint` callbacks — callbacks call global
   functions or invoke Rust via `@global` / imported callbacks.
6. 1.18 ↔ 1.19 **docs drift** — when reading docs online, check which
   version; the local source at `~/prog/android/ui/slint-src` (tag v1.18.1)
   is ground truth for the pinned version. `git grep` beats the docs.
7. `slint::android::init_with_event_listener(app, |event: &PollEvent| ...)`
   re-exports the whole `android-activity` crate as
   `slint::android::android_activity`; `AndroidApp` is also re-exported at
   `slint::android::AndroidApp`.
8. `slint::Image` runtime construction: `Image::from_rgba8_premultiplied`
   (also `from_array`); assign to an `image` property to update a cell.
9. Desktop Linux: needs `libfontconfig1` + `pkg-config` (pure Rust on macOS).
10. Retained-mode redraw: changing a property re-renders only affected
    elements — design bindings so thumbnail delivery touches one property.

## 7. Permissions & platform integration (Android)

Declared in `Cargo.toml` (verified schema in cargo-apk2 1.5 / ndk-build2:
`uses_permission` is **top-level** under `[package.metadata.android]`, and
`Permission` supports only `name` + `max_sdk_version`):
`READ_MEDIA_IMAGES`, `READ_EXTERNAL_STORAGE` (maxSdk 32),
`MANAGE_EXTERNAL_STORAGE`.

Runtime flow (M5), using the `ANDROID_APP` clone stashed in `src/lib.rs`:

1. **MANAGE_EXTERNAL_STORAGE** (primary path, same as passgo): no runtime
   dialog exists — check `Environment.isExternalStorageManager()` via JNI,
   if false launch `Settings.ACTION_MANAGE_ALL_FILES_ACCESS_PERMISSION` via
   `ANDROID_APP.get().run_on_java_main_thread(...)` (attach `JNIEnv` from
   `vm_as_ptr()`; Activity from `activity_as_ptr()`), then **re-check on
   `MainEvent::Resume`** — the hook is already wired in `src/lib.rs`
   (TODO(M5) marker).
2. **READ_MEDIA_IMAGES / READ_EXTERNAL_STORAGE**: `requestPermissions` via
   the same JNI route; the bare `NativeActivity` does not forward
   `onRequestPermissionsResult` to native, so either (a) poll
   `checkSelfPermission` after requesting (good enough), or (b) ship a custom
   `NativeActivity` subclass via cargo-apk2 `java_sources = "android/java"`
   (manifest `activity name` points at it) — the proper callback, reusable
   pattern from the eval.
3. Facts that constrain design (verified in android-activity 0.6.1 source):
   no `NewIntent` event; `poll_events` may only run on the `android_main`
   thread (Slint owns it — don't add a second consumer); `AndroidApp` is
   `Clone` and a clone works from any thread (`run_on_java_main_thread`,
   `vm_as_ptr`, `activity_as_ptr`).
4. **Source discovery** for photos: start with a plain directory scan of
   `/storage/emulated/0/Pictures` + `DCIM` (works with all-files access,
   emulator-friendly, and it's what makes the app testable without
   MediaStore). MediaStore integration (API-aware, scoped to
   `READ_MEDIA_IMAGES`) is a later refinement if directory scanning proves
   insufficient.

## 8. Performance targets & measurement

Targets (emulator is NOT representative for absolute decode times — validate
on a real device with the aarch64 build):

- Grid: 60 fps scrolling at 5k+ items; zero full-size decodes on the hot path.
- First frame: grid of placeholders < 100 ms after launch (metadata from
  sqlite, images async).
- Thumbnail decode (grid size): < 5 ms off-thread per JPEG (measure with
  `std::time::Instant` logs; zune-jpeg turbo should be well under).
- Steady-state memory: in-memory thumb cache bounded (~64–128 MB compressed
  bytes on a phone) + Slint's texture memory for visible tiles only.
- Cold start (emulator, release): compare against the eval baseline
  (Slint sample ≈ 1–2 s; gallery must not regress > 0.5 s).

Test data (M2): generate a synthetic set on the device — a small Rust `bin`
or Python script that writes N=5,000 JPEGs of varied sizes/orientations into
`/storage/emulated/0/Pictures/gallery-test/` (include a few HEICs if
imagepipe works; include rotated EXIF). Reuse it for every perf run.

Measure with: `adb shell am start -W` (cold start),
`dumpsys gfxinfo wow.gmp.gallery framestats`, Perfetto trace (GPU + main +
pool threads), and `logcat` pool timing logs behind a `debug_timing` flag.

## 9. Milestones

- **M0 — scaffold (DONE)**: builds (x86_64 + aarch64 toolchain), installs,
  runs on emulator; permission-aware init + Resume hook; build script;
  manifest with permissions; this document.
- **M1 — metadata**: `db.rs` + `scan.rs`; scan Pictures+DCIM on first launch
  (background thread, progress in UI); sqlite table; list of items as a
  Slint `Model` fed to a placeholder `ListView` (paths as text, no images).
  *Accept: 5k files scanned < 2 s; relaunch reuses DB; mtime-changed files
  refreshed.*
- **M2 — thumbnail engine**: `thumb.rs` + `images.rs`; disk cache;
  cancellation; pool; synthetic 5k test set; timing logs.
  *Accept: first grid fill < 2 s; re-launch fills instantly from disk cache;
  fast scroll causes no backlog (watch pending-queue length in logs).*
- **M3 — grid UI**: ListView grid, cell sizing, placeholder shimmer,
  pull-to-refresh optional; Rust `Model` ↔ Slint; thumbnail delivery →
  cell image swap. *Accept: 60 fps scroll on 5k items (emulator: no dropped-
  frame spikes > 16 ms in gfxinfo).*
- **M4 — viewer**: tap cell → full decode + overlay; pinch/pan (manual
  math), double-tap zoom; back gesture/button; EXIF orientation correct.
- **M5 — permissions**: implement `perm.rs` per §7; launch flow when
  un-granted (settings card → open Settings → recheck on Resume); test the
  exact passgo-style flow on a fresh install.
- **M6 — polish**: dark theme, animations (slide/fade), search/filter
  (name/date), sort; double-tap zoom; battery-friendly (pool idles, no
  busy repaint — Slint is retained, verify with gfxinfo "total frames").
- **M7 — perf & release**: real-device (aarch64) pass; Perfetto; memory
  bounds; release APK size check (expect ~8–9 MB like the sample);
  consider the 1.19 upgrade (verify pointer-scroll fix first).

## 10. Open questions & risks

1. **HEIC on Android**: `imagepipe` is Apple's ImagePipe — its Rust crate
   may only ship Apple prebuilts. **First task of M2: verify an NDK build**
   (`cargo apk2 build` with imagepipe). Fallbacks: `libheif-rs`/libheif
   (C, cross-buildable), or skip HEIC initially and document JPEG/PNG only
   (the emulator test set is JPEG anyway; real device photos from this
   ecosystem may already be JPEG — check passgo's existing files).
2. **Slint pointer-scroll `todo!()` panic** (open issue in the Android
   backend at eval time): could fire on trackpad/touch scroll. Test scrolling
   on M3; if it fires, patch locally (the fix is small) or upgrade to 1.19
   when available.
3. **Max texture size**: full-screen decode of 12 MP+ images (e.g.
   8192 px) may exceed GPU max texture size on some devices — verify on
   M4; fall back to a 2×-max downscale for display.
4. **Slint license**: royalty-free tier covers proprietary mobile/desktop/
   web; embedded devices excluded — irrelevant for us, but the repo README
   should state the chosen tier (royalty-free) once the app ships.
5. **Docs drift**: trust the pinned source (`~/prog/android/ui/slint-src`
   @ v1.18.1) over online docs; bump pin + re-verify after any Slint upgrade.
6. **rusqlite `bundled`** compiles SQLite from source in every target
   (adds ~30–60 s to clean builds) — acceptable; revisit if it hurts.

## 11. Pointers

- Framework eval (full report + raw notes + both sample apps + screenshots):
  `~/prog/android/ui/` — `REPORT.md`, `NOTES.md`, `slint_app/`, `eframe_app/`,
  `slint-src/` (v1.18.1), `egui-src/` (0.37.0).
- Slint docs (version-sensitive!): https://slint.dev/docs — Android guide:
  `docs/guide/platforms/mobile/android.mdx` in the source tree.
- `examples/native-gestures`, `demos/` in `slint-src` — reference code for
  gestures, lists, images.
- passgo (existing Go app using the same all-files-access pattern):
  `~/prog/go/passgo` — its `scripts/build.sh` and manifest handling are a
  proven reference for the permission UX flow.
- Emulator AVD: `pad_avd` (google_apis x86_64, API 35); start command in §3.
- Gitea: push target `git+ssh://gitea@git.wow.st/gmp/gallery.git`
  (repo must exist on the server first, like the other gmp/* repos).