The embedding approach was overkill: it embedded only 'name — kind'
(short strings), spent a 4B model + ~2.5GB VRAM + 4.6GB of vectors + a
one-time 450k-row job to do what indexed SQL does directly.
New default — no embedding model, no extra VRAM:
- The LLM agent is the semantic layer: it maps the user's concept to
OSM kinds + local-language name keywords. It is already in VRAM for
chat, so this costs nothing.
- /kinds: returns the tag vocabulary that actually exists (GROUP BY kind
with counts) so the agent grounds its choices in real data.
- /search: indexed retrieval — kind IN/ILIKE (poi_kind), name FTS
(to_tsvector) + trigram (pg_trgm) for fuzzy/substring, optional
ST_DWithin radius. Ranked by trigram similarity then distance.
- schema.sql: real trigram GIN index (poi_name_trgm_ops); renamed the
misnamed FTS index to poi_name_fts.
- Agent tools: poi_semantic -> poi_kinds + poi_search (both pin results
on the map).
pgvector demoted to an opt-in path (embed/ + /semantic) — still works
if poi_vec is built, but no longer the default. Dropped the half-built
poi_vec and stopped the background embed run.
Ran the full stack (docker now available via sg docker):
- postgis/postgis:16-3.4 up, both extracts imported:
northeast 291,000 POIs (from ~/osm-build/data/northeast.osm.pbf),
colombia 228,446 POIs
- spatiald live on :5005, proxied by the mock server (/spatial/*,
/spatial-status); UI badge + poi_near tool activate automatically
Fixes discovered by running against real data:
- osm2pgsql image: osm2pgsql/osm2pgsql does not exist on Docker Hub —
use iboates/osm2pgsql, bypass its DB-probing entrypoint (--entrypoint
osm2pgsql), password via PGPASSWORD (this build's --password flag
forces an interactive prompt)
- osm2pgsql 2.x schema: planet_osm_point/_polygon (not _node/_way),
geometry in EPSG:3857 'way' column, -k hstore for opening_hours/
fee/website/addr:*, refresh_poi() dedupes relation polygons and
uses hstore -> (not ->>)
- compose file downgraded to v1-compatible 3.8 (host has docker-compose
1.29, no v2 plugin); OSM_DIR exported by import.sh
- spatiald: proper SQL parameterization (@@i@@ tokens, body ..
shifted past the filter args — the first renumber attempt was
unsound), ST_Distance(geography) instead of the geometry-only
ST_Distance_Sphere, 4-arg ST_DWithin for the corridor band, WKT in
lng lat order, kind= accepts exact (amenity=restaurant), bare tag
value (restaurant) or family (tourism)
- mock server /spatial proxy: forward u.search (was dropping it)
- app.js poi_near: 'points' param, semicolons must be %3B-encoded
(Go url.Parse drops the tail of a value containing a raw ';')
- README: osm2pgsql 2.x data notes + kind semantics
Milestone 1 — live web search:
- mock/server.js: /search + /search-status proxy to the self-hosted
SearXNG (third leg of the 3-source blend; engine endpoint stays
server-side), /spatial + /spatial-status proxy to the spatial service
- mock/app.js: background enrichment of suggestion cards — each
candidate gets a web element (discovery cards + "you might like"
rows) that is checked via SearXNG in the background (one in-flight
query per place, 1 h TTL, failures retry in ~5 min) and APPENDS
sourced facts with provenance URLs; never reorders/rewrites the plan.
New LLM tools: web_search (cite URLs) and poi_near (PostGIS). Status
badge gains 'web' / 'spatial'.
- styles.css: .cc-web live-facts blocks, t-web verified chip
Milestone 2 — PostGIS spatial DB (code complete; needs docker access
to run — see spatial/README.md):
- spatial/: docker-compose (postgis/postgis:16-3.4 + osm2pgsql:16
one-shot importer + spatiald), schema.sql (poi table w/ GIST index,
spatial_extract bookkeeping, refresh_poi()), import.sh for the PBF
extracts in ../osm, spatiald (Go, lib/pq): /health, /near
(ST_DWithin), /nearest (KNN), /corridor (ST_Buffer along a route),
with kind/name/extract/limit filters
