trips/spatial/README.md

spatial — PostGIS source of truth

Native spatial queries for the trip planner (DESIGN.md §Data: "PostGIS (Postgres) — import PBF via osm2pgsql; gives SQL spatial queries (radius, nearest, bbox)"). Replaces the flat-file/Overpass approximations.

Stack

Piece	What it is
docker-compose.yml	postgis/postgis:16-3.4 + postgresql-16-pgvector (DB, port 5432) + one-shot iboates/osm2pgsql importer + spatiald (query service, port 5005)
schema.sql	runs on first boot: poi table (named POIs as points, GIST index), spatial_extract bookkeeping, refresh_poi()
20-vector.sql	runs on first boot: CREATE EXTENSION vector (the vector package is baked into the DB image — see db/Dockerfile)
db/Dockerfile	postgis/postgis:16-3.4 + postgresql-16-pgvector (workaround for the EOL bullseye repos — see the file)
import.sh	loads a PBF from ../osm/ via osm2pgsql, refreshes poi
main.go (spatiald)	JSON query service over poi + poi_vec (radius / KNN / corridor / semantic)
embed/main.go (embedpoi)	builds the poi_vec semantic index from poi (zembed embeddings → pgvector, HNSW)
Dockerfile	builds spatiald

Prerequisites

• Docker with the current user in the docker group (on this box: one-time sudo usermod -aG docker $USER, then re-login).
• PBF extracts in ~/trips/osm (already present: nh.osm.pbf, colombia.osm.pbf, US state extracts; override with OSM_DIR).

Usage

docker-compose up -d postgis           # first: runs schema.sql
./import.sh ~/trips/osm/colombia.osm.pbf colombia
./import.sh ~/osm-build/data/northeast.osm.pbf northeast   # merged NE extract
docker-compose up -d spatiald          # query service on :5005

The mock server (mock/server.js) proxies it: GET /spatial/* → :5005, plus GET /spatial-status for the UI badge. When the service is up, the chat assistant gains the poi_near tool automatically.

spatiald endpoints

All return {"count":N,"results":[{extract,osm_id,name,kind,opening_hours, fee,website,addr_city,dist_m},…]}. Common filters:

• extract=nh — one loaded extract
• kind=amenity=restaurant — exact kind, or kind=restaurant (any family)
• name=café — ILIKE substring
• limit=20 (max 200)

Endpoint	Query	SQL core
GET /health	—	extract bookkeeping
GET /near	lat,lng,r(m; default 500)	ST_DWithin(geography, …, r) + KNN ordering
GET /nearest	lat,lng	ORDER BY geom <-> point
GET /corridor	points=lng,lat;…, r	ST_DWithin(geom::geography, ST_Buffer(line::geography, r))
GET /semantic	q (free text), limit	cosine <=> over poi_vec (pgvector HNSW)

/semantic embeds q with the local zembed model (EMBED_BASE / EMBED_MODEL, defaults http://192.168.3.7:1234/v1 + zembed-1-Q4_K_M) and ranks poi_vec by cosine distance. Each result also carries lat, lng and score (1 − cosine dist) so the UI can drop map pins. The same extract/kind/name filters apply.

Examples:

curl 'localhost:5005/near?lat=42.35&lng=-71.06&r=1000&kind=restaurant'
curl 'localhost:5005/nearest?lat=10.40&lng=-75.54&kind=tourism&limit=5'
curl 'localhost:5005/corridor?r=300&kind=fuel&points=-71.06,42.35;-71.07,42.36'
curl 'localhost:5005/semantic?q=medieval%20museums&limit=5'

Semantic index (pgvector)

poi_vec mirrors poi for every named POI family except place=* and stores a 2560-dim zembed embedding of "name — kind". It is a derived table — rebuild it any time with embedpoi (it is resumable: existing keys are skipped, and it drops/recreates the table only if the embedding dimension changed).

go run ./embed -dsn "host=localhost port=5432 user=trips password=trips dbname=trips sslmode=disable"
# restrict to one extract, or be gentler on the shared model host:
go run ./embed -extract colombia -qps 2 -batch 64

• Embeddings come from the shared llama.cpp host (/v1/embeddings), so the host must be up. That host swaps models in/out and 500s briefly while zembed is (re)loaded — embedpoi retries patiently (up to 20× with backoff).
• Zembed uses a query: prompt prefix for search-time queries — spatiald adds it in /semantic; embedpoi stores raw "name — kind" passages.
• After the run it builds poi_vec_hnsw (hnsw (vec vector_cosine_ops)), which is what makes /semantic fast at full scale.
• The DB image is postgis/postgis:16-3.4 + postgresql-16-pgvector (see db/Dockerfile). The stock postgis images for PG16 are bullseye-based and their Debian release files are expired, so the image disables Check-Valid-Until and adds the pgdg repo just to fetch postgresql-16-pgvector.

Data notes

• Built for osm2pgsql 2.x (iboates/osm2pgsql): tables are planet_osm_point / planet_osm_polygon (not the 1.x _node/_way), geometry lives in a way column in EPSG:3857, and the import runs with -k so tags without a fixed column (opening_hours, fee, website, addr:city) land in the hstore tags column that refresh_poi() reads. refresh_poi() also dedupes relation-derived polygons (2.x repeats them per relation with negated ids).
• poi covers every named point/polygon with an amenity/tourism/ shop/leisure/historic/place tag. Unnamed amenities (a nameless kiosk) are out of scope for v1.
• kind filter semantics: kind=amenity=restaurant (exact), kind=restaurant (tag value, any family), kind=tourism (whole family).
• Corridor points must URL-encode the semicolons (%3B) — Go's url.Parse drops the tail of a value that contains a raw ;.
• Multiple extracts coexist in one DB, disambiguated by poi.extract (set automatically by import.sh).
• lib/pq has no []float32 → vector codec, so /semantic sends the embedding as a [f1,f2,…] text literal and casts with ::vector in SQL.
• poi_vec is rebuilt by embedpoi (not import.sh); it is keyed (extract, osm_type, osm_id) to match poi.

Roadmap hooks

• bbox queries: trivial addition (ST_Contains(ST_MakeEnvelope,…)).
• Rerun embedpoi after a new import.sh so poi_vec tracks the new poi rows.
• Routing-graph join: OSRM/GraphHopper geometries can be loaded into the same DB (route_geom table) for true along-route analytics instead of the current buffer-over-polyline.