trips/spatial/README.md
Greg Pomerantz d05ebfa72f PostGIS: live — imports, spatiald, and fixes from real runs
Ran the full stack (docker now available via sg docker):
- postgis/postgis:16-3.4 up, both extracts imported:
  northeast 291,000 POIs (from ~/osm-build/data/northeast.osm.pbf),
  colombia 228,446 POIs
- spatiald live on :5005, proxied by the mock server (/spatial/*,
  /spatial-status); UI badge + poi_near tool activate automatically

Fixes discovered by running against real data:
- osm2pgsql image: osm2pgsql/osm2pgsql does not exist on Docker Hub —
  use iboates/osm2pgsql, bypass its DB-probing entrypoint (--entrypoint
  osm2pgsql), password via PGPASSWORD (this build's --password flag
  forces an interactive prompt)
- osm2pgsql 2.x schema: planet_osm_point/_polygon (not _node/_way),
  geometry in EPSG:3857 'way' column, -k hstore for opening_hours/
  fee/website/addr:*, refresh_poi() dedupes relation polygons and
  uses hstore -> (not ->>)
- compose file downgraded to v1-compatible 3.8 (host has docker-compose
  1.29, no v2 plugin); OSM_DIR exported by import.sh
- spatiald: proper SQL parameterization (@@i@@ tokens, body ..
  shifted past the filter args — the first renumber attempt was
  unsound), ST_Distance(geography) instead of the geometry-only
  ST_Distance_Sphere, 4-arg ST_DWithin for the corridor band, WKT in
  lng lat order, kind= accepts exact (amenity=restaurant), bare tag
  value (restaurant) or family (tourism)
- mock server /spatial proxy: forward u.search (was dropping it)
- app.js poi_near: 'points' param, semicolons must be %3B-encoded
  (Go url.Parse drops the tail of a value containing a raw ';')
- README: osm2pgsql 2.x data notes + kind semantics
2026-09-10 12:23:49 -04:00

3.8 KiB

spatial — PostGIS source of truth

Native spatial queries for the trip planner (DESIGN.md §Data: "PostGIS (Postgres) — import PBF via osm2pgsql; gives SQL spatial queries (radius, nearest, bbox)"). Replaces the flat-file/Overpass approximations.

Stack

Piece What it is
docker-compose.yml postgis/postgis:16-3.4 (DB, port 5432) + one-shot iboates/osm2pgsql importer + spatiald (query service, port 5005)
schema.sql runs on first boot: poi table (named POIs as points, GIST index), spatial_extract bookkeeping, refresh_poi()
import.sh loads a PBF from ../osm/ via osm2pgsql, refreshes poi
main.go (spatiald) JSON query service over poi
Dockerfile builds spatiald

Prerequisites

  • Docker with the current user in the docker group (on this box: one-time sudo usermod -aG docker $USER, then re-login).
  • PBF extracts in ~/trips/osm (already present: nh.osm.pbf, colombia.osm.pbf, US state extracts; override with OSM_DIR).

Usage

docker-compose up -d postgis           # first: runs schema.sql
./import.sh ~/trips/osm/colombia.osm.pbf colombia
./import.sh ~/osm-build/data/northeast.osm.pbf northeast   # merged NE extract
docker-compose up -d spatiald          # query service on :5005

The mock server (mock/server.js) proxies it: GET /spatial/*:5005, plus GET /spatial-status for the UI badge. When the service is up, the chat assistant gains the poi_near tool automatically.

spatiald endpoints

All return {"count":N,"results":[{extract,osm_id,name,kind,opening_hours, fee,website,addr_city,dist_m},…]}. Common filters:

  • extract=nh — one loaded extract
  • kind=amenity=restaurant — exact kind, or kind=restaurant (any family)
  • name=café — ILIKE substring
  • limit=20 (max 200)
Endpoint Query SQL core
GET /health extract bookkeeping
GET /near lat,lng,r(m; default 500) ST_DWithin(geography, …, r) + KNN ordering
GET /nearest lat,lng ORDER BY geom <-> point
GET /corridor points=lng,lat;…, r ST_DWithin(geom::geography, ST_Buffer(line::geography, r))

Examples:

curl 'localhost:5005/near?lat=42.35&lng=-71.06&r=1000&kind=restaurant'
curl 'localhost:5005/nearest?lat=10.40&lng=-75.54&kind=tourism&limit=5'
curl 'localhost:5005/corridor?r=300&kind=fuel&points=-71.06,42.35;-71.07,42.36'

Data notes

  • Built for osm2pgsql 2.x (iboates/osm2pgsql): tables are planet_osm_point / planet_osm_polygon (not the 1.x _node/_way), geometry lives in a way column in EPSG:3857, and the import runs with -k so tags without a fixed column (opening_hours, fee, website, addr:city) land in the hstore tags column that refresh_poi() reads. refresh_poi() also dedupes relation-derived polygons (2.x repeats them per relation with negated ids).
  • poi covers every named point/polygon with an amenity/tourism/ shop/leisure/historic/place tag. Unnamed amenities (a nameless kiosk) are out of scope for v1.
  • kind filter semantics: kind=amenity=restaurant (exact), kind=restaurant (tag value, any family), kind=tourism (whole family).
  • Corridor points must URL-encode the semicolons (%3B) — Go's url.Parse drops the tail of a value that contains a raw ;.
  • Multiple extracts coexist in one DB, disambiguated by poi.extract (set automatically by import.sh).
  • pgvector (semantic POI index, later milestone) gets its own init script + embedding importer — the base image doesn't ship the vector package.

Roadmap hooks

  • bbox queries: trivial addition (ST_Contains(ST_MakeEnvelope,…)).
  • Routing-graph join: OSRM/GraphHopper geometries can be loaded into the same DB (route_geom table) for true along-route analytics instead of the current buffer-over-polyline.