Greg Pomerantz
d05ebfa72f
Ran the full stack (docker now available via sg docker): - postgis/postgis:16-3.4 up, both extracts imported: northeast 291,000 POIs (from ~/osm-build/data/northeast.osm.pbf), colombia 228,446 POIs - spatiald live on :5005, proxied by the mock server (/spatial/*, /spatial-status); UI badge + poi_near tool activate automatically Fixes discovered by running against real data: - osm2pgsql image: osm2pgsql/osm2pgsql does not exist on Docker Hub — use iboates/osm2pgsql, bypass its DB-probing entrypoint (--entrypoint osm2pgsql), password via PGPASSWORD (this build's --password flag forces an interactive prompt) - osm2pgsql 2.x schema: planet_osm_point/_polygon (not _node/_way), geometry in EPSG:3857 'way' column, -k hstore for opening_hours/ fee/website/addr:*, refresh_poi() dedupes relation polygons and uses hstore -> (not ->>) - compose file downgraded to v1-compatible 3.8 (host has docker-compose 1.29, no v2 plugin); OSM_DIR exported by import.sh - spatiald: proper SQL parameterization (@@i@@ tokens, body .. shifted past the filter args — the first renumber attempt was unsound), ST_Distance(geography) instead of the geometry-only ST_Distance_Sphere, 4-arg ST_DWithin for the corridor band, WKT in lng lat order, kind= accepts exact (amenity=restaurant), bare tag value (restaurant) or family (tourism) - mock server /spatial proxy: forward u.search (was dropping it) - app.js poi_near: 'points' param, semicolons must be %3B-encoded (Go url.Parse drops the tail of a value containing a raw ';') - README: osm2pgsql 2.x data notes + kind semantics
89 lines
3.8 KiB
Markdown
89 lines
3.8 KiB
Markdown
# spatial — PostGIS source of truth
|
|
|
|
Native spatial queries for the trip planner (DESIGN.md §Data: "PostGIS
|
|
(Postgres) — import PBF via osm2pgsql; gives SQL spatial queries (radius,
|
|
nearest, bbox)"). Replaces the flat-file/Overpass approximations.
|
|
|
|
## Stack
|
|
|
|
| Piece | What it is |
|
|
|---|---|
|
|
| `docker-compose.yml` | `postgis/postgis:16-3.4` (DB, port 5432) + one-shot `iboates/osm2pgsql` importer + `spatiald` (query service, port 5005) |
|
|
| `schema.sql` | runs on first boot: `poi` table (named POIs as points, GIST index), `spatial_extract` bookkeeping, `refresh_poi()` |
|
|
| `import.sh` | loads a PBF from `../osm/` via osm2pgsql, refreshes `poi` |
|
|
| `main.go` (spatiald) | JSON query service over `poi` |
|
|
| `Dockerfile` | builds spatiald |
|
|
|
|
## Prerequisites
|
|
|
|
* Docker with the current user in the `docker` group (on this box:
|
|
one-time `sudo usermod -aG docker $USER`, then re-login).
|
|
* PBF extracts in `~/trips/osm` (already present: `nh.osm.pbf`,
|
|
`colombia.osm.pbf`, US state extracts; override with `OSM_DIR`).
|
|
|
|
## Usage
|
|
|
|
```bash
|
|
docker-compose up -d postgis # first: runs schema.sql
|
|
./import.sh ~/trips/osm/colombia.osm.pbf colombia
|
|
./import.sh ~/osm-build/data/northeast.osm.pbf northeast # merged NE extract
|
|
docker-compose up -d spatiald # query service on :5005
|
|
```
|
|
|
|
The mock server (`mock/server.js`) proxies it: `GET /spatial/*` → `:5005`,
|
|
plus `GET /spatial-status` for the UI badge. When the service is up, the
|
|
chat assistant gains the `poi_near` tool automatically.
|
|
|
|
## spatiald endpoints
|
|
|
|
All return `{"count":N,"results":[{extract,osm_id,name,kind,opening_hours,
|
|
fee,website,addr_city,dist_m},…]}`. Common filters:
|
|
|
|
* `extract=nh` — one loaded extract
|
|
* `kind=amenity=restaurant` — exact kind, or `kind=restaurant` (any family)
|
|
* `name=café` — ILIKE substring
|
|
* `limit=20` (max 200)
|
|
|
|
| Endpoint | Query | SQL core |
|
|
|---|---|---|
|
|
| `GET /health` | — | extract bookkeeping |
|
|
| `GET /near` | `lat,lng,r(m; default 500)` | `ST_DWithin(geography, …, r)` + KNN ordering |
|
|
| `GET /nearest` | `lat,lng` | `ORDER BY geom <-> point` |
|
|
| `GET /corridor` | `points=lng,lat;…`, `r` | `ST_DWithin(geom::geography, ST_Buffer(line::geography, r))` |
|
|
|
|
Examples:
|
|
|
|
```bash
|
|
curl 'localhost:5005/near?lat=42.35&lng=-71.06&r=1000&kind=restaurant'
|
|
curl 'localhost:5005/nearest?lat=10.40&lng=-75.54&kind=tourism&limit=5'
|
|
curl 'localhost:5005/corridor?r=300&kind=fuel&points=-71.06,42.35;-71.07,42.36'
|
|
```
|
|
|
|
## Data notes
|
|
|
|
* Built for **osm2pgsql 2.x** (`iboates/osm2pgsql`): tables are
|
|
`planet_osm_point` / `planet_osm_polygon` (not the 1.x `_node`/`_way`),
|
|
geometry lives in a `way` column in EPSG:3857, and the import runs with
|
|
`-k` so tags without a fixed column (`opening_hours`, `fee`, `website`,
|
|
`addr:city`) land in the hstore `tags` column that `refresh_poi()` reads.
|
|
`refresh_poi()` also dedupes relation-derived polygons (2.x repeats them
|
|
per relation with negated ids).
|
|
* `poi` covers every **named** point/polygon with an `amenity`/`tourism`/
|
|
`shop`/`leisure`/`historic`/`place` tag. Unnamed amenities (a nameless
|
|
kiosk) are out of scope for v1.
|
|
* `kind` filter semantics: `kind=amenity=restaurant` (exact),
|
|
`kind=restaurant` (tag value, any family), `kind=tourism` (whole family).
|
|
* Corridor `points` must URL-encode the semicolons (`%3B`) — Go's
|
|
`url.Parse` drops the tail of a value that contains a raw `;`.
|
|
* Multiple extracts coexist in one DB, disambiguated by `poi.extract`
|
|
(set automatically by `import.sh`).
|
|
* pgvector (semantic POI index, later milestone) gets its own init script +
|
|
embedding importer — the base image doesn't ship the `vector` package.
|
|
|
|
## Roadmap hooks
|
|
|
|
* `bbox` queries: trivial addition (`ST_Contains(ST_MakeEnvelope,…)`).
|
|
* Routing-graph join: OSRM/GraphHopper geometries can be loaded into the
|
|
same DB (`route_geom` table) for true along-route analytics instead of
|
|
the current buffer-over-polyline.
|