gmp/trips
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
main
trips/SURVEY.md
Greg Pomerantz efde2cc71b maps project: design, survey, mock app, and route-aware planning backend 
- DESIGN.md: full design incl. driving-trip requirements (R1-R4),
  stays model, focus mode, mobile, provenance rules
- SURVEY.md: open-source landscape
- mock/: interaction mock (Florence itinerary, focus mode, stays,
  region stops, mobile layout)
- router/: Go module (stdlib-only) with Router interface
  (Valhalla + OSRM backends), stop_cost, optimize_stops, corridor,
  routectl CLI, bench (5 real NE-corridor tasks, 26 checks passing),
  integration tests, and setup-osrm.sh for the self-hosted router
- osm/: NH+MA+CT+NY PBFs (gitignored) + setup artifacts
2026-09-06 00:05:17 -04:00

128 lines
8.9 KiB
Markdown
Raw Permalink Blame History

# Survey: existing open-source projects in the LLM + maps domain
Compiled 2026-09-04. Categories are ordered by proximity to the target
system (a chat assistant answering map questions: routes/travel times,
businesses, opening hours, products/services).
## 1. Closest matches: LLM tool layers over OpenStreetMap
These provide *part* of the architecture (tools for an LLM), not a complete
product. This is the most active corner of the space.
| Project | What it is | Notes |
|---|---|---|
| [geodaai/openassistant](https://github.com/geodaai/openassistant) | OpenAssistant (open-source LLM assistant framework) with an **OpenStreetMap tool plugin**: geocoding, reverse geocoding, routing, isochrone analysis | Framework + tools, no map-UI product; tools mix with other tools for multi-step tasks. Python. |
| [jagan-shanmugam/open-streetmap-mcp](https://github.com/jagan-shanmugam/open-streetmap-mcp) | **MCP server** exposing OSM to LLM clients: geocoding, POI search, routing | One of several near-identical MCP servers (2025+ wave). |
| [NERVsystems/osmmcp](https://github.com/NERVsystems/osmmcp) | OSM **MCP server, written in Go**: geocoding, routing, nearby places, neighborhood analysis, EV charging stations | Go implementation — reusable pieces for us; same "MCP server" pattern. |
| [visuelconcept/myosm-mcp-server](https://github.com/visuelconcept/myosm-mcp-server) | Another OSM MCP server (geocoding etc.) | |
| [IBM/chuk-mcp-geocoder](https://github.com/IBM/chuk-mcp-geocoder) | MCP geocoder: forward/reverse, batch, nearby places, route waypoints, admin boundaries | Useful geocoding reference. |
| [steveattewell/osm-ai-map](https://github.com/steveattewell/osm-ai-map) | Demonstrator: ChatGPT interprets natural language → **Overpass queries** → results on a map | Old (pre-function-calling era) but exactly the target interaction; demonstrates the pattern on a real map UI. |
| [Aravindak27/LLM-GIS-ASSISTANT](https://github.com/Aravindak27/LLM-GIS-ASSISTANT) | AI-driven GIS assistant; natural language → maps, climate graphs, spatial analysis (Open-Meteo, WorldPop, OSM) | Analyst-oriented, not POI/routing-focused. |
| [DBishal13/geospatial-data-copilot](https://github.com/DBishal13/geospatial-data-copilot) | Natural-language interface over spatial infrastructure data (SF utility poles etc.) | Niche dataset but same idea: NL → structured geospatial query. |
## 2. GIS-workbench agents (QGIS plugins & the like)
A crowded niche, all aimed at *analysts inside QGIS* rather than end-user
map questions. Still useful for prompting patterns (inject layer/CRS context
into the prompt, generate+execute code):
- [opengeos/GeoAgent](https://github.com/opengeos/GeoAgent) — shared AI agent
layer for geospatial Python packages, map widgets, QGIS plugins.
- [xinguangYan/GeoPilot](https://github.com/xingguangYan/GeoPilot) — NL →
QGIS Python code.
- **GIS Chat for QGIS** (zenodo.org/records/18916276) — LLM chat panel in
QGIS, injects live workspace context.
- [gladcolor/LLM-Geo](https://github.com/gladcolor/LLM-Geo),
**IntelliGeo** (intelligeo.org),
[Teakinboyewa/SpatialAnalysisAgent](https://github.com/Teakinboyewa/SpatialAnalysisAgent),
[r-wenger/LLMFileDescribe](https://github.com/r-wenger/LLMFileDescribe).
- [iamtekson/GeoAgent](https://github.com/iamtekson/GeoAgent) (QGIS plugin,
distinct from opengeos/GeoAgent).
**Takeaway:** nobody here targets "ask about a business / route / hours";
they target geodata processing.
## 3. Research code & benchmarks (evaluation assets)
| Project | What it gives us |
|---|---|
| [knowledge-computing/MapQA-dataset](https://github.com/knowledge-computing/MapQA-dataset) | **MapQA**: 3,154 OSM QA pairs (SoCal, Illinois), 9 question types incl. routing & POI attributes; code for both a retrieval-based and an LLM-based approach. Our primary eval set. |
| [OSU-slatelab/MapQA](https://github.com/OSU-slatelab/MapQA) | *Different* "MapQA" — QA on US choropleth maps. Don't confuse. |
| [rohinmanvi/GeoLLM](https://github.com/rohinmanvi/GeoLLM) | ICLR'24: extract geospatial knowledge from LLMs using **auxiliary OSM data in prompts**; models + datasets. Validated that OSM context unlocks LLM geo-knowledge. |
| [Geo-R2LLM/groke](https://github.com/Geo-R2LLM/groke) | GROKE (ACL'26): vision-free, training-free hierarchical LLM reasoning **over the OSM graph** for navigation-instruction evaluation. |
| [makunyang/Geo-Loc](https://github.com/makunyang/Geo-Loc) | GeoKG-Loc: urban knowledge graph from OSM + LLM parsing of spatial constraints → coordinate estimation. |
| [MarcWeberFS/Text-to-sql](https://github.com/MarcWeberFS/Text-to-sql) | **Text-to-PostGIS**: NL → SQL for PostGIS with saved-query memory + benchmark. Direct alternative to (or complement of) the fixed tool catalog: let the LLM write PostGIS SQL. |
| [nl2sql-geospatial-benchmark](https://github.com/2023302141060/nl2sql-geospatial-benchmark) | 200-question NL2GeoSQL benchmark. |
| MapBench / MapReason-OSM / GeoLLM-OSM (papers) | VLM map-reading benchmarks; code scattered, mostly paper-driven. |
| [CityMind-Lab/Awesome-Location-Intelligence](https://github.com/CityMind-Lab/Awesome-Location-Intelligence) | Curated list of geospatial-representation-learning papers/datasets/code — good ongoing pointer. |
## 4. The mature building blocks (everyone builds on these)
- **Routing:** [OSRM](https://github.com/Project-OSRM/osrm-backend) (fast,
C++, table API), [Valhalla](https://github.com/valhalla/valhalla)
(multimodal, isochrones, matrix, elevation, TSP; open global server now
exists), [GraphHopper](https://github.com/graphhopper/graph-hopper)
(easiest to configure, car/bike/foot).
- **Geocoding:** [Nominatim](https://github.com/osmandapp/nominatim),
Photon (MeiliSearch-based, lighter).
- **Map data:** [Overpass API](https://github.com/danimw/overpass-api)
(query engine), `osm2pgsql` / [osmium-tool](https://github.com/osmcode/osmium-tool)
(ingest), Geofabrik extracts.
- **Opening hours:** `osm-opening-hours` (Python), `opening_hours` (JS),
`is-it-open-now` (JS) — the standard evaluators for the OSM hours language.
- **Frontend:** Leaflet / MapLibre GL, OSM tile servers (tileserver-gl).
## 5. Travel-itinerary agents (prior art for the itinerary framing)
All research-grade; none are OSM-native, none treat the itinerary as a
versioned state object, and their toolkits are synthetic sandboxes (or
Google Maps), not production geodata services:
- **TravelPlanner** (OSU-NLP-Group/TravelPlanner, ICML'24) — benchmark: 1,225
planning intents over ~4M records with a tool sandbox. The eval standard
for this task class.
- **TravelAgent** (jiangjiechen.github.io/publication/travelagent, 2024) —
LLM planning with four modules incl. itinerary generation & refinement;
paper-level.
- **AgentTravel** (OpenReview) — knowledge-augmented agent framework for
spatially-feasible itineraries.
- **Trip Weaver** (manolo-alvarez.github.io/projects/TripWeaver) — fine-tuned
small LLM for itineraries on consumer hardware.
- assorted LangGraph/LangChain multi-agent planner demos — thin, not robust.
## 6. Gap analysis — what does *not* exist yet
1. **No turnkey consumer product** "chat with your region's map": business
discovery + opening hours + routes in a chat/map UI. Everything is either
a demo, a tool server (MCP), or an analyst plugin.
2. **The MCP servers (2025 wave) are a signal**: the ecosystem standardized
on "OSM tools as MCP" but they are thin (hosted APIs, no local data,
no hours evaluation, no `along_route`, no semantic POI search, no eval).
We can *consume* osmmcp-style tools rather than rebuild them, or reuse
the Go code.
3. **Opening-hours-aware QA is underserved** — nearly no project treats
"open now / open when I arrive" as a first-class capability.
4. **Eval is research-only** — MapQA is a dataset; no project ships a CI
regression harness for map-QA answers.
5. **Hybrid structured + semantic POI retrieval** (PostGIS + vector index)
appears in the FOS4G 2026 comparison paper but no strong open-source
implementation of it.
6. **No production itinerary-refinement product over real OSM data** with a
deterministic feasibility validator and versioned diffs — the itinerary
agents above are benchmarks/papers, the map-QA projects are Q&A.
## Implications for our design
- **Reuse, don't rebuild:** routing/geocoding/hours libraries and even an
MCP server's tool set are off-the-shelf; our differentiators are (a) the
chat+map product, (b) hours-aware multi-hop tools (`along_route` + hours),
(c) hybrid retrieval, (d) evaluation.
- **Consider text-to-PostGIS as a second tool tier** (Text-to-sql /
PostGISer evidence): fixed tools for the common 90%, an LLM-generated
SQL escape hatch for the long tail, with a read-only DB role as the
safety boundary.
- **MapQA-dataset** is the natural benchmark to wire in from day one.
- **MCP as an interface option:** if we build our tool server in MCP form,
it works with Claude/Cursor/etc. for free *and* with our own agent —
low extra cost, high optionality.
Reference in New Issue View Git Blame Copy Permalink