trips/DESIGN.md

LLM Map Assistant — High-Level Design

An itinerary building & refinement tool: the user chats with the system to build and iteratively refine a multi-stop trip (sightseeing, food, errands, travel between places), and the system answers map questions — routes and travel times, businesses, opening hours, products & services — against authoritative geospatial data. The itinerary is a first-class, versioned state object that the LLM edits; free-form map QA is a sub-feature.

The system carries a trip through the whole planning lifecycle: idea generation & filtering → organization & planning (incl. multi-modal route planning) → changes & refinement → identification & selection of alternatives (see Planning lifecycle).

Deployment: web-first — serious planning happens on a desktop browser (map + timeline + chat, lots of screen). The mobile app is an on-trip companion: read the plan, navigate, check "open now?", make small edits — and it must keep working in areas with limited network coverage (see Interface & offline strategy). The LLM agent + tools + data live on a server; on-device inference is a later fallback, not a target (see Model selection).

Core principle (from the research)

Don't let the LLM "look at" the map. Have the LLM orchestrate deterministic geospatial tools and let the tools produce exact facts (distances, times, coordinates, hours). The LLM contributes:

• understanding and decomposing the user's question,
• choosing/sequencing tool calls (agent loop),
• filtering/ranking retrieved results,
• composing a natural-language answer.

Evidence:

• MapQA (arXiv 2503.07871): open-domain geospatial QA benchmark built on OpenStreetMap; question types include routing, POI attributes, spatial relations. A good evaluation set for us.
• FOS4G 2026 talk "A Systematic Comparison of RAG Architectures for Geographic POI Question Answering": structured/geotemporal RAG with tool access beats naive vector RAG for spatial POI queries.
• MapBench (arXiv 2503.14607) and MapReason-OSM: even strong VLMs fail at reliable path-finding/symbol reading on map images. ⇒ map images are for the user, not the engine.

Itinerary: the core state object

The product is itinerary building & refinement, so the itinerary is a typed, persisted document — not chat history. It is hierarchical: the top level spans countries/regions; each place (a country, city, neighbourhood — free-form, not a fixed category list) carries its own refined objective/"vibe" and contains its own days and stops.

{
  "id": "trip-2026-09",
  "version": 12,                      // every LLM edit = new version (undo, diff)
  "preferences": {                    // the evolving taste model (see Exploration loop)
    "likes": ["slow_mornings", "street_food", "coastal_walks"],
    "rules_out": ["theme_parks", "early_flights"],
    "budget": "moderate",
    "vibe_by_place": {"portofino": "relaxed, food-first",
                      "florence": "art, but not the big 4 museums"}
  },
  "dates": ["2026-09-12", "2026-09-20"],
  "modes": {"default": "foot", "car": false, "transit": true},
  "places": [{
    "id": "PL1",
    "name": "Tuscany — Florence & coast",   // free-form label, any granularity
    "centroid": [43.77, 11.25],              // its geo anchor (city or area)
    "bbox": [43.1, 10.0, 43.9, 11.9],        // for tiles/caching/planning
    "vibe": "art + food, relaxed pace",      // refined objective for this place
    "dates": ["2026-09-12", "2026-09-17"],
    "days": [{
      "date": "2026-09-12",
      // no per-day `base`: the anchor is RESOLVED from `stays` (below) — the
      // stay(s) covering this date. >1 candidate stay on the same date ⇒
      // worst-case anchoring for that date (see rules).
      "stops": [
        {"poi_ref": "P456", "arrive": "09:30", "depart": "10:15",
         "purpose": "visit",
         "suggested_duration": {"value": "02:30", "source": "search"},
         "planned_duration": {"value": "01:00", "source": "user"},  // sticky override
         "price": {"amount": 14, "currency": "EUR", "basis": "per person",
                   "source": "search"},
         "slot": "morning_visit",
         "kind": "point|region|transit",   // region = vague ("lunch in Oltrarno")
         "state": "planned|maybe|alt"},    // one `planned` per (day, slot)
        {"poi_ref": "P789", "arrive": "12:00", "depart": "13:30",
         "slot": "lunch", "kind": "point", "state": "planned",
         "purpose": "meal", "meal": {"kind": "lunch",
           "typical_duration": {"value": "01:15", "source": "search"},
           "typical_cost": {"amount": 25, "currency": "EUR", "source": "search"}}
      ],
      "legs": [                       // filled by router, never by the LLM
        {"from": "P123", "to": "P456", "mode": "foot",
         "duration": "00:12", "distance_m": 900, "geometry_ref": "G1"}
      ]
    }]
  }, {
    "id": "PL2", "name": "Portofino", "centroid": [...], "bbox": [...],
    "vibe": "beach + seafood, one slow day",
    "dates": ["2026-09-17", "2026-09-19"], "days": [...] 
  }],
  "transit": [                        // external transport: opaque, not OSM-routed
    {"type": "train",                 // flight | train | bus | car_rental | hotel | ferry …
     "from_place": "PL1", "to_place": "PL2",
     "date": "2026-09-17", "depart": "08:10", "arrive": "09:55",
     "ref": "Trenitalia 12345", "source": "user_booking",
     "status": "booked"},             // booked|planned|estimated — booked = hard anchor
    {"type": "car_rental",
     "place": "PL2",
     "pickup": {"at": "2026-09-18T15:00",
               "where": "Hertz, via Palazzuolo 12", "poi_ref": "P901"},
     "dropoff": {"at": "2026-09-21T10:00", "where": "…"},
     "ref": "Hertz conf. ABC123", "source": "user_booking",
     "status": "booked"}
  ],
  "stays": [                         // lodging = per-night records, NOT a trip attribute
    {"id": "ST1", "place": "PL1",
     "hotel": {"name": "Hotel Palagio", "poi_ref": "P123"},
     "check_in": "2026-09-12", "check_out": "2026-09-15",
     "ref": "conf. H-77821", "source": "user_booking",
     "status": "booked|candidate",   // candidates ⇒ worst-case anchoring for those nights
     "price_per_night": {"amount": 180, "currency": "EUR", "source": "booking"}},
    {"id": "ST2", "place": "PL2", "hotel": {"name": "hotel Y", "poi_ref": "P555"},
     "check_in": "2026-09-17", "check_out": "2026-09-19",
     "status": "booked"}
  ]
  // router legs (intra-place) may themselves be multi-modal:
  // {"mode": "multimodal", "sub_legs": [{"mode":"foot",…},{"mode":"bus",…}], …}
}

Rules that make this work:

• Hierarchy is data, not ceremony. A place is any {label, centroid, bbox, vibe, dates}. "Italy", "Florence" and "Oltrarno" are the same record shape; the planner just works at the granularity that has days. One country trip = one place; a neighbourhood focus inside it = a child place with its own days. No fixed category taxonomy anywhere.
• External transit is opaque, and bookings are hard anchors. Flights/trains/long hauls are not OSM-routed (rail/air networks aren't in OSM at usable fidelity): they're first-class transit objects with {type, window, source, status}. Anything the user has booked (dropped in as a ticket/confirmation — see Booking ingestion below) is a hard constraint: the planner re-anchors days and legs around it and may never move it, only work around it. car_rental objects additionally widen available modes (car legs allowed inside the pickup→dropoff window at that place). Intra-place movement stays in the router — including multi-modal legs (walk + bus/subway; OSM carries the transit network, GraphHopper/OTP/Valhalla compute the combined leg with sub-legs). The validator checks junctions (arrive PL2 09:55 → first stop 10:30 feasible; transit departs after last stop; car legs only inside the rental window).
• Lodging is a set of stays, not a trip attribute. A stay is {hotel, place, check_in, check_out, status, price_per_night}. A day's anchor resolves to the stay(s) covering its date — one city, one hotel = one stay; a multi-city trip = several stays, each anchoring its own dates. Multiple candidate stays covering the same dates are normal (comparing two hotels before deciding), and they trigger the rule below.
• Slots & states. Every stop belongs to a slot (a free-form mutual-exclusion group per day: lunch, dinner, afternoon_visit… — not a fixed taxonomy) and a state: planned (counts against the day's time budget), maybe (held, time-neutral), alt (candidate for the same slot, rendered as an "or"). One planned stop per (day, slot); choosing a second demotes the incumbent to alt (swap-in is one tap, nothing is lost). Removing the last planned of a slot auto-promotes a remaining alt. A stop's kind is point (decided POI), region (vague — "lunch near Oltrarno"; reserves the slot and time, anchored to a centroid+radius), or transit (booked external transport, a hard anchor). Region stops refine in place (the concrete stop replaces the placeholder at the same position).
• Worst-case anchoring (never plan on false precision). Any leg whose endpoint is a set — multiple candidate stays on that date, or a region stop — is budgeted at the farthest/worst member and flagged in the low-confidence tier ("worst-case until you commit/refine"). Committing a hotel or refining a region recomputes those legs to real values; because downstream decisions were made on worst cases, the plan stays valid. Days end with a return-to-stay leg (included in the budget) except on departure days.
• Exclusion list. "Not for me" / "no " writes to the preferences.rules_out-style list; it is applied to every future search (query level, before anything renders) and fed to the LLM as context.

Booking ingestion (drop a ticket, the plan adapts)

Users drag & drop real-world documents — train tickets, flight confirmations, car-rental emails (PDF, image, screenshot) — onto the app:

1. Parse (LLM): document → structured proposal {type, date, times, origin, destination, reference, carrier}. The LLM extracts; nothing is invented — fields the document doesn't state stay null and are asked for in the UI.
2. Ground: origin/destination names are geocoded to POIs/places; the proposal is matched against the itinerary (which place, which day, near which stop).
3. Propose: rendered as a suggested transit object (status booked) plus the planner's re-anchoring diff: which legs/days move around it, new mode availability (car), junction checks. Example: drop a rental-car confirmation "pick up 15:00, via Palazzuolo" → car appears available from 15:00, afternoon legs re-plan onto car, walking leg to the agency inserted.
4. Confirm: user accepts (commits via apply_edit) or corrects fields inline; the document's reference number is stored on the object and shown on the itinerary.

This is the mechanism by which the itinerary subsumes the user's real commitments instead of competing with them — the app plans around what's already booked, which is the state of the world on day 1 of any trip.

Hotels are special bookings: a lodging booking sets each day's base — the anchor that mornings depart from and evenings return to (drives first/last leg computation and "open near your hotel" suggestions). Hotels also enter the alternatives loop ("a quieter hotel near the old town?"), and changing the hotel re-anchors every affected day.

Entry via booking: the app's first screen is the "Where would you like to go?" prompt — and it doubles as a drop zone. Dragging in a booking (train/flight most common, hotel reservation works too) starts the app: parse → ground → the itinerary is born already anchored to the user's real commitments, with the places implied by the booking pre-selected.

• The LLM proposes, the tools decide. The LLM adds/moves/removes stops and places (POI refs, vibes, date ranges). Every leg (mode, duration, distance, geometry) is computed by the router and written back by a service, never by the LLM. Arrival/departure times are proposed by the LLM and validated by the checker (below); the checker can reject or correct ("Museum A opens 10:00, your 09:30 arrival is impossible — moved to 10:00").
• Deterministic validator after every edit (validate_itinerary):
  • opening-hours check of every stop against its scheduled time,
  • time-budget feasibility (arrive ≤ depart, legs sum to the day),
  • mode/time sanity (no 2 a.m. entries in closed venues, max walking distance per leg, etc.). The validator's warnings are fed back to the LLM and/or the user. This is the itinerary equivalent of the grounding rule for plain QA.
• Versioned diffs → the refinement UX. "Make day 2 more leisurely", "swap the gallery for a pub with live music", "we're staying 1 day longer" become edits that produce a new version; the UI renders old vs new (map + timeline) with accept/reject/undo. Refinement is where the product lives — cheap incremental edits, not full replans.
• Persistence & shareability: itinerary is a JSON doc in Postgres (+ map payload); exportable/shareable; re-validation is a pure function so a shared itinerary can be re-checked when data refreshes.

Suggestion engine: knowledge sources

Suggestions for places/vibes/stops are a three-source blend, in order of latency and trust:

1. OSM ground truth (instant, local): place_stats — what the place actually contains (tag clusters, coastline, parks, transit). Sets the floor: nothing is suggested the place can't deliver.
2. LLM world knowledge (instant): culture, seasonality, "what's worth it and why", rough quality rankings, hidden gems. Fast but may be stale or wrong.
3. Web search (seconds, server-side): augments 1+2 — current events ("restored 2026", "festival in September", closures, new openings), practical detail ("is the cable car running"), per-stop expectations (recommended dwell time, entry price, typical meal duration & cost — see Stop expectations below), and freshness checks on anything the LLM asserts confidently. Implementation: a search backend behind one interface (self-hosted SearXNG for privacy/free, or Tavily/Serper- class APIs for quality); queries derived by the LLM per suggestion, results cached per (place, topic, week), and cited — each search-derived enrichment carries its source URL, shown in the UI.

The blend is per-suggestion, not global: a "visit X" card may be 100% OSM+ LLM (no search needed), while "best time to see the aurora here" is search-heavy. Search augments; it does not gate (below).

Real-time, non-blocking UI (streaming enrichment)

Search is seconds-slow; the chat and cards must feel instant. Pattern:

• Two-phase rendering per turn. The LLM's fast-path output (OSM + world knowledge) renders immediately: vibe chips, suggestion cards with name/pitch/vibe, draft edits. Each card is fully interactive the moment it appears (tap to like/rule-out/add to plan).
• Async enrichment over SSE. While the user reads, search results stream back (server-sent events, one channel per conversation turn) and patch existing cards in place: added image, "opens Sep 2026", source link, a corrected fact. Enrichment never reorders, removes, or rewrites committed content; if a search result contradicts a rendered claim, the card shows the correction inline (note + source) and the next LLM turn sees the corrected state. User actions made while enrichment is in flight are valid — the server resolves conflicts against the itinerary version, not the search results.
• Progressive depth in chat: the assistant answers in layers — direct answer first, then "digging deeper…" details as search lands. The user can steer or cancel at any time; a fast reply the user acted on is never retracted silently.

Imagery in the UI

Pictures guide activity choices (a beach, a monument, a valley). Sourcing, cheapest-first:

1. Wikimedia Commons (free API, no key, per-image license + attribution) — keyed off the place/POI name; covers monuments, nature, cities well.
2. POI-level photos from OSM tags where present (image/photo tags).
3. Optional paid upgrade: Unsplash/Pexels API for consistent lifestyle shots.

Rules: an image is a card decoration with attribution + source link, fetched server-side and cached (keyed by POI/place), lazy-loaded, with a graceful fallback (category icon over a gradient) when nothing is found — an empty image slot must never block card interaction.

Exploration & preference loop (the fluid "where would you like to go?")

Idea generation is a conversation, not a form. The preferences block above is the state of that conversation, and it evolves through two input channels: chat text and UI selections (liking/ruling out suggestions, picking a vibe, dragging a suggested stop into a place).

• Seeding ("what is this place like?") is data-grounded. When the user names a destination, the server pulls its OSM tag statistics (top tourism/amenity/leisure clusters: beaches, trails, museums, nightlife, hiking, food culture; plus coastline/park area) — a small deterministic function — and the LLM turns that into vibe suggestions ("relaxed", "beach", "adventure", "food crawl") for that specific place, with web search (Suggestion engine above) asynchronously adding current events and seasonality ("harvest festival in September") as chips enrich. This kills the failure mode of suggesting "beach days" for a landlocked region: the chips are generated from what the place actually contains, with the LLM adding world knowledge on top.
• Suggestion loop: the LLM surfaces candidate places/days/stops in the UI as cards (not just chat text). Every user reaction — like, pass, "not that, something quieter", a free-text refinement — writes into preferences and the LLM regenerates/re-ranks. This is the same mechanism as the alternatives loop, applied upstream: generation is iterative constrained sampling, and the constraint set is user-observed, never invented.
• Progressive commitment: suggestions are candidate → user picks → tentative → planner fills in → confirmed. The trip takes shape top down (country → places → days → stops) but any level can be edited at any time; committing deeper doesn't lock higher levels.
• Free-form refinement field: always on. Its text goes to the LLM with the current preferences + current suggestion batch; the LLM's reply may include both prose and new suggestion cards. Nothing here requires predetermined categories — the only structured bits are the preference keys, which are open strings.

Stop expectations: dwell times, prices, meals

A bottom-level stop is more than a slot: it carries expectations that the plan is checked against:

• planned_duration (what's scheduled) vs suggested_duration (how long it should take, with source: user|osm|llm|search).
• price {amount, currency, basis (per person/family), source} — OSM fee/average_spend tags first, else background search, else labeled LLM estimate.
• Meal stops add meal {kind, typical_duration, typical_cost}.
• User overrides are sticky and authoritative. If the user sets 1 h at the Palazzo Pitti, source: user — the LLM never silently re-extends it; at most a one-time inline note ("guides usually suggest 2–3 h"), never nagging. The UI always shows planned vs suggested ("1 h (suggested ~2.5 h)") so the trade-off stays visible without being pushy.
• Background search fills missing expectations through the enrichment pipeline (Suggestion engine §3) whenever a stop is added — dwell, price, meal duration & cost — streaming into the card like any other enrichment, cited.
• Meals are a first-class planning target, not an afterthought:
  • purpose: meal stops carry a kind (breakfast/lunch/dinner). After an activity is confirmed, suggest_meals(day, window, near) actively finds restaurants: within a short walk of / along the corridor between the neighboring activities, open during the free window, whose typical_duration + travel fits the gap — each with cost and a one-line character note.
  • Day budgeting books meals at typical durations (lunch ≈ 60–90 min, not 45), so free time isn't silently eaten by eating.
  • Prices are collected per stop and rolled up in the background (per day and trip, by category) — not surfaced in the UI at this stage: no cost totals, no budget bars, no cost review issues. The data model keeps it ready to surface later without rework.

Plan review (overall & detailed — issues, not prose)

Layered, mirroring hard/soft:

1. Deterministic validator (existing): opening hours, feasibility, booked anchors, mode windows — blocking.
2. Expectation-aware checks (new, deterministic): planned vs suggested dwell (45 min booked for a meal typical at 90 → issue), price vs budget, daily walking distance, dead gaps, last-stop vs closing times.
3. LLM review — review_plan(scope: trip|place|day|stop): reads itinerary_summary + the computed expectation deltas and returns structured issues, not prose: {severity: error|warn|info, scope, stop?, kind (meal_too_short | pacing | fatigue | cost | hours_risk | …), message, fix_op?}. The LLM catches what heuristics can't (three museums in a row, dinner at 21:30 with the last train gone, a brutal transition day after a lazy one); the narrative lives in message.

Issues are a UI object: an issues panel (filter by scope/severity; click → zoom to the stop; one-click apply of the attached fix_op via apply_edit). Review runs on demand ("review day 2") and automatically after large edits (badge lights up when new warnings appear — never blocks). Every expectation cited in an issue carries its source, so the user can see whether "typically 90 min" came from search or a guess.

Itinerary tools (added to the catalog below)

• place_stats(location | place_ref) -> tag statistics Deterministic: OSM amenity/tourism/leisure clusters, coastline/park area for the place's bbox. Input to vibe generation — never invented by the LLM.
• suggest_vibes(place_ref) / suggest_places(dates, budget) — LLM-facing wrappers that emit structured suggestion cards (place: name, label, 1-line pitch, vibe, suggested days; stop: POI ref, pitch, trade-off). Cards render in the UI; user reactions are recorded server-side into preferences (the LLM never has to "remember" preferences).
• add_place(name | bbox, dates, vibe), remove_place(ref), move_place_dates(ref, dates) — hierarchical edits; adding a place auto-creates a transit stub from the previous place.
• suggest_meals(day, window, near_stop?) -> [restaurant cards] — nearby restaurants fitting the free window (open at that time, typical_duration
  • travel fits), each with cost, typical duration, hours, one-line character note; expectations background-filled by search.
• review_plan(scope: trip|place|day|stop) -> {issues[]} — deterministic expectation-aware checks + LLM review; issues carry severity, kind, message, and an optional one-click fix_op. See Plan review.
• transit_estimate(from, to, date, mode?) -> {windows, duration, source, cost?} — external rail/air lookup where available, else labeled estimate; never fabricated silently — source is always shown in the UI.
• plan_day(constraints, candidate_pool?) -> draft day — server-side heuristic planner (geo-cluster + hours + mode) that proposes a draft; the LLM reviews/edits instead of freehand-sequencing from scratch. Keeps the LLM's job in its comfort zone (taste/judgment) and the combinatorics in code.
• add_stop(day, poi_ref, slot?), move_stop(...), remove_stop(...), replace_stop(old_ref, constraints) — atomic edits; each triggers leg recomputation + validation.
• validate_itinerary(itinerary) -> {ok, warnings[], fixes[]}
• itinerary_summary(itinerary) -> compact text — the context form the LLM sees (POI names, slots, leg durations; not geometries), keeps the context window small on long trips.
• parse_booking(doc) -> transit proposal — LLM extraction from a dropped ticket/confirmation (PDF/image) → structured fields (nulls asked for, not invented) + geocoded origin/destination; feeds the Booking ingestion flow.
• apply_edit(op) — the single mutation endpoint for ALL itinerary changes, from chat or from the UI (both emit the same op vocabulary: move_stop, set_slot, set_leg_mode, set_route_via, add_stop, remove_stop, add_transit, …). Applies op → recompute affected legs → validate (booked transit = immovable anchors) → version bump → stream new state. Shared undo/redo lives here.
• recompute_legs(day) — router recomputation after edits (also called implicitly by apply_edit).
• web_search(topic, place_ref?, freshness?) -> [results with source URL, date] — the search backend (self-hosted SearXNG, or Tavily/Serper-class API behind the same interface); per-turn budget (e.g. ≤4 queries), cached per (topic, place, week).
• image_for(poi_ref | place_ref | query) -> {url, attribution, license} — Wikimedia Commons → OSM photo tags → fallback icon; cached.

The existing QA tools (geocode/search/hours/route) are the building-block tools the itinerary tools are composed from, and they also serve ad-hoc questions ("anything good near the castle?").

Planning lifecycle (functional scope)

Five capabilities, each mapped to the machinery above:

Phase	User does	System does	Key tools/mechanisms
0. Exploration	"where would you like to go?" + free-form refinement; like/rule-out suggestions	OSM-grounded vibe suggestions per destination; candidate place cards; every reaction updates preferences	OSM tag-statistics function + LLM, suggestion cards, preference state (see Exploration loop)
1. Idea generation & filtering	"we like history and coffee"	proposes candidate POIs (broad recall), surfaces hours/cuisine/tags, filters by constraints & interest match	search_poi, semantic vector path, poi_details, candidate pool with status=candidate
2. Organization & planning	"plan day 2 around these"	clusters geographically, sequences stops, multi-modal route planning (foot/bike/car/train legs, with transfer stops), schedules slots	plan_day, route (per-mode legs), distance_matrix, validator
3. Changes & refinement	"move the museum to the morning"	minimal-diff edits, re-derives affected legs & slots, re-validates	move_stop/add_stop/remove_stop, recompute_legs, versions
4. Alternatives	"anything better for lunch?", "what if it rains?"	generates ranked alternative sets (2–5 options with trade-offs: distance, hours, price, weather-fit) for the user to pick from — the LLM presents, the user decides, the choice commits one option	search_poi + distance_matrix + opening_hours composed into an alternatives payload rendered as selectable cards
5. On-trip execution	"where next?", "open right now?"	serves the synced itinerary, live re-validation (hours, traffic mode), small edits	read of synced itinerary, opening_hours, re-route

Notes:

• Multi-modal legs: a leg is {from, to, mode}; the router returns the best single-mode leg, and a separate transfer planner composes multi-leg chains (e.g. train + walk) with intermediate stops. Transfer chains are stored explicitly so refinement can swap a leg's mode without touching its endpoints.
• Alternatives are first-class UI, not chat text: an alternatives request produces a structured set (options + one-line trade-off rationale each); the user taps one and it commits. This keeps selection a human decision and keeps the LLM out of the final pick.
• Scope honesty for multi-country: intra-place = OSM routing + hours + POIs (strong); inter-place = opaque transit objects (estimates or external rail/air APIs) — the system plans junctions and day structure across countries, it does not pretend to book or exact-route the long hauls.

Data layer

Source of truth: OpenStreetMap (free, structured, has opening_hours, brand, amenity, shop, cuisine, opening_hours etc. per POI).

Component	Options	Notes
Raw map data	Geofabrik PBF extract for the region of interest	one-time download + periodic refresh
Spatial DB	PostGIS (Postgres)	import PBF via osmium/osm2pgsql; gives SQL spatial queries (radius, nearest, bbox)
Geocoding	Nominatim (local) or Photon	address ↔ coordinates; fuzzy place names
Routing	GraphHopper (easy multi-modal) or OSRM (fast, table API) or Valhalla	car/bike/foot/wheelchair profiles, distance + duration, turn instructions
Opening hours	osm-opening-hours lib (Go/Py/JS)	evaluates OSM opening_hours for "now" / "at time X", timezone-aware
(optional) live queries	Overpass API	for regions we don't keep locally; rate-limited

Why local over pure APIs:

• reproducible, fast, no rate limits, works offline;
• PostGIS lets the LLM's tools do real spatial math (e.g. "shops within 500 m of the route"), which no hosted API offers as a single call.

Alternative for v0: skip PostGIS entirely and use Overpass API + a hosted router (GraphHopper API / OSRM demo). Fine to start, but keep the interface so a local backend can be swapped in.

Multi-region note: with multi-country itineraries the data unit is per place (per-region PBF / router graph / tile cache), not one global load. Local per-city GraphHopper is fine; alternatively a hosted router serves any place. Inter-place transit is not OSM routing — it's external (rail/air search APIs where they exist, e.g. open rail timetables, else labeled LLM/user estimates) stored as opaque transit objects.

Architecture

┌────────────────────────────────────────────────────────────┐
│  Frontend: chat UI + map view (Leaflet / MapLibre GL)      │
│  (renders POIs, routes the LLM refers to; click → context) │
└──────────────┬─────────────────────────────────────────────┘
               │
┌──────────────▼─────────────────────────────────────────────┐
│  Agent layer (the LLM, chat + function calling)            │
│  - conversation state (user location, preferences)         │
│  - itinerary state (current version, constraints)          │
│  - tool selection & multi-step planning                    │
│  - answer synthesis with citations (POI ids / coords)      │
└──────────────┬─────────────────────────────────────────────┘
               │ tool calls (JSON)
┌──────────────▼─────────────────────────────────────────────┐
│  Tool / service layer (deterministic, no LLM)              │
│  geocode · search_poi · poi_details · nearest · route ·    │
│  distance_matrix · opening_hours · along_route ·           │
│  route_options · corridor · stop_cost · optimize_stops ·   │
│  place_facts                                               │
│  ┌────────────┬──────────────┬──────────────┬───────────┐  │
│  │  PostGIS   │  Router      │ Geocoder     │ Hours lib │  │
│  └────────────┴──────────────┴──────────────┴───────────┘  │
└────────────────────────────────────────────────────────────┘

Tool catalog (the heart of the system)

Each tool = a JSON-schema function the LLM can call. Keep them narrow and composable.

1. geocode(query) -> [name, lat, lon, type] Resolve "St. James Park", "my location", street numbers.
2. search_poi(name?, category?, tags?, near: {lat,lon,r} | bbox, limit) -> [pois] Category synonyms handled in the LLM prompt ("place to get coffee" → amenity=cafe). Returns id, name, tags of interest.
3. poi_details(poi_id) -> full tags Second-level lookup so list queries stay cheap.
4. nearest(category, from, max_results, max_distance)
5. route(from, to, profile=car|foot|bike|multimodal, via?, depart_at?) -> {distance, duration, geometry, steps, sub_legs?} — multimodal (walk+bus/subway, from OSM transit data) returns sub-legs; via[] waypoints make user-modified routes first-class (drag a waypoint, recompute, done). The same call backs the L3 route editor with live duration updates
6. distance_matrix(places[], mode) -> matrix For "which of these is closest / quickest".
7. opening_hours(poi_id | poi_ref, when=now|ISO-timestamp) -> {open?, closes, weekly}
8. along_route(route_ref, category, max_distance) -> pois "a petrol station along the way" — PostGIS buffer over route geometry. Driving trips upgrade this to corridor search — see below.
9. reverse_geocode(lat, lon) — for "what is near here".
10. route_options(from, to, profile, max_alts) -> [{ref, geometry, duration, tolls, via: [highways], diff_vs_fastest}] Fastest + user-relevant alternatives. A driving trip must not emit any time estimate before this has run — the route is computed, never narrated from LLM memory ("you'll cross Boston at 15:15" is a bug, not an answer).
11. corridor_from_route(route_ref, buffer_km, time_window?) -> corridor Spatial predicate (band around the geometry) + an along-route coordinate system (fraction / minutes-from-origin). User constraints — "south of origin", "in the first 2 hours", "on the way" — are filters in this coordinate space; direction is a fact, not a guess.
12. search_along_route(corridor, slot, tags, prefs) -> [{poi, along_route_pos, detour_min, facts}] OSM POIs inside the band ∪ LLM world-knowledge candidates projected into the band (a remembered trail outside the corridor is filtered or explicitly flagged "off route, +X min"). Exclusion list applied. Every candidate carries its computed detour cost — the sort key.
13. stop_cost(route_ref, stop | poi) -> {detour_min, total_added_min, new_route_ref} (route A→B via stop) − (route A→B) + dwell + declared overhead. The single most important number in trip planning: pure router arithmetic, no LLM. Surfaced as a chip ("+18 min to trip") and used by the validator (arrival-time constraints).
14. optimize_stops(route_ref, candidates[], k, time_budget) -> [ordered stops] Minimizes detour + Σ dwell + Σ overhead over ordered subsets of exactly k (exhaustive permutation search ≤ 10 candidates — also the benchmark ground truth — else greedy best-insertion). "Pick my two 45-min hikes" is this call, not an LLM freehand choice — the LLM frames candidates/tastes, the optimizer picks.
15. place_facts(poi_id) -> {distance, loop_length, elevation_gain, est_duration, dog_policy, parking, season_notes, hours, provenance[]} Sourced activity facts (OSM tags + cached source pages). The answer may only quote numbers that exist here.

Itinerary tools (plan_day, add/move/remove/replace_stop, validate_itinerary, itinerary_summary, recompute_legs) — see the Itinerary section above.

Design rules for tools:

• Return compact JSON; truncate large fields; include a stable id/ map_ref per POI so the frontend can highlight it and the LLM can refer to it without repeating all the data.
• Every tool returns coordinates so the answer is always renderable.
• Time is explicit: now resolved server-side with the location's timezone (IANA), passed through opening-hours evaluation.
• Provenance is enforced, not stylistic. Every time/cost/policy value carries a source tag: computed (router) / measured (OSM) / sourced (booking, official page) / assumed (LLM). A hard output rule: an assumed number may never be presented as fact — it either triggers a tool call or renders in the low-confidence tier. (Failure case that motivated this: a real planning session in which the route, distances and a 40-mile "detour" were all confidently asserted from LLM memory, and the "detour" turned out to be on the actual fastest route.)

Driving trips: the route is a first-class object

Requirements sharpened by a real exercise (Lincoln NH → Queens NY, "two 45-min hikes on the way") where the LLM-only approach failed on route, direction, distance and activity facts while the activity recommendations themselves (given correct spatial facts) were good. The lesson: bad facts poisoned good judgments, and the fix is structural:

1. R1 — Compute the route before speaking. A driving trip has an OD pair; route_options runs first, the user's route choice (fastest vs scenic vs toll-avoiding) becomes the versioned route object, and the corridor derives from it. No suggestion, ETA or "you'll cross X at Y" may be emitted before a route exists.
2. R2 — "Along the route" is a spatial primitive. Search is corridor-constrained by default; direction and position constraints ("south of origin", "first 2 hours") are filters on the corridor's along-route coordinate system, not LLM geometry.
3. R3 — Every stop has a computed cost. stop_cost (detour + dwell + overhead) is the display number, the sort key, and the input to the validator ("detour pushes past your 19:00 arrival"). Adding/removing stops re-derives the route and all costs through the existing mutation pipeline.
4. R4 — Provenance-gated output. See design rules above.

Model impact: the itinerary gains a versioned route per place/trip (geometry ref, profile, chosen-alternative); stops gain corridor_pos (minutes/fraction along route). The validator checks stop costs against user time constraints ("home by 19:00"). UI impact: stop cards get a +X min detour chip (computed tier); focus-mode pins are corridor- constrained by default; the budget bar counts detour minutes, not just dwell.

Evaluation hook: corridor-aware stop optimization is a cleanly bench-markable task class: given OD + k + time budget, ground truth = router-optimal stop set and total time; compare system output. Generate the benchmark from real OSM + router data (TravelPlanner covers leg planning but not this composite). Built: router/ (Go, stdlib-only) — route_options/stop_cost/optimize_stops/corridor over Valhalla or local OSRM, cmd/bench with 5 real NE-corridor tasks and recorded goldens (26 checks passing), integration tests asserting e.g. the fastest Lincoln NH → Queens NY drive stays ≥15 km from Boston.

optimize_stops semantics (implemented): the user asked for k stops, so the objective is total = detour + dwell + overhead over ordered subsets of exactly k. "At most k" without a budget degenerates (zero stops always wins — dwell only adds cost). With a time budget: best feasible exactly-k → best feasible smaller set (flagged relaxed) → best exactly-k ignoring the budget (flagged infeasible). Exhaustive permutation search ≤ 10 candidates (this is the ground truth), greedy best-insertion above.

Retrieval strategy (hybrid)

• Structured path (primary): tools 1–9. Exact, cheap, verifiable.
• Vector path (secondary): embed the POI text (name + tags + any free text) once per POI, index in Postgres pgvector (or SQLite-vec for v0). Used for fuzzy semantic queries the tag vocabulary can't express: "a place that sells artisan sourdough", "quiet pub with a garden". Semantic search returns candidates → poi_details → LLM decides.
• Rerank candidates by distance (PostGIS) before handing to the LLM so the context window always holds the nearest, most relevant few, not ten random hits.

Agent loop

Standard function-calling loop, with the itinerary state injected as context (via itinerary_summary, not the raw doc):

user question
  → LLM (system prompt: you are a map assistant; user location; tool list;
     answer only from tool results; cite POIs; give times as durations,
     don't invent)
  → tool call(s), possibly parallel (geocode A, geocode B)
  → results back → more calls or final answer
  → answer + structured payload {pois:[refs], route: {geometry}}

• Grounding rule: the LLM must never state a travel time/distance/hours value that didn't come from a tool result. This is the main guard against hallucination and the reason the architecture matters.
• Multi-hop questions work naturally: "Is the best-rated bakery near my home open right now?" → search_poi → route (optional) → opening_hours → answer.
• Cap the loop (e.g. 6 tool rounds) and fall back gracefully.
• Two operating modes: QA (question → answer, no state change) and refine (answer plus an itinerary edit). In refine mode the turn commits as a new itinerary version; the validator runs between the edit and the user sees anything.

Interface (web-first; mobile on-trip)

Quality bar: beautiful, responsive, flexible — the map is the hero, the chat is a tool, and every LLM suggestion must be visually inspectable and reversible.

Web app (primary, desktop-optimized):

• Layout: map (hero, ~60–70%) + day timeline rail + chat pane (collapsible to a bottom drawer on narrow screens). Responsive down to tablet/phone — same app, panels reflow; it is not the on-trip app.
• Timeline rail: days → stops in sequence with slots, leg chips (mode icon + duration); drag a stop to reorder (triggers recompute_legs + validator), hover a leg to highlight its geometry.
• Alternatives & candidates render as cards (photo/tagline, hours, trade -offs) alongside the map, with one-tap "put in plan".
• Every agent turn carries a structured payload (POI refs, geometries, itinerary diff). Refine-mode edits render as an accept/reject/undo diff (map + timeline animate old→new); rejection feeds back to the LLM as conversation context.
• Power features for desktop: compare two versions, pin a POI as a conversation anchor, filter layers (hours, price, mode), export (PDF/ICS/GPX), split-screen two days.
• Issues panel: review_plan output lives here — filter by scope/severity, click an issue to zoom to its stop, one-click apply of the suggested fix-op; badge lights up when an edit introduces new warnings (never blocks). Stops show planned vs suggested time ("1 h (sug. ~2.5 h)") and price chips with source attribution.

Focus mode (slot search — the Google-Maps pattern). "Where should we lunch? What for the afternoon? Which hotel?" is a different mental mode from editing the plan, and the UI says so:

• Triggered by chat — "find a lunch spot", "lunch near Oltrarno", "alternative hotels", "find another activity" — and works whether the slot is empty, vague, or already decided: search is always re-openable, and choosing is a swap (incumbent → alt, chosen → planned), so any decision can be revisited later.
• The rail dims; the map becomes the hero with name-labeled teardrop pins on the candidates and nothing else. Click a pin → popup: pitch, distance from the day's anchor (the previous stop — where the user will actually be) and to the hotel, duration, price, actions.
• Chat becomes the filter bar: "cheaper", "near Boboli", "views", "vegan", "no Mario" — pins re-filter live; each reply names the closest remaining option to the anchor. Filters are slot-aware ("cheaper" means ≤ €25 for lunch, ≤ €220 for a hotel).
• Actions per pin: Choose this (slot swap), Maybe, for hotels Keep as option (adds a candidate stay for those nights ⇒ worst-case anchoring) and Make this the hotel (commit ⇒ re-anchor, real legs), and More like this (re-query on the candidate's own tags).
• Candidates come from the 3-source engine (OSM ground truth → LLM world knowledge → web enrichment); the exclusion list is applied on entry.

Mobile (web-first, usable from day one). Below ~860px the 3-pane grid collapses to a single pane with a bottom tab bar (Plan | Map | Chat): each pane full-width, the topbar wraps to two rows with horizontally scrollable day tabs, and the map re-fits to the full viewport. Same app, same state — not a separate mobile build. (A native on-trip shell with an offline trip bundle is a later phase, not a prerequisite.)

Direct editing is first-class — chat is one editor, not the only one. Anything the LLM can do to the itinerary, the user can do by hand, and both paths funnel through the same mutation pipeline:

• Every change — LLM or human — is an atomic edit op (move_stop, set_slot, set_leg_mode, set_route_via, add_stop, …) sent to the server's single apply_edit endpoint. The server recomputes affected legs, runs the validator, bumps the version, and streams back the new state. The LLM never gets a fast path around validation, and a user drag never bypasses leg recomputation.
• Shared undo/redo stack across both editors (each op is a version delta); an LLM suggestion the user rejects is just an unapplied op.
• Hand-editing affordances: drag stop reorder / drag across days; inline slot-time editing (typing a time or dragging on a time ruler — later slots cascade and the validator flags overruns live); per-leg mode switcher (foot/bike/car) with instant duration update; route editing: drag waypoints on the map (or click to add them) to alter a leg's path — for a hike, snap to the actual trail — and duration/distance recompute live; add a stop by clicking the map (reverse-geocode → search → pick); remove/duplicate stops; per-place day-length and budget settings.
• While the user is editing, chat suggestions for the same stops are dimmed, not force-updated (the UI never fights the user's hand).

Time is the product's core value — a first-class display object, not metadata.

• Every stop shows its duration; every leg shows mode + duration (distance on hover); each day shows planned total vs available as a budget bar (7.5 h planned of 11 h waking; amber/red on overrun); places and the whole trip roll the totals up.

• Travel times are never LLM-estimated. Leg durations come from the router (mode-specific profiles) or from explicit transit objects (scheduled or labeled estimate). Dwell times (how long at the museum) are a different beast: they carry planned vs suggested with source labels, and user-set values are authoritative (see Stop expectations). All estimates render with a "~" (dashed chip), distinct from computed values.

• Confidence hierarchy — every time/number estimate carries a level, and the UI encodes each level distinctly:

  Level	Source	UI indicator
  5 (top)	user-set	filled chip + pin/lock glyph
  4	scheduled (booked transit / hotel)	filled chip + ticket glyph
  3	router-computed	plain filled chip
  2	search-derived	"~" + source link
  1 (lowest)	LLM estimate	dashed chip, labeled "estimate"

  Rules: a lower-confidence value never silently replaces a higher one (user overrides are sticky — §Stop expectations); background enrichment that raises confidence (1→2, 2→3) streams into the UI as an update with a small "updated" pulse, never a silent rewrite.

• Time changes animate: a recompute that moves the afternoon is visible (slots slide on the rail, the budget bar moves), so the user always sees the consequence of an edit in minutes, not just in words.

Multi-scale navigation (one UI, four zoom levels):

Level	View	Map shows	Rail shows
L0 Trip	all places + transit	places as nodes, transit arcs, day counts	places → days as a strip
L1 Place / Day (primary working view)	one place or one day	stops + intra-place legs	stops with slots + leg chips
L2 Time window	a slice of a day ("after lunch")	that subset	the subset, rest dimmed
L3 Leg / Activity	a single stop or a single leg	the route editor: the path with draggable waypoints (hike: snap to the trail; urban: walk-only, or multi-modal walk+bus+subway with the router's sub-legs shown as segments), distance + live-updating duration as the user edits it, optional elevation profile (GraphHopper elevation API)	day context, collapsed

Drill down by click (place → day → time window → leg/stop), back via breadcrumb; map zoom and rail scope stay in sync. L3 is the generic route editor — a hike is just a foot-only instance of it; so is a walk through the city, or a "walk + bus + metro" chain (the router returns the multi-modal leg with sub-legs; the user can drag waypoints on any of them or swap a sub-leg's mode and the duration updates live). It must feel instant: leg recompute for short legs is tens of ms on the local router.

The mobile on-trip app uses the same hierarchy (L0–L3) with direct editing scoped to today (reorder, skip, route-edit) and read-mostly elsewhere.

Mobile app (on-trip, secondary):

• Modes: today (current day as a checklist timeline + map, live "open now" badges), navigate (leg-by-leg with mode-aware guidance), ask (chat — server LLM when online), edit small (reorder today's stops, drop a stop; server re-plans legs when connectivity allows).
• Offline trip bundle — the key on-trip requirement (limited coverage on the trip itself):
  • On "prepare for trip" the phone downloads: the itinerary + full stop POI details (hours, tags, contact), vector map tiles for the trip bbox (MapLibre + locally generated tiles, e.g. via osm2vsmartin, or raster MBTiles), and leg geometries with turn-by-turn instructions.
  • Offline the app serves: itinerary, maps, stop details, hours evaluation (the hours library runs fully offline — pure function of local time), and walking/bike re-routing within the trip area if we ship a small local router (GraphHopper can preload a small region; treat as a v2+ stretch — offline re-route degrades gracefully to "show saved leg").
  • Sync model: bundle = snapshot (itinerary version N + tile version); edits made offline queue and reconcile on reconnect (itinerary is a versioned doc, so a simple last-writer + server re-validation works); re-validation on sync flags stops whose hours/legs changed.
  • No LLM offline (except the optional on-device 4B for "is X open?" — see Model selection); the offline experience is plan + navigate, not re-plan.

Hard sub-problems to plan for

1. Opening hours: OSM coverage is patchy (especially for small towns); answer honestly ("no hours listed") instead of guessing.
2. Timezones: evaluate hours in the place's timezone, show the user their timezone.
3. "Open now" at the route level: along_route + opening_hours composition is a differentiator.
4. Ambiguity: geocoding "Main Street" → ask a clarifying question or show top-3 candidates in the UI (structured, not just text).
5. Staleness: schedule PBF refresh (e.g. weekly); show data timestamp.
6. Offline bundles: tile size vs bbox (vector tiles keep this small; a city at z14 ≈ a few hundred MB raster, much less vector); offline-edit conflict resolution (versioned doc + server re-validation is the simple correct answer); bundle validity (re-validate on reconnect).
7. Time is the value proposition: calibrate router profiles per mode where possible; always distinguish scheduled (transit), computed (router) and estimated (LLM/user) times in the UI; L3 route-edit recompute latency must stay <~100 ms for short legs (local router, preloaded graph).
8. Two editors, one truth: chat and hand-editing share apply_edit — conflict resolution is by itinerary version (last op wins, validator re-checks), never by "who asked first".
9. Booking-document parsing: tickets/confirmations arrive as PDFs, screenshots, and carrier-specific layouts — LLM extraction needs per-field confidence (show "did we read the platform right?" for doubtful fields), and station/agency names need robust geocoding ("near via Palazzuolo" → closest matching POI with user confirmation). Car-rental windows are easy to get subtly wrong (timezone, date line) — always echo the parsed window back for confirmation.

Evaluation

• MapQA (arXiv 2503.07871) — 3,154 QA pairs from OSM (Southern California, Illinois): routing + POI attribute questions. Great regression suite for the QA sub-feature.
• TravelPlanner (ICML'24, OSU-NLP-Group/TravelPlanner) — 1,225 curated travel-planning intents against ~4M records with a tool sandbox; the standard benchmark for exactly this task class (constrained, multi-day, transportation + meals + attractions + lodging). Use its constraint-checker idea for our itinerary validator.
• Build a small local eval set over our region with golden itineraries; score: (a) tool-answer correctness (deterministic, easy), (b) validator pass-rate of LLM-edited itineraries (should be high after validation), (c) LLM faithfulness to tool results (LLM-as-judge + spot checks), (d) refinement quality: does a 1-instruction edit touch only the intended stops (diff-locality metric).
• Track: question types, tool-call success rate, hallucinated-number rate (should be ~0), median latency.

Model selection (server-first, on-device as fallback)

Server (primary): any strong function-calling model behind one swappable interface. Candidates from current benchmarks:

• Qwen3.8-27B (dense, ~24 GB) — first choice for this system: it runs well on our hardware. The 27B class buys real headroom exactly where this workload stresses models: long context (whole-trip summaries across places/days), multi-step planning with many tool rounds, and taste-level suggestion writing. Good general/search reasoning; native tool calling.
• Granite 4.2-8B (Aug 2026): dense, Apache 2.0, native OpenAI-format tool calling, agentic RL training, switchable thinking. Strong fit for the refine-loop workload at lower compute; run at Q4 or FP16, thinking off on simple tool turns.
• Qwen3-30B-A3B-Instruct-2507: MoE with 3B active — 8B-class capacity at 3B-class latency; great if we serve on modest GPUs.
• Any hosted API (Claude/GPT-class) as the "quality ceiling" option and for LLM-as-judge in eval.

On-device fallback (optional, later — e.g. offline trip planning on a phone):

• Qwen3-4B-Instruct-2507 (dense 4B, Q4 ≈ 2.5–3 GB) — the reference small tool-calling model; note it is the 4B, not a 3B (the "3B" in the family is the 30B-A3B MoE's active count).
• Granite 4.2-3B — new contender with the same agentic-RL pedigree; worth a head-to-head on our MapQA subset (don't trust vendor BFCL numbers — Granite 4.1-8B's were panned by the community).
• Phi-4-mini (3.8B) — fastest of the class, best if latency dominates.
• Sub-2B models: not reliable agent brains (tool JSON breaks); acceptable only as an intent classifier in front of the main model.
• Quantization floor: never below 4-bit — structured output (tool calls) degrades fastest under aggressive quantization (QuantCall benchmark; Q3 already shows 3–8% PPL + visible reasoning regression), and a 2–3-bit 8B is strictly worse than a 4B at Q4 for the same footprint.

Reality check that shaped the server-first choice: an on-device 4B agent loop takes ~30–60 s per multi-hop question (each tool call is a 100–300 token generation at 8–15 tok/s). Fine as an offline fallback for "is this open?"-class questions; the itinerary planner belongs on the server.

Build plan

Pre-v0 (mock, 1–2 days): before backend work, a static interactive mock of the web UI in the real frontend stack, with fake data behind the same interface the API will implement: map + timeline rail + chat; vibe chips; the landing screen ("Where would you like to go?" + free-form field

• drop zone, incl. a mocked booking drag-in → parse → anchored plan); suggestion cards with simulated streaming enrichment (image + citation appearing after a delay); accept/reject diff animation; like/rule-out reactions; drag-reorder a stop and watch leg durations + the day budget bar update live; L3 route editor (hike: draggable trail with live duration; urban: multi-modal walk+bus leg); drop a (mocked) train ticket → parsed transit object pinned, day re-anchors around it; a stop card showing planned vs suggested time + price; issues panel with one mocked warning ("lunch 45 min, typically ~90 at this venue") and a one-click fix; L0/L1 drill-down (mocked data only). Purpose: validate the interaction model, settle the visual design language, and produce the API data contract by construction. The mock becomes the app shell — only the fake data layer gets swapped for the real server.

v0 (days, all-hosted APIs): chat + Leaflet + day timeline; the hierarchical skeleton from day one (trip → place → days → stops, start with one place); exploration loop: destination → OSM tag stats → vibe chips → suggestion cards → preferences; single-day itinerary over Nominatim (demo) + Overpass + GraphHopper public API + opening-hours lib; QA mode + add/move/remove stop + validator. Validation: the fluid "where would you like to go → vibe → place plan" flow works end-to-end for one city.

v1 (local data + multi-place): PostGIS + local GraphHopper + local Nominatim for the target region; multi-day + multi-place itineraries with opaque transit and junction validation; plan_day heuristic planner, versioning/undo, semantic POI index (pgvector); TravelPlanner-style constraint eval in CI.

v2 (multi-country + mobile on-trip): per-place data loading for true multi-country trips; external rail/air transit APIs; the mobile app with today-view, leg navigation, offline trip bundle (tiles + itinerary + stop data + hours), and offline-edit sync; alternatives UI (structured option cards) in web; clarifying questions; "open on arrival" filters; MapQA

• local eval in CI; optional VLM: send the rendered map view to a VLM only when the user shares/asks about a screenshot; optional on-device 4B for offline "is X open?".

Tech stack suggestion (aligns with your existing Go tooling)

• Go service: agent loop, tools, HTTP API (chi), Postgres driver with PostGIS, GraphHopper/OSRM client, osm-opening-hours equivalent (or shelling to the Python lib / a small sidecar).
• LLM: any function-calling model; keep it behind one interface so the provider is swappable (including a local model for offline use).
• Frontend: small SPA (or Go + templ + htmx if you want minimal) with Leaflet.

References

• MapQA — arxiv.org/abs/2503.07871
• RAG architectures for POI QA — talks.osgeo.org/foss4g-2026/talk/VBGM3W
• MapBench (VLM map reading limits) — arxiv.org/abs/2503.14607
• MapReason-OSM — arxiv.org/abs/2606.22597
• LAMP: A Language Model on the Map — arxiv.org/abs/2403.09059
• OSRM API — project-osrm.org/docs
• Overpass API — wiki.openstreetmap.org/wiki/Overpass_API
• GraphHopper — github.com/graphhopper/graph-hopper
• GROKE (OSM graph reasoning, ACL'26) — github.com/Geo-R2LLM/groke
• TravelPlanner (ICML'24 travel-planning benchmark) — github.com/OSU-NLP-Group/TravelPlanner
• Granite 4.2 (IBM open reasoning models, Aug 2026) — ibm.com/granite/docs/models/granite4-2
• Qwen3-2507 refresh (Instruct/Thinking, 4B–235B) — github.com/QwenLM/Qwen3
• TravelAgent (LLM travel planning, 2024) — jiangjiechen.github.io/publication/travelagent
• QuantCall (quantization effect on tool calling) — github.com/Happynood/quant-toolcall-bench