traffic: live-traffic pipeline (511NY → Overpass match → OSRM segment speeds)
Verified end-to-end on the local OSRM stack: - internal/traffic: 511NY OData client (lenient decode, raw archive), Overpass way matcher (class-window selection: highest-class road within 150m beats a closer service road), incident policy map, CSV builder - cmd/traffic: fetch | match | apply; apply copies the dataset (CoW) and re-customizes the traffic layer in ~15s (v26 customize has no --output-prefix; it writes contract files in place) - Proof: synthetic I-90 closure → 38s (static :5000) vs 142s (traffic :5002) on the same 0.87 km stretch - Policy speeds are 'assumed' provenance; R4: traffic routes must carry computed(traffic, as_of, policy=assumed)
This commit is contained in:
parent
916f9eb503
commit
fc9cf44052
|
|
@ -18,6 +18,8 @@ internal/plan/ the primitives:
|
|||
- optimize.go: optimize_stops (exhaustive ≤10 stops, greedy above)
|
||||
- corridor.go: corridor = buffer band around the route
|
||||
cmd/routectl/ manual CLI: route | stopcost | optimize
|
||||
internal/traffic/ live-traffic pipeline (511NY → Overpass match → speeds)
|
||||
cmd/traffic/ fetch | match | apply (segment-speed CSV → re-customize)
|
||||
cmd/bench/ benchmark runner (bench/tasks.json)
|
||||
bench/tasks.json 5 real NE-corridor tasks with recorded goldens
|
||||
scripts/setup-osrm.sh self-host OSRM over the NH+MA+CT+NY extract
|
||||
|
|
@ -136,6 +138,55 @@ arrival-time math must use one backend consistently per trip and say
|
|||
which; (c) "computed" provenance should carry the profile name,
|
||||
because two computed numbers for the same leg can differ by 20–25%.
|
||||
|
||||
## Live traffic (511NY → OSRM segment speeds) — verified
|
||||
|
||||
OSRM v26 has a first-class mechanism for this: `osrm-customize
|
||||
--segment-speed-file speeds.csv` (CSV = `nodeA,nodeB,km/h`, OSM node
|
||||
pairs) rewrites edge weights and re-customizes the MLD in **~15 s — no
|
||||
re-extract**. The traffic layer is a renamed copy of the dataset
|
||||
(customize writes the contract files in place; v26 has no
|
||||
`--output-prefix`), served by its own `osrm-routed` process.
|
||||
|
||||
`internal/traffic` + `cmd/traffic` implement the whole loop:
|
||||
|
||||
```sh
|
||||
# 1. pull incidents (needs a 511NY key — 511ny.org → Sign Up →
|
||||
# account → API key; the key can't be requested programmatically)
|
||||
go run ./cmd/traffic fetch --key "$FIFTYONE_NY_KEY" --out events.json --raw raw.json
|
||||
|
||||
# 2. match each incident to OSM node pairs (Overpass: ways around the
|
||||
# point; within 150 m the highest-class road wins, so "I-90" beats a
|
||||
# closer service road) and expand to the affected stretch
|
||||
go run ./cmd/traffic match --events events.json --out speeds.csv
|
||||
|
||||
# 3. copy the base dataset (CoW) + re-customize the traffic layer
|
||||
go run ./cmd/traffic apply --csv speeds.csv --data $HOME/osm-build/data
|
||||
# → serve: osrm-routed --algorithm mld --port 5002 $HOME/osm-build/data/northeast-traffic.osrm
|
||||
```
|
||||
|
||||
Verified end-to-end: a synthetic `Road Closed` on a 0.87 km I-90
|
||||
stretch routed at 38 s (static `:5000`) becomes 142 s on the traffic
|
||||
layer (`:5002`) — same path, closure speed. `match` was tested against
|
||||
live Overpass data (real OSM node IDs; class-window selection; unknown
|
||||
event types ignored).
|
||||
|
||||
Notes:
|
||||
- **Incident policy speeds are `assumed`** (policy map in
|
||||
`internal/traffic/policy.go`), not measurements. Routes from the
|
||||
traffic layer must carry provenance
|
||||
`computed(traffic, as_of <fetch time>, policy=assumed)` per DESIGN.md
|
||||
R4 — and the `as_of` age is a product-visible confidence signal.
|
||||
- **Closures at 5 km/h are slow, not blocked.** True blocking needs the
|
||||
`--turn-penalty-file` mechanism (untested) or a graph edit; 511NY
|
||||
"Road Closed" events should probably map to blocking, not 5 km/h.
|
||||
- **Free coverage is patchy**: 511NY (NY) + MassDOT (MA, separate
|
||||
application) give incident-level data; there is no free
|
||||
floating-car feed, so per-leg traffic coverage is a confidence tier,
|
||||
not a boolean.
|
||||
- 511NY API: base `https://511ny.org/api`, OData-v3 shape
|
||||
(`{"d":[...]}`), 10 calls/min throttle. `fetch` decodes leniently and
|
||||
archives the raw body for schema confirmation on first live run.
|
||||
|
||||
## Design notes
|
||||
|
||||
- **The route is computed, never narrated.** No function in this module
|
||||
|
|
|
|||
236
router/cmd/traffic/main.go
Normal file
236
router/cmd/traffic/main.go
Normal file
|
|
@ -0,0 +1,236 @@
|
|||
// Command traffic runs the live-traffic pipeline:
|
||||
//
|
||||
// traffic fetch --key K [--out events.json] # pull 511NY incidents
|
||||
// traffic match --events events.json [--out speeds.csv] # Overpass-matched node-pair speeds
|
||||
// traffic apply --csv speeds.csv --data <dir> # osrm-customize with the speed file
|
||||
//
|
||||
// `fetch` needs a 511NY developer key (511ny.org → Sign Up → API key).
|
||||
// Until a key is available, `match`/`apply` work from any events JSON in the
|
||||
// shape FetchEvents emits (or by hand).
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"flag"
|
||||
"fmt"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"maps/router/internal/traffic"
|
||||
)
|
||||
|
||||
func main() {
|
||||
if len(os.Args) < 2 {
|
||||
usage()
|
||||
}
|
||||
var err error
|
||||
switch os.Args[1] {
|
||||
case "fetch":
|
||||
err = cmdFetch(os.Args[2:])
|
||||
case "match":
|
||||
err = cmdMatch(os.Args[2:])
|
||||
case "apply":
|
||||
err = cmdApply(os.Args[2:])
|
||||
case "help", "-h", "--help":
|
||||
usage()
|
||||
default:
|
||||
usage()
|
||||
}
|
||||
if err != nil {
|
||||
fmt.Fprintln(os.Stderr, "error:", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
func usage() {
|
||||
fmt.Fprint(os.Stderr, `traffic: live-traffic pipeline (511NY → OSM match → OSRM segment speeds)
|
||||
|
||||
traffic fetch --key K [--out events.json] [--raw out-raw.json]
|
||||
traffic match --events events.json [--out speeds.csv] [--radius 3000]
|
||||
traffic apply --csv speeds.csv --data /path/to/data
|
||||
`)
|
||||
os.Exit(2)
|
||||
}
|
||||
|
||||
func cmdFetch(args []string) error {
|
||||
fs := flag.NewFlagSet("fetch", flag.ExitOnError)
|
||||
key := fs.String("key", envOr("FIFTYONE_NY_KEY", ""), "511NY developer API key")
|
||||
out := fs.String("out", "events.json", "normalized events output")
|
||||
raw := fs.String("raw", "", "archive raw response (optional)")
|
||||
fs.Parse(args)
|
||||
if *key == "" {
|
||||
return fmt.Errorf("no --key (get one: 511ny.org → Sign Up → account → API key)")
|
||||
}
|
||||
events, body, err := traffic.FetchEvents(*key)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if *raw != "" {
|
||||
if err := os.WriteFile(*raw, body, 0o644); err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Fprintf(os.Stderr, "raw response → %s\n", *raw)
|
||||
}
|
||||
valid, matched := 0, 0
|
||||
for _, e := range events {
|
||||
if e.Valid() {
|
||||
valid++
|
||||
}
|
||||
if p, ok := traffic.PolicyFor(e.Type); ok {
|
||||
matched++
|
||||
_ = p
|
||||
}
|
||||
}
|
||||
data, _ := json.MarshalIndent(events, "", " ")
|
||||
if err := os.WriteFile(*out, data, 0o644); err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Printf("fetched %d events (%d located, %d with traffic policy) → %s\n", len(events), valid, matched, *out)
|
||||
return nil
|
||||
}
|
||||
|
||||
func cmdMatch(args []string) error {
|
||||
fs := flag.NewFlagSet("match", flag.ExitOnError)
|
||||
eventsPath := fs.String("events", "events.json", "normalized events JSON")
|
||||
out := fs.String("out", "speeds.csv", "segment-speed CSV output")
|
||||
radius := fs.Float64("radius", 3000, "Overpass search radius (m)")
|
||||
overpass := fs.String("overpass", "", "Overpass endpoint (default: public)")
|
||||
timeout := fs.Int("timeout", 15, "Overpass query timeout (s)")
|
||||
fs.Parse(args)
|
||||
|
||||
raw, err := os.ReadFile(*eventsPath)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var events []traffic.Event
|
||||
if err := json.Unmarshal(raw, &events); err != nil {
|
||||
return fmt.Errorf("events json: %w", err)
|
||||
}
|
||||
|
||||
var assignments []traffic.Assignment
|
||||
skipped, noPolicy := 0, 0
|
||||
for i, e := range events {
|
||||
if !e.Valid() {
|
||||
skipped++
|
||||
continue
|
||||
}
|
||||
p, ok := traffic.PolicyFor(e.Type)
|
||||
if !ok {
|
||||
noPolicy++
|
||||
continue
|
||||
}
|
||||
ways, err := traffic.QueryWaysAround(nil, *overpass, e.Lat, e.Lon, *radius, *timeout)
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "event %d (%s): %v\n", i, e.Type, err)
|
||||
skipped++
|
||||
continue
|
||||
}
|
||||
way, a, b, dist, ok := traffic.ClosestNodePair(ways, e.Lat, e.Lon)
|
||||
if !ok {
|
||||
fmt.Fprintf(os.Stderr, "event %d (%s): no highway way found\n", i, e.Type)
|
||||
skipped++
|
||||
continue
|
||||
}
|
||||
pairs := traffic.ExpandToRadius(way, a, b, p.RadiusM)
|
||||
if len(pairs) == 0 {
|
||||
pairs = [][2]int64{{a, b}}
|
||||
}
|
||||
fmt.Fprintf(os.Stderr, "event %d: %q @ %s (%s, %dm to %s/%s) → %d edges @ %d km/h\n",
|
||||
i, e.Type, e.City, way.Ref, int(dist), way.Highway, e.Type, len(pairs), p.SpeedKmh)
|
||||
for _, pr := range pairs {
|
||||
assignments = append(assignments, traffic.Assignment{Pair: pr, SpeedKmh: p.SpeedKmh, Weight: p.Weight})
|
||||
}
|
||||
// be gentle with the public Overpass instance
|
||||
time.Sleep(500 * time.Millisecond)
|
||||
}
|
||||
|
||||
csv := traffic.BuildCSV(assignments)
|
||||
if err := traffic.WriteSpeedFile(*out, csv); err != nil {
|
||||
return err
|
||||
}
|
||||
lines := strings.Count(csv, "\n")
|
||||
fmt.Printf("matched: %d speed rows → %s (skipped %d no-location, %d no-policy)\n", lines, *out, skipped, noPolicy)
|
||||
return nil
|
||||
}
|
||||
|
||||
func cmdApply(args []string) error {
|
||||
fs := flag.NewFlagSet("apply", flag.ExitOnError)
|
||||
csv := fs.String("csv", "speeds.csv", "segment-speed CSV")
|
||||
data := fs.String("data", "/home/gmp/osm-build/data", "OSRM data dir")
|
||||
base := fs.String("base", "northeast", "base dataset prefix (northeast, northeast-us)")
|
||||
prefix := fs.String("prefix", "traffic", "suffix for the traffic layer (base-prefix)")
|
||||
binary := fs.String("osrm", "/home/gmp/osm-build/build/osrm-customize", "osrm-customize binary")
|
||||
skipCopy := fs.Bool("skip-copy", false, "reuse an existing base-prefix dataset")
|
||||
fs.Parse(args)
|
||||
if _, err := os.Stat(*csv); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// OSRM v26's osrm-customize writes the updated contract files IN PLACE
|
||||
// next to the input (there is no --output-prefix), so the traffic layer
|
||||
// is a renamed copy of the base dataset. CoW copy when the filesystem
|
||||
// supports it.
|
||||
trafficPrefix := *base + "-" + *prefix
|
||||
if !*skipCopy {
|
||||
if err := copyDataset(*data, *base, trafficPrefix); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
cmd := exec.Command(*binary, *data+"/"+trafficPrefix, "--segment-speed-file", *csv)
|
||||
cmd.Env = append(os.Environ(), "LD_LIBRARY_PATH=/home/gmp/osm-build/prefix/lib")
|
||||
fmt.Fprintf(os.Stderr, "running: %s %s %s\n", *binary, *data+"/"+trafficPrefix, *csv)
|
||||
start := time.Now()
|
||||
out, err := cmd.CombinedOutput()
|
||||
if err != nil {
|
||||
return fmt.Errorf("osrm-customize: %w\n%s", err, tail(string(out), 1500))
|
||||
}
|
||||
fmt.Printf("re-customized %s in %s\n", trafficPrefix, time.Since(start).Round(time.Second))
|
||||
fmt.Println("serve it: osrm-routed --algorithm mld --port <p> " + *data + "/" + trafficPrefix + ".osrm")
|
||||
fmt.Println("provenance: route responses from this layer must carry")
|
||||
fmt.Println(" source=computed(traffic), as_of=<fetch time>, policy=assumed")
|
||||
return nil
|
||||
}
|
||||
|
||||
// copyDataset copies base.osrm.* → target.osrm.* (reflink when possible).
|
||||
func copyDataset(dataDir, base, target string) error {
|
||||
targetBase := dataDir + "/" + target + ".osrm"
|
||||
if _, err := os.Stat(targetBase + ".ebg"); err == nil {
|
||||
fmt.Fprintf(os.Stderr, "%s.* already exists; use --skip-copy to reuse it\n", target)
|
||||
return nil
|
||||
}
|
||||
matches, err := filepath.Glob(dataDir + "/" + base + ".osrm.*")
|
||||
if err != nil || len(matches) == 0 {
|
||||
return fmt.Errorf("no %s.osrm.* files in %s", base, dataDir)
|
||||
}
|
||||
start := time.Now()
|
||||
for _, m := range matches {
|
||||
name := filepath.Base(m)
|
||||
dst := dataDir + "/" + target + ".osrm" + strings.TrimPrefix(name, base+".osrm")
|
||||
cp := exec.Command("cp", "--reflink=auto", m, dst)
|
||||
if err := cp.Run(); err != nil {
|
||||
if err2 := exec.Command("cp", m, dst).Run(); err2 != nil {
|
||||
return err2
|
||||
}
|
||||
}
|
||||
}
|
||||
fmt.Fprintf(os.Stderr, "copied %d files → %s.* (%s)\n", len(matches), target, time.Since(start).Round(time.Second))
|
||||
return nil
|
||||
}
|
||||
|
||||
func tail(s string, n int) string {
|
||||
if len(s) <= n {
|
||||
return s
|
||||
}
|
||||
return "…" + s[len(s)-n:]
|
||||
}
|
||||
|
||||
func envOr(k, dflt string) string {
|
||||
if v := os.Getenv(k); v != "" {
|
||||
return v
|
||||
}
|
||||
return dflt
|
||||
}
|
||||
143
router/internal/traffic/fiftyone.go
Normal file
143
router/internal/traffic/fiftyone.go
Normal file
|
|
@ -0,0 +1,143 @@
|
|||
package traffic
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Event is a normalized traffic incident from a feed.
|
||||
type Event struct {
|
||||
Raw json.RawMessage `json:"raw"`
|
||||
Type string `json:"type"` // feed-native event type, e.g. "Road Closed"
|
||||
Desc string `json:"desc"` // short description
|
||||
City string `json:"city"`
|
||||
Lat float64 `json:"lat"`
|
||||
Lon float64 `json:"lon"`
|
||||
}
|
||||
|
||||
const fiftyOneNYBase = "https://511ny.org/api"
|
||||
|
||||
// FetchEvents pulls all current incidents from the 511NY REST API.
|
||||
// The service is ASP.NET OData v3, so the JSON payload is {"d":[...]};
|
||||
// we accept that plus plain arrays for forward compatibility.
|
||||
// The raw response body is returned so callers can archive/inspect it —
|
||||
// field names are decoded leniently because we could not inspect a live
|
||||
// response before the first keyed call.
|
||||
func FetchEvents(key string) ([]Event, []byte, error) {
|
||||
client := &http.Client{Timeout: 30 * time.Second}
|
||||
u := fiftyOneNYBase + "/GetEvents?" + url.Values{"key": {key}, "format": {"json"}}.Encode()
|
||||
resp, err := client.Get(u)
|
||||
if err != nil {
|
||||
return nil, nil, fmt.Errorf("511ny: %w", err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
body, err := io.ReadAll(io.LimitReader(resp.Body, 32<<20))
|
||||
if err != nil {
|
||||
return nil, nil, fmt.Errorf("511ny read: %w", err)
|
||||
}
|
||||
if resp.StatusCode != 200 {
|
||||
return nil, body, fmt.Errorf("511ny: status %d: %s", resp.StatusCode, truncate(string(body), 200))
|
||||
}
|
||||
var events []Event
|
||||
var err2 error
|
||||
if events, err2 = decodeEvents(body); err2 != nil {
|
||||
return nil, body, fmt.Errorf("511ny decode: %w", err2)
|
||||
}
|
||||
return events, body, nil
|
||||
}
|
||||
|
||||
// decodeEvents accepts {"d":[...]} (OData v3), a plain array, or
|
||||
// {"results":[...]}.
|
||||
func decodeEvents(body []byte) ([]Event, error) {
|
||||
var pick func(v any) []json.RawMessage
|
||||
switch {
|
||||
case strings.HasPrefix(strings.TrimSpace(string(body)), "{"):
|
||||
var obj map[string]json.RawMessage
|
||||
if err := json.Unmarshal(body, &obj); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, k := range []string{"d", "results", "value"} {
|
||||
if raw, ok := obj[k]; ok {
|
||||
var arr []json.RawMessage
|
||||
if err := json.Unmarshal(raw, &arr); err == nil {
|
||||
pick = func(any) []json.RawMessage { return arr }
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
if pick == nil {
|
||||
return nil, fmt.Errorf("no event array in object (keys: %s)", keys(obj))
|
||||
}
|
||||
default:
|
||||
var arr []json.RawMessage
|
||||
if err := json.Unmarshal(body, &arr); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
pick = func(any) []json.RawMessage { return arr }
|
||||
}
|
||||
var events []Event
|
||||
for _, raw := range pick(nil) {
|
||||
events = append(events, decodeEvent(raw))
|
||||
}
|
||||
return events, nil
|
||||
}
|
||||
|
||||
func keys(m map[string]json.RawMessage) string {
|
||||
var ks []string
|
||||
for k := range m {
|
||||
ks = append(ks, k)
|
||||
}
|
||||
return strings.Join(ks, ",")
|
||||
}
|
||||
|
||||
// decodeEvent extracts location + type leniently. 511NY field names vary
|
||||
// between vintages; we try the documented shapes:
|
||||
//
|
||||
// {"EventType": "...", "Location": {"Latitude": .., "Longitude": ..}, ...}
|
||||
// {"type": "...", "lat": .., "lon": .., ...}
|
||||
func decodeEvent(raw json.RawMessage) Event {
|
||||
e := Event{Raw: raw}
|
||||
var m map[string]any
|
||||
if err := json.Unmarshal(raw, &m); err != nil {
|
||||
return e
|
||||
}
|
||||
e.Type = firstString(m, "EventType", "eventType", "type", "Event", "category")
|
||||
e.Desc = firstString(m, "Description", "description", "EventDescription", "summary")
|
||||
e.City = firstString(m, "City", "city", "LocationDescription")
|
||||
// location: nested object first, then flat fields
|
||||
if loc, ok := m["Location"].(map[string]any); ok {
|
||||
e.Lat = firstFloat(loc, "Latitude", "latitude", "lat")
|
||||
e.Lon = firstFloat(loc, "Longitude", "longitude", "lon", "lng")
|
||||
}
|
||||
if !e.Valid() {
|
||||
e.Lat = firstFloat(m, "Latitude", "latitude", "lat")
|
||||
e.Lon = firstFloat(m, "Longitude", "longitude", "lon", "lng")
|
||||
}
|
||||
return e
|
||||
}
|
||||
|
||||
// Valid reports whether the event carries a usable location.
|
||||
func (e Event) Valid() bool { return e.Lat != 0 && e.Lon != 0 }
|
||||
|
||||
func firstString(m map[string]any, keys ...string) string {
|
||||
for _, k := range keys {
|
||||
if v, ok := m[k].(string); ok && v != "" {
|
||||
return v
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func firstFloat(m map[string]any, keys ...string) float64 {
|
||||
for _, k := range keys {
|
||||
if v, ok := m[k].(float64); ok && v != 0 {
|
||||
return v
|
||||
}
|
||||
}
|
||||
return 0
|
||||
}
|
||||
207
router/internal/traffic/overpass.go
Normal file
207
router/internal/traffic/overpass.go
Normal file
|
|
@ -0,0 +1,207 @@
|
|||
// Package traffic implements the live-traffic pipeline: external incident
|
||||
// feeds → OSM node-pair speeds → OSRM segment-speed CSV → re-customize.
|
||||
//
|
||||
// The matcher works on the OSM side (Overpass returns OSM node IDs directly),
|
||||
// so no OSRM binary parsing is needed: the segment-speed CSV is keyed by OSM
|
||||
// node pairs, exactly what the OSRM updater consumes.
|
||||
package traffic
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"maps/router/internal/geo"
|
||||
)
|
||||
|
||||
const defaultOverpassURL = "https://overpass-api.de/api/interpreter"
|
||||
|
||||
// WayNode is one OSM node on a way.
|
||||
type WayNode struct {
|
||||
ID int64 `json:"id"`
|
||||
Lat float64 `json:"lat"`
|
||||
Lon float64 `json:"lon"`
|
||||
}
|
||||
|
||||
// Way is a highway way with its full node list.
|
||||
type Way struct {
|
||||
ID int64 `json:"id"`
|
||||
Highway string `json:"highway"`
|
||||
Ref string `json:"ref"`
|
||||
Nodes []WayNode `json:"-"`
|
||||
}
|
||||
|
||||
// QueryWaysAround fetches highway ways within radiusM of (lat, lon).
|
||||
func QueryWaysAround(client *http.Client, overpassURL string, lat, lon, radiusM float64, timeoutSec int) ([]Way, error) {
|
||||
if client == nil {
|
||||
client = &http.Client{Timeout: time.Duration(timeoutSec+30) * time.Second}
|
||||
}
|
||||
if overpassURL == "" {
|
||||
overpassURL = defaultOverpassURL
|
||||
}
|
||||
ql := fmt.Sprintf("[out:json][timeout:%d];way(around:%d,%g,%g)[highway];out geom;",
|
||||
timeoutSec, int(radiusM), lat, lon)
|
||||
req, err := http.NewRequest(http.MethodPost, overpassURL,
|
||||
strings.NewReader(url.Values{"data": {ql}}.Encode()))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
|
||||
// public Overpass WAFs reject the default Go user agent
|
||||
req.Header.Set("User-Agent", "voyage-router/0.1 (traffic pipeline; contact: gmp)")
|
||||
resp, err := client.Do(req)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("overpass: %w", err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
body, err := io.ReadAll(io.LimitReader(resp.Body, 64<<20))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("overpass read: %w", err)
|
||||
}
|
||||
if resp.StatusCode != 200 {
|
||||
return nil, fmt.Errorf("overpass: status %d: %s", resp.StatusCode, truncate(string(body), 200))
|
||||
}
|
||||
var doc struct {
|
||||
Elements []struct {
|
||||
Type string `json:"type"`
|
||||
ID int64 `json:"id"`
|
||||
Tags map[string]string `json:"tags"`
|
||||
Geom []struct {
|
||||
Lat float64 `json:"lat"`
|
||||
Lon float64 `json:"lon"`
|
||||
} `json:"geometry"`
|
||||
NodeIDs []int64 `json:"nodes"` // parallel to Geom
|
||||
} `json:"elements"`
|
||||
}
|
||||
if err := json.Unmarshal(body, &doc); err != nil {
|
||||
return nil, fmt.Errorf("overpass json: %w (body: %s)", err, truncate(string(body), 120))
|
||||
}
|
||||
var ways []Way
|
||||
for _, e := range doc.Elements {
|
||||
if e.Type != "way" || len(e.Geom) < 2 || len(e.NodeIDs) != len(e.Geom) {
|
||||
continue
|
||||
}
|
||||
w := Way{ID: e.ID, Highway: e.Tags["highway"], Ref: e.Tags["ref"]}
|
||||
for i, g := range e.Geom {
|
||||
w.Nodes = append(w.Nodes, WayNode{ID: e.NodeIDs[i], Lat: g.Lat, Lon: g.Lon})
|
||||
}
|
||||
ways = append(ways, w)
|
||||
}
|
||||
return ways, nil
|
||||
}
|
||||
|
||||
// classRank orders highway classes by importance (higher = more likely the
|
||||
// intended road for an incident reported near it).
|
||||
func classRank(h string) int {
|
||||
switch h {
|
||||
case "motorway":
|
||||
return 7
|
||||
case "trunk":
|
||||
return 6
|
||||
case "primary":
|
||||
return 5
|
||||
case "secondary":
|
||||
return 4
|
||||
case "tertiary":
|
||||
return 3
|
||||
case "unclassified":
|
||||
return 2
|
||||
default:
|
||||
return 1
|
||||
}
|
||||
}
|
||||
|
||||
// classWindowM is the distance within which a higher-class road beats a
|
||||
// closer lower-class one: an incident reported "at I-90" means the motorway
|
||||
// even when a service road or footway is nearer.
|
||||
const classWindowM = 150.0
|
||||
|
||||
// ClosestNodePair finds the highway segment the incident is on and returns
|
||||
// its two OSM endpoint node IDs. Selection: within classWindowM, the
|
||||
// highest-class road wins (ties → nearest); beyond it, plain nearest.
|
||||
func ClosestNodePair(ways []Way, lat, lon float64) (way Way, nodeA, nodeB int64, distM float64, ok bool) {
|
||||
p := geo.Point{Lat: lat, Lon: lon}
|
||||
type seg struct {
|
||||
way Way
|
||||
a, b int64
|
||||
distM float64
|
||||
rank int
|
||||
}
|
||||
var best *seg
|
||||
for _, w := range ways {
|
||||
rank := classRank(w.Highway)
|
||||
for i := 0; i+1 < len(w.Nodes); i++ {
|
||||
d := geo.DistToPolylineMeters(p, []geo.Point{{Lat: w.Nodes[i].Lat, Lon: w.Nodes[i].Lon}, {Lat: w.Nodes[i+1].Lat, Lon: w.Nodes[i+1].Lon}})
|
||||
if d > classWindowM && best != nil && best.distM <= classWindowM {
|
||||
continue
|
||||
}
|
||||
better := false
|
||||
if best == nil {
|
||||
better = true
|
||||
} else if rank > best.rank && d <= classWindowM {
|
||||
better = true
|
||||
} else if rank == best.rank && d < best.distM {
|
||||
better = true
|
||||
} else if rank < best.rank && best.distM > classWindowM && d < best.distM {
|
||||
better = true // nothing in the window; fall back to nearest
|
||||
}
|
||||
if better {
|
||||
best = &seg{way: w, a: w.Nodes[i].ID, b: w.Nodes[i+1].ID, distM: d, rank: rank}
|
||||
}
|
||||
}
|
||||
}
|
||||
if best == nil {
|
||||
return
|
||||
}
|
||||
return best.way, best.a, best.b, best.distM, true
|
||||
}
|
||||
|
||||
// ExpandToRadius returns node pairs along the matched way covering ±radiusM
|
||||
// of the incident point (incidents affect a stretch, not a single edge).
|
||||
func ExpandToRadius(way Way, hitA, hitB int64, radiusM float64) [][2]int64 {
|
||||
found := -1
|
||||
for i := 0; i+1 < len(way.Nodes); i++ {
|
||||
if way.Nodes[i].ID == hitA && way.Nodes[i+1].ID == hitB {
|
||||
found = i
|
||||
break
|
||||
}
|
||||
}
|
||||
if found < 0 {
|
||||
if hitA != 0 {
|
||||
return [][2]int64{{hitA, hitB}}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
start := found
|
||||
back := 0.0
|
||||
for start > 0 && back+segmentLen(way, start-1) <= radiusM {
|
||||
back += segmentLen(way, start - 1)
|
||||
start--
|
||||
}
|
||||
end := found
|
||||
fwd := 0.0
|
||||
for end+1 < len(way.Nodes)-1 && fwd+segmentLen(way, end+1) <= radiusM {
|
||||
fwd += segmentLen(way, end+1)
|
||||
end++
|
||||
}
|
||||
var pairs [][2]int64
|
||||
for i := start; i <= end; i++ {
|
||||
pairs = append(pairs, [2]int64{way.Nodes[i].ID, way.Nodes[i+1].ID})
|
||||
}
|
||||
return pairs
|
||||
}
|
||||
|
||||
func segmentLen(w Way, i int) float64 {
|
||||
return geo.Meters(geo.Point{Lat: w.Nodes[i].Lat, Lon: w.Nodes[i].Lon}, geo.Point{Lat: w.Nodes[i+1].Lat, Lon: w.Nodes[i+1].Lon})
|
||||
}
|
||||
|
||||
func truncate(s string, n int) string {
|
||||
if len(s) <= n {
|
||||
return s
|
||||
}
|
||||
return s[:n] + "…"
|
||||
}
|
||||
104
router/internal/traffic/policy.go
Normal file
104
router/internal/traffic/policy.go
Normal file
|
|
@ -0,0 +1,104 @@
|
|||
package traffic
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Policy maps a feed event type to the speed override + affected stretch.
|
||||
// Speeds are in km/h, applied to every graph edge on the matched stretch.
|
||||
// These are initial values — they should be tuned against measured traffic,
|
||||
// not asserted as fact (R4: provenance "assumed (policy)").
|
||||
type Policy struct {
|
||||
// SpeedKmh is the speed applied to the affected stretch.
|
||||
SpeedKmh int
|
||||
// RadiusM is how far along the road the effect extends from the point.
|
||||
RadiusM float64
|
||||
// Weight: when multiple incidents hit the same edge, the highest-weight
|
||||
// (most severe) policy wins.
|
||||
Weight int
|
||||
}
|
||||
|
||||
// Policies covers the 511NY event types we expect. Unknown types are
|
||||
// ignored (no override) rather than guessed.
|
||||
var Policies = map[string]Policy{
|
||||
"road closed": {5, 1500, 100},
|
||||
"roadclosure": {5, 1500, 100},
|
||||
"lane closure": {45, 800, 60},
|
||||
"laneclosure": {45, 800, 60},
|
||||
"accident": {25, 1200, 90},
|
||||
"collision": {25, 1200, 90},
|
||||
"road work": {50, 1000, 40},
|
||||
"roadwork": {50, 1000, 40},
|
||||
"construction": {50, 1000, 40},
|
||||
"congestion": {30, 1500, 30},
|
||||
"slow traffic": {30, 1500, 30},
|
||||
"traffic": {30, 1500, 30},
|
||||
"vehicle on road": {30, 800, 50},
|
||||
"disabled vehicle": {25, 800, 70},
|
||||
"incident": {25, 1000, 50},
|
||||
}
|
||||
|
||||
// PolicyFor normalizes a feed event type and looks up its policy.
|
||||
func PolicyFor(eventType string) (Policy, bool) {
|
||||
norm := strings.ToLower(strings.TrimSpace(eventType))
|
||||
if p, ok := Policies[norm]; ok {
|
||||
return p, true
|
||||
}
|
||||
// substring fallback: "Accident - Multi Vehicle" etc.
|
||||
for key, p := range Policies {
|
||||
if strings.Contains(norm, key) {
|
||||
return p, true
|
||||
}
|
||||
}
|
||||
return Policy{}, false
|
||||
}
|
||||
|
||||
// Assignment is one incident's contribution: a stretch of node pairs, a
|
||||
// speed, and a severity weight.
|
||||
type Assignment struct {
|
||||
Pair [2]int64
|
||||
SpeedKmh int
|
||||
Weight int
|
||||
}
|
||||
|
||||
// BuildCSV resolves assignments into OSRM segment-speed CSV rows. For each
|
||||
// unordered node pair, the highest-weight (most severe) speed wins; the OSRM
|
||||
// updater itself keys on (nodeA,nodeB), so conflicts must resolve here.
|
||||
func BuildCSV(assignments []Assignment) string {
|
||||
type key struct{ a, b int64 }
|
||||
type row struct{ pair key; speed, weight int }
|
||||
best := map[key]*row{}
|
||||
for _, as := range assignments {
|
||||
k := key{as.Pair[0], as.Pair[1]}
|
||||
if k.a > k.b {
|
||||
k.a, k.b = k.b, k.a
|
||||
}
|
||||
cur, ok := best[k]
|
||||
if !ok || as.Weight > cur.weight {
|
||||
best[k] = &row{pair: k, speed: as.SpeedKmh, weight: as.Weight}
|
||||
}
|
||||
}
|
||||
rows := make([]*row, 0, len(best))
|
||||
for _, r := range best {
|
||||
rows = append(rows, r)
|
||||
}
|
||||
sort.Slice(rows, func(i, j int) bool {
|
||||
if rows[i].pair.a != rows[j].pair.a {
|
||||
return rows[i].pair.a < rows[j].pair.a
|
||||
}
|
||||
return rows[i].pair.b < rows[j].pair.b
|
||||
})
|
||||
var b strings.Builder
|
||||
for _, r := range rows {
|
||||
fmt.Fprintf(&b, "%d,%d,%d\n", r.pair.a, r.pair.b, r.speed)
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
// WriteSpeedFile writes the CSV to disk.
|
||||
func WriteSpeedFile(path, csv string) error {
|
||||
return os.WriteFile(path, []byte(csv), 0o644)
|
||||
}
|
||||
112
router/internal/traffic/traffic_test.go
Normal file
112
router/internal/traffic/traffic_test.go
Normal file
|
|
@ -0,0 +1,112 @@
|
|||
package traffic
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestPolicyFor(t *testing.T) {
|
||||
cases := []struct {
|
||||
in string
|
||||
ok bool
|
||||
kph int
|
||||
}{
|
||||
{"Road Closed", true, 5},
|
||||
{"ACCIDENT", true, 25},
|
||||
{"Accident - Multi Vehicle", true, 25},
|
||||
{"road work", true, 50},
|
||||
{"Bridge Outage (Unknown)", false, 0},
|
||||
{"", false, 0},
|
||||
}
|
||||
for _, c := range cases {
|
||||
p, ok := PolicyFor(c.in)
|
||||
if ok != c.ok || (ok && p.SpeedKmh != c.kph) {
|
||||
t.Errorf("PolicyFor(%q) = %+v, %v; want ok=%v kph=%d", c.in, p, ok, c.ok, c.kph)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildCSV(t *testing.T) {
|
||||
// same pair, two incidents → most severe (highest weight) wins
|
||||
csv := BuildCSV([]Assignment{
|
||||
{Pair: [2]int64{100, 200}, SpeedKmh: 50, Weight: 40},
|
||||
{Pair: [2]int64{200, 100}, SpeedKmh: 5, Weight: 100}, // reversed order, must dedupe
|
||||
{Pair: [2]int64{300, 400}, SpeedKmh: 25, Weight: 90},
|
||||
})
|
||||
lines := strings.Split(strings.TrimSpace(csv), "\n")
|
||||
if len(lines) != 2 {
|
||||
t.Fatalf("want 2 rows, got %d: %v", len(lines), lines)
|
||||
}
|
||||
if lines[0] != "100,200,5" {
|
||||
t.Errorf("row0 = %q; want 100,200,5", lines[0])
|
||||
}
|
||||
if lines[1] != "300,400,25" {
|
||||
t.Errorf("row1 = %q; want 300,400,25", lines[1])
|
||||
}
|
||||
}
|
||||
|
||||
func TestClassRank(t *testing.T) {
|
||||
if !(classRank("motorway") > classRank("trunk") && classRank("trunk") > classRank("primary") &&
|
||||
classRank("primary") > classRank("residential")) {
|
||||
t.Errorf("classRank ordering wrong")
|
||||
}
|
||||
}
|
||||
|
||||
func TestClosestNodePairPrefersClass(t *testing.T) {
|
||||
// motorway at 120m, service at 20m → motorway wins (within class window)
|
||||
ways := []Way{
|
||||
{
|
||||
ID: 1, Highway: "service",
|
||||
Nodes: []WayNode{{ID: 11, Lat: 45.0000, Lon: -73.0000}, {ID: 12, Lat: 45.0000, Lon: -72.9990}},
|
||||
},
|
||||
{
|
||||
ID: 2, Highway: "motorway",
|
||||
Nodes: []WayNode{{ID: 21, Lat: 45.0011, Lon: -73.0000}, {ID: 22, Lat: 45.0011, Lon: -72.9990}},
|
||||
},
|
||||
}
|
||||
// point 20m from service line (lat 45.0000) and ~120m from motorway line (lat 45.0011)
|
||||
way, a, b, _, ok := ClosestNodePair(ways, 45.0002, -73.0)
|
||||
if !ok {
|
||||
t.Fatal("no match")
|
||||
}
|
||||
if way.Highway != "motorway" || a != 21 || b != 22 {
|
||||
t.Errorf("got %s (%d→%d); want motorway (21→22)", way.Highway, a, b)
|
||||
}
|
||||
}
|
||||
|
||||
func TestClosestNodePairNearestFallback(t *testing.T) {
|
||||
// only a service road exists, 500m away → nearest still wins
|
||||
ways := []Way{
|
||||
{
|
||||
ID: 1, Highway: "service",
|
||||
Nodes: []WayNode{{ID: 11, Lat: 45.0050, Lon: -73.0000}, {ID: 12, Lat: 45.0050, Lon: -72.9900}},
|
||||
},
|
||||
}
|
||||
way, _, _, d, ok := ClosestNodePair(ways, 45.0, -73.0)
|
||||
if !ok || way.ID != 1 {
|
||||
t.Fatalf("got ok=%v way=%+v", ok, way)
|
||||
}
|
||||
if d < 500 || d > 560 {
|
||||
t.Errorf("dist = %.0f; want ~500", d)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDecodeEvents(t *testing.T) {
|
||||
// OData v3 envelope
|
||||
body := []byte(`{"d":[{"EventType":"Road Closed","Description":"I-90 closed","City":"Albany","Location":{"Latitude":42.75,"Longitude":-73.93}},{"EventType":"Accident","lat":42.1,"lon":-73.5}]}`)
|
||||
events, err := decodeEvents(body)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(events) != 2 {
|
||||
t.Fatalf("want 2 events, got %d", len(events))
|
||||
}
|
||||
e0 := events[0]
|
||||
if e0.Type != "Road Closed" || !e0.Valid() || e0.Lat != 42.75 || e0.Lon != -73.93 {
|
||||
t.Errorf("event0 = %+v", e0)
|
||||
}
|
||||
e1 := events[1]
|
||||
if e1.Type != "Accident" || !e1.Valid() {
|
||||
t.Errorf("event1 = %+v", e1)
|
||||
}
|
||||
}
|
||||
Loading…
Reference in New Issue
Block a user