Which LLM router actually picks the right model?

Hypothesis

Model routers promise to send each request to the cheapest model that can handle it. But routing quality is rarely measured. Running a router pilot side-by-side against our existing gateway, on the same real workloads, should show whether "smart routing" earns its complexity.

What we built

A pilot deployment of an open routing layer on our infrastructure, plus a comparison harness that replays the same prompts through both routes and diffs cost, latency, and answer quality.

Learnings

• Self-hosted routers have sharp edges: our instance hung on first boot until restarted, and keyless relay upstreams rejected our datacenter IP outright. "Works on the demo" and "works from a server" are different claims.
• Routing is only as good as the eval behind it — without a harness of your own workloads, you're trusting someone else's benchmark.

Log

• 2026-07-10 — Comparison harness consuming live traffic samples.
• 2026-07-08 — Pilot deployed; first-boot hang and datacenter-IP relay rejections diagnosed.