Which LLM router actually picks the right model?
OmniRoute Pilot
Thrivbe piloted an open LLM routing layer against its existing gateway on identical real workloads, diffing cost, latency and answer quality, because routing quality is usually promised and rarely measured. Self-hosted routers have sharp edges: the instance hung on first boot, and keyless relay upstreams rejected the datacenter IP.
- OmniRoute
- Comparison harness
Relevant services: Thrivbe AI
Hypothesis
Model routers promise to send each request to the cheapest model that can handle it. But routing quality is rarely measured. Running a router pilot side-by-side against our existing gateway, on the same real workloads, should show whether "smart routing" earns its complexity.
What we built
A pilot deployment of an open routing layer on our infrastructure, plus a comparison harness that replays the same prompts through both routes and diffs cost, latency, and answer quality.
Learnings
- Self-hosted routers have sharp edges: our instance hung on first boot until restarted, and keyless relay upstreams rejected our datacenter IP outright. "Works on the demo" and "works from a server" are different claims.
- Routing is only as good as the eval behind it — without a harness of your own workloads, you're trusting someone else's benchmark.
Log
- 2026-07-10 — Comparison harness consuming live traffic samples.
- 2026-07-08 — Pilot deployed; first-boot hang and datacenter-IP relay rejections diagnosed.