How far can you get on the free tier of every LLM?

Hypothesis

Dozens of LLM providers offer free tiers and relays. Aggregated behind one OpenAI-compatible endpoint with smart routing and rate-limit awareness, they should cover a surprising share of background workloads — sentiment scoring, summaries, reflections — at zero marginal cost.

What we built

A self-hosted router that fronts many free providers as one API: model aliasing, automatic failover, and rate-limit autotuning that learns each provider's real ceilings instead of hardcoding them. It powers our trading fleet's news scoring, report reflections, and other batch jobs.

The hard-won part is a privacy policy layer: a "safe" routing class that only uses models empirically verified as not training on prompts — because some free relays do. Any workload touching personal data pins to that class; one popular relay was removed entirely.

Learnings

• Free tiers are real capacity, but treat them like weather — the router's job is graceful degradation, never a hard dependency.
• Aliases lie. A router that normalises model names can serve your request from a different platform than the name suggests; if privacy depends on where a model runs, probe the live served identity, don't trust the label.
• Separating workloads by data sensitivity (free-anything vs verified no-train) is a cleaner policy than trying to make everything private.

Log

• 2026-07-11 — Privacy routing class live: verified no-train cascade for any personal-data workload; one relay removed.
• 2026-06-25 — Rate-limit autotuning shipped.
• 2026-06-10 — First aggregation gateway online.