Skip to main content
The cheapest capable model that can do the job, chosen per task — and one place where that choice can actually be governed.
Running several AI agents at once creates a problem that does not exist with one: every agent has its own model, its own credential, and its own bill. Nobody can see the total, nobody can cap it, and the same work gets done at wildly different prices depending on which agent happened to pick it up. Model routing is the answer to that. This page is the concept; the specific endpoints, credentials, and model inventory are deliberately not here.

​
One endpoint, many backends

The core move is to stop letting each tool hold its own model configuration and put a single OpenAI-compatible endpoint in front of everything instead. Every consumer points at that one endpoint. Adding a backend, retiring one, or repointing a tier becomes a change in one config file rather than an edit to every tool that might use it. This is worth doing even when all the backends are local, because the win is not about where the models run. It is about having exactly one place where routing, observability, and policy can live. The cost of the pattern is that the endpoint sits in the path of every model call. That is only acceptable if it cannot become a bottleneck — which in practice means keeping it stateless, with its configuration rendered from version control rather than authored in a UI or stored in a database. Several identical instances behind a health check can then serve interchangeably, and losing one costs nothing. Keeping it stateless has a consequence worth planning for: features that need to remember something across requests — per-caller spend ledgers, revocable per-consumer keys — need somewhere to store that state. Statelessness and per-caller accounting pull in opposite directions, and that trade should be made deliberately rather than discovered later.

​
Ask for a role, not a model

Consumers should request a stable alias that names a job — the default reasoning model, the judge, the tool-calling tier — rather than a specific model identifier. The reason is drift. Model identifiers change constantly: a new quantization, a better checkpoint, a swap for something cheaper. Every place that hardcoded the old identifier now needs finding and editing, and the ones nobody finds fail at the worst moment. An alias moves once, centrally. This extends to documentation. A page that lists model names is wrong the moment the list changes, and it is wrong silently. The right pattern is a single machine-readable registry that the routing config is generated from, and docs that point at it — or at the endpoint’s own live model listing — instead of restating it. That is why this page names no models.

​
Delegation: the cheapest capable tier

Once every agent can reach a shared endpoint, an agent no longer has to do all its own thinking on its own subscription. It can hand a bounded subtask to whichever tier can do it most cheaply, in roughly this order:
  1. A local tier. No marginal cost, no data leaving the network, no credential. This covers most bounded subtasks and should be the default.
  2. An external provider, when a local tier genuinely cannot do the job — a context window it cannot hold, or a capability it does not have.
  3. The agent’s own model. The fallback, not the default. An agent that delegates and ends up here has saved nothing.
Two properties make external access safe enough to leave switched on. It should be opt-in by explicit model identifier, so nothing routes to a paid provider implicitly. And it should be excluded from fallback chains, so a rate-limited or flaky external provider can never degrade a local call that would otherwise have succeeded.

​
Advisory controls are not enforcement

This is the distinction that matters most, and the one most easily blurred. A control that runs at the caller — an instruction in a prompt, a budget an agent tracks for itself — binds a well-behaved caller and nothing else. An agent that ignores it, misunderstands it, or is compromised simply proceeds. It is a convention. A control that runs at the routing endpoint — an allowlist of permitted models, a hard spend ceiling, a rate limit — binds every caller, because the caller is not the thing enforcing it. Both are useful. The failure is writing documentation that describes the first as if it were the second, because that produces false confidence in a control that was never load-bearing. If a cap is tracked by the agent, the honest word is advisory, and it should stay that word until enforcement actually moves. An allowlist is often the pragmatic middle: it constrains what can be reached without needing anywhere to store spend, so it works on a stateless endpoint. It bounds reach rather than cost — a permitted model can still be expensive — but against a broad credential it removes most of the exposure.

​
Failing honestly

A shared endpoint is a shared dependency, so agents need defined behaviour when it is unreachable: a bounded timeout and a clear report. The one genuinely bad outcome is a silent fallback — the agent quietly does the work on its own model and presents the result as though delegation happened. That hides both the cost and the fact that the routing layer is down. If delegation failed, the report says delegation failed.
  • Repo boundaries — where routing rules live relative to plugins and docs.
  • MCP tool plane — the same “register once, reach from everywhere” idea, applied to tools rather than models.
  • Local LLM — the serving side: what runs the local tiers.