Anyone building this looks at what exists first. Here is what we found, grouped by what the tool believes routing is.
Access layers that can also route
LiteLLM, and the proxy modes of several hosted products. A hundred providers behind one call, with fallback lists and simple cost-based selection on top.
Best at: reaching anything, quickly. The per-provider correction work is enormous and already done.
The routing is a list you configure. That is proportionate for most users, and it puts the policy at the call site as a model string — which is where it should be when the choice is obvious, and the wrong place when a rule depends on it.
Difficulty routers
RouteLLM and the research line around it. Train a classifier to predict whether a strong model is needed; send the rest to a weak one. Published results are good — large cost reductions at small quality loss on standard benchmarks.
Best at: cost reduction on homogeneous traffic where quality is a scalar.
The axis is the limitation. A difficulty score cannot express "this must not leave our infrastructure", and bolting the constraint on beside the score means the routing logic and the binding rule live in different places — with the rule in the margin.
Semantic routers
semantic-router and similar. Embed the query, match against named routes, dispatch. Structurally close to what we do — utterances play the role of exemplars.
Best at: fast intent classification with no model call, which is genuinely the right primitive.
It stops at the decision. What it dispatches to is yours: no notion of a model having a residency or a tier, no filtering before ranking, no fallback chain that inherits the constraints, no decision log. Which is fair — it is a classification library, not a router — but the gap between "I know what this request is" and "I can prove what it was allowed to reach" is where most of our code lives.
Framework routing
LangChain and friends, where routing is a chain component. Maximum flexibility, and the policy becomes application code — so what routes where is answered by reading a program rather than a file.
The thing none of them makes easy
Proving that a constraint held.
Every one of these can express "prefer the cheap model". None makes it straightforward to say "this class of request may only ever reach these models, and here is the record showing it did" — and, more importantly, to keep that true once stickiness, fallback and caching are added.
That is not an oversight. It is a different problem, and it only matters if you have a rule that must never be broken. Most people do not, which is why most tools do not solve it.
What we would tell someone choosing
If your problem is access, use LiteLLM or a hosted gateway. Do not build it.
If your problem is cost on uniform traffic, a difficulty router is the highest-leverage thing available and the research is real.
If your problem is classification, semantic-router is a good primitive and you will write the dispatch yourself, which is fine.
If your problem is a rule that must never be broken for one kind of work — and you would want to answer an auditor with a per-request record rather than a policy document — that is the case we built for, and it is a narrower case than the category name suggests.
What we are not
Not an access layer: two providers, five models, configured by hand. Not a marketplace. Not a difficulty router, and not better at cost reduction than one.
We chose one axis and are trying to be right about it. Anyone comparing on breadth will find us worse, and they should — it is not the thing being attempted.