Here is a rule that sounds obvious: constraints must be satisfied before preferences are considered. Residency, criticality, context window, required capabilities — these gate. Cost, quality and latency rank whatever is left.
Every system agrees with this rule. Plenty of them break it anyway, and they break it the same way.
How it erodes
Nobody sets out to let a preference override a constraint. It happens in three moves, each defensible on its own:
- The ranker is given the full pool and a set of filters to apply.
- Someone adds a stickiness rule: keep a conversation on the model it started on. It is a good rule and it makes results more consistent.
- Someone adds a fallback: if the chosen model errors, try the next one.
Each of those three is now reading from the pool. Which means each of them can select a model the filters excluded, and two of them eventually will — the sticky model whose conversation began before the request was marked critical, and the fallback that walks past the end of the filtered list into whatever is next.
You will not notice. The system keeps working. It serves a money-path request from a vendor-hosted model, correctly, quickly, and against the one rule you thought you had.
Holding it structurally
The fix is not to be more careful. It is to make the pool unreachable.
In our selector, filtering happens first and produces a survivor set. Ranking, stickiness and fallback are all handed that set. None of them receives the pool, so none of them can return something the filters excluded — not because they are careful, but because the excluded models are not in the data they were given.
The fallback chain inherits this for free. A critical request that exhausts its in-infrastructure candidates is refused, because the chain it is walking is the survivor list and the survivor list has ended. There is no next element that happens to be a vendor model.
The test that says so
The invariant is worth a test that fails loudly if someone re-plumbs it:
tests/router/test_router.py::TestFailOpen
::test_failing_open_does_not_mean_failing_unconstrained
An unclassifiable request routes to the default concept rather than erroring — open means unclassified, not unconstrained. The hard filters still run, on the default concept's constraints. That test exists because the two meanings of "fail open" are one careless sentence apart.
Why this is a property and not a promise
The distinction matters more than it sounds. A promise is a thing you write in a document and then have to keep. A property is a thing the code cannot easily violate, so keeping it is the default and breaking it takes work.
We would rather have the second kind, because the first kind has a shelf life measured in the tenure of whoever wrote it down.