Writing
Notes from building it
Routing by concept, the constraints that make it hold, and what a marketplace around it actually costs to build. Mostly things we got wrong first.
-
5 September 2026
What OpenRouter solved, and the question it leaves open
One API for many models is a genuinely hard problem and it is solved. The question underneath it — which model should this request go to, and who decides — is a different one.
-
5 September 2026
The open-source router landscape, honestly
Four approaches, what each is actually good at, and the one thing none of them makes easy.
-
5 September 2026
The case against routing entirely
The strongest argument against this product, made properly, and the narrow conditions under which it fails.
-
5 September 2026
Semantic caching — the wins, and the quiet correctness cost
Returning a cached answer to a similar question is the highest-leverage optimisation available and the one most likely to be wrong in a way nobody notices.
-
5 September 2026
Residency as a routing constraint rather than a contract clause
Most data residency is enforced by procurement. Making it a property the router checks per request is a different kind of guarantee — and we cannot currently serve it.
-
5 September 2026
Reading LiteLLM's provider abstraction
A hundred providers behind one signature, and the shape of the work is nothing like what the README suggests.
-
5 September 2026
Gateways, routers and proxies — three things called the same word
Most arguments about LLM infrastructure are two people using one word for different jobs. Here are the three jobs.
-
5 September 2026
Cascades and verifiers — when answering twice is cheaper
Try the cheap model, check the answer, escalate if it fails. It only works where "check" is something a machine can do, which is rarer than it sounds.
-
5 September 2026
Benchmarks that measure the benchmark
Published routing results are real and they are about a distribution that is not yours. Which makes them useful for a narrower purpose than they are usually put to.
-
5 September 2026
What we would need before earned rankings beat authored ones
The candidate order in every policy row is a human's opinion. Replacing it with measurement is obviously right and needs three things we do not have.
-
4 September 2026
The security pipeline — what blocks a merge and what only reports
Two workflows with the same tools and opposite rules. A scanner that can fail your build on somebody else's disclosure is a scanner people learn to skip.
-
4 September 2026
What a release bundle should carry besides the site
A scratch image with no entrypoint, holding the built site, the compose file, the edge config and one deploy script. Each addition was a bug we had already had.
-
4 September 2026
Read-only by construction, asserted by walking the route table
A test that iterates the application's own routes and fails if any admin endpoint accepts anything but GET. The value is not the assertion — it is what it does to the next conversation.
-
4 September 2026
Pushing CI state instead of pulling it
The console shows what the pipeline did. Getting that data by asking GitHub would mean a standing credential on the server and a page that breaks when GitHub is slow.
-
4 September 2026
Product-as-code — intent that fails the build
Features, specs and decisions are files in the repository, validated in CI. A pull request that breaks the graph does not merge.
-
4 September 2026
Operators as configuration, not as a database column
The conventional design lets anything with write access to one table grant itself admin. This one has no code that grants it at all.
-
4 September 2026
Why a feature is not "shipped" until a monitor watches it
Everything in this repository is `building`. Several of those features are in production and working. The gap between those two sentences is the whole argument.
-
4 September 2026
Diagnostic codes are an API — retire, never renumber
A code in a build log is referenced by scripts, waivers, commit messages and someone's memory. Reusing one is worse than leaving a gap.
-
4 September 2026
Deploying from CI without handing CI a shell
A GitHub secret is readable by anyone who can merge a workflow change. So the deploy key opens four verbs and refuses everything else.
-
4 September 2026
A console that is an authorisation boundary first
It reads key metadata, balances, email addresses and consent records. Deciding who may open it came before deciding what it shows.
-
3 September 2026
Uppercase in an image reference
Moving the repository to an organisation broke the pipeline in a way that only affected the references we built by hand.
-
3 September 2026
Two secrets, one of which quietly did nothing
A config value needs four things to be true before it works. Three settings shipped with one of them missing, and only the first announced itself.
-
3 September 2026
The dependency that was never declared
Every typecheck in the project's history had passed against a node_modules that happened to contain something the lockfile did not.
-
3 September 2026
Sending mail from a domain you have not verified
The contact form was built with a From address that would have been rejected on every send. The failure would have looked like an outage.
-
3 September 2026
Every bug in this series was found by a guard we had just written
Nine mistakes in one build. Almost none was found by thinking harder, and the pattern in how they surfaced is more useful than any of them.
-
3 September 2026
The workflow that was broken on main for a day, invisibly
An empty expression in a comment. GitHub parses comments. And it only parses a workflow when you need it.
-
3 September 2026
The forced command that could set a trap it could not clear
A boolean input wrote 0 when true and nothing when false. Nothing is not the opposite of 0 when the receiving script merges.
-
3 September 2026
A test suite that was really testing its own ordering
Three tables, three times, same bug. The fix that stuck was deleting the list and asking the database instead.
-
3 September 2026
A monitor that could not parse its own assertion
Twenty-seven consecutive failures. The service was fine every time. The monitor had never once run.
-
2 September 2026
The spend gate belongs at authentication
The hop that already knows who you are is the hop that should know whether you can pay. Anywhere else is a second lookup and a second thing to keep in agreement.
-
2 September 2026
The gap we documented, and then closed without noticing
We wrote down why chargebacks were out of scope. The reason stopped being true in the same pull request, and the note sat there for a day being wrong.
-
2 September 2026
Seven ledger kinds, each with a known sign
We started with four and added three, and each addition was an argument about whether a motion was really new. The temptation each time was to widen an existing kind.
-
2 September 2026
Metering off the request path, and the overshoot we accept
Billing reads the log the router already writes. An account can end slightly below zero, and we wrote that down rather than discovering it.
-
2 September 2026
Idempotency you get from an index, not from care
A unique index and one INSERT. The version that checks first is the one that looks correct and is not.
-
2 September 2026
Free credit that a card refund cannot claw back
New accounts get a dollar. Making that safe meant asking what a refund is allowed to take, and the answer was already written in a query nobody had revisited.
-
1 September 2026
Why the routing budget is 5%, and what it buys
A router that adds 200ms to a 400ms request has not optimised anything. The number is arbitrary; having one is not.
-
1 September 2026
The classification ladder — explicit, cache, embedding, LLM, default
Five rungs, cheapest first, and each one exists because the rung below it is not always available. Most requests never reach the third.
-
1 September 2026
Session stickiness, and the guarantee it does not make
One conversation keeps one model. Except when it must not — and the exception is the reason the feature is safe to have.
-
1 September 2026
Fallback chains that cannot reach past a filter
A critical request that exhausts its candidates is refused, not handed to the next model down. The chain is the survivor list, which is why.
-
1 September 2026
Failing open without failing unconstrained
An unclassifiable request must not error. It must also not become a request with no rules. Those two sentences are one careless word apart.
-
1 September 2026
Explaining a decision without serving it
/v1/explain stops before the provider is called. It is a debugging tool, a demo, a monitor and a test fixture, and it is all four because it costs nothing.
-
1 September 2026
Adding a model should be a config edit, not a deploy of new code
Four artifacts, kept separate on purpose. Adding a model is a row and an id; adding a route is four sentences. Nothing is retrained and no concept file moves.
-
31 August 2026
What a concept is, and why three example sentences beat a fine-tune
A concept is a file somebody wrote, not a cluster somebody discovered. That is a choice about who is accountable for the routing.
-
31 August 2026
Why the balance moves on the webhook and never on the redirect
A redirect is the customer's browser claiming a payment happened. A signed webhook is Stripe saying so. We shipped the rule and then found out how it fails.
-
31 August 2026
The balance is the sum of the ledger, or it is nothing
We cache the balance on the account row. There is deliberately no function that sets it, and that absence is the whole design.
-
31 August 2026
Integer micro-cents, and the float that nearly shipped
Money is a signed integer of 1e-8 dollars everywhere it is stored. The router still computes cost as a float, and the boundary between those two facts is the whole of it.
-
29 August 2026
How a refund invents a debt you do not owe
We shipped refund handling, then found the state where reversing a payment creates a debt from nothing. The fix turns on a distinction we had not needed until that afternoon.
-
28 August 2026
The deploy that succeeded and changed nothing
Files on the box, a green deploy, and a site still serving last week's build. A bind mount follows the inode, not the path.
-
26 August 2026
Hard filters gate, preferences rank
The rule is easy to state and easy to erode. We hold it structurally instead — the ranker is handed the survivors and never sees the pool.
-
24 August 2026
Routing by difficulty is the wrong axis
Every router we looked at sorted requests into easy and hard. That axis cannot express the one thing our traffic actually cared about.