The Model You Pinned Is Gone

Providers retire models. Not often, and not without notice, but they do it — and when they do, every integration that pinned that exact model id starts failing. The request is well-formed, the credentials are valid, the service is up. The model simply is not there any more.

Until this week, Nyquest handled that correctly and unhelpfully: it returned a clean error naming a live alternative, and stopped. Correct, because we will not quietly send your prompt somewhere you did not ask for. Unhelpful, because your integration is now down and the fix requires a human to notice, read the error, and ship a change.

What made us look harder was not the error. It was the silence around it.

Two months, thirty-six requests, zero successes

One of our own API keys was pinned to a model that had been delisted. It made thirty-six requests over its entire lifetime. All thirty-six failed. Not one ever succeeded, and nobody noticed for two months.

The reason is worth sitting with, because it generalises. A caller-side 4xx does not look like a platform problem. It looks like the caller's problem. It does not page anyone, it does not dent an uptime number, and it does not appear in any dashboard measuring whether we are healthy — because by every metric we had, we were. The integration was completely broken and the platform was completely fine, simultaneously, for two months.

There is an older, uglier version of this bug that we closed earlier in July and should name here. A pinned dead model used to burn a call to the direct provider and a fallback call to OpenRouter before failing, then hand the raw upstream error body straight back to the caller — including our own OpenRouter account identifier. That path is gone: the request is now rejected before dispatch, against the model catalogue, and the caller sees our structured error rather than somebody else's internals.

Serving the successor

The dispatcher already knew the answer. To write a useful error it looks up the newest live model from the same provider, so it can say try this instead. It had the replacement in hand and was throwing it away.

So now it can serve it. When a caller pins a model we have retired, Nyquest routes the request to that same-provider successor and the integration keeps working.

Never silently

This is the part we spent the most time on, because a substituted model is not a free win. A different model has different behaviour and, more pointedly, a different price. Swapping one in without saying so would be a correctness surprise and a billing surprise at the same time, and "we helpfully charged you a different rate" is not a sentence we want to write.

So the swap is always visible:

• x-nyquest-model-substituted reports both ids — what you asked for, and what actually ran.
• x-nyquest-model and the response body carry the model that actually served the request, so OpenAI-compatible clients reading the model field see the truth without changing a line.
• Prefer the old strictness? One configuration flip restores hard rejection, with no redeploy. Callers who would rather fail than be rerouted keep that option.

It is off in code and enabled by configuration, which means the behaviour is reversible in seconds rather than in a build.

The trap in our own fix

Here is the bit that nearly got us, and it is the most interesting thing in this post.

Fixing the dead end would have destroyed the evidence. Once a retired-model request succeeds via its successor, our usage records store the model that ran — the successor. The record of anyone asking for the retired model disappears. The very change that keeps the integration alive also removes the only trace that it is still pinned to something that no longer exists.

We would have fixed the symptom and blinded ourselves to the cause, which is a worse position than the one we started in: broken and visible beats working-by-accident and invisible.

So the fix and the detection were designed together. The dispatcher now emits one canonical log line in both outcomes — whether it rerouted or rejected — carrying the model that was requested, the substitution decision, and which account asked. A daily scan reads that alongside our usage records and reports any caller still pinned to a retired model, naming the key and its owner. It reports as information, never as a failure: somebody else's stale integration is not a production incident, and turning the daily red for it would only teach us to ignore the colour.

Run against the two months we missed, the scan names the offending key and owner immediately. Had it existed, that would have been a day-one finding instead of a day-sixty one.

What deliberately didn't change

Defaults. An integration that sends nothing new behaves exactly as it did before — same models, same prices, same responses. Aliasing only ever engages for a model that is genuinely retired, and it never reaches past the original provider to find a substitute.

We also did not touch the catalogue's self-healing behaviour or the auto-routing guard that already prevents Nyquest from ever selecting a delisted model on its own. Those work. This change is only about the case where you named the model yourself.

The general lesson is not about model deprecation. It is that the failures which survive longest are the ones that look like somebody else's fault. A platform can be green on every dashboard it owns while an integration built on it has not worked once. Worth asking what your own monitoring would call a success.

Try Nyquest →

No account needed to start chatting — or bring your own API key.

← All posts