The best thing about compression on Nyquest is that it compounds: the longer a conversation runs, the more of its history the engine can condense, and the more you save — up to 75% of tokens on long chats. The worst thing about it, until this week, was that the very feature doing that saving charged a toll: on conversations long enough to qualify, condensing the history added about a second to the request. Every turn.
This post is about deleting that second. Token savings are unchanged. The work just stopped happening while you wait.
When a conversation gets long, the engine summarizes the older turns into a compact digest and keeps the recent ones verbatim. That summarization is itself a model call, so we cached it — and here is the embarrassing part in hindsight: the cache treated the entire history so far as one unit. Add a single new exchange and the unit changed, so the cache missed, so the whole history was re-summarized from scratch. On the request path. Roughly one second, on every turn of exactly the conversations where compression matters most.
A cache that cannot hit in its primary use case is not a cache. It is a ritual.
Two things shipped together in this week's engine release, and they compose.
First, the cache now understands that conversations only grow. A chat history is append-only: everything condensed yesterday is still the beginning of the conversation today. So instead of re-summarizing everything when new turns arrive, the engine reuses the summary it already produced and folds in only what's new — a far smaller, far faster operation. To keep quality honest over many folds, it periodically re-summarizes from scratch, so digests never degrade across generations. Fact retention is watched by the same judge loop that already audits compression quality daily.
Second, the engine stopped waiting for you. The moment a turn is served, the engine already knows nearly everything the next request will contain — the new reply always lands in the recent window that is kept verbatim, so the part that needs condensing is fully known ahead of time. It now does that work in the background, right after responding. By the time you send your next message, the condensed history is already sitting in cache.
Measured end-to-end on the production engine, three consecutive turns of a qualifying conversation:
| Turn | Before | Now |
|---|---|---|
| First qualifying turn | ~1s (full condensation) | ~1s once (unchanged) |
| Next turn | ~1s again | 15ms |
| Turn after that | ~1s again | 11ms |
That is roughly a 70× reduction in request-path condensation cost on warmed turns. The background work that replaces it is smaller than what it replaced, because it also benefits from the fold-forward cache.
Being precise about the claim: this release saves time, not additional tokens. The savings curve — 15% at 7 messages climbing to 75% at 19 — is exactly as it was. The first turn that crosses the condensation threshold still pays the full cost once. And if you walk away long enough for the cache to move on, the next turn pays a reduced cost, not zero. Everything here is measured, and the daily test suite grew three assertions to pin the new behavior (112 checks, every morning, results emailed whether they flatter us or not).
Long conversations are where real work happens — debugging sessions, research threads, documents that take shape over an afternoon. They were already the cheapest place to work on Nyquest. Now they are also no slower than short ones. The platform gets cheaper the longer you talk, and as of this week, you stop paying for that in seconds.