The Web Doesn't Know Your Hours

Live web grounding is one of those features that looks unambiguously good. Ask about something that happened this morning and the model shouldn't be guessing from training data — it should go and look. Nyquest has done that for a while: a fast, zero-cost detector reads the question, and when the answer is likely to be fresher than the model, it searches, reads the top results, and hands them to the model as context.

Then we built an assistant whose entire job was to answer from one source of truth — a single organisation's own content — and watched live grounding confidently talk over it. This post is about the fix, because the underlying mistake is a general one and worth naming.

The failure, concretely

Picture an assistant on a restaurant's website. Its instructions are unambiguous: answer only from this site's content, never invent hours or prices, say you don't know if the site doesn't say. The site says dinner is served Monday to Saturday from 4:30pm.

A visitor asks: "are you open today?"

That word — today — is a textbook freshness signal, so grounding fires. The search returns what searches return for a business name: directory listings and aggregator pages that other people maintain. The assistant now has two sources: the restaurant's own site, and a page someone else last updated at an unknown date. And it answers in the restaurant's voice, because that's whose website it's on.

The failure isn't that the search happened. It's that nothing in the system said which source wins.

Position is precedence

Retrieved web context was being inserted at the front of the conversation — ahead of the developer's own system prompt. In a chat transcript, order is not cosmetic: it's a strong signal about what governs what. The instructions defining the assistant's entire purpose were arriving as a footnote behind a block of text pulled off the open internet ninety milliseconds earlier.

Web context is now injected after the caller's own instructions and still ahead of the conversation itself. It reads as what it is: reference material in service of the assistant's purpose, not a replacement for it.

Reference material, not marching orders

The injected block also carried its own instructions, and they were doing real damage. The text told the model to base your answer primarily on the search results above. Automatically retrieved third-party text was, in plain language, instructing the model to outrank whatever the developer had asked for.

That wording is gone. The block now says explicitly that it does not override instructions or context already present in the conversation, that it's there for what those don't cover, and that anything drawn from it should be attributed to its source rather than stated in the assistant's own voice. Same information; correct standing.

A switch for callers who already know

Inferring from phrasing is the right default for a general-purpose chat client, where the system genuinely can't know whether a question needs live data. It's the wrong default for an integration that already knows exactly what its assistant is for. An assistant embedded on an organisation's own site should never answer questions about that organisation from third-party pages, no matter how the question is phrased.

So that's now the caller's decision, via a request header or a field in the request body:

• off — never ground, whatever the phrasing suggests. Your context is the only context.
• auto — the detector decides. This is the default, and it is exactly what every existing integration already gets.
• force — ground every substantive request, for callers who know their domain is always live even when the wording looks static. A pasted URL still wins here: fetching the page someone pointed at beats searching for words about it.

The bug we found on the way

In the spirit of honest numbers: while testing the switch we discovered that the response headers reporting grounding — whether it happened, how many sources, which ones — were only ever sent when a request belonged to a stored conversation. Grounding itself ran for every request. So a stateless API call, which is what most integrations are, could receive a fully web-grounded answer with no way whatsoever to know it.

We caught it the unglamorous way: the server log said grounding injected sources=5 and the response carried no grounding headers at all. The reporting had been quietly scoped to the one kind of caller least likely to need it, and the callers most likely to care were the ones flying blind. Those headers now travel with every response, along with a new one reporting which of the three modes was applied.

What deliberately didn't change

The detector's trigger rules are untouched. They encode a series of specific, hard-won fixes — the day a bare "now" inside the word "know" fired a real web search, the day finance questions silently stopped grounding — and every one of those regression tests still passes. Sharpening when grounding fires is a real and separate piece of work with its own risks; it isn't a thing to bundle into a change about precedence.

The defaults are unchanged too. An integration that sends nothing new behaves exactly as it did before this shipped. The difference is that one which does know better can now say so.

The general lesson is not really about search. Any system that automatically injects context into someone else's prompt is making a claim about authority, whether or not it means to — in where it puts the text and in how that text describes itself. Getting retrieval right is the easy half. Deciding what loses is the other one.

Try Nyquest →

No account needed to start chatting — or bring your own API key.

← All posts