Engine · open source · v3.2.8 · MIT
Drop-in compression proxy for LLMs. 532 regex rules, bounded LRU cache, local LLM semantic stage, 15–75% token savings. This is the public v3.2.8 release, with a published Docker image. Development has continued since: the hosted platform runs a later, proprietary build of the same lineage.
No telemetry · No phone-home · Privacy mode by default
Same lineage as production
No "lite" version: the public v3.2.8 release is the full engine, sanitized for
release (hardcoded keys and internal paths removed). Production has continued
past it on proprietary builds of the same lineage, so the two are no longer
identical.
v3.2 adds a RegexSet
prefilter, zero-allocation rule misses via Cow<str>, a bounded
LRU compression cache, per-rule hit telemetry, a regression suite of 47 passing
tests, and a multi-stage Dockerfile auto-published to GHCR.
Measured on production traffic, July 11 2026. Savings scale with conversation length — from ~15% at seven messages to 75% on long sessions — and are verified continuously by a shadow-eval loop that re-runs sampled prompts uncompressed and has independent judges compare the answers. On top of compression, provider-side prompt caching now cuts the cost of resent context by 50–90% on two major vendors. The week in review →
Quickstart
All three converge on a running proxy at localhost:5400. Health
check, web dashboard, OpenAI-compatible endpoint, Anthropic-native endpoint —
all there.
After install: curl localhost:5400/health · dashboard at localhost:5400/dashboard · cache snapshot at localhost:5400/admin/v2/compression/cache
Pipeline
Every request flows through this pipeline. Stages 2 and 5 are toggleable; stages 1, 3, 4, 6 always run.
Stages run in microseconds (1, 3, 4) to a few hundred ms (5, GPU). Hot path stays under 1ms (p50 <1ms measured) · cached identical content returns in ~19µs (~3,730× speedup vs cold).
Compression levels
A single 0.0–1.0 knob. Server default is 0.7 (balanced). Override
per-request with x-nyquest-level.
| Level | Strategy | Rules active | Typical savings |
|---|---|---|---|
0.0 | Pass-through (metrics only) | — | 0% |
0.2 | Filler removal | ~95 rules | 5–12% |
0.5 | Structural compression | ~250 rules | 15–28% |
0.7 | Default · balanced | ~390 rules | 18–33% |
1.0 | Aggressive + format + minify | 532 rules | 23–35% rules · up to 75% with semantic |
Hardware tiers
The preflight command (nyquest preflight -v) inspects your machine
and recommends a tier. You can always force a different one in nyquest.yaml.
Pure regex pipeline. No semantic stage. Ridiculously fast hot path — sub-millisecond p50.
2+ CPU cores · 512 MB RAM · No GPU required · Runs on a Raspberry Pi 4
Rules + Qwen 2.5 1.5B on a small GPU. 200–350ms semantic latency. The production tier.
4+ CPU cores · 6 GB RAM · 2+ GB VRAM (GTX 1080+ class) · Ollama installed
No GPU? CPU runs the semantic stage at 1–4 seconds per call. Same savings, slower path.
4+ CPU cores · 8 GB RAM · No GPU · Ollama installed
Benchmarks
Measured live on the v3.2 Docker container (Ubuntu 24.04 host,
--network host). Full breakdown in
CHANGELOG.md.
| Test | Result | Detail |
|---|---|---|
| Reference prompt (358 tokens) @ L0.3 | 26.6% saved | 319 → 234 tokens |
| Reference prompt @ L0.5 | 28.2% saved | 319 → 229 tokens |
| Reference prompt @ L1.0 | 34.8% saved | 319 → 208 tokens |
| Aggregate across 8 realistic scenarios @ L1.0 | 22.6% mean | 2,201 → 1,703 tokens · Haiku 4.5 conservative profile |
| Semantic stage — system prompts | 55.9% saved | 200–350ms on GPU |
| Semantic stage — conversation history | 75% saved | 200–350ms on GPU |
| Health throughput (ab -n 5000 -c 1) | 8,266 req/s | p50 <1ms · host networking |
| Concurrent (ab -n 5000 -c 20) | 18,287 req/s | p50 1ms · p95 1ms · p99 2ms |
| Cache hit (warm replay) | ~3,730× speedup | ~70.9 ms cold → ~19 µs hit on ~500-char prompt |
| Failed requests | 0 / 10,000 | across single-thread + concurrent runs |
| Memory footprint | 48.5 MB RSS | 15 threads · Docker container |
| Test suite | 47 / 47 passing | 16 safety-invariant snapshots · 6 measurement reports · 25 role personas |
Compression rows measured against Claude Haiku 4.5, which uses the engine's
conservative profile (raised in v3.2 with specific-marker
matching so -mini / -flash / -nano / -light
always win over family-name substrings). Frontier models (Opus, Sonnet, GPT-5.5,
Grok-3) use the aggressive profile and typically see
6–10 percentage points higher savings on the same prompts.
Production cumulative
Lifetime stats from the live engine's /metrics endpoint, processing
real traffic behind app.nyquest.ai:
Average is low because production traffic skews toward short prompts — the long-context savings are where the engine earns its keep.
8 real-world scenarios
Per-prompt compression on Haiku 4.5 (conservative profile). Each scenario is a
verbose, professional system prompt of the kind real users send. Re-measured on
v3.2 (tests/v320_compression_report.rs).
| Scenario | @ 0.5 | @ 0.7 | @ 1.0 |
|---|---|---|---|
| Customer Support | 28.1% | 33.4% | 34.7% |
| Legal Review | 14.1% | 16.0% | 27.5% |
| Data Science | 14.6% | 16.5% | 18.1% |
| Travel Planning | 22.9% | 23.3% | 25.9% |
| Code Review | 11.7% | 13.8% | 20.7% |
| Financial Advisor | 12.5% | 10.9% | 17.5% |
| HR Policy | 11.6% | 13.6% | 18.2% |
| Medical Education | 9.3% | 10.8% | 14.3% |
| AGGREGATE (mean) | 15.9% | 17.7% | 22.6% |
Cost impact at scale
Savings calculated against published provider rates. With BYOK + the engine in front of your app, these come out of your bill, not Nyquest's.
| Model | Price / 1M input | 100M tok/mo cost | Saved @ 0.7 | Saved @ 1.0 |
|---|---|---|---|---|
| Claude Haiku 4.5 | $1.00 | $100 | $17.70 | $22.60 |
| Claude Sonnet-class | $3.00 | $300 | $53.10 | $67.80 |
| Claude Opus 4.8 | $5.00 | $500 | $88.50 | $113.00 |
| GPT-5.5 | $5.00 | $500 | $88.50 | $113.00 |
| Grok-3 | $3.00 | $300 | $53.10 | $67.80 |
Illustrative provider list rates (May 2026) — live rates in the app. Savings calculated against v3.2 aggregate mean (8-scenario suite, conservative profile). Add the semantic stage and the savings double or triple on long-context workloads.
Integration
Point your existing OpenAI or Anthropic client at localhost:5400
instead of the cloud endpoint. Done. The proxy auto-detects which provider
you're calling and translates between formats as needed.
Features
Filler removal, verbose phrases, imperative conversion, clause collapse, code minify (Python/JS/Bash), JSON→CSV, AI-output noise, and more. Tier-gated by compression level. Every rule has an atomic hit counter exposed via the admin API.
Each category builds a regex::RegexSet at startup. On every call, only candidate rules run — rules that can't match are skipped without a full text scan. Combined with Cow<str> zero-allocation misses, the hot path stays well under 1ms.
Compression results keyed by sha256(content) + level + profile + mode. Default 2048 entries / 64KB max entry size. Tunable via NYQUEST_CACHE_CAPACITY / NYQUEST_CACHE_MAX_ENTRY_SIZE. Warm replay: ~3,730× speedup on a ~500-char prompt.
Qwen 2.5 1.5B via Ollama. Condenses long system prompts (~56%) and conversation history (~75%) with provider-aware fallback to extractive compression on timeout.
7-strategy pipeline for autonomous agents: tool result pruning, schema minimization, thought-block compression, error dedup, sliding window, cache injection, file-view condensation.
Aggressive (Opus, Sonnet, GPT-5), Balanced (unknown), Conservative (Haiku, Mini, Flash, Nano, Light). v3.2 fixed selector ordering so specific markers (-mini, -flash, -nano, -light) always win over family-name substrings.
tool_use and image blocks pass through byte-identical. Python tuples preserved. Four-digit dates preserved. Role declarations rewritten as Role: X. instead of stripped. Fail-closed length contract: returns original if compression produced a longer string.
/admin/v2/compression/rules exposes per-category and per-rule hit counts (no prompt content). /admin/v2/compression/cache exposes cache snapshot (entries, hits, misses, evictions). /dashboard for live request stats.
Multi-stage Dockerfile (Rust builder + Debian-slim runtime, non-root user, tini PID-1). docker-compose.yml with --network host and a prod profile (read-only root FS, resource caps). Auto-published to ghcr.io/nyquest-ai/nyquest-engine on every v* tag.
47 / 47 passing. 16 regression snapshots for safety invariants (Python tuple syntax, four-digit dates, profile-detection ordering, byte-identical tool_use pass-through, non-English text). 6 measurement reports. 25 role personas across 7 categories.
No telemetry. No phone-home. Prompt content is never logged — admin endpoints expose hit counts and regex sources, not data. Run it air-gapped.
Optional AES-256-GCM encryption for API keys at rest. Keys never logged, never written to request logs, only decrypted in memory at dispatch.
CLI
| Command | What it does |
|---|---|
nyquest install | Interactive 11-section setup wizard |
nyquest install --defaults | Headless mode with defaults (CI / Docker) |
nyquest preflight -v | OS, CPU, RAM, disk, GPU, Ollama, network — verbose with tier rec |
nyquest doctor | 10-point health check (config, port, keys, dashboard, logs, systemd) |
nyquest config show | Display all resolved config values |
nyquest config set <key> <value> | Set a single value (dot-notation) |
nyquest serve | Start the proxy server (default if no command) |
v3.2.8 is the published release. Pull the Docker
image from ghcr.io/nyquest-ai/nyquest-engine:3.2.0 or build from
source. PRs welcome — see
CONTRIBUTING.md.