From two APIs to one Smart API Router: rules, a neural gate and Deep Analyze behind a single endpoint

EngineeringJohn Silver9 min read

Updated on 2026-10-17: marked as a historical post, with a pointer to the current architecture, and removed the plan limits and the figures that no longer hold.

Historical post. This describes the router as it was on 2026-09-12: the neural model, the Deep Analyze tier and the plan limits of that date. Since 12 October 2026 the cloud check serves Galvanize 2 (threshold 0.662), and the router levels are rules, advanced and deep. For current plans and limits, see pricing.

Until this week, zn shipped two APIs. You picked one per request, and your plan limits were split across both. That was a design tax on every integration: the wrong choice was a false sense of security, and the right choice required understanding our internals better than you should have to.

Today there is one endpoint. POST /analyze routes every request through three escalating layers — deterministic rules, the neural gate, and Deep Analyze — and tells you which one decided.

TL;DR

  • POST /analyze is now a Smart API Router (SAR): Tier 1 deterministic rules (<50 µs), Tier 2 neural int8 (~15–18 ms), Tier 3 Deep Analyze (mmBERT v8, ~166 ms warm) for the ambiguous band.
  • /v30/analyze still works as a compatibility alias. The product path POST /prod/analyze stays rules-only for callers who want the ultra-low-latency gate.
  • The router is fail-open: if Deep Analyze times out (6 s budget) or errors, you still get the neural verdict. A security gateway must never become the outage.
  • Plan limits changed after this post. The current ones are on the pricing page.
  • We don't keep the text you send on any plan unless you opt into Contributor Mode. We keep evidence metadata, including a SHA-256 fingerprint of the input, for 90 days. The response includes tier, decided_by, latency_ms and an evidence_id you can audit.

Why two APIs was the wrong shape

The first endpoint, POST /analyze, is a deterministic rules engine: normalization plus a catalogue of injection patterns (pi-direct and friends). It answers in microseconds, it is trivially auditable, and when a rule fires it is almost never wrong. But rules only catch what you have seen before.

The second endpoint, POST /v30/analyze, runs a neural gate: a multilingual transformer, exported to ONNX and quantized to int8, that scores the raw text and blocks above a calibrated threshold (τ = 0.950 after the multilingual hardening pass). It is a different trade: ~15–18 ms p50 on CPU, dramatically better coverage, still nowhere near an LLM guard's cost or latency.

Both are honest products. Together, they were a bad interface. Customers had to know which threat model they were in to pick an endpoint per call, quotas were split, and evidence from the two paths was not directly comparable. In practice, people defaulted to the fast path and silently lost the neural coverage they were paying for.

One endpoint, three brains

The router decides per request, in the open:

  1. Tier 1 — Rules. Normalize, match deterministic patterns. If a rule fires, return. Cost: microseconds.
  2. Tier 2 — neural. If rules are inconclusive, score with the int8 model. High-confidence benign or attack returns here.
  3. Tier 3 — Deep Analyze. Only when the neural score lands in the uncertain band (0.50–0.935) does the request go to Deep Analyze, the mmBERT v8 model that exists for exactly this: text that looks benign-ish to a small encoder but deserves a second opinion.

The verdict fusion is simple on purpose: the router combines the neural score with the Deep Analyze score and blocks at ≥0.935. Every response now carries tier (rules, advanced, deep), decided_by, latency_ms and evidence_id, so you can measure your own traffic by decision path.

Before: two separate endpoints. After: one Analyze endpoint routed by the Smart API Router across rules, neural and Deep Analyze tiersTop panel shows the old POST /analyze rules endpoint and POST /v30/analyze neural endpoint requiring callers to choose. Bottom panel shows a single POST /analyze routed by the Smart API Router to Tier 1 rules under 50 microseconds, Tier 2 int8 about 15 milliseconds, and Tier 3 Deep Analyze mmBERT about 166 milliseconds warm, with a fail-open budget of 6 seconds and a response containing tier, latency and evidence id.FIGURE 1: TWO ENDPOINTS → ONE SMART API ROUTERBEFORE · CHOOSE PER REQUESTPOST /analyzeDeterministic rules · <50 µs · pattern coverage onlyPOST /v30/analyzeneural int8 · ~15–18 ms · split quotasAFTER · ONE CALLPOST /analyze→ SMART API ROUTERTIER 1 · RULES<50 µs · deterministicexact patterns, auditableTIER 2 · NEURAL~15–18 ms · int8 ONNXmultilingual, high coverageTIER 3 · DEEP ANALYZE~166 ms warm · mmBERT v8only the 0.50–0.935 bandresponse: verdict · tier · decided_by · latency_ms · evidence_idfail-open: Deep gets 6 s, then the neural verdict ships

Deep Analyze: the tier that had to earn its place

Deep Analyze is our multilingual mmBERT v8 detector, exported to ONNX int8 and served from a container Lambda (3 GB memory, 4 GB ephemeral, 120 s ceiling). Warm it answers in ~166 ms. Cold starts take ~24 s, which is why it never sits on the synchronous edge of an unprotected path: it only runs for the ambiguous band, inside a hard 6 s router budget, and the router falls back to the neural verdict if it misses.

Getting a model good enough to be worth that slot took longer than the router itself. Our earlier posts told the story in full: the dataset that quietly lied to us and seventeen failed training runs where the model scored 0.498–0.509 on everything because the loss never saw a label. The fix was ordinary — supervised end-to-end training, froze the split, stop tuning on the test set — and the result was a working gate on the frozen test split.

Decision ladder and latency budget on a logarithmic time axis: rules under 50 microseconds, the neural gate about 15 milliseconds, Deep Analyze about 166 milliseconds, router budget 6 secondsLogarithmic axis from 10 microseconds to 10 seconds. Tier 1 rules bar ends near 50 microseconds. Tier 2 neural bar ends near 15 milliseconds. Tier 3 Deep Analyze bar ends near 166 milliseconds. A dashed limit line marks the 6 second fail-open budget. Deep Analyze runs only when the neural score falls in the 0.50 to 0.935 band; 0.935 and above blocks.FIGURE 2: DECISION LADDER · LATENCY BUDGET (LOG SCALE)neural band 0.50–0.935 → Deep Analyze · score ≥0.935 → blockTIER 1 · RULESconclusive match returns~50 µsTIER 2 · NEURALint8 ONNX, p50~15–18 msTIER 3 · DEEP ANALYZEmmBERT v8, warm~166 ms6 s FAIL-OPEN LIMIT10 µs100 µs1 ms10 ms100 ms1 s10 sEvery tier logs tier, decided_by, latency_ms and evidence_id.Fail-open: a Deep timeout never turns into a failed request.

Shipping it without breaking production

A router that sits in front of every request is a scary deploy. We shipped it in three moves: shadow mode first, where both the live path and the router decided and we only compared — Deep Analyze scoring alongside the production verdict without touching it. Then canary at 50% behind the alias, with CloudWatch alarms on Lambda errors, Deep fail-opens and p99 latency, wired to SNS. Then 100%.

The boring guarantees are the point: rollback is an alias version switch, the router has 39/39 unit tests including an equivalence suite that proves the routed path returns the same verdict as the direct call, and the fail-open path is exercised in tests, not just hoped for.

Plans

The plan limits described when this post was published have changed. See pricing for the current plans and quotas.

Retention does not change: input is kept only in Contributor Mode. We keep the verdict, the tier and the hashes needed for the Evidence Vault, with 90-day retention. Contributor Mode is opt-in and is the only case where input is stored, anonymized, and only because you opt in to community telemetry for model training. If you stay out of Contributor Mode, we do not use your prompts for training. (Edited on 15 October 2026.)

What changes for you

If you were calling /v30/analyze, nothing breaks: same path, same response shape, now with tier and decided_by telling you where each verdict came from. If you were calling /analyze expecting rules-only, move to /prod/analyze for the deterministic path — or stay, and get the full router for the same price. One key, one endpoint, three brains.

If you want to see the decision path on your own traffic before trusting it, the Free plan (10,000 calls a month) is the way to do that: one POST, an API key, and evidence for every block.

Updated 14 October 2026: the data-handling wording above now describes what the package and the API do today.

Share