From two APIs to one Smart API Router: rules, v29 and Deep Analyze behind a single endpoint

zn10 min read

Until this week, zn shipped two APIs. You picked one per request, and your plan limits were split across both. That was a design tax on every integration: the wrong choice was a false sense of security, and the right choice required understanding our internals better than you should have to.

Today there is one endpoint. POST /analyze routes every request through three escalating layers — deterministic rules, the v29 neural gate, and Deep Analyze — and tells you which one decided.

TL;DR

  • POST /analyze is now a Smart API Router (SAR): Tier 1 deterministic rules (<50 µs), Tier 2 v29 neural int8 (~15–18 ms), Tier 3 Deep Analyze (mmBERT v8, ~166 ms warm) for the ambiguous band.
  • /v30/analyze still works as a compatibility alias. The product path POST /prod/analyze stays rules-only for callers who want the ultra-low-latency gate.
  • The router is fail-open: if Deep Analyze times out (6 s budget) or errors, you still get the v29 verdict. A security gateway must never become the outage.
  • Plans now carry explicit DeepAnalyze quotas per tier, from 25/month on the 48 h trial up to 30,000/month on Growth, with overage packs from $9.
  • Zero data retention on paid tiers. The response includes tier, decided_by, latency_ms and an evidence_id you can audit.

Why two APIs was the wrong shape

The first endpoint, POST /analyze, is a deterministic rules engine: normalization plus a catalogue of injection patterns (pi-direct and friends). It answers in microseconds, it is trivially auditable, and when a rule fires it is almost never wrong. But rules only catch what you have seen before.

The second endpoint, POST /v30/analyze, runs v29: a multilingual transformer, exported to ONNX and quantized to int8, that scores the raw text and blocks above a calibrated threshold (τ = 0.950 after the multilingual hardening pass). It is a different trade: ~15–18 ms p50 on CPU, dramatically better coverage, still nowhere near an LLM guard's cost or latency.

Both are honest products. Together, they were a bad interface. Customers had to know which threat model they were in to pick an endpoint per call, quotas were split, and evidence from the two paths was not directly comparable. In practice, people defaulted to the fast path and silently lost the neural coverage they were paying for.

One endpoint, three brains

The router decides per request, in the open:

  1. Tier 1 — Rules. Normalize, match deterministic patterns. If a rule fires, return. Cost: microseconds.
  2. Tier 2 — v29 neural. If rules are inconclusive, score with the int8 model. High-confidence benign or attack returns here.
  3. Tier 3 — Deep Analyze. Only when the v29 score lands in the uncertain band (0.50–0.935) does the request go to Deep Analyze, the mmBERT v8 model that exists for exactly this: text that looks benign-ish to a small encoder but deserves a second opinion.

The verdict fusion is simple on purpose: the router combines the v29 score with the Deep Analyze score and blocks at ≥0.935. Every response now carries tier (rules, advanced, deep), decided_by, latency_ms and evidence_id, so you can measure your own traffic by decision path.

Before: two separate endpoints. After: one Analyze endpoint routed by the Smart API Router across rules, v29 neural and Deep Analyze tiersTop panel shows the old POST /analyze rules endpoint and POST /v30/analyze neural endpoint requiring callers to choose. Bottom panel shows a single POST /analyze routed by the Smart API Router to Tier 1 rules under 50 microseconds, Tier 2 v29 int8 about 15 milliseconds, and Tier 3 Deep Analyze mmBERT about 166 milliseconds warm, with a fail-open budget of 6 seconds and a response containing tier, latency and evidence id.FIGURE 1: TWO ENDPOINTS → ONE SMART API ROUTERBEFORE · CHOOSE PER REQUESTPOST /analyzeDeterministic rules · <50 µs · pattern coverage onlyPOST /v30/analyzev29 neural int8 · ~15–18 ms · split quotasAFTER · ONE CALLPOST /analyze→ SMART API ROUTERTIER 1 · RULES<50 µs · deterministicexact patterns, auditableTIER 2 · v29 NEURAL~15–18 ms · int8 ONNXmultilingual, high coverageTIER 3 · DEEP ANALYZE~166 ms warm · mmBERT v8only the 0.50–0.935 bandresponse: verdict · tier · decided_by · latency_ms · evidence_idfail-open: Deep gets 6 s, then the v29 verdict ships

Deep Analyze: the tier that had to earn its place

Deep Analyze is our multilingual mmBERT v8 detector, exported to ONNX int8 and served from a container Lambda (3 GB memory, 4 GB ephemeral, 120 s ceiling). Warm it answers in ~166 ms. Cold starts take ~24 s, which is why it never sits on the synchronous edge of an unprotected path: it only runs for the ambiguous band, inside a hard 6 s router budget, and the router falls back to v29 if it misses.

Getting a model good enough to be worth that slot took longer than the router itself. Our earlier posts told the story in full: the dataset that quietly lied to us and seventeen failed training runs where the model scored 0.498–0.509 on everything because the loss never saw a label. The fix was ordinary — supervised end-to-end training, froze the split, stop tuning on the test set — and the result was a gate with FPR 0.82%, FNR 8.20%, AUROC 0.9958 on the frozen test split, at 17.9 ms p50 int8.

Deep Analyze pushes that further on the hardest slice. On our internal probes it separates adversarial inputs at 0.9999999998 while benign-but-suspicious text sits at 1.35e-7. Those are not marketing probabilities; they are the numbers our own gateway sees when it decides, logged with an evidence_id for every block.

Decision ladder and latency budget on a logarithmic time axis: rules under 50 microseconds, v29 about 15 milliseconds, Deep Analyze about 166 milliseconds, router budget 6 secondsLogarithmic axis from 10 microseconds to 10 seconds. Tier 1 rules bar ends near 50 microseconds. Tier 2 v29 bar ends near 15 milliseconds. Tier 3 Deep Analyze bar ends near 166 milliseconds. A dashed limit line marks the 6 second fail-open budget. Deep Analyze runs only when the v29 score falls in the 0.50 to 0.935 band; 0.935 and above blocks.FIGURE 2: DECISION LADDER · LATENCY BUDGET (LOG SCALE)v29 band 0.50–0.935 → Deep Analyze · score ≥0.935 → blockTIER 1 · RULESconclusive match returns~50 µsTIER 2 · v29 NEURALint8 ONNX, p50~15–18 msTIER 3 · DEEP ANALYZEmmBERT v8, warm~166 ms6 s FAIL-OPEN LIMIT10 µs100 µs1 ms10 ms100 ms1 s10 sEvery tier logs tier, decided_by, latency_ms and evidence_id.Fail-open: a Deep timeout never turns into a failed request.

Shipping it without breaking production

A router that sits in front of every request is a scary deploy. We shipped it in three moves: shadow mode first, where both the live path and the router decided and we only compared — Deep Analyze scoring alongside the production verdict without touching it. Then canary at 50% behind the alias, with CloudWatch alarms on Lambda errors, Deep fail-opens and p99 latency, wired to SNS. Then 100%.

The boring guarantees are the point: rollback is an alias version switch, the router has 39/39 unit tests including an equivalence suite that proves the routed path returns the same verdict as the direct call, and the fail-open path is exercised in tests, not just hoped for.

Plans: bigger limits, explicit DeepAnalyze quotas

The old plans counted one bucket of calls and left Deep Analyze as an experiment. The new canon gives every tier a standard quota and a DeepAnalyze quota in the same place, and raises the ceilings:

Subscription limits: trial 100 standard and 25 DeepAnalyze, contributor 50,000 and 500, hobby 100,000 and 1,000, starter 500,000 and 5,000, growth 2,500,000 and 30,000, enterprise unlimitedEach row shows the subscription tier, its monthly standard analyze calls and its monthly DeepAnalyze calls, right aligned with thin separators. Paid tiers retain zero data retention; the free Contributor tier is telemetry opt-in.FIGURE 3: SUBSCRIPTIONS · MONTHLY LIMITSSTANDARDDEEPANALYZETrial · 48 h10025Contributor · free forever50,000500Hobby · $19100,0001,000Starter · $49500,0005,000Growth · $1992,500,00030,000Enterprise · customunlimitedunlimitedEvery subscription includes the full router: rules, v29 and Deep Analyze.Paid tiers keep zero data retention · Contributor is telemetry opt-in.

Retention does not change: paid tiers store zero input. We keep the verdict, the tier and the hashes needed for the Evidence Vault, with 90-day retention. The Contributor tier — free forever — is the only one that stores input, anonymized, and only because you opt in to community telemetry for model training. If you use any paid tier, your prompts are not in our training set. Ever.

What changes for you

If you were calling /v30/analyze, nothing breaks: same path, same response shape, now with tier and decided_by telling you where each verdict came from. If you were calling /analyze expecting rules-only, move to /prod/analyze for the deterministic path — or stay, and get the full router for the same price. One key, one endpoint, three brains.

If you want to see the decision path on your own traffic before trusting it, that is what the free 48 h trial is for: one POST, an API key, and evidence for every block.

Share