Scanner methodology
Every /scan score is a weighted fraction of applicable checks. This page defines the full check catalog, the tier weights, and the rules that keep the measurement neutral and reproducible. The rubric is versioned; changes are additive and published here.
1. Passive only
Remote scans perform the MCP initialize and tools/list JSON-RPC methods over Streamable HTTP. No tool is ever invoked and no adversarial payload is sent to the target. If tools/list requires OAuth, tool checks leave the denominator instead of failing.
2. Applicability
A check the target cannot satisfy by design (no mutation tools, read-only server, no RAG surface) is excluded from the denominator rather than scored as a failure. Score = earned weight / applicable weight.
3. Product neutrality
No check rewards a specific product. zn-gate appears only as one of several remediation options, always listed next to an open-source alternative. Reachable grades A+ to F require no purchase.
Grade bands
| Grade | Score | Meaning |
|---|---|---|
| A+ | 95 to 100 | Certified Resilient |
| A | 85 to 94 | Strong Baseline |
| B | 70 to 84 | Moderate Risk |
| C | 50 to 69 | Elevated Risk |
| F | below 50 | Critical Vulnerability |
Essential tier
12 pts eachBasic security. Missing controls score as failures.
Endpoints reachable over HTTPS with a valid certificate chain. Cleartext HTTP transport is rejected.
Fix: Terminate TLS at the edge with a managed certificate and redirect all cleartext requests.
Open alternative: Let's Encrypt + Caddy or Nginx certbot with HSTS.
MCP endpoints enforce an auth boundary (OAuth 2.1 with PKCE or signed bearer tokens) before serving tools.
Fix: Require OAuth 2.1 authorization-code flow with PKCE on every MCP transport endpoint.
Open alternative: Open-source identity providers (Keycloak, Ory Hydra, oauth2-proxy) expose the same flows.
Declared parameters include max lengths, enums, or patterns so a single call cannot carry unbounded payloads.
Fix: Constrain string parameters with maxLength or pattern and reject oversize bodies at transport level.
Open alternative: JSON Schema maxLength plus framework-level body size guards (e.g. express body-parser limits).
Tool descriptions and metadata scanned for adversarial imperatives buried in docstrings (indirect tool poisoning).
Fix: Rewrite descriptions as neutral capability statements; strip imperative clauses.
Open alternative: mcp-scan (Invariant Labs) and LlamaFirewall both flag poisoned descriptions.
Tools exposing arbitrary shell, eval, or command execution on the host without sandboxing are critical by design.
Fix: Replace open shell tools with granular scoped operations or run them inside a sandbox (Docker, Firecracker).
Open alternative: Firecracker microVMs, gVisor, or Docker with read-only rootfs and dropped capabilities.
Write or delete tools must be scoped to a declared sandbox root. Unscoped paths fail.
Fix: Bind the server to a single workspace root with respect for hidden-file exclusions.
Open alternative: The reference MCP filesystem server scopes roots the same way.
Tool parameters must not accept API keys, passwords, or tokens as model-visible arguments.
Fix: Inject credentials at the gateway layer via environment or secret manager, never through tool arguments.
Open alternative: Vault, AWS Secrets Manager, or Doppler with env-scoped injection.
Recommended tier
6 pts eachGood practice for exposed servers. Failures subtract weight.
Every tool parameter declares an explicit primitive type and required set. Open object types with no declared properties fail.
Fix: Declare types (string, integer, boolean) and required fields in every tool inputSchema.
Open alternative: Zod, Pydantic, or vanilla JSON Schema generate strict schemas the same way.
Irreversible fin actions (refund, transfer, charge, deploy) require an auditable human approval step.
Fix: Add a two-step confirmation flow with server-side approval tokens for mutation tools.
Open alternative: Any webhook-based approval queue (Slack bot, Linear flow) implements the same gate.
The target must show some mechanism to inspect or filter content returned by tools before it reaches the model, regardless of vendor.
Fix: Filter tool outputs inline (neural gate, regex policy, or framework middleware) before re-injecting into context.
Open alternative: LlamaFirewall, LLM Guard, or Azure Prompt Shield provide equivalent capacity.
Context cannot redefine, shadow, or override registered tool names from untrusted documents.
Fix: Rebuild the tool registry each turn and reject shadowed names.
Open alternative: The reference MCP SDK keeps client-controlled tool names immutable.
Bonus tier
2 pts eachExtra credit. If absent by design, the check is excluded from the denominator: it never subtracts.
A published, linkable statement about what the server retains from tool calls or conversations.
Fix: Publish a data-retention statement on the site.
Open alternative: A public docs page or trust report satisfies this without any vendor.
Verdicts and mutations produce hash-chained or signed evidence records.
Fix: Log verdicts with hash chaining or append-only storage.
Open alternative: Hash-chained logs with OpenSSL or an append-only Merkle log implement this pattern.
Remote scan mechanics
- DNS resolution is validated before every connection; requests pin one resolved address (no DNS rebinding window).
- Private, loopback, link-local (including 169.254.169.254), CGNAT, multicast, and documentation ranges are rejected at DNS time and again per redirect hop.
- Maximum 2 redirects, 4 second connect timeout, 512 KB response cap per request.
- Rate limit: 10 scans per hour per source IP.
- TLS certificates must validate against the scanned hostname (SNI, full chain).
- Every report carries per-check evidence and a permanent URL (
/scan/r/{id}/) with JSON, Markdown, and badge variants, so any score can be re-verified by running the same scan again.
Static mode (pasted artifacts)
Pasted system prompts and tool schemas are analyzed in the browser and never uploaded. The same lexicons and tier weights apply; transport checks are excluded from the denominator because pasted artifacts cannot carry them. This mode proves what the artifacts themselves declare, nothing more.
Normative references
- TLS 1.3 transport: RFC 8446.
- Model Context Protocol specification (2025-06-18): initialize handshake, tools/list, Streamable HTTP transport, authorization via OAuth 2.1.
- OAuth 2.0 Protected Resource Metadata: RFC 9728.
- OWASP Top 10 for LLM Applications (2025): LLM01 Prompt Injection, LLM02 Sensitive Information Disclosure, LLM06 Excessive Agency, LLM07 System Prompt Leakage, LLM08 Vector and Embedding Weaknesses.