Mesedi
Start free →

Docs / Security / Data leakage

data_leakage

Mesedi scans outbound LLM prompts, tool-call arguments, and tool return values against a registry of regex rules matching secrets and PII. Hits are redacted before storage (Mesedi never stores the raw secret) and clustered into the data_leakagefailure class. The detector runs in-band at event ingest using Go's RE2 engine — zero ReDoS risk by construction.

What this detector catches

16 built-in rules. Mesedi's DLP severity model has three tiers internally — critical, high, and medium. The 16 built-ins currently use only critical (14 rules — real credentials and signed tokens) and high (2 rules — SSN and credit card; PII with non-trivial false-positive risk). The medium tier is reserved for custom rules and future built-in expansion. All three tiers are scanned + redacted at ingest; promotion to a failure_group is governed by the per-project severity_policy (default ["critical", "high"]). The rules are tuned to minimize false positives at the expense of some recall — production deployments are expected to extend the baseline with custom per-project rules via the Custom security patterns primitive.

The 16 built-in rules:

  • anthropic_api_key — matches sk-ant-... (modern + legacy formats).
  • openai_api_key — matches sk-... (legacy 48-char) + sk-proj-... (project-scoped).
  • gemini_api_key — matches AIza-prefixed 39-char keys.
  • aws_access_key — matches AKIA... 20-char access-key IDs.
  • aws_temporary_token — matches ASIA... 20-char STS tokens.
  • gcp_service_account_json — matches thetype: service_account block shape.
  • github_personal_token — matches ghp_... and github_pat_... formats.
  • github_oauth_token — matches gho_... + ghs_....
  • slack_token — matches xoxb-... / xoxa-... bot + user tokens.
  • stripe_live_secret_key — matches sk_live_....
  • stripe_live_publishable_key — matches pk_live_....
  • stripe_live_restricted_key — matches rk_live_....
  • jwt — matches the 3-segment header.body.signature shape.
  • private_key_pem — matches any -----BEGIN ... PRIVATE KEY----- header.
  • ssn_us — matches US Social Security Numbers (NNN-NN-NNNN format).
  • credit_card_pan — matches 13-19 digit primary account numbers (Luhn-validated).

Where it scans

Exact field names per event type (useful when grepping raw events on the dashboard):

  • llm_call: system_prompt, user_message, response_text, exception_message (provider exception text — may carry API keys returned in the error response or request IDs with embedded customer IDs).
  • tool_call: arguments, return_value, error, exception_message. Nested JSON in arguments / return_value is re-marshalled to text before scanning so regex hits on stringified inner values still fire. return_value is subject to the per-project tool_return_value_max_bytes cap.
  • validator_result: reason and message(validator failures often echo the failing slice of agent output verbatim into the rejection text).
  • exception_event: message (defensive catch-all; canonical crash path hashes before persist).

What to do when it fires

Open the failure-group detail page. The Playbook walks through three diagnostics: identify which rule_id fired (the signature carries the rule name), trace which event type produced the hit (LLM payload vs tool argument vs tool return), and decide between rotation (the secret actually leaked → rotate immediately) and policy fix (the secret was intended to flow but shouldn't have → tighten upstream code path). The raw secret is never stored — the dashboard shows redacted markers only.

When NOT to investigate

  • Test fixtures — if a synthetic-customer harness or an integration test legitimately emits canary tokens (e.g. Stripe test keys, fixture SSNs), the detector will fire. Filter on project ID + tag the synthetic project in your webhook-routing config.
  • Documentation embedded in prompts — some agents pass policy docs (PII redaction policy, SOC2 references) into the system prompt that contain example SSN-shaped strings. Use a custom pattern to allowlist the specific known-safe context, or rewrite the prompt to avoid the pattern.
  • Cohere API keys — there is no built-in rule for Cohere keys because their 40-char bare-alphanumeric format produces too many false positives (collides with SHA-1 hex, git SHAs, Stripe IDs). Cohere customers can add a context-aware custom rule via the custom-patterns primitive (e.g. require a nearby cohere keyword).

Custom rules

Per-project custom DLP rules ship via the same primitive that backs the prompt_injection + sandbox_escape custom patterns. See Custom security patterns for the editor + REST surface. Custom DLP patterns scan, redact in place (replaced with [REDACTED:custom-<pattern_id>]), and promote to a failure_group per the same severity_policy as built-in rules. Pattern severity maps as: low → medium tier, medium → high tier, high → critical tier (matches the dashboard chip-color semantics).

Related docs