Docs / Observability / Detector thresholds

Detector thresholds

Six of Mesedi's detectors expose per-project tunable thresholds. Defaults match the historical hardcoded values, so customers who don't tune see zero behavior change. Tune to push alerting sensitivity for a specific workload — tighter detection on production agents, looser detection on noisy roleplay agents.

What you can tune

The six detectors and the values exposed:

  • semantic_loop revisit_threshold (int, default 3). How many times the same checkpoint state must repeat before semantic_loop fires. Raise it for agents that legitimately re-check work; lower it for tighter loop detection.
  • token_waste prefix_window_chars (int, default 2048) and min_repeats (int, default 3). The prefix window is the leading characters hashed per user_message for exact-prefix clustering. The min repeats is how many repeated prefixes trigger the detector.
  • tool_schema_drift min_history_calls (int, default 10). How many historical calls per tool the detector needs before evaluating drift on a new shape. Higher values delay first signal but reduce false-positive priming noise on low-volume tools.
  • grounding_failure mean_floor (float, default 0.5). Below this rolling-mean evaluator score, grounding_failure fires (when the evaluator did not return passed=false explicitly). Raise it for stricter alerting on borderline scores.
  • drift — three lexical-drift cutoffs: lexical_threshold_low (default 0.45), lexical_threshold_medium (default 0.55), lexical_threshold_high (default 0.70). Bucket cutoffs for the cosine-distance signal. The three must satisfy low < medium < high; violating the ordering reverts to defaults (see Defensive fallback below).
  • context_overflow high_pct (float, default 0.90) and critical_pct (float, default 1.00). Fractions of the model context window at which the HIGH and CRITICAL buckets fire. Both must be in [0.5, 1.0] and high_pct < critical_pct; violation reverts to defaults.

How to tune (dashboard)

Go to Settings and scroll to the "Detection thresholds" sections (one per detector). Each row shows the threshold key, description, current value (with a Default badge when unchanged), and Save + Revert-to-Default buttons. Server-side validation rejects out-of-bound values with a precise message; the editor renders the error inline next to the row.

How to tune (API)

Prefer scripting? All operations are available via REST:

# List all tunable thresholds for a detector.
curl -H "Authorization: Bearer $MESEDI_API_KEY" \
  https://api.mesedi.ai/me/detector-thresholds/semantic_loop

# Set a custom value.
curl -X PUT -H "Authorization: Bearer $MESEDI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"value":5}' \
  https://api.mesedi.ai/me/detector-thresholds/semantic_loop/revisit_threshold

# Revert to default.
curl -X DELETE -H "Authorization: Bearer $MESEDI_API_KEY" \
  https://api.mesedi.ai/me/detector-thresholds/semantic_loop/revisit_threshold

Tier caps

Only one threshold has a per-tier cap: token_waste.prefix_window_chars. Wider windows mean hashing more bytes and a larger shingle set per execution-close — both real CPU vectors on the detector hot path. Caps:

  • Hobby — max 4,096 chars (4 KB)
  • Team — max 16,384 chars (16 KB)
  • Enterprise — max 65,536 chars (64 KB)

The other nine thresholds are pure alerting-sensitivity knobs (raising / lowering them just changes when the detector fires, with no asymmetric cost to Mesedi) and ship with global bounds only, no tier discrimination. Same doctrine as the existing cost_velocity threshold.

Defensive fallback

Detectors defend against bad stored values rather than crash:

  • Out-of-bound values (caught at write time by the server) revert to the registry default on read.
  • drift: if low < medium < high ordering is violated, all three values revert to defaults (0.45 / 0.55 / 0.70). Bucketing chaos is worse than ignoring a broken override.
  • context_overflow: if high_pct < critical_pct is violated, both values revert to defaults (0.90 / 1.00).
  • Store-read errors during execution-close revert all of that detector's thresholds to defaults; the execution close proceeds.

Every fallback writes a durable audit event under the same config_fallback action used by the other per-project primitives. The Settings dashboard surfaces a counter so you see when your config is being silently ignored.

Signature stability

Customers who don't tune see signature strings byte-identical to historical behavior (e.g. lexical_drift_0.45+, context_overflow:warn:claude-opus-4-6, semantic_loop:abc12345). Customers who tune drift get signatures embedding their cutoff values (lexical_drift_0.60+) so dashboard filters stay meaningful. Existing alert routing keyed on signature strings continues to work — the clustering namespace is unchanged.

Design choices you should know

  • Defaults never change unless you opt in. Backward compatibility is non-negotiable — customers who never touch the editor see the same detection behavior they had before this feature shipped.
  • Hardcoded ranges are intentional. Global bounds (e.g. revisit_threshold in [2, 1000]) prevent both alerting silence (1 would fire on every repeat) and silent false-negatives (10,000 would never fire on real workloads). The bounds reflect Mesedi's tested range, not your tier's plan economics.
  • No retroactive re-detection. Changing a threshold affects future executions only. Past failure_groups stay clustered under their original signatures.
  • hitl_rejection_spike is intentionally NOT tunable. Its 40% / 30% / 5-spike floors are statistical guardrails, not customer policy. Re-evaluation post- launch if production data shows real need.