token_waste

What this detector catches

Three-layer detection pipeline runs on every llm_call's leading prompt window:

• Variable-prefix strip — normalizes 5 known drifting shapes (ISO/RFC3339 timestamps, hex UUIDs, request_id/trace_id/correlation_id key-value prefixes, leading numeric counters, and "Turn N" / "Step N" labels) so payloads that differ only in these prefixes still cluster together.
• Exact 2048-char SHA-256 hash — the canonical signal. When 3+ calls share the same hash, fires under token_waste:<hex8>.
• Shingle-Jaccard near-duplicate fallback — runs only when exact hash misses. k=8 character shingles; Jaccard threshold 0.85. Catches the structurally-similar-but-lexically-distinct cases the strip can't reach (customer IDs embedded mid-prefix, account labels, conversation-history drift). Fires under token_waste:near_dup:<hex8>. The hex8 suffix is deterministic across re-runs.

The detector emits one signature per execution — either an exact match or a near_dup, never both. The exact-match path runs first; near_dup runs only when exact found nothing. The field read from each llm_call payload is user_message(the LAST user-role message in the conversation as the SDK's instrument_* modules ship it).

Example failures it groups

• An agent retries a failed LLM call by re-running the exact same prompt (exact-hash group).
• A RAG agent embeds "current_time: 2026-06-23T14:23:01Z" in the system prompt and each iteration ticks. Strip normalizes timestamps; exact-hash group fires.
• A multi-step planner embeds the request_id mid-prompt; each call has a different ID. Strip handles key-value prefixes; clusters as one group.
• An agent sends nearly-identical prompts with a customer ID baked into the middle of the system prompt. Strip can't reach a mid-prefix variable; shingle-Jaccard catches it and fires token_waste:near_dup.

What to do when it fires

Open the failure-group detail page. The Playbook walks through the diagnostic: read the prefix being repeated; decide whether the repeats represent (a) retry logic that should add prompt caching or fewer retries, (b) a loop that should be bounded, (c) a streaming-pattern where each call legitimately rebuilds the same context. Cost-aware customers will also want to enable provider-side prompt caching for the leading prefix — token_waste is often the leading indicator that prompt caching would pay back immediately. The parallel loops detector may also fire on the same execution under its own failure_class — by design, each detector answers a different question about the same underlying calls.

When NOT to investigate

• Provider prompt caching is enabled — if you're using Anthropic or OpenAI prompt caching on the leading prefix, the actual cost of the repeated prefix is much lower than the gross token count suggests. The detector still fires on token count; treat as informational rather than actionable.
• Bulk-processing agents where the same prompt template legitimately applies to many items in sequence. Each call is independent work; the repeated prefix is the price of the template pattern.
• Low-volume agents under the 3-repeat threshold — the detector requires at least 3 repeats by default. Tune min_repeats down if you need earlier detection on a hot path.

Tunable thresholds

Two per-project knobs via the Detector thresholds primitive: prefix_window_chars (chars hashed per user_message; default 2048; tier-capped — Hobby 4KB, Team 16KB, Enterprise 64KB; bigger window = more CPU per execution-close) and min_repeats (minimum repeats before firing; default 3; pure alerting sensitivity, no tier cap, global bounds [2, 100]). No allowlist support today.

token_waste vs. semantic_loop: which fires when?

Both detectors catch "the agent is doing the same thing repeatedly" but at different layers of the execution. They can fire on the same execution by design — and frequently will when the underlying loop has both shapes. The right way to think about them:

• token_waste fires on the llm_calllayer: same leading prompt prefix sent 3+ times in one execution. It catches the "agent re-sent the same prompt" pattern that wastes input tokens. If your loop has byte-level repetition in the prompts going to the model, token_waste sees it.
• semantic_loop fires on the checkpointlayer: same canonical-state hash revisited 3+ times in one execution. It catches the "agent re-derived the same conclusion via different surface text" pattern — the prompts look different to token_waste, but the agent is logically going in circles. Requires the customer to call mesedi.checkpoint() at logical-state boundaries.

When both fire on the same execution: the loop has both shapes — start with token_waste because fixing the prompt-level repetition usually also fixes the state-level one. If only semantic_loop fires, the agent is varying surface text but stuck in the same logical state; the fix is at the planning layer, not the prompt layer. If only token_waste fires, the agent is sending identical prompts but actually making progress at the state layer (rare; usually means a retry-without-backoff bug rather than a stuck-agent bug).

Related docs

• loops — the parallel signal on the same calls. loops surfaces the same data under the "repetitive behavior" framing; token_waste surfaces it under the "wasted spend" framing.
• semantic_loop — the checkpoint-state sibling. token_waste operates on llm_call payloads; semantic_loop operates on checkpoint events.
• Detector thresholds — tune prefix_window_chars + min_repeats.