Mesedi
Start free →

Docs / Observability / Context overflow

context_overflow

The agent's cumulative input tokens crossed a fraction of the model's context window. Mesedi tracks per-model high-water marks per execution and fires under two severity levels (warn at 90% and fail at 100% by default) when the configured utilization thresholds are crossed. Pure-deterministic detector; fail-safe on unknown models (skips rather than guesses).

What this detector catches

On every llm_call event, the backend resolves the model name to its context window from the registry, computes input_tokens / window_size as a utilization fraction, and tracks the per-model high-water mark for the execution. At execution close, the detector fires under context_overflow:warn:<model> when the high-water mark crossed high_pct (default 90%) or context_overflow:fail:<model> when it crossed critical_pct (default 100%). The signature carries the model name so a regression that flips a model from warn → fail surfaces as a distinct new cluster (the prior warn cluster keeps its identity). Both percentage thresholds are per-project tunable.

Note: the per-project knob names use high_pct / critical_pct, but the emitted signature levels are warn / fail (historical naming preserved so existing dashboard signature filters keep working).

Supported models

Registry covers 50+ models across:

  • Anthropic — Claude 3 / 3.5 / 4 / 4.5 / Opus / Sonnet / Haiku families.
  • OpenAI — GPT-4 / GPT-4o / GPT-5 / o1 / o3 / o4 families.
  • Google — Gemini 1.5 / 2.0 / 2.5 Pro and Flash.
  • Cohere — Command-R / Command-R-Plus / Command-Light.
  • Meta + Mistral — Llama 3 / 3.1 / 3.2 / 3.3 + Mistral Large / Medium / Small / Mixtral.

Routed identifiers are normalized before lookup. Bedrock prefixes (e.g. anthropic.) and date/version suffixes (e.g. -20240620-v1:0) are stripped; Vertex version suffixes (e.g. -001) are stripped. anthropic.claude-3-5-sonnet-20240620-v1:0 resolves the same window as claude-3-5-sonnet.

What it does NOT catch

  • Azure OpenAI deployment names — deployment names are customer-chosen and non-deterministic (e.g. my-team-gpt4-prod); automatic normalization is impossible without a customer-supplied mapping. Use the per-project custom_model_windows tunable (see below) to declare your deployment names and their window sizes — the detector will resolve via the override and produce accurate signals. Without an override, Azure deployments fall through the fail-safe path (detector skips silently rather than guessing).
  • Unknown / fine-tuned models without an override — the detector skips when the resolved model name isn't in the static registry AND has no entry in custom_model_windows. Fail-safe is the right posture (a guess could fire false-positives en masse). Declare the window via the override or request a registry addition.
  • Token-counting drift across providers — different providers count tokens differently (e.g. tokenizer boundary rules); the detector uses whatever input_tokens the provider reports. Gross comparison across providers may show small offsets.

What to do when it fires

Open the failure-group detail page. The Playbook walks through context-management strategies: bound chat history with a sliding window; cap retrieved-chunk counts in RAG agents; summarize previous turns instead of carrying them verbatim; switch to a larger-window model if your task legitimately needs more context. high_pct surfaces typically warn-tier; critical_pct surfaces typically fire-tier because they indicate the provider may truncate the prompt (silent context loss).

Tunable thresholds

Three per-project knobs via the Detector thresholds primitive (none tier-capped):

  • high_pct (default 0.90; bounds [0.5, 1.0]). Soft alarm threshold — emits the warn signature when crossed.
  • critical_pct (default 1.00; bounds [0.5, 1.0]). Hard alarm threshold — emits the fail signature when crossed. Must satisfy high_pct < critical_pct; violation reverts both percentages to defaults.
  • custom_model_windows (default empty map; max 50 entries; values bounded [1024, 10,000,000] tokens). JSON map of model_id → window for models not in the static registry (Ollama tag-style identifiers, fine-tuned models, Azure deployments) OR for known models where you want to enforce a tighter window (e.g. behind a proxy that caps inputs). Overrides win over the static registry even for known models.

Related docs