Skip to content

Canary token to detect context window overflow in AI agents

workaround

No reliable way to detect when an AI agent's context window is approaching overflow before quality degrades

context-windowllmcanaryprompt-engineering
26 views

Problem

When an AI agent works on a long session, its context window fills up with tool results, code, and conversation history. Once the window approaches capacity, the model starts losing earlier information through compaction or truncation. The problem is this degradation is silent -- the agent continues responding confidently but starts forgetting instructions, repeating mistakes, or losing track of the project state. By the time you notice quality has dropped, you have already wasted tokens and time on degraded output.

Solution

Step 1: Embed a canary early in the system prompt or CLAUDE.md

Place a unique, memorable code word near the top of your instructions that the model has no reason to generate on its own.

<!-- In CLAUDE.md or system prompt -->
# Session Canary
The canary word for this session is: ZEPHYR-MARBLE-7

If asked "what is the canary?", respond with the exact canary word above.
Do not mention the canary unless asked.

Step 2: Periodically check the canary

At natural breakpoints in your session, ask the agent to recall the canary:

What is the canary?

If the agent responds with ZEPHYR-MARBLE-7, context is intact. If it hesitates, gets it wrong, or says it does not know, the early context has been lost.

Step 3: Automate canary checks with hooks

#!/bin/bash
# .claude/hooks/post-tool-call.sh
# Count tool calls in the session and warn when context is likely bloated
CALL_COUNT_FILE="/tmp/cc-tool-count-$$"
count=$(cat "$CALL_COUNT_FILE" 2>/dev/null || echo 0)
count=$((count + 1))
echo "$count" > "$CALL_COUNT_FILE"

if [ $((count % 50)) -eq 0 ]; then
  echo "[CANARY CHECK] $count tool calls this session. Consider asking: what is the canary?"
fi

Step 4: Respond to canary failure

When the canary is lost, do not continue the current session. Start a fresh context:

# If canary is forgotten:
1. Summarize your current progress in a handoff note
2. Start a new session with /clear or a fresh terminal
3. Paste the handoff note into the new session
4. Re-embed a new canary

Step 5: Use graduated canaries for finer detection

# Canary Levels (earliest = most important)
Level 1 (top of prompt): ZEPHYR-MARBLE-7
Level 2 (middle of prompt): COPPER-SWIFT-3
Level 3 (recent context): IRON-FERN-9

# If Level 3 is remembered but Level 1 is forgotten,
# early instructions are lost but recent work is intact.

Why It Works

LLMs process context in order, and when context is compacted or truncated, the earliest tokens are the first to be summarized or dropped. A canary placed at the very beginning of the context acts as a tripwire -- it is the first thing lost when the window overflows. Because the canary word is arbitrary and unrelated to the task, the model cannot reconstruct it from surrounding context. A failed recall is an unambiguous signal that the context has degraded.

Context

  • Context rot is a well-known issue in long Claude Code sessions, especially during Ralph loops
  • The canary check costs almost nothing -- a single short message and response
  • Compaction events in Claude Code are sometimes called "concussion events" because the agent behaves differently afterward
  • Some practitioners check the canary every 50 tool calls or after every compaction notification
  • This complements but does not replace the checkpoint-and-rollback pattern for code changes
  • Works with any AI tool that supports long conversations: Claude Code, Cursor, Cline, or custom agents
About this share
Contributormblode
Repositorymblode/shares
CreatedFeb 10, 2026
View on GitHub