Context Guardian

Your local-model session hits the context limit and dies. This stops that.

mcp-name: io.github.LuminariSoftwares/context-guardian

Quick start · Works with · Client settings · Examples · Install both · Why · Two ways to run it · Install the proxy · Install in DSH · See it work · Compatibility · Changelog

Compaction for local models that actually fires, and never takes the conversation down with it. Two front doors, one idea: a proxy that sits in front of any OpenAI-compatible backend (Ollama, LiteLLM, Headroom, vLLM, LM Studio) for any CLI or agent, and a native engine for DeepSeek Harness. Both keep the full original on disk, and both fail open.

Measured: in a 48-prompt DeepSeek Harness session on qwen3-coder:30b (Ollama, 64K window) with codenames changed mid-session, 0.1.0-alpha.9 kept 12/12 facts current with 0 stale answers and brought back 3/3 early details in every run (n = 3), where pins alone brought back 0/3 (n = 2). Details: Measured.

Works with

The proxy needs only an OpenAI-compatible /v1/chat/completions; your CLI points OPENAI_BASE_URL at it. "Expected" means not yet tested; please open an issue with what you saw. Full matrix: Compatibility.

Quick start

pip install context-guardian && context-guardian   # proxy on :8786 in front of Ollama on :11434; set OPENAI_BASE_URL=http://localhost:8786/v1
context-guardian-doctor                            # second terminal: checks the install and the running proxy, prints a fix: line for anything wrong

Another backend or window: see examples/ or Configure. Your client's settings: Point your client at the proxy. DeepSeek Harness users: install the native engine below.

After a compaction, context-guardian-recall "<term>" searches the archived originals (the summary the model receives names the exact command).

Point your client at the proxy

The proxy compacts OpenAI-style POST /v1/chat/completions requests. Every client below is configured from its own documented settings; none of these four has been tested against the proxy yet, so please open an issue with what you see.

<model> is the name your backend serves (for Ollama, what ollama list shows). The proxy forwards the API key unchanged; a local backend ignores it. Then set GUARDIAN_NUM_CTX to that model's real window (Configure).

An illustration of what the DSH engine (0.1.0-alpha.9) does, not a DSH screen. Inside DSH you see it as DSH's own "Context compacted" row and context meter, /context and /guardian for pressure and the last compaction, and the [context-guardian recall] message in the conversation. What happens: at 85 % of the window it compacts, the facts you asked it to keep lead every checkpoint, the old turns stay on disk, and when a later question needs a detail that was compacted away, stale recall puts the original line back before the model answers. The results line is the benchmark in Measured. The proxy has its own monitor: see See it work.

Install both (DeepSeek Harness, about 5 minutes)

Context Guardian and Tool Guardian are two halves of one problem: Tool Guardian keeps tool schemas and tool results from filling the window, and Context Guardian compacts the conversation before it fills and keeps what matters. They share no files and install separately. You need DSH 0.1.2-alpha.2 or later, Node.js ^22.19 or >=24, and Python 3.9+ on PATH for Tool Guardian's router.

# 1. Add both bundles to the DSH profile you use (`web` is the one `dsh web` uses)
dsh plugin --profile web add dsh-tool-guardian
dsh plugin --profile web add dsh-context-guardian

# 2. Tool Guardian: copy in the MCP servers you already use (Claude Desktop / Cursor / Windsurf / .mcp.json), then check them
cd ~/.dsh/profiles/web/node_modules/dsh-tool-guardian          # Windows: cd %USERPROFILE%\.dsh\profiles\web\node_modules\dsh-tool-guardian
npm run setup                  # asks before it writes ~/.tool-guardian/mcp.json, then prints a doctor report

# 3. Context Guardian: add its compaction row to your agent preset (a dry run until --apply)
cd ~/.dsh/profiles/web/node_modules/dsh-context-guardian       # Windows: cd %USERPROFILE%\.dsh\profiles\web\node_modules\dsh-context-guardian
npm run setup
npm run setup -- --apply

# 4. Start DSH and open a NEW session with the preset setup named
dsh web

In that session, type /guardian. It shows Context Guardian's engine revision and your model's window. Then type /toolguardian. It shows each MCP server Tool Guardian started, the tokens the router saves on every request, and whether an update is out.

Not on DSH? Context Guardian's proxy (python context_guardian.py, see its README) works with any OpenAI-compatible CLI. Tool Guardian runs as a plain MCP server for Claude Code, Cursor or any MCP client (see its README).

Two ways to run it

Hand-off files (DSH engine). At every compaction the engine writes handoff_<session>.json and .md, plus handoff_latest.json, beside memory.json: the session's goal, its current pins and its decisions, so a new session or another agent can pick the work up from them alone. GUARDIAN_HANDOFF=0 (or handoff: false on the engine row) turns them off; see docs/dsh-integration.md.

flowchart LR
    subgraph P["Proxy: any OpenAI-compatible harness"]
      A1["CLI / agent"] --> G1["Context Guardian<br/>proxy :8786"] --> B1["Ollama / LiteLLM / vLLM"]
      G1 -. "over budget" .-> S1["summarise older turns<br/>keep recent verbatim"]
    end
    subgraph D["Engine: inside DeepSeek Harness"]
      A2["DSH agent"] --> C2["compaction-basic"] --> E2{"LLM summary<br/>fits and works?"}
      E2 -->|yes| K2["LLM checkpoint"]
      E2 -->|no| X2["deterministic checkpoint<br/>with seq pointers"]
    end
    S1 --> Z[("span archive on disk")]
    X2 --> Z
    K2 --> Z

The rest of this page is the proxy. The DSH engine has its own five-minute guide: docs/dsh-integration.md.

Compatibility

"Tested on" means the repo's test suites plus daily use on the machine this was built on. "Expected to work" is not tested; please open an issue if it does not work for you.

Measured: does it keep what matters? (2026-10-01)

A 48-prompt session on qwen3-coder:30b (Ollama, 64K window) under the DeepSeek Harness: 12 project codenames are given in the first 6 prompts, 40 filler reading tasks force 3-7 compactions, then the model is asked for all 12 (null allowed, guessing counted as wrong). Fresh random codenames every run.

What changed: facts the user asks to keep are pinned verbatim; the summary call is also asked for a ## Must Keep section whose bullets are pinned for the rest of the session; every checkpoint is checked for its pins; the idle trigger is off by default.

0.1.0-alpha.9 (2026-10-04): early details come back too. Same session, but codenames change mid-session and the last questions ask for details from the first documents that nobody flagged. Stale recall (on by default) adds one [context-guardian recall] message with up to 3 matching original lines when your message names something only the compacted turns still hold.

Why this exists

Claude-Code-style coding CLIs (Claude Code itself, and OpenAI-compatible-backend tools like OpenClaude) ship with a built-in auto-compact feature. That feature depends on accurate, real-time token-usage accounting coming back from the API in the exact shape the CLI expects. Point one of these tools at a local model through an OpenAI-compatible bridge — Ollama's /v1 endpoint, a LiteLLM proxy, a Headroom proxy — and that accounting is frequently missing, wrong, or shaped differently, so auto-compact silently never fires.

The visible symptom: the session just runs until the backend hard-rejects the request ("token limit reached"), you're forced to close and reopen, and there's no partial-compaction attempt in between — you just lose your place.

Context Guardian is a small, deliberately simple fallback for that specific gap. It estimates the running token count itself, and once a conversation crosses a configurable threshold, it asks the same backend to condense the older portion of the conversation into one summary message before forwarding the request onward. Recent messages are always kept verbatim. If the summarization call itself fails, Guardian fails open — it forwards the original, uncompacted request rather than risk silently dropping history.

Where it sits in your stack

This is a new link in an existing chain, not a replacement for anything you already have:

Your CLI / agent (Claude Code, OpenClaude, etc.)
    -> Context Guardian        (this project)
    -> your existing OpenAI-compatible backend
       (Ollama directly, a LiteLLM proxy, Headroom, vLLM, ...)

Point your CLI's OPENAI_BASE_URL at Context Guardian instead of directly at your backend, and set GUARDIAN_UPSTREAM_URL to wherever your backend actually lives. Guardian is a pure passthrough for everything except POST /v1/chat/completions, which gets the compaction check — every other route, including streaming responses, is forwarded byte-for-byte, untouched.

If you use MCP servers or an agentic CLI, read this

Guardian counts your tools array against the context budget. It did not before 0.2.0, and that was a real bug — see the changelog.

Tool definitions are usually invisible in a way message history is not. You do not type them, they do not scroll past, and your CLI's context display often does not break them out. But they are in every single request. On the setup this was developed against, seven MCP servers came to 28,689 tokens — 87.6% of a 32,768-token window — before the first user message.

Guardian cannot compact them. It summarizes conversation history; tool definitions are a fixed floor underneath it. So there are two different problems and only one of them is Guardian's:

Guardian will now tell you which one you have. It logs a tool_budget event the first time it sees a given tool payload, and warns outright when the tool definitions alone meet or exceed the whole window:

[ContextGuardian] TOOL DEFINITIONS ALONE (2671) EXCEED THE ENTIRE CONTEXT
WINDOW (1000). Nothing this proxy does can fix that -- send fewer tools.

If you see that, no proxy setting will help you. Most MCP-capable CLIs let you scope which servers load per session — Claude Code and OpenClaude both accept --mcp-config <file> together with --strict-mcp-config, which makes that file the only source of MCP servers for the session.

Companion project — Tool Guardian (dsh-tool-guardian on npm, or tool-guardian as a plain MCP server; see its README) does the other half of this. It fronts your MCP servers behind three generic tools and reveals the rest on demand, so the tool definitions stop being re-sent on every request in the first place. Context Guardian can't compact that fixed tool floor — Tool Guardian removes it. Use them together: one trims the conversation, the other trims the tools.

One consequence worth expecting: after upgrading, Guardian compacts sooner and more often. It is measuring the whole request now instead of a fraction of it. If that feels aggressive, the honest reading is that your window was already this full and you could not see it.

What this does not do

• It doesn't replace or duplicate compression your backend already does (e.g. Headroom, prompt caching). It forwards to your backend as-is once it's decided whether to compact first — the two are complementary, not competing.
• It doesn't fix your CLI's own context-usage display. Your CLI doesn't know this proxy exists, so its own token counter will drift from reality after a compaction happens. What matters is that the session keeps working instead of hard-stopping — a slightly-wrong displayed number afterward is an accepted tradeoff of doing this invisibly at the proxy layer, since the CLI itself usually isn't something you can modify.
• It is not a tokenizer-accurate counter. Token count is estimated from character length (~3.5 chars/token by default), not a real tokenizer, so it triggers a little early rather than late. Treat it as a safety-margin trigger, not a precise measurement.

Install

git clone https://github.com/LuminariSoftwares/context-guardian.git
cd context-guardian
python -m venv .venv
source .venv/bin/activate   # Windows: .venv\Scripts\activate
pip install -r requirements.txt

Then run python configure.py (see Configure below) before starting Guardian for the first time.

Configure

Run the interactive setup script instead of hand-editing a config file — it asks you a handful of questions about your specific hardware/backend (most importantly, your model's real context window) and writes the answers to .env for you:

python configure.py

Every question has a sensible default shown in [brackets] — press Enter to accept it. You can re-run configure.py any time to change your answers, or just edit .env directly afterward.

The one setting that actually matters per-person is GUARDIAN_NUM_CTX. This project was originally built and tested on a 16GB card (RTX 4070 Ti Super) running a model configured for a 32K context window — that number is specific to that hardware, not a universal default. Your correct value depends entirely on your own GPU/VRAM budget and which model you're running, so configure.py asks for it explicitly rather than silently assuming everyone's setup looks the same. If you're not sure what your real number is:

• Ollama: run ollama ps while your model is loaded — the CONTEXT column shows the live value actually in use (not necessarily the model's theoretical max).
• LM Studio / vLLM / other servers: check whatever context-length setting you configured when loading the model — Guardian has no way to auto-discover this, so it needs to match what you actually set.
• If you're unsure or haven't set one explicitly: start conservative (the configure.py default of 32768 is a reasonable, widely-safe starting point on a single consumer GPU) and raise it later once you've confirmed your backend can actually sustain it without running out of VRAM.

Setting this too high means Guardian won't compact soon enough and your backend can still hard-error before Guardian steps in. Setting it too low just means Guardian compacts a bit more often than strictly necessary — safe, just not optimal.

If you'd rather skip the wizard, copy .env.example to .env and edit it by hand:

cp .env.example .env

A note on timeouts: local "thinking"/reasoning models can go silent for a long time before their first output token. If you see 500 errors appear only on real (non-trivial) requests after a long pause, raise GUARDIAN_UPSTREAM_TIMEOUT before assuming something is broken — the default 5-second timeout most HTTP clients ship with is sized for ordinary REST APIs, not local LLM inference, which is exactly the bug this project's own commit history caught during development.

Run

python context_guardian.py

Then point your CLI's OPENAI_BASE_URL at http://localhost:8786/v1 (or whatever port you configured).

Check your setup: cg_doctor.py

One command says what is configured, what is wrong, and how to fix each problem:

python cg_doctor.py                          # from the repo, or the installed npm package folder
context-guardian-doctor                      # after `pip install context-guardian` (0.7.0+)
python cg_doctor.py --preset path/to/agent.cordis.yml   # also check the DSH preset that mounts engine.js
python cg_doctor.py --json                   # the same findings as JSON

It prints one OK / WARN / FAIL line per check, a fix: line under every problem, and a count line (doctor: 5 ok, 1 warnings, 0 errors); it exits 1 if anything FAILed. It checks:

• the package version (package.json) and the proxy's own version (context_guardian.py);
• your .env — found or not, and every numeric GUARDIAN_* value actually parses (a 32k or a GUARDIAN_RESERVE_OUTPUT that is not below GUARDIAN_NUM_CTX is a FAIL with the fix);
• the context window in use — GUARDIAN_NUM_CTX if you set it, else a numCtx pinned in the preset you pass with --preset (a WARN: since 0.1.0-alpha.5 the engine reads your model's real window from DSH, so a copied numCtx: 32768 usually just shrinks it), else the window DSH reports for the session;
• whether the proxy answers on GUARDIAN_HOST:GUARDIAN_PORT (a WARN, not a FAIL — the DSH bundle works without it), and whether a running proxy was started with a different window than you now have set;
• whether the Python bridge works — it starts modules/cg_bridge.py with the same interpreter the plugin would pick (GUARDIAN_PYTHON, then a .venv beside the package, then python/python3 on PATH) and requires Python 3.10+.

Standard library only; it never changes a file.

Testing before you trust it with a real session

1. Start your real backend (Ollama, LiteLLM, Headroom, whatever you use) the way you normally would.
2. Start Guardian: python context_guardian.py
3. Send one manual request at it instead of your real CLI, to confirm plain passthrough works before testing compaction specifically:
   bashCopy

   curl http://localhost:8786/v1/chat/completions \
     -H "Content-Type: application/json" \
     -d '{"model":"<your-model>","messages":[{"role":"user","content":"say hi"}]}'
   

4. Check GET http://localhost:8786/guardian/stats for the running token estimate and compaction count — or open http://localhost:8786/guardian/health in a browser for the live dashboard (window usage, compactions, tokens/cost saved, and any update notice), which just renders that same JSON on a 3-second refresh.
5. Force a compaction test: temporarily set GUARDIAN_NUM_CTX and GUARDIAN_COMPACT_THRESHOLD low (e.g. NUM_CTX=2000, THRESHOLD=0.5), then send a conversation with several long messages. Confirm a compaction log entry appears at GUARDIAN_LOG_PATH and the request that actually reaches your backend is smaller than what was sent in.
6. Only after that, point your CLI's OPENAI_BASE_URL at Guardian and test with a real session.

Running multiple models with different context windows

On Ollama, Guardian asks /api/show for each model's num_ctx (see GUARDIAN_OLLAMA_CONTEXT), so a model whose Modelfile sets num_ctx is budgeted against its own window. Otherwise GUARDIAN_NUM_CTX is fixed for the lifetime of one running instance. If you switch between models with meaningfully different context windows, either:

• run a second Guardian instance on a different GUARDIAN_PORT with its own GUARDIAN_NUM_CTX, or
• keep one instance and accept that its threshold is tuned to whichever model has the smaller/more-constrained window (safer than the alternative, since it just means Guardian compacts a bit earlier than strictly necessary for the larger-window model).

MCP server

context-guardian-mcp (installed with the PyPI package) is a stdio MCP server, so an MCP client can see what the proxy is doing without a terminal. It has three tools, and all three only read:

Claude Desktop (claude_desktop_config.json) or any client that takes the same shape:

{ "mcpServers": { "context-guardian": { "command": "context-guardian-mcp" } } }

Claude Code: claude mcp add context-guardian -- context-guardian-mcp

Benchmark

scripts/bench_proxy.py drives a scripted agent conversation (user turns, tool calls with large results, assistant replies; the client resends its whole history every turn) through the real proxy app against an in-process stub upstream, and prints one line. Same --turns and --seed give the same token figures; latency depends on the machine. Nothing touches the network or the repository.

python scripts/bench_proxy.py --turns 40 --seed 1

Measured against a stub upstream (not a real model; the stub summarises instantly, so added_ms is the proxy's own overhead, not summarisation time), 2026-10-14, Python 3.11, Linux container:

turns=40 tokens_before=1130722 tokens_after=160310 compactions=27 p50_added_ms=5.07 p95_added_ms=7.20

Development / running tests

pip install -r requirements-dev.txt
pytest
npm install --no-package-lock --legacy-peer-deps
npm run check && npm run test:node

CI runs both suites on Ubuntu, Windows and macOS (tests, node).

See it work

The proxy's /guardian/health compaction monitor: a request that arrives above the compaction threshold (GUARDIAN_COMPACT_THRESHOLD, 85 % of the window) is compacted before it is forwarded, so the backend never sees an over-long request.

• Proxy: open http://localhost:8786/guardian/health while a session runs. The compaction monitor replays every compaction as a before → after bar, with a per-compaction dropdown and an advice box when your fixed floor (tools + system prompt) is the real problem. GET /guardian/events returns the same records as JSON.
• DSH: type /context in a session for pressure, cache hits and what compacting now would save; /recall 3-7, /recall result 42 or /recall find <text> to read originals; logs/guardian_dsh.jsonl for the decision trail (idle-check → compaction/start → deterministic or llm → compaction/end).

Acknowledgements

Context Guardian stands on other people's ideas, and says so:

• dsh-compaction-instant (TsFreddie, MIT) — vendor/compiler.js and vendor/region.js are vendored unmodified from 0.1.4, with their original headers and the MIT licence text in vendor/LICENSE.dsh-compaction-instant. The recall / search contract follows theirs.
• VCC (lllyasviel) — the conversation-compiler principle that compiler ports: compile the log into a compact view made only of original tokens, with a pointer back to every elided part.
• dsh-openwolf (MIT) — the idea of snapshotting session state right before a compaction. The engine's precompact-<seq>.json and the FILES WRITTEN list are an independent implementation of that idea; no openwolf code is included.
• DeepSeek Harness — summarize() is the hook its compaction engine documents for exactly this.

Full third-party notices: THIRD_PARTY_NOTICES.md.

License

MIT — see LICENSE.