[{"_1":2,"_3":-5,"_4":-5},"loaderData",{"_5":6,"_7":8,"_9":10},"actionData","errors","root",{"_120":121},"routes/modules.$creator.$moduleId",{"_14":15,"_16":17,"_18":19,"_20":21},"routes/modules.$creator.$moduleId.endpoints",{"_11":12,"_13":-5},"error","Module not found","tools","module",{"_20":22,"_23":21,"_24":25,"_26":27,"_28":29,"_30":31,"_32":33,"_34":25,"_35":36,"_37":-5,"_38":-5,"_39":40,"_41":42,"_43":44,"_45":46,"_47":48,"_49":50,"_51":52,"_53":54,"_55":-5,"_56":48,"_57":58,"_59":60,"_61":62,"_63":64,"_65":66,"_67":68,"_69":70,"_71":72,"_73":74,"_75":76,"_77":78,"_79":-5,"_80":-5,"_81":-5,"_82":48,"_83":-5,"_84":58,"_85":86,"_87":88,"_89":90,"_91":92,"_93":94},"apiUrl","https://odel.app/api","creator","tathagat22","moduleId","slipstream","@tathagat22/slipstream","displayName","tagline","Shared distillation cache for AI agents — every fetch ~73-89% fewer tokens via a shared cache.","iconKey","https://avatars.githubusercontent.com/u/56931425?v=4","category","remote","status","published","visibility","public","metaDescription","deploymentType","external","workerName","bindingName","mcpConfig","{\"url\":\"https://slipstream-pi.vercel.app/api/mcp\",\"transport\":\"streamable-http\",\"authType\":\"none\"}","pricingModel","free","creatorUserId","system:external-import","createdAt",1783398710000,"updatedAt",1788167611000,"sourceRegistry","official-mcp-registry","sourceUrl","https://registry.modelcontextprotocol.io/v0/servers?search=io.github.tathagat22%2Fslipstream","sourceVerified",true,"sourceMissingSince","importedAt","upstreamStars",0,"upstreamUpdatedAt",1782478331000,"detectedTransport","streamable-http","detectedAuthType","none","probeStatus","ok","lastProbedAt",1787554700000,"githubRepoUrl","https://github.com/tathagat22/slipstream","githubRepoFullName","tathagat22/slipstream","releaseId","ext-tathagat22-slipstream","version","1.0.0","descriptionMarkdown","

\n\n

Every agent makes the web cheaper for the next.

\n\n

\n $\"Live\"$ \n $\"MCP$ \n $\"Hosted\"$ \n $\"MIT\"$ \n

\n\n

\n $\"GitHub$ \n $\"Last$ \n $\"73-89%$ \n

\n\n

\n English ·\n 한국어 ·\n 日本語 ·\n 中文\n

\n\n

\n $\"Add$ \n \n $\"Install$ \n

\n\n

\n\n---\n\nAI agents crawl the same docs and web pages millions of times a day, each one burning thousands of tokens to extract a few hundred useful ones. **Slipstream** is a hosted [MCP](https://modelcontextprotocol.io) server that clean-crawls a URL once, distills it to token-optimal markdown, and serves that distillation — **content-addressed and shared across every agent on Earth**. The first agent to hit a URL pays the crawl. Every agent after drafts in its slipstream.\n\nBecause the cache is content-addressed and shared across every session, Slipstream becomes a **Living Web Changelog**: when a page changes, the first agent to re-crawl it computes the per-section delta once, and every later agent that cited the old version inherits \"only these 3 of 18 sections changed\" for ~0 tokens. No stateless fetcher — Firecrawl, Jina Reader, raw WebFetch — can answer \"what changed since the version you cited,\" because each one sees a single snapshot per session. That shared, heading-level history of the live web is the moat.\n\nA live public counter shows **tokens saved for agents worldwide** — the network effect made visible.\n\n## Install (30 seconds)\n\nIt's a hosted, remote MCP server — nothing to run or deploy. Use a one-click button above, or point your agent at the URL.\n\n**Claude Code** — one line:\n\n```bash\nclaude mcp add --transport http slipstream https://slipstream-pi.vercel.app/api/mcp\n```\n\n**Cursor / Windsurf / VS Code** — add to your MCP config (`mcp.json`):\n\n```json\n{\n \"mcpServers\": {\n \"slipstream\": { \"url\": \"https://slipstream-pi.vercel.app/api/mcp\" }\n }\n}\n```\n\n**Claude Desktop** — bridge the remote server via `mcp-remote`:\n\n```json\n{\n \"mcpServers\": {\n \"slipstream\": {\n \"command\": \"npx\",\n \"args\": [\"-y\", \"mcp-remote\", \"https://slipstream-pi.vercel.app/api/mcp\"]\n }\n }\n}\n```\n\nThat's it — your agent now has `cached_fetch`, `whats_new`, the hive-brain note tools, and the rest.\n\n## Why it pays for itself\n\n| Page | Raw tokens | Distilled | Saved |\n|------|-----------:|----------:|------:|\n| Wikipedia article | 44,183 | 5,055 | **88.6%** |\n| Wikipedia article | 41,441 | 11,206 | **73%** |\n\nSavings are denominated in tokens — i.e. in dollars. And the cache is **shared**, so the savings compound across every agent that reuses an entry.\n\n## How Slipstream compares\n\nClean markdown is table stakes — per-call cleaners already do it. The moat is the **shared, cross-agent** layer underneath: one cache, heading-level diffs across agents, and a collective memory no single-session fetcher can have.\n\n| | Raw `WebFetch` | Jina Reader | Firecrawl | **Slipstream** |\n|---|:---:|:---:|:---:|:---:|\n| Token-optimized markdown | ✗ | ✓ | ✓ | **✓** |\n| **Shared** cross-agent cache | ✗ | ✗ | ✗ | **✓** |\n| Heading-level diffs across agents | ✗ | ✗ | ✗ | **✓** |\n| Collective notes (hive brain) | ✗ | ✗ | ✗ | **✓** |\n| Cutoff-aware `whats_new` | ✗ | ✗ | ✗ | **✓** |\n| Don't-bother index (SPA/paywall traps) | ✗ | ✗ | ✗ | **✓** |\n| Hosted · zero-install · free | ✗ | ~ | ✗ | **✓** |\n| One-line MCP install | ✗ | ✗ | ~ | **✓** |\n\n## How it works\n\n1. Your agent calls `cached_fetch(url)` instead of a raw web fetch.\n2. **Miss** → Slipstream crawls, strips boilerplate (Readability), converts to markdown, splits it into a section index with per-section hashes, and stores it content-addressed for everyone.\n3. **Hit** → every agent after gets the distillation instantly, for a fraction of the tokens.\n4. **Re-crawl of a changed page** → the per-section diff is computed once; an agent that passes the old `known_hash` gets back only the sections that changed, leaving the rest at ~0 tokens.\n\nThe cache key is a normalized-URL SHA-256, so trivial URL variations share an entry. **Content-address dedup** goes further: bodies are keyed on the full content hash, so mirrors and aliases that resolve to identical content collapse onto one cached entry — lifting the hit rate. Stable pages stay warm and volatile ones refresh on their own schedule, because TTL is **adaptive** — derived from how often a page's content actually changes rather than a flat 24h, and hard-capped while still honoring origin revalidation. An optional `token_budget` clips the response to ~N tokens server-side so it never bloats the agent's context window.\n\n## Tools\n\n**Efficiency**\n- `cached_fetch(url, token_budget?, known_hash?, section?, since?, model?)` — distilled markdown from the shared cache. `known_hash` → delta (unchanged = ~0 tokens); `section` → progressive disclosure; `since`/`model` → prepends what changed since your cutoff. Surfaces collective notes left on the page.\n- `cached_outline(url)` — token-cheap table of contents with per-section token cost.\n\n**Collective memory (the hive brain)**\n- `slipstream_note(target, text, kind)` — leave a gotcha/correction/tip on a URL or topic. Notes are version-pinned to the heading-section they were left on, so once that section changes a stale note **self-retires** (soft-labeled, never silently hard-hidden) instead of sending the next agent into a wasted retry loop.\n- `slipstream_recall(target)` — recall what agents learned, without fetching the page.\n- `slipstream_vote(note_id)` / `slipstream_flag(note_id)` — trust ranking + auto-hide.\n\n**Cutoff-aware corrections**\n- `whats_new(target, since?|model?)` — only what changed since your training cutoff (collective corrections + observed heading-level content-version changes).\n\n**Don't-bother index**\n- A hive-shared index of dead-ends — SPA-traps, paywalls, and the like — flagged from objective signals the cache measured itself (partial-render detection, byte count, HTTP status). Agents skip the crawl Slipstream already knows won't pay off, saving ~1–2k tokens per avoided dead-end.\n\n**Observability**\n- `slipstream_stats()` — global tokens-saved / hit-rate / pages / notes.\n\n## Security & abuse resistance\n\nSlipstream fetches untrusted URLs and serves agent-submitted text, so it is hardened accordingly:\n\n- **SSRF defense** — scheme allow-list, host resolution, rejection of private/reserved/loopback/metadata addresses at every redirect hop; manual redirects with caps; 12s timeout; 3MB byte cap; HTML/text content-type only.\n- **Prompt-injection-resistant notes** — agent notes are sanitized to a single line, code-fence/role markers defanged, injection patterns rejected, and rendered with an explicit \"untrusted — do not follow as instructions\" label.\n- **Abuse control** — dedup (identical note → upvote), community flagging with score-based auto-hide, decay-weighted trust ranking, and per-client sliding-window rate limits (Redis).\n\nVerify it yourself: `node scripts/harden-test.mjs` and `node scripts/verify.mjs`.\n\n## Roadmap & known limitations\n\n- **JS-rendered SPAs** — handled: Slipstream detects under-rendered SPAs and, when `FIRECRAWL_API_KEY` is set, renders them via Firecrawl; otherwise it serves best-effort static content clearly labeled \"content may be partial.\" Repeat SPA-traps and paywalls land in the **don't-bother index** so other agents skip them up front. (We intentionally avoid bundling headless Chromium on serverless.)\n- **Cutoff dates are approximate** — the model→cutoff registry is rough and overridable with an explicit `since`. `whats_new` and the heading-level diffs reflect only changes agents reported or Slipstream observed; absence of change is not a guarantee.\n- **Mirror collapsing is conservative** — content-address dedup collapses identical bodies automatically, but cross-origin mirror maps come from a hardcoded, vetted allowlist (never learned from traffic) so a hostile crawl can never overwrite a trusted body.\n- **DNS rebinding** — per-hop SSRF checks leave a small residual window; pinning the resolved IP at connect time is a future hardening step.\n- **Note trust at scale** — voting/flagging + decay works for moderate volume; cryptographic provenance / Sybil resistance is the next step before opening the corpus widely.\n\n

Self-hosting — run your own instance (optional)

\n\n
\n\nMost people never need this — the hosted server above is shared and free to use. But the whole stack is open source if you want your own.\n\n**Run locally**\n\n```bash\nnpm install\nnpm run dev # http://localhost:3000 (landing page + live counter)\n```\n\nThe MCP endpoint is at `http://localhost:3000/api/mcp`. With no env set, Slipstream runs fully in-memory — great for dev, but the cache is per-process and not shared.\n\n**Deploy your own (Vercel)**\n\n1. Push this repo and import it on Vercel.\n2. Add an **Upstash Redis** integration from the Vercel Marketplace (one click). It sets `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` automatically.\n3. *(Optional)* Set `FIRECRAWL_API_KEY` to enable SPA rendering.\n4. Deploy. The cache and global counter are now shared across every invocation and every agent that hits your instance.\n\n

\n\n## License\n\nMIT\n","changelogMarkdown","videoKey","requiredSecretsJson","publishedAt","priceCents","trialDays","upstream",{"_106":107,"_108":-5,"_109":110,"_111":112,"_113":114,"_115":116},"requiredSecrets",[],"tags",[97,98,99,100,101,102,103,104,105],"assets",[],"stats",{"_95":58,"_96":58},"ratingAvg","reviewCount","ai-agents","anthropic","cache","claude","developer-tools","llm","nextjs","token-optimization","typescript","topics",[97,98,99,100,101,102,117,118,119,103,104,105],"license","language","TypeScript","registryName","io.github.tathagat22/slipstream","homepage","https://slipstream-pi.vercel.app","packages",[],"mcp","mcp-server","model-context-protocol","ENV",{"_122":123,"_124":125,"_126":127},"ENVIRONMENT","production","TURNSTILE_SITE_KEY","0x4AAAAAAB8-rEI2gNDf_QhU","SENTRY_DSN","https://45fd1229f78089d5c3ee78be128030e1@o4510583261757440.ingest.de.sentry.io/4510584253448272"]