Odel
TheCrawler

TheCrawler

Local
@manchittlab2TypeScriptAGPL-3.0Updated 2mo ago

Universal web scraper with LLM-ready markdown, RAG chunking, PDF/DOCX support.

View on GitHub

Runs locally over stdio

This server isn't hosted — your MCP client launches it from a package registry. Use one of the commands below, or drop the config into your client (e.g. Claude Desktop).

npmthecrawlerv0.5.8

Run

npx -y thecrawler

MCP client config

{
  "mcpServers": {
    "thecrawler": {
      "command": "npx",
      "args": [
        "-y",
        "thecrawler"
      ]
    }
  }
}