Darwin RAG

A local-first RAG engine that ingests documents, indexes them with BM25 + dense embeddings, and exposes search via an MCP server for AI agent integration.

• Ingestion — PDF, Markdown, HTML, images (OCR), CSV, Excel, ODS, URLs
• Indexing — BM25 keyword + dense embedding hybrid index with configurable chunking strategies
• Search — Hybrid, semantic, or keyword retrieval with reranking, diversity rerank, and structural penalties
• Generation — LLM-backed answer synthesis via LiteLLM (OpenAI, Anthropic, Gemini, etc.)
• Observability — Structured logging with per-session history, queryable via MCP
• Isolation — Multiple independent stores for tenant/project separation
• Deployment — stdio (AI agent subprocess), SSE, or Streamable HTTP; Docker-ready
• Local-first — Everything runs locally, fully offline-capable after setup

Built by BrightDotDev.

Quick Start

1. Install

pip install darwin-rag

2. Set up models

# Interactive — detects hardware, pick your models
darwin-admin setup interactive

# Or one-shot (embedding-only, no prompts)
darwin-admin setup --preset required

3. Start the MCP server & connect

# stdio mode — for AI agent subprocess (Claude Desktop, Cursor, etc.)
darwin mcp

# Or HTTP mode — for remote clients
darwin mcp --http --port 8765

Configure your MCP client:

{
  "mcpServers": {
    "darwin": {
      "command": "darwin",
      "args": ["mcp"],
      "env": {
        "OPENAI_API_KEY": "sk-..."  // At least one LLM provider key
      }
    }
  }
}

Or generate config automatically:

darwin config claude          # Claude Desktop config
darwin config cursor          # Cursor config
darwin config all --copy      # All clients + copy to clipboard

MCP Tools

Full documentation: docs/mcp.md

Remote / HTTP Mode

Start the server on a network-accessible endpoint:

# SSE transport (legacy)
darwin mcp --sse --host 0.0.0.0 --port 8765

# Streamable HTTP transport (recommended for production)
darwin mcp --http --host 0.0.0.0 --port 8765

Configure your MCP client with the URL:

{
  "mcpServers": {
    "darwin": {
      "url": "http://your-host:8765/mcp"  // or /sse for SSE mode
    }
  }
}

Environment Variables

* At least one LLM provider key is required for answer generation. Search/indexing works without any.

Setup Details

See docs/setup.md for the full walkthrough including Docker, from-source install, and API key configuration.

CLI Reference

darwin — User CLI

darwin-admin — Power-user CLI

See docs/admin.md for the full command reference.

Python API

For embedding darwin-rag as a library in your own app:

from core import Darwin

d = Darwin()
d.ingest("./papers", recursive=True)
results = d.search("what is this paper about")
records = d.search_records(filters={"status": "active"})

Full reference: docs/api.md

Documentation

Contributing

Found a bug? Want to add something? You're welcome here.

• Issues — open one at github.com/BrightDotDev/DARWIN/issues
• Code — fork, branch, PR. Keep it minimal.
• AI-generated code is fine — but you own what you ship. Test it before submitting.

Read CONTRIBUTING.md for the full guidelines.

License

MIT with Attribution — see LICENSE.

Core architecture and implementation by BrightDotDev.