AI evaluation & benchmarks MCP servers

A server here evaluates, tests or benchmarks AI models or agents, or provides benchmark tasks for them.

Not here: testing ordinary software.

752servers
439 remote313 localTagged by Odel's AI, which can make mistakes

Best fit

400 made for this, ranked by starsRemoteLocal
  1. Csoai @csoai-org/csoaiSigned deterministic AI-governance measurement: EU AI Act cards + agent-callable record verify.5% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from PyPI.3 GitHub stars.
  2. Reelier @seldonframe/reelierSnapshot tests and CI for agent workflows: record a run, replay it at 0 tokens, diff for drift.3% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from npm.3 GitHub stars.
  3. CompletionKit @completionkit/evalsPrompt evals over MCP: run a prompt on your dataset, score each output 1-5 with an LLM judge.4% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.3 GitHub stars.
  4. Palimpsest — censorship, China economy and model-eval observatory @beepboop2025/palimpsestRights-aware censorship, China-economic status, and tamper-evident AI evaluation tools.5% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.3 GitHub stars.
  5. Browserbash @pramoddutta/browserbashPlain-English browser tests for AI agents: real Chrome, deterministic verdicts. Free, no API keys.5% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from npm.3 GitHub stars.
  6. Forkmind @medhovarsh/forkmindBranchable LLM history DAG + encrypted context capsules agents can save, restore, and crypto-shred.4% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from npm.3 GitHub stars.
  7. Embodify @yidayang/embodifyGive your agent a body: observe and control simulated robots (LIBERO, RoboDojo) over MCP.4% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from PyPI.2 GitHub stars.
  8. jevnav @dtduc-git/jevnavReplayable browser automation: Jev picks the element, a gate blocks risky actions, every run traces.6% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from PyPI.2 GitHub stars.
  9. LLM Preflight @feronovak/llm-preflightLocal, cross-provider preflight checks for LLM integration changes.5% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from PyPI.2 GitHub stars.
  10. Dsh Verify @263311487-ux/dsh-verifyAgent-built web app quality gate. Real browser is the judge. PASS/FAIL with receipts.4% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from npm.2 GitHub stars.
  11. ChimeraForge @sahil170595/chimeraforgeLocal-first LLM deployment planner: GPU/VRAM sizing, cost and latency, with provenance3% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from PyPI.2 GitHub stars.
  12. WhitePact @guruprasath-annadurai/whitepactAI governance MCP server: trust scoring, guardrails, bias/hallucination detection, compliance.10% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; you sign in with your account.2 GitHub stars.
  13. Memtrust @rudrendupaul/memtrustIndependent, reproducible benchmark harness for agent-memory backends.3% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from PyPI.2 GitHub stars.
  14. aboard @untype/aboardFalsifiable claims about systemic problems: causal graph, forecasts, dossiers.4% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.2 GitHub stars.
  15. Hotato @attenlabs/hotatoOpen-source, self-hosted conversation QA for voice agents. MIT.3% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from PyPI.2 GitHub stars.
  16. Arkheiaai MCP @arkheiaai/mcp-serverDetect fabrication and hallucination in any LLM output. 30+ models supported.15% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from npm.2 GitHub stars.
  17. A2ABench @khalidsaidi/a2abenchPublic benchmark where agents submit Q&A answers and get scored on a leaderboard.5% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.2 GitHub stars.
  18. VerifiMind PEAS - RefleXion Trinity @creator35lwb-web/verifimind-genesisX-Z-CS AI validation with auditable reasoning. 8 active tools; 5 in maintenance. v0.5.6414% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.2 GitHub stars.
  19. Clousd @clousd-android/clousd-mcpA real Android phone in the cloud for MCP agents: observe the screen, act, reset to a saved state.6% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.1 GitHub stars.
  20. Testers.ai CARBON Test Harness Lite @jarbon/carbon-liteLocal QA workflows, test evidence, screenshot maps and confidence reports for AI coding agents.Not assessed yet.Runs on your machine, installed from MCPB.1 GitHub stars.
  21. Jev Decision Gate by BPJ @f-tiger/jev-decision-gateLocal Issue triage with optional Jev judgments and calibrated acceptance gates.Not assessed yet.Runs on your machine, installed from MCPB.1 GitHub stars.
  22. Harvey Verify @meltingpixelsai/harvey-verifyPost-transaction outcome verification. LLM-as-judge for agent service quality.Not assessed yet.Runs on your machine, installed from npm.1 GitHub stars.
  23. LLM Audit @llmaudit/llmauditMeasure whether AI assistants actually recommend a brand, from measured answers.11% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.1 GitHub stars.
  24. Groundwork @nickjlamb/groundworkDeployment-readiness checks for document-QA and extraction AI: grounding, extraction, CI gate.Not assessed yet.Runs on your machine, installed from npm.1 GitHub stars.