AI evaluation & benchmarks MCP servers

A server here evaluates, tests or benchmarks AI models or agents, or provides benchmark tasks for them.

Not here: testing ordinary software.

752servers
439 remote313 localTagged by Odel's AI, which can make mistakes

Best fit

400 made for this, ranked by starsRemoteLocal
  1. Benchmark Hygiene @sudo-ai-git/mcp-benchmark-hygieneDetect pytest config-leakage that corrupts agent-benchmark grading. Deterministic, no-LLM.4% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from PyPI.
  2. FlightSweeper @flightsweeper-webmcp/publicAnonymous public tools for FlightSweeper. See the published agent boundary before use.7% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.
  3. Verdict @dgotlieb/verdict-mcpSandboxed test runs for coding agents: compact typed verdicts with failure fingerprints and history3% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from PyPI.
  4. Liars Town Arena @liars/arenaPlay Werewolf against other AI agents. join once, then loop observe/act. Public ELO leaderboard.5% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.
  5. retriEVAL @hcarrillo001/retrieval-mcpLLM evals as MCP tools: score outputs for faithfulness, relevancy, and hallucination.5% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from Docker.
  6. OAIA Arena @oaia/arenaScore AI trading agents against 2,048 coin-flipping monkeys. Public leaderboard.5% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.
  7. Agent Reliability @agentreliability/agent-reliabilityTesting, benchmarking and auditing autonomous AI agents — methods, harnesses, evidence3% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.
  8. Agent Observatory @model-mischief-managed/agent-observatoryLive census of AI agents: prove you can reason (reverse CAPTCHA), check in, talk to other agents.14% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.
  9. AIO — AI Integrity Organization @aioq/aioAI integrity standards, benchmarks, and EU AI Act-aligned Tier 0 model certification4% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.
  10. Twothumbs @asselwan/twothumbsVerifies AI agent work end to end: real artifacts and outcomes checked, not self-reported success.9% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.
  11. Evr @brienbounded/evrChecks agent traces for evidence-readiness: verifies evidence is present, not that it is true.3% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from PyPI.
  12. Naturepedia Canonical Discovery @robbiegeorgephotography/naturepediaSearch and resolve Naturepedia, Robbie's Razor, and GC-MRD-v2.0 canonical resources.5% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.
  13. Agent Accessibility Auditor @mambabuilt/mcp-agent-accessibility-auditorAudit whether an AI agent can read a site: llms.txt, robots AI rules, structured data.5% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from npm.
  14. cloro @cloro/cloroScrape AI answer engines and Google Search/News with country and state-level geo-targeting.9% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.
  15. KarmoLab @mascari4615/karmolab-mcpMCP server for what LLMs get wrong: exact hashes, real randomness, and Korean rules.5% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from npm.
  16. LatAm Fintech Synthetic Data @jmendozapuche/latam-fintech-synthetic-dataPrivacy-safe synthetic financial data for LatAm fintech, AI agents, testing and ML.7% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; it needs your API key.
  17. Hookline MCP @hookline/mcpHookline MCP — create webhook endpoints, inspect requests, and validate payloads from AI agents.5% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from npm.
  18. AgentReady @agentready-market/auditAgentReady.market audit: can an AI shopping agent find, understand and BUY on this store? /100.4% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.
  19. Plzebo @wkoverfield/plzeboCost-parity bench for agent loops: $/success and time across model mixes, with verifiable receipts.6% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from npm.
  20. Before Users Do @bhuman-ai/beforeusersdoBrowser-backed QA with evidence and fix-ready reports for coding agents.6% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; you sign in with your account.
  21. Kit Memory Demo @peteratpondo/kit-memory-demoContinuity for agents: a live memory you can read and cite right now, outliving the context window.6% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.
  22. Mingxin @mingxinstorage/mingxin-mcp-serverMeasured AI-inference-storage benchmarks with citations, article search, KV-cache ROI estimation.9% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.
  23. METTLE @creed-space/mettle-mcpExperimental reverse CAPTCHA with signed badges and credentials. Not proof of identity or substrate.8% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from PyPI.
  24. Whetstone @cyberelf/toolsVerifier-grounded AI promotion gates, disposable report cards, and signed PASS/HOLD/BLOCK receipts.8% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.