AI evaluation & benchmarks MCP servers

A server here evaluates, tests or benchmarks AI models or agents, or provides benchmark tasks for them.

Not here: testing ordinary software.

752servers
439 remote313 localTagged by Odel's AI, which can make mistakes

Best fit

400 made for this, ranked by starsRemoteLocal
  1. WebMCP Eval @webmcp-eval/webmcp-evalEvaluate one public URL for WebMCP readiness: intent, tools observed, recommended, do-not-expose.16% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; you sign in with your account.
  2. First Dollar Test @first-dollar-test/first-dollar-testBase wallet-agent beta: deposit 1 USDC; PASS returns 1 USDC; terminal failure is refunded.17% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.
  3. ALPNAI — Agent Performance Tools @fredericmagnathy-ops/alpnaiAgent cost, latency and quality analysis, report delivery and order status. Crypto payments paused.7% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.
  4. LLM Red-Team Scanner @manhliemcn4euwlu/llm-red-team-scanner35-probe LLM/agent security red-team scan (injection, jailbreak, MCP abuse) with report.27% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.
  5. GardenArena GardenScript @gardenarena/gardenscriptSimulate farms, validate GardenScript and compare policies with Rust. No outbound LLM calls.4% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.
  6. Tada: automated QA for AI-written code @taada/tadaYour coding agent asks Tada to work through a feature in a real browser and gets back a verdict.5% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from npm.
  7. MarketAIVerse @marketaiverse/marketaiverseA benchmarked market for AI agents: what runs on your GPU, trust checks, encrypted agent rooms.7% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.
  8. Japanese Order Eval @shigoto-dogu/order-evalRegression checks for Japanese order automations: six free cases, strict JSON scoring and CI gates.4% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.
  9. Agent Canary @dorianchn/agent-canaryFree V1: inert MCP decoys and canary tokens detect AI-agent compromise signals. Local stdio server.8% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from a package registry.
  10. AimRank @aimrank/aimrankRead public pairwise-preference rankings and their statistical quality evidence. No API key.3% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.
  11. Compass by 402rates @402rates/compassFree agent tests (no account or payment); paid ping.5% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.
  12. Agentic RL: Credit Assignment and CLI Agents @hf/agentic-rlFilter agent RL methods by supervision, critic and task setting; retrieve source links and BibTeX.3% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.
  13. Human4AI @human4ai/collaborationSynthetic support cases and owner-reviewed Agent feedback. Anonymous reads; scoped writes.4% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.
  14. Scholium Live Needle @sudobot99/scholium-live-needleGET a public URL now and report whether an exact needle is still in the body.12% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.
  15. Batcave Detective Basic @bjorn2-sol/batcave-detectiveEvidence-bound second-opinion audit of an agent conclusion against caller-supplied evidence.11% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.
  16. Kawal @pugarhuda/kawalEvidence about ERC-8004 agents on BNB Smart Chain, gathered by calling them. No key.6% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.
  17. DidWork @didwork/didworkVerify that an agent's claimed outcome actually happened, using independently gathered evidence5% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from npm.
  18. ComputeSage StackBench @computesage/stackbenchGPU and LLM inference benchmarks, hardware evidence, deployment recommendations, and launch configs.6% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.
  19. Open for Agents Website Visibility @mdotk/open-for-agents-website-visibilityCheck how well AI agents can discover and understand a public website.7% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; you sign in with your account.
  20. RatchetX Forecast Arena @3esign/ratchetShared Pyth context and live Solana forecasting arena for calibration, proof, and agent ranking.9% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.
  21. Sichta Web @gigliotti/sichta-webCheck whether AI agents can read any website: 18 deterministic checks, each with its fix.9% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.
  22. Agent Arena @agentarena/arenaAgents play Connect 4, Battleship and duels for real USDC. Every move published. First match free.6% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.
  23. ScoreIA Open Chamber @scoreia/benchTest your AI agent over MCP in the Open Chamber: a timed public trial with signed cards.6% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.
  24. Assurly MCP @assurly/mcp-serverPre-deploy ship gate AI agents call before shipping. Also audits the agent's own MCP config.7% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from npm.