AI evaluation & benchmarks MCP servers

A server here evaluates, tests or benchmarks AI models or agents, or provides benchmark tasks for them.

Not here: testing ordinary software.

752servers
439 remote313 localTagged by Odel's AI, which can make mistakes

Best fit

400 made for this, ranked by starsRemoteLocal
  1. LLM Redteam @4hmetuyar/llm-redteamActive jailbreak/extraction/obfuscation red-teaming for live LLM endpoints16% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from npm.1 GitHub stars.
  2. The Open House @gregbenza/openhouseThe Open House: a locker, a verifier, a fair coin, a job board, and rooms that give nothing back.4% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.1 GitHub stars.
  3. Ticket Demo @markusvankempen/mcp-ticket-demoMCP server demo — tickets, schema discovery, auth, and PII gating. stdio + HTTP.4% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from npm.1 GitHub stars.
  4. Zhijiangyun Cloud (智匠云) @artibot/zhijiangyunRobot embodied-AI cloud for GPU training, inference, benchmarks, and robot data collection.7% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; it needs your API key.1 GitHub stars.
  5. The Meeting Place @gregbenza/meetA second opinion, in one call, from a desk where an answer is allowed to be no.4% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.1 GitHub stars.
  6. TERM @term/forumA forum whose members are AI agents. Publish verifiable findings, enter scored challenges.5% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.1 GitHub stars.
  7. ios-agent @emazaheri/ios-agentDrive a real iPhone or an iOS Simulator with an AI agent, through XCUIAutomation.7% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from PyPI.1 GitHub stars.
  8. ProfitPlay @jarvismaximum-hue/profitplay-mcpMCP server for the live ProfitPlay BTC five-minute AI agent sandbox9% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from Docker.1 GitHub stars.
  9. Sentinel Scan @ventrova/sentinel-scan-cliMCP security scanner: detect tool poisoning, prompt injection & rug-pulls - 10 OWASP heuristics.7% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from npm.1 GitHub stars.
  10. Asked Thrice @askedthrice/askedthriceMeasure whether AI assistants actually recommend a brand, from measured answers.4% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.1 GitHub stars.
  11. RuleDrift @ri7in/rule-driftTest whether AI agents retain critical instructions as conversations grow.5% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from npm.1 GitHub stars.
  12. Evolveguard @rudrendupaul/evolveguardWraps the evolveguard CLI as a single generic MCP tool for agent-skill file safety checks.3% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from PyPI.1 GitHub stars.
  13. EdgeGate @frozo-ai/edgegate-mcpRegression-test AI models on real Snapdragon and Jetson hardware, with signed evidence for CI.5% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from npm.1 GitHub stars.
  14. DCL Trust Oracle — AI/LLM Output Audit (x402 MCP) @fronesislabs/dcl-trust-oracleAI/LLM agent output audit MCP: policy eval, tamper-evident chain, AI safety, x402 USDC on Base.12% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.1 GitHub stars.
  15. GAUNTLEX @sanjoy1234/gauntlexConcurrent Builder + Breaker agents generate code and adversarial tests at once, gated in CI.7% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from PyPI.1 GitHub stars.
  16. QueryPilot @nickklos10/querypilotSafe SQL gateway for AI agents: SELECT-only validation, access policies, masking, audit trail, evals4% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from PyPI.1 GitHub stars.
  17. VerifyAX @verifyax/mcp-serverEvaluate, benchmark, and simulate AI agents on the VerifyAX agent-evaluation platform.8% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; it needs your API key.1 GitHub stars.
  18. Rotifer Protocol MCP @rotifer-protocol/mcp-serverSearch, compare, and install self-evolving AI Agent Genes ranked by Arena fitness7% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from npm.1 GitHub stars.
  19. Holomime @productstein/holomimeBehavioral therapy for AI agents — self-diagnosis, alignment, and training via MCP7% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from npm.1 GitHub stars.
  20. Opengate @nickjlamb/opengate-mcpCheck whether an AI answer is grounded in its context — deterministic, no LLM judge.5% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from npm.1 GitHub stars.
  21. AWT (AI Watch Tester) @ksgisang/awtAI-powered E2E testing MCP server. Detects and auto-fixes UI bugs via DevQA Loop and Vision AI.6% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from PyPI.1 GitHub stars.
  22. Trustmodel @karlmehta/trustmodel-mcpScore any AI for trust across 10 dimensions; evaluate, monitor & govern LLMs and agents.5% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from npm.1 GitHub stars.
  23. Index9 @index9-org/mcpDiscover, shortlist, compare, cost-model, and live-test 300+ AI models from your editor5% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from npm.1 GitHub stars.
  24. FetchSandbox @fetchsandbox/mcpA deterministic verification engine for agents. Proves a fix: fails on the old code, passes on new.4% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.1 GitHub stars.