- Nonobench @mauricekleine/nonobenchAn open-source benchmark of how well LLMs solve nonogram puzzles, from 5x5 to 20x20.Not assessed yet.A remote server; no sign-in.7 GitHub stars.
- CoinRithm Agent Trading @coinrithm/mcp-tradingKeyless prediction-market data across 12 venues plus paper-trading of crypto spot, futures, and PM.4% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.4 GitHub stars.
- CompletionKit @completionkit/evalsPrompt evals over MCP: run a prompt on your dataset, score each output 1-5 with an LLM judge.4% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.3 GitHub stars.
- Palimpsest — censorship, China economy and model-eval observatory @beepboop2025/palimpsestRights-aware censorship, China-economic status, and tamper-evident AI evaluation tools.5% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.3 GitHub stars.
- WhitePact @guruprasath-annadurai/whitepactAI governance MCP server: trust scoring, guardrails, bias/hallucination detection, compliance.10% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; you sign in with your account.2 GitHub stars.
- aboard @untype/aboardFalsifiable claims about systemic problems: causal graph, forecasts, dossiers.4% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.2 GitHub stars.
- A2ABench @khalidsaidi/a2abenchPublic benchmark where agents submit Q&A answers and get scored on a leaderboard.5% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.2 GitHub stars.
- VerifiMind PEAS - RefleXion Trinity @creator35lwb-web/verifimind-genesisX-Z-CS AI validation with auditable reasoning. 8 active tools; 5 in maintenance. v0.5.6414% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.2 GitHub stars.
- Clousd @clousd-android/clousd-mcpA real Android phone in the cloud for MCP agents: observe the screen, act, reset to a saved state.6% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.1 GitHub stars.
- LLM Audit @llmaudit/llmauditMeasure whether AI assistants actually recommend a brand, from measured answers.11% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.1 GitHub stars.
- The Open House @gregbenza/openhouseThe Open House: a locker, a verifier, a fair coin, a job board, and rooms that give nothing back.4% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.1 GitHub stars.
- Zhijiangyun Cloud (智匠云) @artibot/zhijiangyunRobot embodied-AI cloud for GPU training, inference, benchmarks, and robot data collection.7% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; it needs your API key.1 GitHub stars.
- The Meeting Place @gregbenza/meetA second opinion, in one call, from a desk where an answer is allowed to be no.4% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.1 GitHub stars.
- TERM @term/forumA forum whose members are AI agents. Publish verifiable findings, enter scored challenges.5% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.1 GitHub stars.
- Asked Thrice @askedthrice/askedthriceMeasure whether AI assistants actually recommend a brand, from measured answers.4% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.1 GitHub stars.
- DCL Trust Oracle — AI/LLM Output Audit (x402 MCP) @fronesislabs/dcl-trust-oracleAI/LLM agent output audit MCP: policy eval, tamper-evident chain, AI safety, x402 USDC on Base.12% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.1 GitHub stars.
- VerifyAX @verifyax/mcp-serverEvaluate, benchmark, and simulate AI agents on the VerifyAX agent-evaluation platform.8% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; it needs your API key.1 GitHub stars.
- FetchSandbox @fetchsandbox/mcpA deterministic verification engine for agents. Proves a fix: fails on the old code, passes on new.4% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.1 GitHub stars.
- SEAR @igor-holt/searBenchmark-first release surface with a read-only MCP endpoint and operator CLI.4% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.1 GitHub stars.
- Agentic Platform @andysalvo/agentic-platformFree MCP tools: the only MCP linter, health checks, cost estimation, and trust evaluation.10% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.1 GitHub stars.
- AgentSIM MCP @agentsim/mcpControl plane for agent auth walls. Connector 0 handles SMS OTP with US test numbers.23% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; it needs your API key.1 GitHub stars.
AI evaluation & benchmarks MCP servers
A server here evaluates, tests or benchmarks AI models or agents, or provides benchmark tasks for them.
Not here: testing ordinary software.
439 remote313 localTagged by Odel's AI, which can make mistakes
Most popular in AI evaluation & benchmarks
By GitHub stars, at most two per publisher- Number 1: Ask Jev@pollinations/ask-jevEvaluate state with typed choice, score, and probability questions.Not assessed yet.A remote server; it needs your API key.5,183 GitHub stars.
- Number 2: Opik@comet-ml/opik-mcpRead and score Opik traces, experiments, datasets and prompts from your AI assistant.3% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; you sign in with your account.220 GitHub stars.
- Number 3: LLM Latency Tracker@llmlatency/llm-latency-trackerMeasured latency, time to first token and uptime for ~45 AI inference APIs, by region.2% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.12 GitHub stars.
