- OrcaReplay @continuum-ai-corp/orcareplayRead, replay and fork recorded coding-agent runs.4% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from npm.283 GitHub stars.
- Opik @comet-ml/opik-mcpRead and score Opik traces, experiments, datasets and prompts from your AI assistant.3% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; you sign in with your account.220 GitHub stars.
- Evalview @hidai25/evalview-mcpRegression testing for AI agents. Golden baselines, CI/CD, LangGraph, CrewAI, OpenAI, Claude.4% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from PyPI.137 GitHub stars.
- Hwatu @hongnoul/hwatuVerification browser for coding agents: headless WebKit, DOM eval, screenshots, pixel-diff.6% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from MCPB.115 GitHub stars.
- Proctor @pulsemcp/proctorMCP server for running Proctor exams against MCP servers with exam execution and result management.6% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from npm.81 GitHub stars.
- Misata: synthetic data library @rasinmuhammed/misataLocal, offline: multi-table synthetic data with joins that hold and exact totals. Seeds databases.4% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from PyPI.69 GitHub stars.
- DOS — the trust substrate for agent fleets @anthony-chaudhary/dos-kernelVerify what agents actually shipped, arbitrate file collisions, refuse with structured reasons.4% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from PyPI.20 GitHub stars.
- RAGScore @hzyai/ragscoreGenerate QA datasets & evaluate RAG systems with failure diagnosis. Any LLM.5% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from PyPI.15 GitHub stars.
- LLM Latency Tracker @llmlatency/llm-latency-trackerMeasured latency, time to first token and uptime for ~45 AI inference APIs, by region.2% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.12 GitHub stars.
- Glassbox Framework @thebarmaeffect/glassbox-frameworkRuntime constitutional verification for AI answers — claim reasoning, ECS, red team, audits.8% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from npm.11 GitHub stars.
- Ducklab @jrullan/ducklabMulti-LLM dev harness, MCP-operable: bugs, cycles, gates. Verdicts are exit codes, never opinions.6% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from MCPB.9 GitHub stars.
- Iris @iris-eval/mcp-serverStop shipping agents on vibes. Score every agent output for quality, safety, and cost.5% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from npm.9 GitHub stars.
- Metrillm @metrillm/metrillmBenchmark local LLM models — speed, quality & hardware fitness verdict from any MCP client7% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from npm.8 GitHub stars.
- Nonobench @mauricekleine/nonobenchAn open-source benchmark of how well LLMs solve nonogram puzzles, from 5x5 to 20x20.Not assessed yet.A remote server; no sign-in.7 GitHub stars.
- GroundLens @groundlens-dev/groundlensVerify AI answers and executions and seal signed, offline-verifiable evidence records.5% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from PyPI.7 GitHub stars.
- designfit @as9978/designfitValidate AI-built front-ends against Figma by tokens and geometry, not pixels.4% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from npm.6 GitHub stars.
- Assurance @i-ops-hq/assurance-mcpChecks what an agent did NOT read. Coverage over a retrieval step or a folder.5% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from PyPI.6 GitHub stars.
- PokéArena @shaumik/pokearenaPlay a real 6v6 Pokémon battle. Runs entirely in-process: no server, no API key, no setup.5% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from MCPB.4 GitHub stars.
- CoinRithm Agent Trading @coinrithm/mcp-tradingKeyless prediction-market data across 12 venues plus paper-trading of crypto spot, futures, and PM.4% chance this is malicious. Assessed by AI, which can make mistakes.A remote server; no sign-in.4 GitHub stars.
- chauffeur @yashnaj/chauffeurDrive the iOS Simulator from a coding agent, with every action verified.4% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from MCPB.3 GitHub stars.
- Jev @aitejiu/jevJev judgments for agents: injection scanning, shell risk gating, ranking7% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from PyPI.3 GitHub stars.
AI evaluation & benchmarks MCP servers
A server here evaluates, tests or benchmarks AI models or agents, or provides benchmark tasks for them.
Not here: testing ordinary software.
439 remote313 localTagged by Odel's AI, which can make mistakes
Most popular in AI evaluation & benchmarks
By GitHub stars, at most two per publisher- Number 1: Ask Jev@pollinations/ask-jevEvaluate state with typed choice, score, and probability questions.Not assessed yet.A remote server; it needs your API key.5,183 GitHub stars.
- Number 2: Polymarket Paper Trader@agent-next/polymarket-paper-traderPolymarket paper trading for AI agents: $10k paper money, real order books, official fees.6% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from PyPI.453 GitHub stars.
- Number 3: Watch Skill@oxbshw/watch-skillWatch video and live sessions, keep timestamped evidence, and verify an agent's own work.7% chance this is malicious. Assessed by AI, which can make mistakes.Runs on your machine, installed from PyPI.438 GitHub stars.
