ResiliReplay

Break an MCP server on purpose. Replay the failure. Ship the regression.

ResiliReplay is the MCP reliability engine behind proof-carrying recovery changes: inspect one reviewed target, inject a deterministic fault, bound retries and duplicate effects, preserve evidence, and turn the failure into an executable regression.

npx --yes resilireplay@0.7.1 mcp demo

ResiliReplay MCP demo

✓ Clean MCP tool call
✓ Deterministic failure reproduced
✓ Recovery bounded to one retry
✓ Duplicate effects observed: 0
✓ Regression generated
✓ Regression executed

MCP reliability check passed.
Evidence: sha256:4d2479b98453732d6011c699c061d023353c8d5cf4159bdf6c5b096961f94c06

The demo is local, deterministic, credential-free, and cleaned up after it runs. It needs Node.js 22 or 24—no repository checkout, configuration, account, API key, paid model, or remote MCP server. This quickstart pins the released 0.7.1 artifact for reproducibility. Later @latest examples are convenient for following current documentation; pin an exact version in CI and retained evidence.

Product site · npm package · real MCP CI example · machine-readable capabilities

Ten-second before and after

What it does

• Inject deterministic MCP failures.
• Verify bounded recovery and duplicate-effect behavior.
• Generate executable regressions for CI.

Try your MCP server

Start with an Inspector-compatible mcp.json. A dry-run reads and sanitizes the selected entry but starts no process, opens no socket, calls no tool, and writes nothing.

npx --yes resilireplay@latest mcp test --config ./mcp.json --server my-server --tool echo --safety inert --dry-run

Review the selected server, transport, tool, fault, retry/time bounds, and plan SHA-256. Execution requires that exact digest:

npx --yes resilireplay@latest mcp test --config ./mcp.json --server my-server --tool echo --safety inert --approve <plan-sha256>

Use the same approved plan in CI and request concise machine-readable evidence:

npx --yes resilireplay@latest mcp test --config ./mcp.json --server my-server --tool echo --safety inert --approve <plan-sha256> --json

mcp test connects with the MCP SDK, discovers only the reviewed operation, runs a clean call, injects one declared result-boundary fault, applies the bounded retry, counts duplicate effects, generates a causal regression, executes it, and closes owned resources. mcp audit remains available with its existing options for compatibility.

CI

The repository contains a complete packed-package example using the official MCP Everything reference server:

name: MCP reliability
on: [pull_request]
permissions:
  contents: read
jobs:
  test:
    runs-on: ubuntu-latest
    timeout-minutes: 10
    steps:
      - uses: actions/checkout@v6
      - uses: actions/setup-node@v6
        with:
          node-version: 24
          package-manager-cache: false
      - run: node scripts/verify-packed-mcp-example.mjs

See the standalone example for its exact package integrity, SDK/runtime, protocol revision, inert echo operation, expected output, and generated regression. It installs the packed CLI in a clean temporary npm project on Windows and Linux CI. This is product-owned field validation, not an independent adopter claim.

Safety boundaries

• Local demo: no network target, telemetry, credentials, account, or paid model.
• Real targets: one reviewed config entry and one explicit tool allowlist entry.
• Execution: an exact plan digest, finite timeouts, and at most 10 retries; examples use one.
• Evidence: metadata and hashes replace tool arguments and result bodies before persistence.
• Filesystem: contained paths, link-escape rejection, exclusive regression publication, and cleanup.

ResiliReplay is a reliability tester, not a security certification, sandbox, authorization layer, or claim that every recovery is safe. Only invoke tools whose effects and retry semantics you own and understand. Remote targets require the existing explicit ownership controls.

MCP support

Evidence labels describe what was executed; they do not imply vendor endorsement. Read the MCP test guide, Inspector compatibility guide, and limitations.

MCP-RES v0.2

MCP-RES v0.2 is the project-defined, open reliability evidence standard behind the result vocabulary. Its versioned profiles cover identity, bounded recovery, duplicate effects, cleanup, integrity, and executable causal evidence. ResiliReplay is a reference implementation, not a required dependency.

MCP-RES is independent of the official MCP specification. It is not an official MCP standard, security certification, or endorsement. The immutable v0.1 and v0.2 trees, schemas, vectors, conformance tools, governance, and limitations remain available from the standards landing page.

Secondary agent-runtime support

ResiliReplay can also capture sanitized failures from supported agent runtimes and compile them into regressions. This is a secondary workflow; it does not change the MCP-first product path.

Genuine local runtime coverage exists for LangGraph 1.4.9 and OpenAI Agents SDK 0.14.3 using deterministic, no-key models. Claude Code and Codex integrations are installation- and fixture-verified; Hermes is installation-verified. Other named surfaces are documented only. No authenticated hosted model, billed provider call, production API behavior, or vendor endorsement is claimed.

Read framework evidence, agent compatibility, plugin operations, and the framework support policy.

CLI map

MCP reliability appears first in resilireplay --help:

mcp demo       Try a deterministic local MCP reliability test
mcp test       Test one reviewed MCP tool with bounded recovery
mcp validate   Validate a test configuration without starting it
mcp serve      Run ResiliReplay as a local stdio MCP server
mcp audit      Preserve the existing lower-level audit workflow

Existing campaign, replay, reporting, adapter, agent capture, and Studio commands remain available. The root demo command is retained as a hidden compatibility alias for mcp demo; mcp audit is not removed or weakened.

Artifact behavior

By default, mcp demo runs in an isolated temporary directory, executes its regression, removes the directory, and prints one evidence digest. Retain a deterministic bundle only when requested:

npx --yes resilireplay@latest mcp demo --keep
npx --yes resilireplay@latest mcp demo --output ./my-evidence
npx --yes resilireplay@latest mcp demo --json

--keep writes .resilireplay/demo/. Explicit output must stay inside the current project. An identical existing bundle is accepted; a mismatch fails without changing it. The completion manifest is written last, and JSON output contains only relative artifact paths.

Exit codes

Campaign-specific codes remain documented in the campaign schema guide.

Installation

Run without a global install:

npx --yes resilireplay@latest --version

Supported runtimes are Node.js 22 and 24 on current Ubuntu and Windows GitHub-hosted runners. The npm package is Apache-2.0 licensed and published from a protected GitHub release through npm trusted publishing with OIDC and provenance; no long-lived npm token is accepted by the release workflow.

Security and privacy

Capture is off by default. ResiliReplay sends no telemetry and persists no raw prompt, transcript, environment value, authorization header, token, or unrestricted tool body by default. Review SECURITY.md, THREAT_MODEL.md, and docs/LIMITATIONS.md before testing a stateful tool.

Please report vulnerabilities through the repository's private security-reporting path, not a public issue. Reliability failures and compatibility gaps can use the public issue templates.

Contributing

Focused bug fixes, MCP compatibility evidence, and bounded reliability cases are welcome. Read CONTRIBUTING.md and the case contribution rules. Do not submit credentials, private traces, production tool bodies, or evidence you are not authorized to publish.

Development

Maintainer development uses the repository-pinned toolchain:

pnpm install --frozen-lockfile
pnpm quality
pnpm mcp:example:verify

The first-time user path never requires a checkout or pnpm. Release gates additionally inspect the packed tarball, test clean installs, scan generated evidence, exercise the cross-platform matrix, and verify immutable MCP-RES v0.1/v0.2 content.

License

Apache-2.0. See LICENSE.