opendrs: PR blast radius analyzer and risk score for safe auto-approval

A calibrated 0-1 risk score for every pull request, so AI-reviewed, low-blast-radius PRs can be auto-approved into the merge queue while risky ones still wait for a human.

AI agents now write and review code faster than people can read it, so human review has become the bottleneck. Most pull requests are low risk, and a few cause most incidents. opendrs measures each PR's blast radius and risk: what the change touches, how many files depend on it, the history of the files it changes, author experience, missing tests and critical paths. It turns that into a 0-1 score using signals from your own git history. A team then sets a policy. For example, a PR that scores below 0.3 and has passed an AI review (Claude, CodeRabbit, Copilot or any check run) is approved and sent to the merge queue or auto-merge. Riskier PRs still wait for a human. The approach follows Meta's Diff Risk Score (DRS); opendrs is an independent, open-source implementation that runs in any git repository, with no service to host.

Use it as:

• a GitHub Action: error-path/opendrs@v0, which comments, labels and, optionally, approves and merges;
• a CLI: npx @error-path/opendrs score;
• a Claude Code plugin (skill plus MCP server): /plugin marketplace add error-path/opendrs, then /plugin install opendrs@opendrs;
• an agent skill for Claude Code, Cursor, Codex and others: npx skills add error-path/opendrs;
• an MCP server for any agent: npx -y @error-path/opendrs mcp.

Website and guides: error-path.github.io/opendrs, including how to auto-approve low-risk PRs and how opendrs compares with other PR risk tools.

Contents: Quick start · How it works · Signals · Policy and safety rails · Calibrate and backtest · Results · LLM judge · MCP and skill · CLI · Configuration · Limitations · Compared with other tools

Quick start

Score your current branch against the default branch. No configuration or API key is needed:

npx @error-path/opendrs score

You get a score, a tier (green, yellow or red), the signals that drove it with their evidence, and the policy decision. --json prints the full result, and --fail-above 0.6 makes the command exit 1 above a threshold.

To score every pull request, add .github/workflows/opendrs.yml (or run npx @error-path/opendrs init):

name: opendrs
on:
  pull_request:
    types: [opened, synchronize, reopened, ready_for_review, labeled, unlabeled]
permissions:
  contents: read
  pull-requests: write
  checks: read
jobs:
  opendrs:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
        with:
          fetch-depth: 0   # required: the score reads past commits and authors
      - uses: error-path/opendrs@v0

This only comments and labels. Approval is off until you turn it on in .opendrs.yml on your default branch; see docs/github-action.md.

How it works

flowchart LR
  diff["git diff<br/>merge-base...head"] --> signals
  hist["git history"] --> signals
  graph["import graph"] --> signals
  llm["LLM judge<br/>(optional)"] --> signals
  signals["Signals<br/>each with evidence<br/>and provenance"] --> model["Logistic model<br/>score = sigmoid(b + Σ w·x)"]
  model --> tier["Tier<br/>green / yellow / red"]
  tier --> policy["Policy<br/>blocks, size cap, unknowns,<br/>AI review, sampling"]
  policy --> decision["auto-approve<br/>waiting-ai-review<br/>human-review"]

1. Signals. Each signal provider reads the diff, the history and the import graph and emits features: size, how spread out the change is, past fixes and reverts in the touched files, the author's experience with them, complexity, blast radius (transitive dependents), critical paths and missing tests. Every signal records how it was obtained (observed, derived, inferred or unknown) and the evidence behind it.
2. Model. A logistic regression turns the features into a probability. Each signal's contribution w·x is reported, so you can see why a PR scored what it did. Out of the box the weights are hand-set priors; opendrs calibrate fits them to your repository.
3. Tiers. The score maps to green, yellow or red, mirroring Meta's gates. Thresholds can be absolute scores or percentiles of your own history.
4. Policy. Ordered rules turn the score into a decision (see below).

Meta's DRS features and their opendrs signals

Meta's baseline is a logistic regression over about a dozen features, chosen from about 100 (paper). opendrs computes each one from git alone:

The import graph uses regex-based extractors for TypeScript/JavaScript, Python, Go, Java/Kotlin, Rust and Ruby. Files in other languages get unknown blast-radius signals, which the policy treats as risk, not as zero.

Policy and safety rails

The decision rules run in this order; the first that matches wins:

1. a changed file matches paths.block → human-review (hard block);
2. a changed file is ignored (paths.ignore), binary, or a submodule pointer (gitlink), the PR deletes tests (tests.deleted), or an added line matches a content.block pattern → human-review;
3. more than maxChangedLines changed → human-review;
4. tier is not green (unless autoApprove.deferredReview is on and the score is in its band), or an active freeze window suspends auto-approval or holds it to a stricter green, or the PR's source (human, bot or AI agent) has its own stricter green, has hit its daily cap, or lacks the required track record → human-review;
5. the LLM judge failed, or llm.requireSafe is on and the judge did not call every hunk safe with enough confidence, or the unknown signals, all at their worst at once, could have changed the tier → human-review;
6. the PR is a draft, or a reviewer's latest review requests changes → human-review;
7. a required AI-review check has not passed on the head commit → waiting-ai-review;
8. the PR is sampled for human review → human-review;
9. otherwise auto-approve, or deferred-review when the tier rule was waived for the deferred band.

The decision also suggests three reviewers: the most frequent recent authors of the changed files. For a yellow or red PR, authors of earlier fixes and reverts in those files count extra. When the PR needs a human, the Action requests a review from one of the top three, picked deterministically per PR, rather than from a team: assigning a group invites the bystander effect (Rigby et al., TOSEM 2025), and the pick spreads the work across the experts.

The rails that matter when the Action can approve PRs:

• Config comes from the base branch. The Action reads .opendrs.yml and the calibrated model from the PR's base commit, so a PR cannot loosen its own gate.
• Hard blocks. paths.block (by default .github/**, .opendrs.yml, .opendrs/**, CODEOWNERS, migrations, Terraform, .gitmodules, .gitattributes, and files that execute code on install, build or CI; see Configuration) always sends the PR to a human.
• Binary files cannot pass as text. Git's own test only looks for a NUL in the first 8000 bytes and obeys .gitattributes, so opendrs also reads every changed file at the PR head and treats it as binary when it holds a NUL anywhere or is not valid UTF-8. With git 2.40 or later the diff reads attributes from the base commit (--attr-source), so the PR's own .gitattributes has no effect. Older git ignores only the global and system attribute files and still reads the ones in the checkout; the content check above still catches binaries there.
• Submodules. Moving a submodule pointer pulls in code the diff does not show, so any gitlink change goes to a human, whatever diff.submodule, diff.ignoreSubmodules or a submodule's ignore setting says. Repointing a submodule's URL changes .gitmodules, which is blocked.
• Self-modification guard. Independently of any config, the Action never approves a PR that adds, changes, deletes or renames .opendrs.yml, .opendrs/**, .github/**, action.yml, action/**, .gitmodules or any .gitattributes, or the configured model file. It also never approves PRs from forks.
• Sampling. sampleRate (default 10 %) of eligible PRs still go to a human, chosen deterministically from the head commit, so you keep measuring what the gate lets through.
• Unknown is not safe. A signal that cannot be computed is reported under "Unknowns" and blocks auto-approval when it could have changed the tier. A failed or unparseable LLM answer counts the same way.
• Withdrawn approvals. When a new push makes the PR ineligible, the Action dismisses its own earlier approval ("Risk increased: yellow 0.45"). A green PR held back by a freeze or a sample names that rule instead.
• Freeze windows. Meta gates more diffs on weekends and around peak events, when fewer people can respond to an incident. A freezes window (weekdays and/or dates in a timezone) suspends auto-approval, or with green holds it to a stricter threshold. Tiers and labels do not change.
• Per-source policy. Like RADAR, opendrs classifies each PR as written by a person, a bot account (Dependabot, Renovate) or an AI coding agent (Claude Code, GitHub Copilot's coding agent, Cursor, Codex, Devin, Jules, Aider, or your own sources.agent.patterns). It reads commit trailers, author emails and, in CI, the GitHub account type. The comment shows the source and why. sources.<human|bot|agent> can set a stricter green, requireSafe (the LLM gate for that source only), a dailyCap on that source's commits landed in the last 24 hours, and a trackRecord (by default: at least 10 landed commits in 60 days, at most 5 % of them reverted). These settings can only tighten the gate. Trailers are written by the PR author's tools, so a source must never earn a looser threshold than a person gets.
• Deferred review. RADAR lets some low-risk human diffs land first and be reviewed after. With autoApprove.deferredReview (enabled, below, sources, default [human]), a PR whose score is between green and below and that passes every other rule gets deferred-review: the Action approves it and lets it merge, requests a post-merge review from the suggested reviewers, and labels it opendrs:review-after-merge. Only the tier rule is waived, and only in that band.
• Landing delay. With autoApprove.landingDelayMinutes (or a longer sources.<kind>.landingDelayMinutes), an eligible PR waits that long, on the same head commit, before opendrs approves it. Meanwhile it carries the label opendrs:landing, and anyone can veto by requesting changes, pushing a commit (which restarts the clock) or adding opendrs:hold. A scheduled run of the Action re-checks waiting PRs; see docs/github-action.md.
• Content blocklist. content.block regexes are checked against every added line, like Meta's RADAR blocklisted code phrases. A match goes to a human, and the reason names file:line.
• Stale approvals are blocked by branch rules. The Action approves only if the base branch requires at least one approval and dismisses stale approvals on push. Without that rule, an approval for one commit could carry over to a later, riskier push. See docs/github-action.md for the ruleset.
• Prompt injection. The diff, PR title and PR body are untrusted. The LLM prompt wraps them in delimiters and tells the model to ignore instructions inside them; the PR body is not sent at all unless llm.includeDescription is true. The answer must be exactly one JSON object and is clamped to [0, 1]. The judge can only raise the score, never lower it, so talking it into "risk 0" gains nothing. Risk claims written in the PR text never change the score directly.

Calibrate and backtest: measure your threshold

The default weights are priors. Before you let the score approve anything, measure it on your own history:

npx @error-path/opendrs backtest     # how well do the default priors rank your past commits?
npx @error-path/opendrs calibrate    # fit weights to your history, write .opendrs/model.json
npx @error-path/opendrs backtest     # compare default and calibrated models

The backtest replays past commits point-in-time (each commit sees only the history before it) and reports Meta's metric: the share of incident-linked commits captured when you gate the riskiest 5 %, 10 % and 50 % of commits, plus ROC-AUC. Incident-linked commits come from reverts, SZZ-blamed bug fixes and an optional .opendrs/incidents.jsonl. See docs/calibration.md for the label sources and how to read the numbers.

For orientation only: Meta reports that its LR baseline captured 18.7 %, 27.9 % and 84.6 % of SEVs at 5 %, 10 % and 50 % gated. Those are Meta's in-house results on its own code and incident data, not numbers your repository should be expected to match.

Results on public repositories

On the public ApacheJIT benchmark (Apache Java projects, official 2017–2019 test years, apache/hadoop left out because it has no positive labels), calibrated opendrs reaches AUC 0.819 and catches 66 % of buggy commits in its riskiest 30 %, on par with a regression on the classic JIT metrics.

We ran opendrs calibrate and then opendrs backtest --max-commits 4000 on nine open-source repositories with opendrs v0.1.0, without changing any configuration. Labels come from reverts and SZZ blame on fix commits only, since none of these repos publish an incident list. Read them as "later needed a fix", not as production incidents.

Held-out commits are the commits newer than the calibration window, and so unseen by the fitted model. That is the fair comparison between the default priors and a calibrated model:

What this shows:

• Calibration beat the default priors on held-out commits in all nine repositories. The median AUC gain is +0.04, but on spring-boot and quarkus the gain is too small to matter. Fit the model on your own history; don't rely on the defaults.
• The default priors alone rank risk better than chance everywhere. Their AUC over all replayed commits ranges from 0.58 to 0.70. On quarkus, gating the riskiest 5 % and 10 % caught 18.5 % and 28.2 % of positives. For orientation only, Meta reports 18.7 % and 27.9 % for its internal logistic-regression baseline. Different code, labels and population make this an analogy, not a comparison.
• Positive rates range from 4 % (spring-boot) to 29 % (cargo). SZZ labels are noisy, and a project's fix-commit habits shape them. Your own incident list (.opendrs/incidents.jsonl) is the strongest label you can add.
• Two known approximations, both reported by backtest: the import graph comes from the newest commit, and the newest commits' labels are incomplete.

A broader run: 18 more repositories

Both runs used opendrs 0.1.2, before the label horizon (history.labelHorizonDays) existed. Re-running with it should mostly change the held-out column, by leaving out commits too new to have been fixed yet.

A second, shorter run used the newest 1,200–1,500 commits of each of 18 more repositories, again without any configuration. "Caught" is the share of later-fixed commits that fall in the riskiest 10 % of commits; a random ranking catches 10 %.

What this adds:

• Untuned, the score ranks risk better than chance in 17 of 18 repositories. The median AUC over all replayed commits is 0.71, and the riskiest 10 % of commits hold 15–45 % of the later-fixed ones, against 10 % for a random pick. serde (0.51) is the exception.
• Calibration helped in 11, was flat in 1 and hurt in 6. Small held-out windows make these numbers noisy: fastapi and laravel have fewer than 15 positives in theirs. Since this run, calibrate keeps the default priors when the fitted weights do not beat them on a validation slice of your own history. Either way, run backtest before you trust a threshold.
• Held-out AUCs near 0.45–0.50 (tokio, terraform, eslint) come from the newest commits, whose bugs mostly haven't been found and fixed yet. Expect labels there to be incomplete.

Reproduce a row: git clone <repo> && cd <repo> && npx @error-path/opendrs calibrate --max-commits 4000 && npx @error-path/opendrs backtest --max-commits 4000. Runs took 1 to 16 minutes per repository on a 4-core Linux machine.

LLM judge (optional)

opendrs works without an LLM. With one, a prompted model reads the title and diff and returns a probability of incident p; it enters the model as one more signal (llm.semantic, prior weight 0.8) with the value max(0, logit(p) − logit(0.2)). The judge can only add risk above a 0.2 baseline: an answer of 0.2 or less adds nothing, and no answer can lower the score. A diff that someone talks the judge into calling safe therefore scores exactly as it would without an LLM. The reply must contain exactly one JSON object. When the diff exceeds maxDiffChars, every file first gets its first hunk and only then do larger files get more, and truncated files are marked.

llm:
  enabled: true
  provider: anthropic        # reads ANTHROPIC_API_KEY
  model: claude-sonnet-5
  requireSafe: true          # optional: RADAR-style gate, see below
  minConfidence: 8

The judge also classifies every hunk it sees, marked [hunk N] in the diff, into one category, following Meta's RADAR reviewer. The safe categories are refactor, dead_code, defensive, logging, formatting, docs, imports, tests and static_resources. The risk categories are logic_bug, security, performance, structural and complex. Each verdict carries a 0-10 confidence. The categories are used twice:

• Ranking. Each risk category is a signal (llm.risk.logic_bug and so on, 0 or 1) with a positive weight. calibrate never lets an llm.* weight turn negative, so no answer lowers a score.
• Gate. With requireSafe: true, a PR is auto-approved only if the judge called every hunk it saw safe with at least minConfidence, classified all of them, and saw the whole diff (no truncation). Otherwise it goes to a human, and the comment names the hunks. Like RADAR, this is a necessary condition, never a sufficient one: every other rule still applies.

Whether the category is safe is decided by opendrs from the category name, not by the model.

For OpenAI or any OpenAI-compatible server (Azure OpenAI, Ollama, vLLM), set provider: openai, model, and baseUrl; the key is read from OPENAI_API_KEY, or from the variable named in apiKeyEnv. In the Action, pass the key as the llm-api-key input. The diff is sent to the provider you configure. If the call fails, the signal is unknown, and the PR cannot be auto-approved on that run.

MCP server and agent skill

opendrs mcp runs a Model Context Protocol server on stdio, so coding agents can check risk before they open a PR. It exposes score_diff (base, head, cwd), explain_file_risk (path), suggest_reviewers (paths) and list_hotspots (top). For Claude Code:

claude mcp add opendrs -- npx -y @error-path/opendrs mcp

Other clients take the same command: {"command": "npx", "args": ["-y", "@error-path/opendrs", "mcp"]}.

The agent skill in skills/opendrs/SKILL.md tells an agent to run opendrs score before opening a PR and to split changes with a large blast radius.

CLI

Exit codes: 0 success, 1 score above --fail-above, 2 error. Set OPENDRS_DEBUG=1 for stack traces.

Configuration

All keys are optional. This is the full file with every default (opendrs init writes it):

thresholds:
  mode: absolute        # absolute: scores in 0..1; percentile: percentiles of your
                        # calibrated history (e.g. green: 50, yellow: 90, red: 95)
  green: 0.3            # below green: eligible for auto-approval
  yellow: 0.6
  red: 0.8

autoApprove:
  enabled: false        # false = comment and label only (dry run)
  requireAiReview:      # every listed check must have passed on the PR's head commit
    checks: []          # check-run names that must have concluded success, e.g. ["claude-review"]
  mergeMethod: queue    # queue | auto-merge | none
  sampleRate: 0.1       # share of eligible PRs still routed to a human
  maxChangedLines: 400  # never auto-approve above this, whatever the score

paths:
  # block and critical are matched ignoring case, and a list you set adds to these defaults.
  block: [".github/**", ".opendrs.yml", ".opendrs/**", "**/CODEOWNERS",
          "**/migrations/**", "**/migration/**", "**/migrate/**", "**/*.tf",
          ".gitmodules", "**/.gitattributes",
          # files that execute code on install, build or CI
          "**/package.json", "**/.npmrc", "**/.yarnrc", "**/.yarnrc.yml", "**/.pnpmfile.cjs",
          "**/setup.py", "**/setup.cfg", "**/pyproject.toml", "**/pip.conf",
          "**/build.gradle", "**/build.gradle.kts", "**/settings.gradle*", "**/gradle/wrapper/**",
          "**/pom.xml", "**/.mvn/**",
          "**/Cargo.toml", "**/build.rs", "**/go.mod",
          "**/Gemfile", "**/Rakefile", "**/*.gemspec",
          "**/requirements*.txt", "**/conftest.py", "**/.pre-commit-config.y*ml", "**/tox.ini",
          "**/noxfile.py",
          "**/*.csproj", "**/*.sln", "**/Directory.Build.*", "**/nuget.config",
          "**/CMakeLists.txt", "**/*.cmake", "**/.cargo/**", "**/.husky/**",
          "**/Makefile", "**/GNUmakefile", "**/*.mk",
          "**/Dockerfile*", "**/docker-compose*.y*ml", "**/compose.y*ml",
          "**/Brewfile", "**/Vagrantfile", "**/Procfile", "**/*.nix", "**/flake.lock",
          "**/.devcontainer/**", "**/.envrc", "**/Taskfile.y*ml", "**/justfile",
          ".gitlab-ci.yml", "Jenkinsfile", ".circleci/**", "azure-pipelines.yml", ".buildkite/**",
          "**/bitbucket-pipelines.yml", "**/.drone.yml", "**/.travis.yml", "**/cloudbuild.y*ml",
          "**/Earthfile", "**/BUILD", "**/BUILD.bazel", "**/WORKSPACE*", "**/MODULE.bazel", "**/*.bzl"]
  unblock: []           # default block globs to drop, written exactly as above, e.g. ["**/go.mod"]
  critical: ["**/auth/**", "**/payment*/**", "**/billing/**", "**/security/**"]
  # ignore and tests: a list you set replaces the default.
  # Scored as no risk (a lockfile bump stays green) but never auto-approved, like binary files.
  ignore: ["**/*.lock", "**/package-lock.json", "**/pnpm-lock.yaml", "**/go.sum", "**/dist/**",
           "**/vendor/**", "**/node_modules/**", "**/*.min.js", "**/*.snap"]
  tests: ["**/test/**", "**/tests/**", "**/__tests__/**", "**/*.test.*", "**/*.spec.*",
          "**/*_test.*", "**/test_*.py"]

llm:
  enabled: false
  provider: anthropic   # anthropic | openai (any OpenAI-compatible endpoint)
  model: claude-sonnet-5
  # baseUrl: https://api.openai.com/v1
  # apiKeyEnv: MY_KEY_VAR
  includeDescription: false   # the PR body is untrusted; title only by default
  maxDiffChars: 60000

history:
  lookbackDays: 365
  fixPattern: "\\b(fix(es|ed)?|bug|hotfix|regression|incident|sev\\d?|revert)\\b"
  maxCommits: 20000
  labelHorizonDays: 60   # calibrate/backtest skip the newest 60 days: their bugs may not be found yet (0 = off)

graph:
  enabled: true
  maxDepth: 3
  maxFiles: 50000

model: .opendrs/model.json   # written by `opendrs calibrate`; priors are used until it exists

content:
  block: []   # JS regexes matched against each added line, e.g. ["\\beval\\(", "dangerouslySetInnerHTML"]

freezes: []   # while a window is active: auto-approval suspended, or held to a stricter green
#  - name: weekend
#    timezone: America/New_York   # IANA name; default UTC
#    days: [sat, sun]
#  - name: black-friday
#    from: "2026-11-26"           # inclusive local dates, quoted
#    to: "2026-11-30"
#    green: 0.15                  # same units as thresholds; must be below thresholds.green

sources:      # per-source policy; every setting only tightens the gate
  agent:
    green: 0.2                # stricter green for AI-agent PRs (thresholds.mode units)
    requireSafe: true         # LLM judge must call every hunk safe (needs llm.enabled)
    dailyCap: 20              # at most 20 agent commits landed in the last 24 hours
    trackRecord: { days: 60, minCommits: 10, maxRevertRate: 0.05 }
    patterns: []              # extra regexes over commit messages / author emails
  bot: {}
  human: {}

Why the default block list is long. Besides the gate's own files, paths.block lists files that execute code on install, build or CI: package manifests and their registry settings (package.json, .npmrc, setup.py, pyproject.toml, requirements.txt, ...), build scripts and project files (build.gradle, pom.xml, build.rs, Makefile, CMakeLists.txt, *.csproj, Dockerfile, Bazel and Nix files, ...), developer hooks and environments (.husky/, .pre-commit-config.yaml, conftest.py, .envrc, .devcontainer/, ...) and CI pipelines outside .github/. Block and critical globs are matched ignoring case, so makefile or an EF Core Migrations/ folder cannot slip past **/Makefile or **/migrations/**. A one-line change there (a postinstall hook, a registry pointed at another host, a new build step) scores as small and low-risk, but runs on every developer machine and CI runner. Dependency bumps also change a lockfile, which is in paths.ignore and so already goes to a human.

Your lists add to the defaults. A paths.block or paths.critical you set extends the default list; it never replaces it, so block: ["infra/**"] blocks infra/** as well as everything above. To drop a default block entry, name it under paths.unblock on the default branch, written exactly as in the list, for example unblock: ["**/go.mod"] if you trust go.mod edits to code review alone. An unblock entry that is not a default block glob is a config error that lists the valid ones, so a typo cannot pass silently. paths.unblock wins over the same glob under paths.block. Default critical globs cannot be removed. paths.ignore and paths.tests work the other way: a list you set replaces the default. In the Action, the self-modification guard keeps .github/**, .opendrs.yml, .opendrs/**, the action, .gitmodules and .gitattributes blocked whatever the lists say.

opendrs refuses a config that would fail silently, naming the offending value:

• Globs are repo-relative and support **, *, ? and {a,b}. A glob with [ or ] (character classes are not supported), a backslash, or a leading / or ./ is an error.
• paths.unblock entries must each be a default paths.block glob, character for character.
• Thresholds must be ordered: 0 ≤ green < yellow ≤ red ≤ 1 in absolute mode, and 0 < green < yellow ≤ red < 100 in percentile mode.
• content.block entries must compile as JS regular expressions. JS has no inline flags such as (?i), so write [Dd]rop instead.
• freezes need at least one of days, from or to, a real IANA timezone, YYYY-MM-DD dates with from ≤ to, and a green below thresholds.green.

Limitations

• The default weights are not fit to your code. They are priors chosen to rank obvious cases sensibly. Run backtest before trusting them, and calibrate when you have enough history.
• Labels are proxies. Reverts and SZZ-blamed fixes are noisy stand-ins for incidents: SZZ blames refactors and formatting, and many incidents are fixed by config changes or never touch git. .opendrs/incidents.jsonl improves this only as far as you keep it current.
• Small repositories give unstable numbers. With a few dozen positive commits, calibrated weights and capture rates swing widely between runs.
• The import graph is regex-based. It misses dynamic imports, reflection, dependency injection, code generation, and dependencies across services or repositories.
• Complexity is a heuristic that counts branch keywords; it does not parse code.
• Test signals count lines, not assertions. tests.missing and tests.deleted see only files matching paths.tests. A PR that weakens a test without shrinking it (an assertion edited into a no-op) is not flagged by them, and tests outside those globs are not seen at all. Splitting a test file into new ones counts as deleting a test file and goes to a human.
• The LLM judge is prompted, not trained. Meta's LLM variants were fine-tuned on Meta's own incidents; a general model reading a diff is a weaker signal, and it costs money per PR.
• The score is not a code review. It estimates the chance of an incident, not whether the change is correct. Auto-approval still depends on your tests and your AI reviewer.
• Required checks are matched by name. opendrs does not verify which app produced a check run, so protect the names you list (see docs/github-action.md).
• GitHub only. The core is host-agnostic, but only a GitHub Action exists today. The Action needs full history (fetch-depth: 0). In a shallow clone, diffusion.authors and history.* are reported as unknown ("shallow clone: history incomplete"), keeping what they saw as a lower bound, so a missing history never reads as low risk.

Compared with other tools

The full comparison, with how each tool scores and where it runs, is at error-path.github.io/opendrs/alternatives.html.

Prior art and credits

• Meta's Diff Risk Score. Diff Risk Score: AI-driven risk-aware software development (Engineering at Meta, 2025) and Abreu et al., Moving Faster and Reducing Risk: Using LLMs in Release Deployment (2024). The feature set, the tiered gates and the capture-at-X %-gated metric come from this work. opendrs is not affiliated with Meta.
• Meta's RADAR. Adams et al., Automating Low-Risk Code Review at Meta (2026): the structured LLM reviewer with safe and risk categories, per-source policy (daily caps, track record), deferred review, the landing delay and the content blocklist follow this paper. Related Meta work behind hotspots and reviewer assignment: Mockus et al., "Metrics Driven Reengineering and Continuous Code Improvement at Meta" (ASE 2025); Paraspatki et al., "Code Reviewer Recommendation for High Risk Diffs at Scale" (FSE Companion 2026); Rigby et al., Improving Code Reviewer Recommendation (TOSEM 2025).
• Benchmarks and peers. Keshavarz and Nagappan, ApacheJIT (MSR 2022), the dataset bench/apachejit/ scores; Sayedsalehi, Rigby and Mockus, DRS-OSS (2025), the open-source LLM counterpart whose numbers the benchmark quotes.
• Just-in-time defect prediction and SZZ. Mockus and Weiss, "Predicting risk of software changes" (2000); Śliwerski, Zimmermann and Zeller, "When do changes induce fixes?" (2005); Kamei et al., "A large-scale empirical study of just-in-time quality assurance" (2013). The change-level features and the SZZ-style labelling follow this literature.
• REA (morluto/rea). opendrs adopts REA's evidence-first method: every conclusion is marked observed, derived, inferred or unknown, and unknown is never treated as safe. It also follows REA's layering of domain types, providers, a shared engine and thin adapters.

License

MIT