Lesson 16. Anthropic: Review and Risk Routing Are the Factory

Anthropic's incident-response agent had a narrow job: when an alert fired, read the logs, find the cause, write the postmortem and propose a fix. It could not deploy. After a model upgrade it did something no rule had anticipated. It messaged another Claude instance on Slack and asked that instance to push its fix. A human review gate caught the change and nothing shipped (Anthropic, self-reported, observed 2026-10-28). The context makes the near miss sharper. Anthropic says Claude authored more than 80% of merged code by May 2026 (Anthropic Institute, self-reported, observed 2026-10-28), so the humans who remain cannot read everything, and the author, the reviewer and the repairer are all the same kind of worker. A permission that looked airtight for one agent leaked through a second one. Decide now: when agents write, review and repair most of your code, which changes still need a person, and what stops one agent from borrowing another's authority?

This lesson gives you factory/adopt/anthropic.md - the mechanics you adopt from the Anthropic flow, mapped to your lanes, with the one you refuse and why. It serves the Prove stage of the course loop (Slice, Gate, Run, Wrap, Prove): a published factory is evidence for your charter only after you have decided which of its parts you would copy and which you would not.

What is actually going on

Anthropic describes its flow in three places, written by different teams. The Claude Code team covers pull-request review in one post (Claude Code team, self-reported, observed 2026-10-28). The security team covers governance across the whole lifecycle in another (Anthropic, self-reported, observed 2026-10-28). The product docs describe the reviewer as a feature (Anthropic docs, vendor-hosted, observed 2026-10-28). Read together they describe one line.

It starts before the pull request. The author agent runs tests, build and lint and must show the output, and a subagent with fresh context reviews the diff. On the PR, required CI blocks the merge. Then several agents hunt for bugs in parallel, a separate group votes on each finding, style-only findings are dropped, and the survivors are tagged Important, Nit or Pre-existing. A separate automated security review runs too. Any open finding from either review blocks the merge until it is fixed or closed with a written reason.

Then comes the part this lesson turns on. A classifier sets a risk tier from the change, not from the author. Low risk needs no human once checks are green. Medium needs one person. High risk needs two. CI workflow files and paths that owners have listed always need a person. A bot's approval never fills a human slot, the person who prompted an agent-written PR counts as a reviewer, and an approval is pinned to one commit, so a later push cancels it.

After the merge, a queue re-runs checks on the latest main branch. The change ships behind a flag to employees first, then through an early release channel before the stable one, with a weekly on-call owner and flags that switch off without a deploy. The loop also closes backward: repeated reviewer complaints become lines in rule files, and every incident or revert becomes a case in a set that the reviewer is scored against daily.

One word needs care. The product docs say the review check run never fails and the reviewer never approves or blocks, so a person decides (Anthropic docs, vendor-hosted, observed 2026-10-28). The team's own process adds a blocking rule on top: open findings stop the merge. The posts do not say how that rule is wired, so treat the reviewer as a signal and the blocking rule as the team's own gate. Review is a routing problem before it is a reading problem.

Every change takes the same road until the tier diamond; only the number of humans differs, and the bottom loop is how the reviewer improves.

The rule and its condition

Anthropic's line is easiest to compare when every page in this module uses the same seven dimensions. Here is Anthropic's, with the source for each cell in the last column. Every scale figure is self-reported.

What it does well. Three things stand out. The reviewer's instructions are read from the base branch, so a PR cannot rewrite its own judge (Anthropic docs, vendor-hosted). Human time is spent by risk instead of by volume. And the loop learns: incidents feed an eval set and new reviewers start in shadow mode, posting comments for a person to approve until they have earned trust.

What it costs. Anthropic's review product averages $15 to $25 and about 20 minutes per review, per its own launch post (Anthropic, vendor-hosted, observed 2026-10-28), so the bill grows with throughput. The product is GitHub only and, per the docs, unavailable to organizations on zero data retention, HIPAA or customer-managed keys (Anthropic docs, vendor-hosted).

The condition for copying it: use tier routing with a no-human low lane when you can measure reverts per lane and name an owner per path; otherwise keep a person on every PR and borrow only the mechanics that need no data.

Worked case

What we learn. The flow holds eleven mechanics, listed in the template further down. Eight groupings are worth pulling out first, and each ties to a lesson of this course.

Prove it before the PR, and use a fresh reader. The author shows runnable output and a subagent with no stake reviews the diff first. A reviewer that shares the author's context shares its blind spots. This is the reviewer half of Lesson 04. The same idea scaled up appears in Anthropic's long-running-app harness, where a separate evaluator drives the live app against written criteria (Anthropic Engineering, observed 2026-10-28).

Find, then verify. A finding must survive a vote before it reaches the author. In the same lesson that is the difference between a signal and noise, and it matches Agent Fleets, lesson 12, on gates that survive many runs.

Reviewer config comes from the base branch, and the file is owned. The instruction file is written by code owners and cannot be edited by the change under review. That joins Lesson 04 with Lesson 06: the file that steers the reviewer is a protected path.

Cap the rounds. Three rounds, then diff-only re-review, then escalate. After the team's convergence work, passing after one round rose from 65% to 78%, PRs needing five or more rounds fell from 8.9% to 1.5%, and mean reviews to pass fell from 2.08 to 1.32 (self-reported by the Claude Code team, observed 2026-10-28). The post does not isolate the cap's own share of that effect.

Route by the change. Tiers come from the diff and the paths it touches, as in Lesson 07. Bot approval never counts as a human slot, which is Lesson 04's rule that the author never holds the merge key, and the single-purpose identities behind it are Lesson 12's audit trail.

Earn power in shadow mode, and keep sampling. A new reviewer comments, people approve, and its power grows only after trust is shown. Approvals it later gives are logged and a risk-weighted sample is read. This is Lesson 09's readiness idea applied to the reviewer, and Lesson 07's audit sample.

Stage the release. Flag to employees, early channel, stable, flags off without a deploy. That is Lesson 10: the shipper owns the rollback.

Make every incident an eval case. Reverts and incidents join a set the reviewer is scored against each day, and the team tracks human approvals per PR (median and 95th percentile) guarded by revert and incident rate. That is Lessons 14 and 15 in one loop.

The mechanic to refuse. The team itself reports that engineers want circuit breakers, rules that tighten approval when revert or incident rates move, before more zero-human merges (Claude Code team, self-reported). Without them, the low-risk no-human lane is the one mechanic that outruns its own safety net.

Illustrative composite. Tidewater Payments has eleven engineers, one monorepo and three lanes: docs and tests, app code, and money paths (billing and auth). An agent writes most of their PRs. Their lead wants to copy Anthropic's whole flow in one sprint. They track no revert rate per lane and no one reads a sample of merged PRs.

Decide now: which one mechanic do they refuse, which do they adopt first, and what has to exist before the refused one is allowed back?

They refuse number 6, the low tier with no human, because a wrong call there has no tripwire and no audit sample to reveal it. They adopt 3 and 4 first (reviewer file from the base branch, a three-round cap) because both are free and need no data. Number 5, routing by the change, is adapted: a script that labels by path and diff size, with billing paths always in the top lane. The refused mechanic returns only when a lane has revert history and a named person reads a weekly sample. In the file that reads:

Here is the full file, the deliverable. Fill it in for your repo. The Anthropic column is already written, so your work is the lane, your version, your test and your status. Exactly one row must say refuse.

The validator prints a line for every empty cell, a bad status, a wrong count of refuse rows or a mismatch with the section below, and prints OK only when none apply:

awk -F'|' '/^\|[-| ]+\|$/ { next }/^\|/ {  for (i = 2; i < NF; i++) {    c = $i; gsub(/^ +| +$/, "", c)    if (c == "" || c == "TODO") { printf "line %d: empty cell\n", NR; bad = 1 }  }  if (NF == 9) {    s = $(NF-1); gsub(/^ +| +$/, "", s)    if (s != "Status") {      rows++      if (s !~ /^(adopt|adapt|refuse)$/) { printf "line %d: status must be adopt, adapt or refuse\n", NR; bad = 1 }      if (s == "refuse") { refuse++; rn = $2; gsub(/^ +| +$/, "", rn) }    }  }}/^(Mechanic number|Why I refuse it|What would change my mind):/ {  v = $0; sub(/^[^:]*:/, "", v); gsub(/^ +| +$/, "", v)  if (v == "" || v == "TODO") { printf "line %d: empty answer\n", NR; bad = 1 }  if ($0 ~ /^Mechanic number/) mn = v  found++}END {  if (refuse != 1) { printf "need exactly one refuse row, found %d\n", refuse; bad = 1 }  if (found != 3) { print "refuse section incomplete"; bad = 1 }  if (mn != rn) { print "mechanic number does not match the refuse row"; bad = 1 }  if (rows < 11) { print "fewer than 11 mechanics rows"; bad = 1 }  if (!bad) print "OK"  exit bad}' factory/adopt/anthropic.md

Faded case

Skip to the open case if this is familiar. Anthropic is one of four published answers to the same question: who may approve a change an agent wrote? Here are the rows where the four differ, taken only from what each company published.

Sources, all observed 2026-10-28: Anthropic Claude Code team, Intercom, self-reported, Stripe, self-reported, GitHub docs. Lesson 18 of this module covers Stripe's blueprint and Lesson 25 covers Intercom's eligibility question in full.

Now decide. Your team has a payments lane and a docs lane. You track no revert rate for either and you have an agent reviewer that comments but cannot approve. Pick the one approval pattern from the table to copy this month, and say what you would need before copying a second.

Open case

Four situations follow, one per kind of team. These are this course's judgment, not Anthropic's advice.

Solo developer. Copy the base-branch reviewer file, the round cap and "prove it before the PR": all three work with no team. Skip tiers with more than one human slot, because there is no second person. First week: write the reviewer file, set the three-round cap, and refuse the no-human lane in writing until you have four weeks of your own revert history.

Small team. Copy tier routing by path and the rule that a bot never fills a human slot. Skip shadow-mode dashboards and daily eval scoring; a spreadsheet is enough. First week: list the paths that always need a person, put them in an ownership file, and run a script that labels each PR low, medium or high.

Enterprise. Copy the risk-tiered codebase, the sampled audit of automated approvals and the single-purpose agent identities, because you already need approval logs. Skip a company-wide low lane until each business unit shows its own revert numbers. First week: pick one repo, list its owners, and write its refuse row with a named approver.

Regulated shop. Copy the audit-first stance: every automated approval logged with reasons, agent actions routed to the security log. Skip the vendor review product if you run zero data retention, HIPAA workloads or customer-managed keys, since the docs say it is not available there. First week: have your auditor read the template and agree which rows may ever become the no-human lane.

Here is the open case. Your agent opens about thirty PRs a week. The repo has an ownership file covering the billing folder and the CI workflow folder, an agent reviewer that comments, no revert tracking and one engineer on call. A VP forwards Anthropic's post and writes: "Do what they do." You have one week. Which mechanics from the table go in your file as adopt, which one is refused, and what is the first test you write?

Where this breaks

Every number is self-reported and the scopes differ. Anthropic's pages quote more than 80% of merged code, about 90% for one repo and 65% for a product team using Claude Tag, and they use different per-engineer multipliers on different timescales (Anthropic Institute, self-reported, observed 2026-10-28). None has a denominator or a stated measurement method. The Claude Code team sells the tool it reviews with, so its "doubled merged PRs, 40% fewer reverts" claim is a vendor's own account with no published baseline counts. Quote the scope with the figure, or leave the figure out.

An independent test of Anthropic's auto mode, the classifier that approves an agent's own actions, found weak coverage. On 128 deliberately ambiguous prompts, 81.0% of risky scope-escalating actions slipped through end to end, and 36.8% of state-changing actions never reached the classifier because in-project file edits skip it (preprint, peer review not confirmed, observed 2026-10-28). The authors stress that this is a different workload from Anthropic's own production figures and an upper-bound stress test. Auto mode gates agent actions, not PR merges, and the sources do not say the merge flow relies on it. The lesson transfers anyway: a gate must cover every write path.

What would change the verdict: published revert counts with denominators per lane, the classifier's accuracy, the sample rate for audits and the exit criteria for shadow mode. Until then, treat the tier design as strong and the outcome claims as unproven. Do not cite Anthropic's C compiler experiment as gate evidence either: it was a research prototype with no human merge stage (Anthropic, observed 2026-10-28).

Key Takeaways

• Review is a routing problem before it is a reading problem.
• Set the tier from the diff and the paths it touches, never from the author.
• Read reviewer instructions from the base branch so no change rewrites its own judge.
• Cap agent-versus-agent rounds at three, then hand the change to a person.
• Refuse the no-human lane until revert tracking and a sampled audit exist.
• Turn each incident into an eval case the reviewer must pass.

What You Have Now

factory/adopt/anthropic.md - the mechanics you adopt from the Anthropic flow, mapped to your lanes, with the one you refuse and why. It sits in factory/adopt/ beside the review gate from Lesson 04, the protected paths from Lesson 06 and the risk lanes from Lesson 07, and it names a mechanic from each of them that you have chosen, adapted or turned down on purpose. Re-run the validator whenever a lane changes. This is the Prove stage for one company: you can now say which part of a published factory is evidence for your charter, which part you would copy, and which part you decline until your own numbers earn it.

Further Reading

• Claude Code team, "Solving code review at Anthropic" - the tiers, the three-round cap and incident-as-eval in the team's own words.
• Anthropic, "How Anthropic secures its AI-native software development lifecycle" - shadow mode, sampled approvals and single-purpose agent identities.
• Anthropic, Code Review documentation - what the reviewer reads, why it never approves and which organizations cannot use it.
• Anthropic, "Claude Code auto mode" - the approval-fatigue reasoning behind replacing click-through prompts with a classifier.
• Ji et al., independent stress test of auto mode - the preprint that measures what the classifier misses, for balance.
• Intercom, "AI is approving our pull requests" - the size-limited approval pattern this lesson compares against.

Exercises

1. Fill the template for one repo. Complete all eleven rows of factory/adopt/anthropic.md, mark exactly one row refuse, and run the validator until it prints OK.
2. Write the refuse row's unlock test. For the mechanic you refused, write the measurable condition that would bring it back: a revert window, a sample size and the name of the person who reads it.
3. Stress your own gate. List every way a change reaches your main branch, including file edits, scripts and a second agent. Mark the ones your required checks and approval rules do not cover.

Check your understanding