Lesson 23. Cloudflare: Review as a Tiered Coordinator With Specialists

Cloudflare engineers merge code across thousands of repositories, and before any AI touched the process the median wait for a first human review was often measured in hours (Cloudflare, self-reported, observed 2026-10-28). The first automated attempt was the obvious one: paste the diff into a single prompt and ask for bugs. The comments came back vague, some named syntax errors that did not exist, and some told authors to add error handling to functions that already had it. A panel of specialist reviewers fixed the noise, and then created a different problem. Seven frontier-model sessions are a poor purchase for a one-line README fix, and the same rule that spares the typo cannot be allowed to spare a small change in an authentication directory. Copying the full MR context into every session would also have multiplied token cost by seven, by Cloudflare's own estimate. Decide now: with one budget per review and a stream that mixes typo fixes with security edits, how many reviewers does each change get, and what stops a cheap path from hiding a dangerous change?

This lesson gives you factory/adopt/cloudflare.md - risk tiers that decide how many reviewer agents run, with a cost ceiling. It serves the Prove stage of the course loop (Slice, Gate, Run, Wrap, Prove): a published case study, read for the mechanics you can copy, feeding the gate you already drafted in Lesson 4.

What is actually going on

Cloudflare publishes a review line, not a coding line. The flow below is taken from its two engineering posts, and every number in this lesson is self-reported by Cloudflare unless it sits in a labelled composite.

An engineer opens a merge request on GitLab. A CI component, added to a repository with a single include line, starts a coordinator agent built on OpenCode, an open-source coding agent (Cloudflare, self-reported, observed 2026-10-28). Before any model reads code, a break-glass check runs. If a human has commented the override phrase, approval is forced and the event is logged. Then a noise filter strips lockfiles, minified assets and source maps, and drops generated files, with one exception: database migrations stay in, because migration tools stamp schema changes as generated.

Next comes the tier. Lines changed, file count and security-sensitive paths set it, and anything under paths such as auth or crypto always goes to the full tier. Trivial runs two agents, lite runs four, full runs seven or more. The specialists cover code quality, security, performance, documentation, release, compliance with Cloudflare's engineering standards, and whether the repo's agent-instructions file is still accurate. Each works in its own session and reads only the patch files for its domain. They share one context file on disk instead of seven private copies.

Then the coordinator acts as judge. It dedupes, moves findings to the right section, drops speculative ones, and reads the source when unsure. A fixed rubric turns its verdict into a GitLab call: approve, approve with comments, revoke the bot's earlier approval, or request changes, which blocks the merge. The stated bias is toward approval, so one warning in an otherwise clean change still approves. On every new push an incremental re-review sees its own earlier findings and closes threads when the developer replies "won't fix".

The diagram traces one merge request from the first comment check to the verdict.

Every path ends in a re-review, not in a merge: the line decides how much review a change gets and leaves the merge itself to whatever the repository already requires.

The rule and its condition

Use tiers when your review stream mixes tiny and large changes and each review has a price; otherwise run one reviewer and skip the machinery. Set the tier from the diff and its paths, never from who wrote it, and let a protected path overrule the line count.

The condition is observable in your own data. If your review runs are not metered, or if nearly every change is large, a tier table saves nothing and one reviewer is simpler. If a meaningful share of changes are small, tiers are where the saving is.

What it does well: effort follows the diff, so cost follows risk. At the published tier averages, a trivial review cost about $0.20 and a full review about $1.68 (Cloudflare, self-reported, observed 2026-10-28). The break-glass hatch is itself measured, so overrides show up in the telemetry instead of in a side channel. A control plane, a small config service, can reroute models or switch off a provider within seconds, without a code change.

What it costs: seven concurrent model calls need circuit breakers, fallback chains and timeouts, and the post devotes whole sections to that plumbing. Cloudflare also says large merge requests are inherently expensive to review (Cloudflare, self-reported, observed 2026-10-28). The coordinator is the most capable and most expensive model in the stack, so the cheapest saving is running it on fewer changes.

Worked case

Six mechanics are worth lifting from Cloudflare, and each maps to a lesson you already hold.

1. The tier is a property of the diff (Lesson 7). Cloudflare sets it from lines, files and security paths. Its published tiers are two agents for 10 lines or fewer, four for up to 100, and seven or more above that, with more than 50 files or any security path forcing full (Cloudflare, self-reported, observed 2026-10-28). Lesson 7 decides when a human reads a change. Cloudflare's tier decides how many machines read it first. They are different axes, and a full tier does not mean a human must read.
2. A judge pass sits between findings and verdict (Lesson 4). The coordinator dedupes and drops speculation before anything reaches the author. Cloudflare reports about 1.2 findings per review, which it describes as deliberately low (Cloudflare, self-reported, observed 2026-10-28). Lesson 4 says to judge a reviewer by noise and resolution; this is the same idea expressed as a prompt design.
3. "What not to flag" is as important as "what to flag" (Lesson 4). Each specialist has an explicit exclusion list, such as theoretical risks and suggestions to swap libraries. Cloudflare says the exclusions carry most of the prompt value.
4. The escape hatch is measured (Lessons 7 and 12). Break glass was used 288 times, 0.6% of merge requests (Cloudflare, self-reported, observed 2026-10-28). A hatch that anyone can use unlogged is a hole; a logged one is a signal. Cloudflare's post lets a human reviewer use it, and the template below narrows that to code owners, which is this course's choice.
5. One choke point carries identity and cost (Lesson 12). All model calls go through one proxy and gateway, no API keys sit on laptops, and users are mapped to anonymous identifiers for cost tracking (Cloudflare, self-reported, observed 2026-10-28). Org-wide settings are pushed down at login with local overrides winning. The post does not say who can resolve an anonymous identifier back to a person, which Lesson 12 would want answered.
6. Context has owners (Lesson 11). Generated agent-instruction files for about 3,900 repositories were opened as merge requests for the owning teams to review, and a dedicated reviewer flags stale ones (Cloudflare, self-reported, observed 2026-10-28). Lesson 11 asks for one owner and one version per skill; this is the same discipline for context files. No source measured that it improves outcomes.

The worked case turns the first mechanic into a spend rule with a ceiling.

Illustrative composite. A 40-engineer lending platform on GitLab. About 300 review runs a week, re-reviews included. Modeled mix: 25% trivial, 30% lite, 45% full. Modeled costs per run borrowed from Cloudflare's published tier averages: $0.20, $0.67 and $1.68. Finance allows $1,500 a month for review. All figures are modeled, not measured.

Three designs are on the table. Design A: run the full panel on every change. Design B: tier by size alone, as in the simplest reading of the post. Design C: tier by size, add paths that always go to full, and put a cost ceiling on each tier. Decide now: which design, and what is the first thing that breaks in each one you reject?

Design A breaks at the invoice. Modeled on Cloudflare's published full-tier average of $1.68 per review (Cloudflare, self-reported, observed 2026-10-28), it costs $504 a week, about $2,180 a month, against a $1,500 allowance. The usual response is a blunt cut, such as skipping weekends or dropping repositories, so the budget decides where review stops instead of the risk. Design B breaks on the first small change in the rates module: six lines, trivial tier, one generalist, no security specialist. It is cheap and wrong in the same way, and the cause is that line count measures effort to read, not harm if wrong. Design C costs about $302 a week, roughly $1,310 a month, modeled, which leaves headroom for retries and outliers. A cheap tier is only safe if a path can overrule the line count.

The ceilings need a basis. Cloudflare's own 99th-percentile cost per review was $0.74 for trivial, $1.95 for lite and $5.05 for full (Cloudflare, self-reported, observed 2026-10-28). The template starts just above each, at $0.75, $2.00 and $5.50. Replace them with your own 99th percentile after two weeks of shadow runs. A ceiling that you copy from another company's logs is a guess.

This is the deliverable. It is a template: fill the header table for your repo, then run the validator.

The validator fails on any empty or placeholder cell. It fails on the unfilled template and passes once every cell has a value.

# Fails (exit 1) and prints the line number of any empty or <placeholder> table cell.awk -F'|' '/^\|/ && !/^\|[-| ]+\|$/ { for (i=2;i<NF;i++) { c=$i; gsub(/^[ \t]+|[ \t]+$/,"",c); if (c=="" || c ~ /<[^>]*>/) { printf "line %d: empty or placeholder cell\n", NR; bad=1; break } } } END { exit bad }' factory/adopt/cloudflare.md

The cost of the rejected designs is the month-end argument in A and the first missed security edit in B. The cost of C is an afternoon to write the path list, and the standing duty to read the break-glass log.

Faded case

Skip to the open case if this is familiar. Cloudflare's design is one of four published review lines in the research. The table lines up the ones that differ, using only facts from their own posts. Figures are self-reported unless noted, and the products and scopes differ, so the cost rows are not a ranking.

Sources: Cloudflare, Anthropic, Atlassian, Datadog via OpenAI.

Your team has the tier table from the template in draft. You may borrow one more mechanic first, and you can afford only one this month: Anthropic's rule that reviewer instructions come from the base branch, Atlassian's acceptance criteria from the linked ticket, or Datadog's replay of past incident changes through the reviewer. Decide which to adopt first and why.

Open case

These are this course's judgments, not Cloudflare's advice. Each says what to copy, what to skip and what to do in the first week.

Solo developer. Copy the tier idea at its smallest: one cheap reviewer for small changes, a second pass on paths you name as dangerous. Skip the seven-specialist panel and the control plane. First week: list three protected paths and add a monthly spend cap.

Small team. Copy the always-full path list and the break-glass log. Skip the model failback chains until a provider outage hurts. First week: run the reviewer in advisory mode on every merge request, log the tier it would have chosen, and read every break-glass use.

Enterprise. Copy the single proxy with anonymous cost attribution and the owner-reviewed context files. Skip nothing until you have your own 99th-percentile cost per tier. First week: name who can resolve a cost identifier to a person, because the posts do not.

Regulated shop. Copy the verdict mapping and add one row: a bot approval never fills a required human slot (Lesson 4). Skip bot-forced approval entirely and keep break glass for code owners. First week: write down which branch the reviewer's instructions come from.

Illustrative composite. A 9-person team in a regulated lender runs one AI reviewer on every change and wants to cut spend by 40%. All figures are modeled.

Their proposed rule: changes of 10 lines or fewer go to a cheaper two-agent review. Today a four-line change to the interest-calculation module is waiting. Write the first rule you would add to their table, before the quiz grades it.

Where this breaks

Everything above comes from Cloudflare describing itself. No outside party measured the reviewer, and the posts publish no false-negative rate, no precision against human reviewers and no defect-escape figure. The counts show the system is used and cheap, not that it catches what matters. The growth in weekly merge requests, from about 5,600 to over 8,700, was observed alongside adoption with no control group, so it cannot be credited to the reviewer (Cloudflare, self-reported, observed 2026-10-28).

The failure modes in the post are a useful checklist. The naive diff-in-a-prompt version produced vague and invented comments. Very large merge request descriptions overflowed the command-line argument limit until they were passed on standard input. Users cancelled slow jobs that looked hung until a 30-second "model is thinking" line was added. Seven concurrent calls forced circuit breakers because rate limits and provider outages were certain. Cloudflare also lists limits in its own words: weak on architecture, on cross-system impact and on timing-dependent concurrency bugs.

Three things would change the verdict. A published recall or escape rate would let you compare the reviewer with human review. A statement that bot approval satisfies a required approver slot would collide with Lesson 4, which keeps that slot human. And a size-based tier can be gamed by splitting one change into many small ones, which is why the template adds a split guard that Cloudflare does not describe.

Key Takeaways

• Tier every review from the diff and its paths, never from the author.
• A cheap tier is only safe if a path can overrule the line count.
• Write an exclusion list for every specialist, because noise teaches authors to ignore the reviewer.
• Log every break-glass use and read the log on a schedule.
• Set each tier's cost ceiling from your own 99th percentile, then abort and escalate at the ceiling.
• Treat every Cloudflare figure as self-reported, with recall and post-merge outcomes unpublished.

What You Have Now

factory/adopt/cloudflare.md - risk tiers that decide how many reviewer agents run, with a cost ceiling. Keep it next to the review gate from Lesson 4 and the risk lanes from Lesson 7, and add its protected-path list to the file from Lesson 6. It gives your reviewer a spend rule that follows risk, a ceiling that fails safe, and a logged escape hatch. It also names, in its tripwires, the evidence you must collect yourself: break-glass rate, ceiling hits and incidents traced to cheap tiers. The case study is Cloudflare's; the values are yours.

Further Reading

• Cloudflare, "Orchestrating AI Code Review at scale" - the primary source for the flow, the tiers and every figure used here.
• Cloudflare, "The internal AI engineering stack" - the proxy, gateway, context files and adoption numbers around the reviewer.
• Cloudflare, "Code Orange: Fail Small" - the resilience plan that the review post says it belongs to.
• Anthropic, Claude Code Review docs - a contrasting design with an advisory check and instructions read from the base branch.
• Atlassian, "How we cut PR cycle time with AI code reviews" - a first-reviewer flow that checks ticket acceptance criteria.
• OpenAI, Datadog case study - the incident-replay method, vendor-hosted, for validating a reviewer on your own history.

Exercises

1. Write your path list. Name five paths in your repo where a small change can do large harm, and add them to the always-full table of factory/adopt/cloudflare.md.
2. Fill and validate the template. Fill the header table for one real repo, run the validator until it prints nothing, then change one cell to empty and confirm it fails.
3. Replay your history. Take your last five reverted or incident-linked changes, assign each the tier the template would have given it, and record any that land in trivial or lite. For each, write the path rule that would have caught it.

Check your understanding