# Coding agents missed more bugs when the answer lay beyond the changed lines

> A reviewer may need to know what a distant part of the program expects before it can spot a mistake in today's change.

Published 2026-11-08

Canonical: https://machines-on-machines.anth.us/posts/coding-agent-bug-review-misses/

A proposed code change can look right in isolation and still break a rule that another part of the project depends on. A 2026 preprint reports that review agents were less effective at finding bugs when the job required that kind of reasoning beyond the changed lines.

::image{
  src="assets/coding-agent-bug-review-misses/crbug-visibility.png"
  layout="inline"
  alt="The coding agents detected roughly 70 percent of bugs visible in the changed lines and roughly 48 percent of bugs that needed extra context."
  caption="The researchers compared 459 tasks whose defects were visible in the diff with 536 tasks that required extra context. The bars show detection rates, and the numbers beside them show task counts."
  credit="Chenyang Yang and coauthors, CRBugBench, figure 7. Cropped to the defect visibility panel. CC BY 4.0."
}

[Image source](https://arxiv.org/html/2610.13668v1#S4.F7) · [Reuse terms](https://creativecommons.org/licenses/by/4.0/)

The researchers assembled 1,000 review tasks from public projects and 800 from proprietary repositories at one company. Each task paired a known bug with the change that introduced it. The best tested setup caught 71.9 percent of bugs within its first five comments. Performance dropped on the private repositories and on tasks that needed more context.

The benchmark asks whether an agent can rediscover bugs people already know about. The missed cases explain why a strong overall score can be a poor reason to treat every change alike.

[Read the source](https://arxiv.org/abs/2610.13668).
<!-- papyrus-source-version: 3 -->