Agent versions from a SWE-bench run, the left panel of Figure 3. Panel reproduced unchanged. Image source. Licence.
The Darwin Gödel Machine can make changes to the software that turns a language model into a coding agent. The interesting choice was to keep a collection of earlier agents instead of building every new version from the current winner. A variant that fell behind could still provide the starting point for a later improvement.
The researchers used a foundation model to write changes to the surrounding software and ran programming benchmarks to test the resulting agents. Changes included different ways to edit code, manage long inputs and check work. The strongest versions raised the reported SWE-bench score from 20 percent to 50 percent.
The result measures performance on coding benchmarks in a controlled experiment. The model could revise parts of the system that directed it, while researchers set the environment and what counted as success. An archive helped the system explore alternatives, and the tests determined which improvements it could demonstrate.