Audit the Scaffold, Not the Checkpoint: A Stationarity Dichotomy for Recursive Self-Improvement in Agentic Coding
Sebastian Bobadilla-Suarez, Bob Suh, Ryan Fortin
cs.LG
Sep 28, 2026 · v2
cs.AI cs.SE
TL;DR
Theoretical propositions on refinement saturation, orchestration ceilings, and majority-vote bounds are machine-checked in Lean 4 with Mathlib, without sorry or added axioms.
Abstract
Rules for AI self-improvement often key on the model itself: has it been retrained, or are its weights frozen? We argue that, for AI coding agents, this check is incomplete, and we propose, but do not run, the experiment that would test it. An agent that keeps revising its own work runs into diminishing returns as long as the set of changes it can reach stays fixed, and can escape them only if that set grows. Our hypothesis is that the set can grow with the model untouched, because the software around the model, its scaffold of tools, tests and task breakdown, can be rewritten. If so, a frozen model guarantees a ceiling eventually but not steady behavior on the way there, and audits should track changes to the scaffold, not only to the model. The same criterion separates three things often lumped together: more search within a fixed setup, retraining the model during use (which can raise the ceiling), and rewriting the scaffold, which sits between them. Adding more agents does not lift the ceiling. Picking the best of several agents' answers reaches the best single agent's ceiling exactly, and only gets there faster. A majority vote could do better, but only if the agents fail on different tasks. In our tests, 30 agents from one model family failed together: some task defeated all of them, and a majority failed 23/55 (42%) of tasks. Our evidence is descriptive. On a public coding benchmark, the gain from each extra round of revision shrinks toward zero, though the unsolved tasks left to measure are also fewer and harder. Across 401 development sessions, the amount of code changed falls off from commit to commit, but programmers working before AI tools show the same pattern, so the data show that current practice levels off without showing why. We reach the criterion by treating repeated revision as a known machine-learning method, boosting, and note where real agents depart from it.
Problem
Rules for AI self-improvement often treat frozen model weights as evidence of bounded behavior. The paper asks when iterative self-modification by coding agents saturates, and whether a frozen checkpoint guarantees that it has.
Approach
Agentic code refinement is modeled as gradient boosting on residual errors, which yields a refinement game and a stationarity dichotomy: improvement saturates when the reachable set of edits is fixed and can run away only when that set expands. The algebraic results are machine-checked in Lean 4 with Mathlib. These include the saturation bound, the orchestration ceilings, and a majority-vote bound built on Mathlib's product measures. The theory is compared against SWE-bench Lite refinement experiments with Claude agents and against 401 production development sessions.
Results
Best-of-k orchestration reaches exactly the best single worker's ceiling, a fact confirmed after Lean produced a counterexample to a conjectured strict inequality. Thirty same-family agents failed in a correlated way, with a majority failing 23/55 (42%) of tasks. Per-round refinement gains shrink toward zero, but a pre-AI human baseline shows the same pattern, so the cause of saturation remains open.