Fix, Not Just Resample: Bounded Compiler-Guided Repair in Lean 4
Haotian Deng, Chris Farber, Sonal Narasimha, Samskrithi Sivakumar, Yexiang Xue
cs.AI
Oct 18, 2026 · v1
TL;DR
Adds compiler-error-guided tactic repair to a Lean 4 neural proof search, evaluated on LeanDojo and Lean Workbook benchmarks.
Abstract
Automated theorem provers often rely on sampling language models, but proof assistants like Lean reject each failed step with a precise error message — information that pure sampling discards. To overcome this, we introduce a bounded, online step-level repair module. Our search samples $K$ candidate tactics at each step and repairs a failed tactic at most $R$ times. Instead of discarding failed tactics, we feed the compiler's error message to a pretrained repair model to fix individual rejected steps in place. This approach improves the solve rate over a sampling-only baseline across all tested candidate budgets ($K \in \{1,3,5,8\}$), achieving up to a $32\%$ relative gain and solving $82$ additional theorems on a $1{,}990$-theorem subset of the LeanDojo benchmark. Repair is not free: it trades extra runtime for reachable proofs. But the trade is predictable: repair pays most when sampling is constrained; its return is front-loaded, with the first repair attempt capturing $84\text{--}96\%$ of the total gain; and the gain comes almost entirely from shallow tactic-execution failures rather than structural errors such as type mismatches. This targeted repair successfully replicates on a second benchmark (Lean Workbook), nearly doubling the solve rate with zero regressions.
Problem
Neural provers in the LeanDojo/ReProver family discard tactics that Lean rejects and simply resample. This throws away Lean's precise error messages. Resampling also gives diminishing returns, so many provable theorems go unsolved.
Approach
A bounded, online step-level repair module is inserted into a priority-queue proof search. At each step the search samples K candidate tactics. Each failed tactic and its Lean compiler error are passed to a pretrained repair model, which fixes that step in place up to R times. Runs over K∈{1,3,5,8} and R∈{0..3} are compared on a common theorem set, with McNemar tests on paired outcomes.
Results
Repair beats the sampling-only baseline at every K on 1,990 LeanDojo theorems, by up to 4.1 Pass@1 points and up to 82 net theorems (32% relative gain). The first repair attempt captures 84–96% of the total gain, and the gains come mostly from shallow tactic-execution failures. On Lean Workbook, repair nearly doubles the solve rate with zero regressions.
| K | P@1 R=0 | P@1 R=3 | Net gain |
|---|
| 1 | 7.1 | 9.4 | +45 |
| 3 | 20.9 | 24.3 | +68 |
| 5 | 29.9 | 34.0 | +82 |
| 8 | 34.8 | 38.8 | +80 |
LeanDojo Pass@1 (%) on 1,990 theorems, baseline (R=0) vs. R=3 repair
| Split | R | P@1 | Fix | Brk |
|---|
| val (98) | 0 | 21.4% | – | – |
| val (98) | 2 | 42.9% | 21 | 0 |
| test (89) | 0 | 19.1% | – | – |
| test (89) | 2 | 31.5% | 11 | 0 |
Lean Workbook results