Learning Lean Feedback for Efficient Proof-Candidate Evaluation on GPUs
Lazar Milikic, Etienne Bamas, Emmanuel Abbe, Viktor Kunčak
cs.LG
Oct 15, 2026 · v1
TL;DR
Trains neural models to predict Lean 4 verdicts and compiler feedback on 4.84M Lean-checked programs, combining them with Lean checks for GRPO rewards.
Abstract
Training language models for formal theorem proving requires large numbers of verifier calls. Slow checks and timeouts often leave GPUs waiting and withhold rewards from potentially correct proofs. We train neural evaluators to predict Lean 4 verdicts and compiler feedback on GPUs using 4.84M labelled programs. We compare two validity classifiers with a feedback model that annotates proof attempts with predicted goal states and error messages. A two-stage evaluator that screens candidates with a classifier and scores its positive predictions with the feedback model reaches 87.8% accuracy in predicting whether Lean accepts a candidate proof, evaluated on a 68,932-program benchmark collected from public provers. The combined evaluator processes 3.5 candidates per second on one H100, 4.2x the throughput of our 64-worker Lean baseline with environment reuse. Finally, we combine our evaluator with Lean verdicts to obtain a hybrid reward system that shortens GRPO training steps by approximately 31%. After one GRPO epoch with hybrid rewards, the prover achieves nearly the same miniF2F pass@1 as with Lean-only rewards (62.2% versus 62.4%) and a pass@32 of 72.4% versus 71.3%.
Problem
Reinforcement learning for Lean provers needs many verifier calls. Slow Lean checks and timeouts leave GPUs idle and withhold rewards from proofs that may be correct; in this setup Lean scoring took about 40% of each GRPO step.
Approach
A training corpus of 4.84M programs labelled by Lean v4.15.0 was built from NuminaMath-LEAN, Goedel-Prover, APRIL, and sampled proof attempts. Three evaluators were trained after continued pretraining on Lean code: a ModernBERT validity classifier, a Qwen3-8B one-token classifier, and a Qwen3-4B model that predicts goal states and error messages. A cascade passes candidates through the one-token classifier and then scores its positive predictions with the feedback model. A hybrid reward keeps the Lean verdicts that finish in time and uses cascade predictions for the remaining candidates.
Results
On a 68,932-program benchmark built from outputs of public provers, the cascade reaches 87.8% accuracy at 3.5 candidates/s on one H100, 4.2x the throughput of a 64-worker Lean pool. Hybrid rewards cut GRPO step time by about 31% with nearly unchanged miniF2F pass@1 (62.2% vs 62.4% for Lean-only) and better pass@32 (72.4% vs 71.3%).
| Reward source | Pass@1 (%) | Pass@32 (%) | Step time (s) |
|---|
| Initial checkpoint | 53.8±0.6 | 72.8±0.3 | — |
| Lean-only, 60 s timeout | 62.4±0.6 | 71.3±0.8 | 1,050 |
| Hybrid rewards | 62.2±0.8 | 72.4±0.4 | 720 |
| Lean-only, 13 s timeout | 59.8±0.6 | 69.7±1.1 | 720 |
GRPO training with different reward sources (miniF2F)