h1: A Taste of the Bitter Lesson in Automated Theorem Proving
Skander Moalla, Ahmad Rammal, Niket Patel, Remi Munos, Julia Kempe
cs.LG
Oct 25, 2026 · v1
cs.LO
TL;DR
Builds h1, an agentic harness giving LLMs an efficient Lean 4/Mathlib interface, evaluated on Lean benchmarks and used to settle open FormalConjectures problems.
Abstract
While progress in informal mathematics has come largely from scaling the chain-of-thought and agentic reasoning of frontier language models, formal theorem proving has increasingly relied on human-designed workflows around these models, such as planning, decomposition, and orchestrating specialized provers. This raises a bitter-lesson question: can a simple agentic loop that scales test-time compute match these workflows? The answer matters as formal verification must scale to keep up with frontier models producing research mathematics, yet new workflows rarely settle it, as their gains are often confounded with stronger models or more compute. We study this question with h1, a simple harness that gives a model an efficient interface to Lean 4 and keeps it working through verifier feedback and self-reflection, without imposing a decomposition or search structure. h1 reaches state-of-the-art results on PutnamBench, FATE-X, and FormalConjectures100-Solved, settles eight open problems in FormalConjectures, and, at matched models, matches or outperforms specialized workflows such as Goedel-Architect and AlphaProof Nexus, surprisingly showing that a single attempt (pass@1) can often scale to solve open problems. Through ablations and matched-model comparisons with Claude Code and AxProverBase, we find that simplicity alone is not enough: an efficient Lean interface largely determines cost and solve rate. We thus recommend that future work evaluate frontier models with efficient execution feedback before prescribing workflows, at the risk of bitter lessons.
Problem
Formal theorem proving has come to rely on hand-designed workflows such as planning, decomposition, and orchestration of specialized provers. It is unclear whether a simple agentic loop that scales test-time compute can match these workflows at the same model and comparable cost.
Approach
h1 is a simple harness in which an LLM agent edits a persistent Lean 4 file that imports all of Mathlib. It gets compiler feedback and open goals after each edit, along with library search, scratch execution, and undo tools. Periodic self-reflection compacts the context so work can continue, and no decomposition or search structure is imposed. Final proofs are checked with SafeVerify in isolated sandboxes, and ablations vary tools, prompts, reflection frequency, and reasoning effort.
Results
h1 reaches state-of-the-art results with single runs per problem: 670/672 on PutnamBench (GPT-5.5), 90/100 on FATE-X and 78/100 on FC100-solved (GPT-5.6 Sol), and all six IMO 2026 problems. It also settles eight open FormalConjectures problems. At matched models it matches or beats Goedel-Architect and AlphaProof Nexus, and the efficiency of the Lean interface largely determines cost and solve rate.
| Benchmark | Model | Solved |
|---|
| FATE-X | GPT-5.6 Sol | 90/100 |
| FC100-solved | GPT-5.6 Sol | 78/100 |
| FormalQualBench | GPT-5.6 Sol | 14/23 |
| PutnamBench | GPT-5.5 | 670/672 |
| IMO 2026 | GPT-5.6 Sol | 6/6 |
Selected results with GPT-5.6 Sol / GPT-5.5 in h1 (single run per problem)