Do Theorem Provers Need Theorem-Proving Agents? mini-Lean-agent: A Coding-Agent Baseline for Lean
Tomas Ortega, Simon Park, Ayush Khaitan, Liam Fowl, Alex Kontorovich, Sanjeev Arora
cs.AI
Oct 12, 2026 · v1
cs.LO
TL;DR
Builds a bash-only coding agent that searches Mathlib and invokes Lean 4 to prove Putnam 2025 problems, with a separate trusted Lean checker validating proofs.
Abstract
General-purpose coding agents use standard shell and file interactions, yet formal theorem-proving agents often rely on specialized, ad-hoc tooling. This makes it difficult to determine which components of theorem-proving agents provide benchmark gains: an improved base model, a specialized harness, a specific tool, or a combination of any of them. We introduce mini-Lean-agent, a deliberately minimal agent for Lean tasks whose only model-visible tool is bash. The model must use bash to inspect and edit the repository, search local libraries, invoke Lean, and manage its own intermediate work, while a separate trusted evaluator verifies the final artifact. We compare mini-Lean-agent against AxProverBase and Numina-Lean-Agent on the Putnam 2025 benchmark. Experiments with Gemini 3.8 Flash as the base model show that mini-Lean-agent solves 7/12 problems at \$2 per problem limit, compared with AxProverBase's 4/12. With Astra as the base model, mini-Lean-agent solves 12/12 problems at an average cost of \$1.28 per problem. This suggests that a generic coding agent with a minimal harness can already achieve strong performance on Lean benchmarks. Our results motivate using mini-Lean-agent as a standard baseline when evaluating theorem-proving infrastructure.
Problem
Theorem-proving agents for Lean often rely on specialized harnesses and tools. This makes it hard to tell whether benchmark gains come from the base model, the harness, or the tools.
Approach
mini-Lean-agent is built on mini-SWE-agent and exposes only bash to the model. Through bash the model inspects and edits files, greps Mathlib, runs `lake env lean`, and keeps scratch files. A separate trusted evaluator verifies the final proof. It is compared with AxProverBase (same model and budget) and with reported Numina-Lean-Agent results on the twelve Putnam 2025 problems, using Lean and Mathlib v4.28.0-rc1 in sandboxed Docker containers.
Results
With Gemini 3.8 Flash and a $2 per-problem limit, mini-Lean-agent verified 7/12 problems versus 4/12 for AxProverBase, at lower total charge. With the Astra model it solved 12/12 at an average of $1.28 per problem; Fable 5.1 solved 11/12 at much higher cost.
| Harness | $0.50 | $1.00 | $2.00 | Charge |
|---|
| mini-Lean-agent | 0/12 | 4/12 | 7/12 | $14.64 |
| AxProverBase | 0/12 | 0/12 | 4/12 | $20.97 |
Problems verified by budget with Gemini 3.8 Flash
| Model | Verified | Charge | Calls | Minutes |
|---|
| Astra | 12/12 | $15.37 | 106 | 51.3 |
| Fable 5.1 | 11/12 | $120.29 | 133 | 99.9 |
mini-Lean-agent with stronger models