← All papers
First page of Hilbert: Recursively Building Formal Proofs with Informal Reasoning

Hilbert: Recursively Building Formal Proofs with Informal Reasoning

Sumanth Varambally, Thomas Voice, Yanchao Sun, Zhifeng Chen, Rose Yu, Ke Ye

cs.AI Sep 26, 2025 · v1
Presents Hilbert, an agentic framework that generates and verifies Lean 4 proofs via recursive decomposition and verifier feedback.
Large Language Models (LLMs) demonstrate impressive mathematical reasoning abilities, but their solutions frequently contain errors that cannot be automatically checked. Formal theorem proving systems such as Lean 4 offer automated verification with complete accuracy, motivating recent efforts to build specialized prover LLMs that generate verifiable proofs in formal languages. However, a significant gap remains: current prover LLMs solve substantially fewer problems than general-purpose LLMs operating in natural language. We introduce Hilbert, an agentic framework that bridges this gap by combining the complementary strengths of informal reasoning and formal verification. Our system orchestrates four components: an informal LLM that excels at mathematical reasoning, a specialized prover LLM optimized for Lean 4 tactics, a formal verifier, and a semantic theorem retriever. Given a problem that the prover is unable to solve, Hilbert employs recursive decomposition to split the problem into subgoals that it solves with the prover or reasoner LLM. It leverages verifier feedback to refine incorrect proofs as necessary. Experimental results demonstrate that Hilbert substantially outperforms existing approaches on key benchmarks, achieving 99.2\% on miniF2F, 6.6\% points above the best publicly available method. Hilbert achieves the \textbf{strongest known result} from a publicly available model on PutnamBench. It solves 462/660 problems (70.0\%), outperforming proprietary approaches like SeedProver (50.4\%) and achieving a 422\% improvement over the best publicly available baseline. Thus, Hilbert effectively narrows the gap between informal reasoning and formal proof generation. Code is available at https://github.com/Rose-STL-Lab/ml-hilbert.

Current prover LLMs specialized for formal languages solve substantially fewer problems than general-purpose LLMs operating in natural language, leaving a gap between informal mathematical reasoning ability and formal proof generation.

Hilbert is an agentic framework that orchestrates four components: an informal LLM that excels at mathematical reasoning, a specialized prover LLM optimized for Lean 4 tactics, a formal verifier, and a semantic theorem retriever. Given a problem the prover cannot solve directly, Hilbert uses recursive decomposition to split it into subgoals, solves them with the prover or reasoner LLM, and leverages verifier feedback to refine incorrect proofs.

Hilbert achieves 99.2% on miniF2F (6.6 percentage points above the best publicly available method) and solves 462/660 problems (70.0%) on PutnamBench, the strongest known result from a publicly available model.