← All papers
First page of A Combinatorial Identities Benchmark for Theorem Proving via Automated Theorem Generation

A Combinatorial Identities Benchmark for Theorem Proving via Automated Theorem Generation

Beibei Xiong, Hangyu Lv, Haojia Shan, Jianlin Wang, Zhengfeng Yang, Lihong Zhi

cs.AI Feb 25, 2025 · v1
Builds LeanComb, a Lean benchmark of combinatorial identities, plus a 260K-theorem Lean dataset generated by an automated theorem generator.
Large language models (LLMs) have significantly advanced formal theorem proving, yet the scarcity of high-quality training data constrains their capabilities in complex mathematical domains. Combinatorics, a cornerstone of mathematics, provides essential tools for analyzing discrete structures and solving optimization problems. However, its inherent complexity makes it particularly challenging for automated theorem proving (ATP) for combinatorial identities. To address this, we manually construct LeanComb, combinatorial identities benchmark in Lean, which is, to our knowledge, the first formalized theorem proving benchmark built for combinatorial identities. We develop an Automated Theorem Generator for Combinatorial Identities, ATG4CI, which combines candidate tactics suggested by a self-improving large language model with a Reinforcement Learning Tree Search approach for tactic prediction. By utilizing ATG4CI, we generate a LeanComb-Enhanced dataset comprising 260K combinatorial identities theorems, each with a complete formal proof in Lean, and experimental evaluations demonstrate that models trained on this dataset can generate more effective tactics, thereby improving success rates in automated theorem proving for combinatorial identities.

Automated theorem proving for combinatorial identities is limited by scarcity of high-quality formalized training data. No benchmark existed specifically for combinatorial identities in Lean.

The authors construct LeanComb, the first formalized benchmark for combinatorial identity theorem proving in Lean. They develop ATG4CI, an Automated Theorem Generator that combines candidate tactics from a self-improving LLM with Reinforcement Learning Tree Search for tactic prediction. The system generates theorems with complete formal proofs iteratively.

ATG4CI produces LeanComb-Enhanced, a dataset of 260K combinatorial identity theorems each with a complete Lean proof. Models trained on this dataset generate more effective tactics and improve success rates on combinatorial identity proving.

CategoryTrainingTest
Generalized Binomial Coefficient4216
Absorption Identity3512
Binomial Inversion466
Alternating Binomial Sums236
Inclusion-Exclusion275
LeanComb benchmark categories