KAPPA: Checking the Proofs in Statistics Papers with Lean 4 and Language Model Agents
Zihua She, Xiao Wang
stat.CO
Oct 26, 2026 · v1
cs.AI cs.LO
TL;DR
Language model agents translate statistics papers' statements and proofs into Lean 4 with Mathlib, which then checks each result along the paper's own route.
Abstract
Papers in statistical theory rest on long proofs that few readers check line by line. We present KAPPA (Kit of Agents for Proof-checking Papers with a proof Assistant), open-source software that organizes language model agents, run by Claude Code, to translate the proofs of a LaTeX paper into the proof assistant Lean 4 with its library Mathlib and check them there, result by result. Each numbered result receives one of four verdicts: verified unconditionally; verified conditionally, with every cited or unproved input stated as a named hypothesis; issue recorded, with its place in the paper and the side at fault; or unfinished, with the reason. The kit follows the paper's own proof step by step, rather than finding a new one. Separate agents write and review, each statement is compared with the paper before it is proved and fingerprinted afterwards, and every verdict rests on evidence that can be rerun without a language model. The kit runs unattended for hours, waits out usage limits, re-checks only what a revision changed and reports its cost by role, model and result. We describe the design, walk through a complete run, and apply the kit to four example papers with answer keys and to three published statistical papers, checked blind in their versions from before public corrections. In all three, the kit found the published error at the step that the correction names, together with further problems that we confirmed by hand. Given the same papers directly, the same language model working with Lean classified fewer results correctly on the longest paper and reported some statements narrower than the paper's as proved; no verdict of the kit claimed more than its Lean evidence showed.
Problem
Statistics papers rest on long appendix proofs that few readers check line by line, and published errors often sit in a single checkable step. Expert formalization takes weeks, and a language model's informal review is just another argument that someone has to check.
Approach
KAPPA is open-source software that organizes Claude Code agents to translate each numbered result of a LaTeX paper, and the paper's own proof, into Lean 4 with Mathlib. Separate agents draft, gate-review statements against the paper, prove, audit, and confirm issues, and each statement is fingerprinted after review. Each result receives one of four verdicts: verified unconditionally, verified conditionally with cited inputs as named hypotheses, issue recorded with a class and the side at fault, or unfinished. Every verdict rests on Lean evidence and mechanical audits that can be replayed offline without a language model.
Results
The kit reached the expected verdicts on four example papers with answer keys. On three published statistics papers, checked blind in their versions from before public corrections, it found the published error at the step the correction names, plus further problems confirmed by hand. Compared with the same model used directly as an agent or a chat, the kit classified all 23 results correctly and never claimed more than its Lean evidence showed.
| Paper (results) | Correctly classified | Faithful in Lean | Hidden or without Lean evidence |
|---|
| A (5) | 5 / 5 / 5 | 3 / 3 / 0 | 0 / 2 / 3 |
| B (12) | 12 / 9 / 10 | 10 / 5 / 0 | 0 / 4 / 8 |
| C (6) | 6 / 6 / 6 | 2 / 4 / 0 | 0 / 0 / 2 |
| Total (23) | 23 / 20 / 21 | 15 / 12 / 0 | 0 / 6 / 13 |
Kit / direct agent / direct chat on three published papers