CausalForge: A Formally Grounded, Self-Improving Agentic Framework for Automated Research in Causal Inference
Jiyuan Tan, Vasilis Syrgkanis
stat.ML
Jul 24, 2026 · v1
cs.AI cs.LG econ.EM
TL;DR
Introduces Causalean, a Lean 4 library of 7,035 machine-checked causal-inference declarations, plus an agentic pipeline that formalizes and proves self-proposed results in Lean.
Abstract
Automating theoretical research is constrained not only by the generation of candidate results, but also by their reliable evaluation. A common approach is to close the research loop with a large language model (LLM) reviewer. However, such reviewers remain empirically unreliable: they may accept fabricated papers and detect them at rates close to chance (Bad Scientist, 2025). We present CausalForge, a framework for automated theoretical research in causal inference grounded in the Lean proof assistant. CausalForge combines Causalean, a foundational Lean library for causal inference containing 7,035 machine-checked declarations developed with language-model assistance under human design and review, with CausalSmith, a self-improving agentic pipeline that selects research topics, proposes results, formalizes statements, constructs proofs, and presents the resulting artifacts for human inspection. Because a machine-checked proof establishes only that a formal statement follows from its assumptions, not that the statement faithfully captures the intended scientific claim, the pipeline augments kernel verification with a statement audit that compares each formal theorem against the informal claim it is intended to express. We evaluate the system using artifacts produced by completed autonomous research runs. The source code, formal library, and run records are available at
https://github.com/Jiyuan-Tan/CausalForge.
Problem
Automating theoretical research is bottlenecked by unreliable evaluation, since LLM reviewers accept fabricated papers at high rates. Machine-checked proofs establish soundness but not that a formal statement faithfully captures the intended claim.
Approach
CausalForge combines Causalean, a foundational Lean 4 library of causal inference (7,035 declarations built with LLM assistance under human review), with CausalSmith, a self-improving agentic pipeline. The pipeline selects topics, proposes results, formalizes statements, constructs kernel-checked Lean proofs, and presents artifacts. It augments kernel verification with a statement audit comparing each formal theorem against its intended informal claim, and a library feedback loop promotes reusable lemmas.
Results
The run record holds 123 runs: 9 accepted (sound, novel, proved in Lean 4), 44 downgraded, and 70 failed. Accepted results span causal discovery, estimation, panel methods, and experimentation, while identification and SCM results appeared only in lower tiers.
| Disposition | Runs | Gate that determined it |
|---|
| Accepted | 9 | sound, novel at tier, proved in Lean 4 |
| Downgraded | 44 | sound, below novelty target |
| Failed | 70 | rejected at proposal or mathematics |
| Total | 123 | |
Run catalogue: 123 recorded CausalSmith runs by disposition.
| Cluster | Files | Lines | Defs | Thms/Lem |
|---|
| Estimation | 168 | 54,246 | 298 | 621 |
| Stat | 202 | 47,165 | 213 | 894 |
| PO | 138 | 41,415 | 592 | 947 |
| SCM | 88 | 36,342 | 208 | 523 |
| Total | 973 | 261,940 | 7,035 declarations | |
Library clusters and declaration counts.