← All papers
First page of Neural Theorem Proving for Verification Conditions: A Real-World Benchmark

Neural Theorem Proving for Verification Conditions: A Real-World Benchmark

Qiyuan Xu, Xiaokun Luan, Renxi Wang, Joshua Ong Jun Leang, Peixin Wang, Haonan Li, Wenda Li, Conrad Watt

cs.AI Jan 26, 2026 · v1
NTP4VC benchmark generates verification-condition proving tasks across Isabelle, Lean, and Rocq, evaluating LLMs and hammers on Lean among the three.
Theorem proving is fundamental to program verification, where the automated proof of Verification Conditions (VCs) remains a primary bottleneck. Real-world program verification frequently encounters hard VCs that existing Automated Theorem Provers (ATPs) cannot prove, leading to a critical need for extensive manual proofs that burden practical application. While Neural Theorem Proving (NTP) has achieved significant success in mathematical competitions, demonstrating the potential of machine learning approaches to formal reasoning, its application to program verification--particularly VC proving--remains largely unexplored. Despite existing work on annotation synthesis and verification-related theorem proving, no benchmark has specifically targeted this fundamental bottleneck: automated VC proving. This work introduces Neural Theorem Proving for Verification Conditions (NTP4VC), presenting the first real-world multi-language benchmark for this task. From real-world projects such as Linux and Contiki-OS kernel, our benchmark leverages industrial pipelines (Why3 and Frama-C) to generate semantically equivalent test cases across formal languages of Isabelle, Lean, and Rocq. We evaluate large language models (LLMs), both general-purpose and those fine-tuned for theorem proving, on NTP4VC. Results indicate that although LLMs show promise in VC proving, significant challenges remain for program verification, highlighting a large gap and opportunity for future research.

Neural theorem proving has shown success in mathematics competitions, but its application to proving verification conditions from real-world program verification remains unexplored, and no benchmark specifically targets this task.

NTP4VC introduces the first real-world multi-language benchmark for neural VC proving. From projects including the Linux kernel and Contiki-OS, industrial pipelines (Why3, Frama-C) generate semantically equivalent test cases across Isabelle, Lean, and Rocq. The benchmark covers algorithms, data structures, memory operations, and loops. General-purpose and theorem-proving LLMs are evaluated.

Automated theorem provers achieve a baseline pass rate of around 20-25% on the VC benchmark. LLMs show promise but leave a large gap, highlighting that program-verification VCs pose distinct challenges compared to competition mathematics. The benchmark comprises 600 VCs across multiple categories from real C verification projects.

Category# of VCsATP pass rate
Algorithm5520.00%
Data Structure7319.18%
Calculation6619.70%
Engineering5420.37%
Competition5219.23%
Function8123.46%
Loop8124.69%
Memory7424.32%
ATP and VC category breakdown