Harnessing Code Agents for Automatic Software Verification
Shuangxiang Kan, Shuanglong Kan, Sebastian Ertel
cs.FL
Jul 7, 2026 · v1
cs.AI cs.SE
TL;DR
A code-agent verification harness (Aria) proves 72 not-yet-ported lemmas in iris-lean, the in-progress Lean 4 port of Iris, checked by the Lean kernel.
Abstract
Formal verification offers the strongest guarantee of software correctness, but it does not scale: the proofs demanded by interactive theorem provers such as Coq require enormous expert effort. Large language models (LLMs) promise to generate these proofs automatically, yet existing approaches wire a fixed, human-designed proof strategy into the system and constrain the model to follow it (retrieving premises and predicting tactics one step at a time, or splitting goals by divide-and-conquer), and still prove only a fraction of their target theorems. We show that imposing such a strategy is unnecessary and limiting. Handing the whole lemma to a general LLM code agent (for example, Claude Code), free to choose its own approach, and wrapping it in a verification harness is both simpler and more effective, achieving full coverage: every targeted lemma proved, with no failures and no Coq expert intervention. The agent writes the proofs under feedback and hard constraints from the harness that keep each one sound (accepted only when the prover's kernel closes it), complete (no obligation left unproved or silently dropped), and terminating (no divergent tactics). We evaluate this harness plus code agent along three dimensions. (1) Core logic: on Iris, the state-of-the-art separation logic for concurrent and memory-manipulating programs, Aria proves all 4,257 lemmas of the four core modules and the 217 lemmas verifying Rust's standard libraries built on it, fully automatically. (2) Comparison with prior LLM provers: on reglang, where prior provers manage barely one in eight, Aria proves all 318. (3) Generality: on iris-lean, the unfinished Lean 4 port of Iris, it proves 72 not-yet-ported lemmas, showing the approach is not specific to Coq. A state-of-the-art model (Claude Opus 4.7) can write proofs for verified software development fully and automatically.
Problem
Formal verification proofs in interactive theorem provers require enormous expert effort and do not scale. Existing LLM-based provers hard-wire a fixed proof strategy and prove only a fraction of targeted theorems.
Approach
Aria hands a whole lemma to a general LLM code agent (Claude Opus 4.7) free to choose its own approach, wrapped in a verification harness. The harness enforces soundness (kernel-checked), completeness, and termination, providing feedback and retries. A declarative Harness Hook Language (HHL) governs allowed actions and required verification steps, with multi-agent roles (Extractor, Prover, Fixer, Polish). Proofs are accepted only when the prover's kernel closes them.
Results
On Iris (Coq) it proved all 4,257 core lemmas plus 217 Rust-library lemmas with no failures, and all 318 reglang lemmas where prior provers managed one in eight. On iris-lean, it proved 72 not-yet-ported Lean 4 lemmas, showing the approach is not specific to Coq.
| Metric | Value |
|---|
| Distinct lemmas proved | 4,257 |
| Source files covered | 113 |
| Failures | 0 |
| First-try success | 3,373 (79.2%) |
| Mean retries per lemma | 0.51 |
Iris core campaign results