← All papers
First page of LEANGUARD: Lean-Verified Guardrails for Tool-Using Agents

LEANGUARD: Lean-Verified Guardrails for Tool-Using Agents

Sreehari Sankar, Debargha Ganguly, Vikash Singh, Mohsen Hariri, Weicong Chen, Yanyan Zhang, Shouren Wang, Wang Yang, Chaoda Song, Alexander Nemecek, Osama Zafar, Idan Davidovich, Yuting Shao, Kai Ye, Biyao Zhang, Xinpeng Li, Joe Stubbs, Erman Ayday, Jing Ma, Vipin Chaudhary

cs.AI Nov 2, 2026 · v1 cs.CL
Implements a Lean 4 runtime that checks agent tool calls against Lean policies, proves the decision procedure equivalent to LeanLTL semantics, and uses metaprogramming for proof-guided repair.
Tool-using language-model agents often execute actions that their domain policies forbid. We present LeanGuard, a runtime that checks every proposed tool call against a policy written in Lean4 before the call reaches the environment. Policies combine state predicates with scoped past-time temporal operators over the execution history, evaluation is fail-closed, and the decision procedure is proved equivalent to a LeanLTL semantics. A denied call can be corrected in two ways. With feedback, the agent receives the violated rule and a hint; with proof-guided repair, Lean metaprogramming searches for a nearby call that preserves the user's intent and proves that the policy admits it, without the model. On Tau-Bench with five Qwen models (0.8B-27B), unguarded agents execute at least 33 calls that the Lean policy forbids, while LeanGuard executes none; with feedback, task success does not change significantly. On DTap, LeanGuard reduces direct prompt-injection success from 57.9% to 5.9%, and in telecom, where benign tasks need no writes, benign success is unchanged. Feedback reduces repeated blocked calls and raises task success for four of five models over a bare denial. On replayed denials, repair proves an admissible edit for 88 of 252 denials caused by argument values, in a median of 2,ms.

Tool-using language-model agents often execute actions that their domain policies forbid, especially state-changing calls. Prompt injection through tool outputs makes this worse.

LeanGuard is a Lean 4 runtime that sits between the agent and its tools and checks each proposed call against policies written in Lean. Policies combine state predicates with scoped past-time temporal operators over the execution history, and evaluation is fail-closed. The admission procedure is proved equivalent to a LeanLTL temporal semantics. A denied call is corrected in one of two ways: the agent receives feedback naming the violated rule plus a hint, or Lean metaprogramming searches for a nearby intent-preserving call and returns it with a kernel-checked proof that the policy admits it.

On tau^2-bench with five Qwen models (0.8B-27B), unguarded agents execute at least 33 policy-forbidden calls while LeanGuard executes none, and task success with feedback does not change significantly. On DTap, LeanGuard cuts direct prompt-injection success from 57.9% to 5.9%. In offline replay, proof-guided repair proves an admissible edit for 88 of 252 argument-value denials, with a median time of 2 ms.

MetricNoneLG+F
tau^2-bench task success (%)72.770.9
tau^2-bench judge-flagged (%)9.76.5
DTap direct attack success (%)57.95.9
DTap indirect attack success (%)22.33.8
Aggregate results across all models (None = unguarded, LG+F = LeanGuard with feedback)