Sage: Formalization with Semantic Correction
Thomas Hirtz, Farzad Jafarrahmani, Abdelmouksit Sagueni, Xiang Zhou, Wenping Deng, Liang Zhang
cs.LG
Sep 16, 2026 · v2
cs.AI cs.CL cs.LO
TL;DR
Autoformalizes natural-language competition problems into Lean 4 statements with Mathlib, using Lean compiler diagnostics plus LLM semantic feedback in a correction loop.
Abstract
While neural theorem provers have achieved impressive milestones in formal mathematics, they largely operate on the assumption that faithful Lean 4 formal statements are already provided. Translating informal natural language into a formal language is a critical data bottleneck plagued by an "illusion of rigor": standard type-checkers accept statements that compile but drop hypotheses, introduce vacuous truths, or subtly alter mathematical bounds. To resolve this, we introduce Sage (Semantic Agent-Guided Formalization Engine), an agentic framework that replaces monolithic translation with a four-stage decomposed generation pipeline coupled with a dual-signal semantic correction loop. By pairing Lean 4 compiler diagnostics with multi-dimensional semantic feedback, our correction loop enforces mathematical fidelity alongside syntactic validity. By explicitly accounting for the gap between open-ended queries and declarative formal targets, our pipeline prevents models from achieving high formalization rates by guessing unverified answers (exhibiting a 70.9% answer leakage rate in monolithic baselines). Consequently, Sage suppresses leakage to 2.7% while achieving 73.3% pass@4 joint compilation and semantic fidelity on the Omni-MATH without proofs (compared to 42.0% for a fine-tuned Goedel-Formalizer-V2 baseline). Finally, on IMO-Unformalized, a novel frontier of 175 unformalized International Mathematical Olympiad problems, Sage demonstrates effective zero-shot generalization with 87.4% pass@4 verified fidelity compared to just 19.4% for the baseline, winning over 79% of blind pairwise evaluations.
Problem
Translating informal math problems into faithful Lean 4 statements is a bottleneck. Compiling statements can still drop hypotheses, be vacuous, or leak guessed answers into open-ended questions.
Approach
Sage is an agentic framework that decomposes formalization into four role-isolated LLM stages: distiller, preprocessor, formalizer, and formatter. It then applies a dual-signal correction loop that combines Lean 4 compiler diagnostics with a reference-free semantic rater. It distinguishes answer-aware from answer-agnostic regimes to prevent answer injection. It also introduces IMO-Unformalized, a set of 175 IMO problems with no existing Lean formalization.
Results
On answer-agnostic Omni-MATH, Sage reaches 73.3% pass@4 joint compile and semantic fidelity, versus 42.0% for Goedel-Formalizer-V2, and cuts answer leakage from 70.9% to 2.7%. On IMO-Unformalized it reaches 87.4% pass@4, versus 19.4% for the baseline.
| Method | pass@1 | pass@4 |
|---|
| Monolithic (Zero-Shot) | 20.0 | 33.1 |
| Goedel-Formalizer-V2 | 10.3 | 19.4 |
| Sage | 60.0 | 87.4 |
Formalization on IMO-Unformalized (N=175), Compile ∧ Goedel-SM