Aligning Information Theory and Formal Verification: A Lean Formalization of Rate-Distortion Theory
Fubin Yan, Ruxuan Dong, Ruoxuan Shi, Duolei Wang, Kenneth W. Shum
cs.IT
Oct 14, 2026 · v1
TL;DR
Formalizes finite-alphabet rate–distortion theory in Lean 4 on top of Mathlib, proving both the converse and achievability parts of the rate–distortion theorem.
Abstract
Large language models have recently demonstrated an ability to generate substantial mathematical proofs and formalizations, including machine-checked arguments for major open problems and classical theorems. These developments highlight a growing workflow in which AI systems search for proofs while proof assistants certify their correctness. Within information theory, however, formalized libraries remain limited, and existing formalized theorems focus almost exclusively on Shannon's foundational lossless theorems. This paper presents the first Lean formalization of core components of rate-distortion theory, a central framework for lossy compression and modern machine-learning analysis. We identify the alignment challenges that arise when translating information-theoretic definitions, such as distortion measures, typicality and achievability, into a proof-assistant setting, and we develop a reusable library of definitions and lemmas faithful to textbook conventions. Our formalization provides a foundation for future AI-assisted proofs in information theory and enables subsequent mechanized developments to build on a coherent, verified infrastructure rather than generating long proofs from scratch.
Problem
Formalized libraries for information theory are limited and mostly cover Shannon's lossless theorems. Lean proofs produced in recent AI-assisted information-theory work are hard to reuse in other projects. Rate–distortion theory, the framework for lossy compression, had no Lean formalization.
Approach
The authors build a reusable Lean library on Mathlib's PMF, ENNReal and NNReal types, written to follow textbook conventions. It defines entropy, mutual information, distortion measures, test channels, block rate–distortion codes, expected distortion under i.i.d. sources, and asymptotic achievability. The paper discusses alignment issues in this translation, including zero-probability conventions, extended nonnegative reals, and feasibility constraints. It then explains how the formal proof of the theorem maps onto the textbook argument.
Results
The direct and converse parts of the finite-alphabet rate–distortion theorem are proved in Lean. The development also proves achievability at R = R_I(D) when the rate bound is required only in the limit superior. The source code is released as a foundation for future AI-assisted proofs in information theory.