← All papers
Formalization of Algorithms for Optimization with Block Structures
math.OC
Mar 24, 2025 · v1
TL;DR
Formalizes convergence of block coordinate descent and ADMM for block-structured optimization in Lean4, with subdifferentials and the KL property.
Abstract
Block-structured problems are central to advances in numerical optimization and machine learning. This paper provides the formalization of convergence analysis for two pivotal algorithms in such settings: the block coordinate descent (BCD) method and the alternating direction method of multipliers (ADMM). Utilizing the type-theory-based proof assistant Lean4, we develop a rigorous framework to formally represent these algorithms. Essential concepts in nonsmooth and nonconvex optimization are formalized, notably subdifferentials, which extend the classical differentiability to handle nonsmooth scenarios, and the Kurdyka-Lojasiewicz (KL) property, which provides essential tools to analyze convergence in nonconvex settings. Such definitions and properties are crucial for the corresponding convergence analyses. We formalize the convergence proofs of these algorithms, demonstrating that our definitions and structures are coherent and robust. These formalizations lay a basis for analyzing the convergence of more general optimization algorithms.
Problem
Convergence analyses of block coordinate descent (BCD) and ADMM for nonsmooth, nonconvex block-structured problems had not been formally verified.
Approach
Using Lean4, the authors formalize subdifferentials and the Kurdyka-Lojasiewicz property, represent the algorithm update schemes via structure types, and prove convergence of proximal-linearized BCD and of ADMM.
Results
Formal proofs that BCD converges to critical points and ADMM converges to KKT points of the problem, providing a framework extensible to more general optimization algorithms.
