← All papers
First page of Formalization of Algorithms for Optimization with Block Structures

Formalization of Algorithms for Optimization with Block Structures

Chenyi Li, Zichen Wang, Yifan Bai, Yunxi Duan, Yuqing Gao, Pengfei Hao, Zaiwen Wen

math.OC Mar 24, 2025 · v1
Formalizes convergence of block coordinate descent and ADMM for block-structured optimization in Lean4, with subdifferentials and the KL property.
Block-structured problems are central to advances in numerical optimization and machine learning. This paper provides the formalization of convergence analysis for two pivotal algorithms in such settings: the block coordinate descent (BCD) method and the alternating direction method of multipliers (ADMM). Utilizing the type-theory-based proof assistant Lean4, we develop a rigorous framework to formally represent these algorithms. Essential concepts in nonsmooth and nonconvex optimization are formalized, notably subdifferentials, which extend the classical differentiability to handle nonsmooth scenarios, and the Kurdyka-Lojasiewicz (KL) property, which provides essential tools to analyze convergence in nonconvex settings. Such definitions and properties are crucial for the corresponding convergence analyses. We formalize the convergence proofs of these algorithms, demonstrating that our definitions and structures are coherent and robust. These formalizations lay a basis for analyzing the convergence of more general optimization algorithms.

Convergence analyses of block coordinate descent (BCD) and ADMM for nonsmooth, nonconvex block-structured problems had not been formally verified.

Using Lean4, the authors formalize subdifferentials and the Kurdyka-Lojasiewicz property, represent the algorithm update schemes via structure types, and prove convergence of proximal-linearized BCD and of ADMM.

Formal proofs that BCD converges to critical points and ADMM converges to KKT points of the problem, providing a framework extensible to more general optimization algorithms.