A Fixed-Penalty Linearized Augmented Lagrangian Method with Classical Multiplier Updates
Benqi Liu, Kangkang Deng, Zichen Wang, Zaiwen Wen
math.OC
Aug 20, 2026 · v1
TL;DR
All theoretical convergence and complexity results for the proposed linearized augmented Lagrangian optimization method are formalized in Lean 4.
Abstract
Augmented Lagrangian methods are effective for nonlinear equality-constrained optimization, but solving their nonlinear primal subproblems can be expensive. For smooth nonconvex problems with deterministic or stochastic objectives, we propose a nonlinear-residual linearized augmented Lagrangian method (NR-LALM) that replaces this subproblem by a regularized Gauss-Newton-type step while retaining the classical multiplier update based on the nonlinear constraint residual. The resulting step is computed from one symmetric positive-definite linear system, but the mismatch between the linearized primal model and the nonlinear-residual update produces a quadratic constraint-linearization error in the multiplier identity. We show that this error can be controlled under local regularity; multiplier boundedness and trajectory localization are derived rather than assumed. With fixed, accuracy-independent parameters, deterministic NR-LALM finds an $\varepsilon$-approximate Karush-Kuhn-Tucker (KKT) pair in $O(\varepsilon^{-2})$ iterations and first-order oracle evaluations. For stochastic objectives, a projected stochastic path-integrated differential estimator with safeguarded restarts requires, in expectation, $O(\varepsilon^{-3})$ stochastic-gradient evaluations and $O(\varepsilon^{-2})$ constraint and Jacobian evaluations. Compactness and a Kurdyka-Lojasiewicz condition further yield finite-length convergence of the deterministic primal-dual sequence. An optional minimum-norm second-order correction reduces the constraint-linearization error from second to fourth order without changing the complexity orders. All theoretical results are formalized in Lean 4. Numerical experiments confirm the predicted error orders and show favorable performance on high-dimensional deterministic and stochastic problems.
Problem
Augmented Lagrangian methods for nonlinear equality-constrained nonconvex optimization have expensive nonlinear primal subproblems. The question is whether a linearized variant with classical nonlinear-residual multiplier updates can retain complexity guarantees without assuming multiplier or trajectory bounds.
Approach
The authors propose NR-LALM, replacing the nonlinear subproblem with a regularized Gauss-Newton-type step computed from one SPD linear system while keeping the classical multiplier update. They analyze the quadratic constraint-linearization error under local regularity, deriving multiplier boundedness and trajectory localization. A stochastic variant uses a projected SPIDER estimator with safeguarded restarts, and an optional second-order correction reduces the error order. All theoretical results are formalized in Lean 4.
Results
Deterministic NR-LALM finds an ε-KKT pair in O(ε^{-2}) iterations; the stochastic variant needs O(ε^{-3}) gradient and O(ε^{-2}) Jacobian evaluations. The SOC reduces constraint-linearization error from second to fourth order. Numerical experiments confirm the predicted error orders (log-log slopes 2.001 and 4.022).