← All papers
First page of A Fixed-Penalty Linearized Augmented Lagrangian Method with Classical Multiplier Updates

A Fixed-Penalty Linearized Augmented Lagrangian Method with Classical Multiplier Updates

Benqi Liu, Kangkang Deng, Zichen Wang, Zaiwen Wen

math.OC Aug 20, 2026 · v1
All theoretical convergence and complexity results for the proposed linearized augmented Lagrangian optimization method are formalized in Lean 4.
Augmented Lagrangian methods are effective for nonlinear equality-constrained optimization, but solving their nonlinear primal subproblems can be expensive. For smooth nonconvex problems with deterministic or stochastic objectives, we propose a nonlinear-residual linearized augmented Lagrangian method (NR-LALM) that replaces this subproblem by a regularized Gauss-Newton-type step while retaining the classical multiplier update based on the nonlinear constraint residual. The resulting step is computed from one symmetric positive-definite linear system, but the mismatch between the linearized primal model and the nonlinear-residual update produces a quadratic constraint-linearization error in the multiplier identity. We show that this error can be controlled under local regularity; multiplier boundedness and trajectory localization are derived rather than assumed. With fixed, accuracy-independent parameters, deterministic NR-LALM finds an $\varepsilon$-approximate Karush-Kuhn-Tucker (KKT) pair in $O(\varepsilon^{-2})$ iterations and first-order oracle evaluations. For stochastic objectives, a projected stochastic path-integrated differential estimator with safeguarded restarts requires, in expectation, $O(\varepsilon^{-3})$ stochastic-gradient evaluations and $O(\varepsilon^{-2})$ constraint and Jacobian evaluations. Compactness and a Kurdyka-Lojasiewicz condition further yield finite-length convergence of the deterministic primal-dual sequence. An optional minimum-norm second-order correction reduces the constraint-linearization error from second to fourth order without changing the complexity orders. All theoretical results are formalized in Lean 4. Numerical experiments confirm the predicted error orders and show favorable performance on high-dimensional deterministic and stochastic problems.

Augmented Lagrangian methods for nonlinear equality-constrained nonconvex optimization have expensive nonlinear primal subproblems. The question is whether a linearized variant with classical nonlinear-residual multiplier updates can retain complexity guarantees without assuming multiplier or trajectory bounds.

The authors propose NR-LALM, replacing the nonlinear subproblem with a regularized Gauss-Newton-type step computed from one SPD linear system while keeping the classical multiplier update. They analyze the quadratic constraint-linearization error under local regularity, deriving multiplier boundedness and trajectory localization. A stochastic variant uses a projected SPIDER estimator with safeguarded restarts, and an optional second-order correction reduces the error order. All theoretical results are formalized in Lean 4.

Deterministic NR-LALM finds an ε-KKT pair in O(ε^{-2}) iterations; the stochastic variant needs O(ε^{-3}) gradient and O(ε^{-2}) Jacobian evaluations. The SOC reduces constraint-linearization error from second to fourth order. Numerical experiments confirm the predicted error orders (log-log slopes 2.001 and 4.022).