The main results on ridgelet reconstruction and the null space of neural-network synthesis are formalized in Lean 4, with a public blueprint.
Abstract
We study parameter nonuniqueness in continuous-width depth-two fully connected neural networks. Our main contribution is a direct method for solving the neural-network equation $S[γ]=f$. Starting from the Fourier expression of the synthesis operator, separation of variables produces a ridgelet particular solution and identifies every homogeneous direction. To isolate the argument, we first prove an abstract reconstruction formula for unitary factorizations, yielding the adjoint, normalized right inverse, and orthogonal solution geometry. We then specialize this formula to neural-network synthesis: for tempered-distribution activations such as ReLU, we equip the activation class $A_{s,t}$ with a Hilbert structure, construct compatible coefficient and parameter Hilbert spaces $H_{s,t}$ and $G_{s,t}$, and prove that $S:G_{s,t}\to L^2(\mathbb R^m)$ is bounded. The resulting ridgelet expansion exhausts the null space and the complete solution set and identifies the unique minimum-norm parameter distribution. Concrete examples give adjoint ridgelet functions for standard activations. Further developments show that finite-measure null elements admit normalized width-$N$ discretizations with $O(N^{-1/2})$ output error and characterize how additive parameter perturbations can reveal information encoded in the null space. A Lean 4 blueprint for the main results is available at https://shosonoda.github.io/lean-ridgelet/ .
Problem
Continuous-width depth-two neural networks S[γ]=f admit non-unique parameter distributions. The paper aims to characterize the full solution set, including the infinite-dimensional null space, for activations such as ReLU.
Approach
An abstract reconstruction theorem for unitary factorizations is proved first, giving the adjoint, a normalized right inverse, and the orthogonal solution geometry. This is then specialized to neural-network synthesis by building Hilbert spaces for activations, coefficients and parameters (A_{s,t}, H_{s,t}, G_{s,t}), proving that S is bounded, and applying ridgelet expansions. The main results are formalized in Lean 4, with a blueprint published online.
Results
The ridgelet expansion covers the entire null space and the complete solution set, and identifies the unique minimum-norm parameter distribution. Adjoint ridgelet functions are computed for ReLU, tanh, Gaussian CDF and Gaussian activations. Finite-measure null elements admit width-N discretizations with O(N^{-1/2}) output error, and numerical experiments show reconstructions that vanish for null ridgelet functions.