Constructive Universal Approximation by Infinite-Dimensional Neural Networks via Gaussian-Weighted Ridgelet Transform
Sho Sonoda, Luca Galimberti, Yuanchao Xu, Isao Ishikawa, Masahiro Ikeda
cs.LG
Oct 16, 2026 · v1
math.FA stat.ML
TL;DR
Every theorem, lemma, and example in the paper is formalized and machine-checked in Lean 4 with Mathlib, with a blueprint linking manuscript to Lean statements.
Abstract
Neural networks whose inputs are functions, as in operator learning, are usually shown to be universal by existence arguments, either by reduction to finite-dimensional networks or by Stone–Weierstrass-type density; these prove that approximants exist but say little about how the parameters depend on the target. We take a constructive route. We extend the ridgelet transform, an integral transform related to the Fourier, Radon, and wavelet transforms, to continuous maps on a separable infinite-dimensional Hilbert space, replacing the Lebesgue measure that is unavailable in infinite dimensions by Gaussian weights with a homogeneous mixture on the direction space. The resulting transform has a Plancherel identity and reconstruction formulas incorporating the frame operator, the correction induced by the weights, and it assigns to each target an explicit coefficient measure, one for every admissible analysis filter. From these representations we obtain uniform approximation on compact sets for all continuous non-polynomial activations of polynomial growth, including ReLU, and, by Monte Carlo sampling of the coefficient, finite-width networks with a dimension-free $O(N^{-1/2})$ error in the width $N$ for globally Lipschitz activations and coefficients with finite variation and second moments. Explicit examples include Gaussian targets and neural-operator layers. A companion Lean 4 formalization provides machine-checked proofs, with its correspondence to the manuscript documented at
https://shosonoda.github.io/lean-operator-ridgelet/ .
Problem
Universality of neural networks whose inputs are functions is usually proved by existence arguments. These arguments say little about how the network parameters depend on the target map on an infinite-dimensional Hilbert space.
Approach
The ridgelet transform is extended to continuous maps on a separable infinite-dimensional Hilbert space. Gaussian weights and a homogeneous mixture of Gaussian direction measures stand in for the Lebesgue measure, which is unavailable in infinite dimensions. The construction yields a Plancherel identity, reconstruction formulas with a frame-operator correction, and an explicit coefficient measure for each target. Monte Carlo sampling of that measure gives finite-width networks, and every result is machine-checked in Lean 4 with Mathlib.
Results
The paper proves uniform approximation on compact sets for all continuous non-polynomial activations of polynomial growth, including ReLU. For Lipschitz activations it gives a dimension-free O(N^{-1/2}) error rate in the width N, with numerical experiments consistent with that rate. All stated items are formalized in Lean; two clauses are carried there in a weaker form.