Loading...
Mathematical Connections between Convolutional Neural Networks and the Scattering Transformation
Hastings, Adam
Hastings, Adam
Citations
Altmetric:
Abstract
Convolutional neural networks (CNNs) are used in visual recognition tasks because they learn parameters, or weights, that are composed of hidden layers to form one classification in many settings. Standard CNN architectures are surprisingly stable, even though the stability is not ensured a priori. The goal of this paper is to consider a mathematically rigorous architecture that uses steps analogous to those in the neural network architecture. Stéphane Mallat, in his paper “Group Invariant Scattering,” provides one of the earliest attempts by creating the scattering transformation and its finite approximation, the windowed scattering. Importantly, it is invariant to diffeomorphic translation. We review and rederive many of his proofs and constructs, adding context and details where pertinent. We also briefly review a construction of wavelets and basic principles of measure theory, which are used in both the scattering and the windowed scattering. Next, we review extensions of this theory: representing the scattering transformation on a stochastic process, ensuring the scattering works over translations/rotations in a Lie Group, and creating the duality argument, which attempts to avoid the restrictions imposed by the scattering. Finally, we implement the scattering transformation on both the CIFAR-10 dataset and a publicly available dataset from TensorFlow (datasets.bee_dataset). Results include similar (though at times decreased) accuracy but improved stability compared to that of two Keras Conv2D and MaxPooling2D layers.
Description
Date
2025-05-31
Journal Title
Journal ISSN
Volume Title
Publisher
University of Kansas
Collections
Files
Research Projects
Organizational Units
Journal Issue
Keywords
Applied mathematics, Mathematics, Computer science, convolutional neural network, Fourier transform, harmonic analysis, scattering, signal processing, wavelets