Nigel Goldenfeld – University of California San Diego
Wednesday, September 30, 2026
Zoom opens: 10:30AM EDT
Seminar begins: 10:45AM EDT
Emergence and Generalization in Machine Learning
The remarkable ability of modern neural networks to generalize improves with increasing network capacity, even when the number of model parameters or effective degrees of freedom exceeds the number of training data points. Here we use dynamical mean field theory to show, in a simple setting of linear regression, that this surprising behavior is the outcome of a phase transition in the stochastic field theory describing the training process. We calculate the critical exponents and scaling function of the double descent phase transition, and show that it is marked by a breakdown of the fluctuation-dissipation theorem associated with broken ergodicity. We describe how this phenomenon is an example of emergent behavior, and that the appropriate response function has the same functional form as the simple London model of the superconducting transition, with the rigidity of the wave function corresponding to the neural network's ability to generalize accurately. Our results are distinct from earlier work, because we calculate the time-dependence specifically, not just the least norm equilibrium solutions. This is what enables us to identify the origin of the emergent behavior. Our work provides a specific framework with which to understand emergent behavior in artificial neural networks, and I outline some future directions.
Work performed in collaboration with Chan Li.