Machine Learning Seminar: Gong -- Shared principles of biological and artificial neural networks

The details about the machine learning seminar are as follows: 

Time: Mon 16 March (1:00 - 2:00pm): 

Location: SMRI Seminar Room (A12-03-301) A12 Macleay Building, Level 3, Room 301.  

Speaker: Pulin Gong (USYD) 

Title: Shared principles of biological and artificial neural networks 

Abstract: Biological neural networks in the brain and deep neural networks (DNNs) in AI
are both built from the interactions of large numbers of basic units (neurons), from
which powerful information-processing capabilities emerge.  In this presentation, I will
discuss several principles shared by these two classes of complex systems.  First, I
will show that heterogeneous synaptic weights, as observed in the brain and in
pretrained DNNs, give rise to a distinct dynamical regime in which neural
representations display a multiscale mixture of localized and delocalized features.
This suggests a common organizational principle underlying representation and
computation in both biological and artificial networks.  I will then turn to a second
shared principle: rich neural sampling dynamics.  In the brain, such dynamics support
flexible cognitive functions such as attention, while in deep learning they help
optimizers navigate complex loss landscapes and find solutions with good generalization
performance.