Modelling Complex-cells and topological structure in the visual cortex of mammals using Sparse Predictive Coding
Abstract
Cells in the primary visual cortex of mammals (V1) have historically been divided into two classes: simple and complex. Simple cells exhibit a rectified linear response to oriented visual stimuli while complex cells show various degrees of invariance with respect to the stimulus’ phase (position). The existence of these two populations can be explained by hierarchical models where simple cells feed information into complex cells through a non-linear spatial pooling [1]. Nevertheless, how the brain develops this structure remains an open question. One of the most successful theories to model hierarchical processing in the brain is Predictive Coding (PC): a framework introduced by Rao & Ballard [2] that exploits feedback and feedforward connectivity to solve a Bayesian inference problem. We extended the classical PC to account for a sparse representation of the input data (natural images) and a convolutional structure to allow translation invariance. We demonstrate that this framework, called Sparse Deep Predictive Coding (SDPC) [3], can easily replicate complex-like neurons when a non-linear pooling is included between the layers. In particular, we show that a large population of complex-like neurons, showing various degrees of phase invariance, emerges in the 2nd layer of the model when the pooling function is extended to include not only neighboring spatial locations but also neighboring neurons with different tuning properties. We trained various networks on natural images (STL-10 data-set). To quantify the complex behavior of the model neurons, we used the modulation ratio F1/F0[4]: if F1/F0 ≥ 1 the cell is identified as simple-like, if F1/F0<1 the cell is complex-like. In all the tested settings the 1st layer of the network exhibits simple-like neurons, while the 2nd layer always presents a fraction of complex-like cells. We observed the emergence of different behaviors by introducing different types on non-linear pooling. A striking emergent property of our model is that this non-linearities induce a topographical structure on the neurons of the network. This organization shows qualitatively strong similarities with that found in V1. Importantly, this organization is solely a consequence of the feedback connection coming from the 2nd layer: enforcing the pooling across neighboring neurons constrains neighboring simple-cells to encode for similar features. In particular, edge-like filters with similar orientations and frequencies, but different phases, tend to be grouped together. We were able to reproduce similar effects with two different types of pooling (l2-pooling, max-pooling) and different network sizes, obtaining different degrees of topographical organization and different ratios of complex-like cells. A possible explanation of this phenomenon lies in the computational bottleneck caused by a reduction in the network layers’ size. This is also in line with previous studies that showed how cortical networks with the same functional characterization can explain different behavior with different structural parameters [5]. The novelty of this model lies in its ability to highlight the link between structure and function in a neural network. This study addresses a long-debated question on the function and role of the diversity of topographical structure in the visual cortex of mammals across species. References: [1] Ko Sakai and Shigeru Tanaka. Spatial pooling in the second-order spatial structure of cortical complex cells. In: Vision Research 40.7 (2000), pp. 855–871. [2] Rajesh PN Rao and Dana H Ballard. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. In: Nature neuroscience2.1 (1999), p. 79. [3] Victor Boutin et al. Sparse Deep Predictive Coding captures contour integration capabilities of the early visual system. In: arXiv preprint arXiv:1902.07651 (2019). [4] Bernt C Skottun et al. Classifying simple and complex cells on the basis of response modulation. In: Vision research 31.7-8 (1991), pp. 1078–1086. [5] Jaeson Jang, Min Song, and Se-Bum Paik. Classification of columnar and salt-and-pepper organization in mammalian visual cortex. In: bioRxiv (2019), p. 698043
- see the follow-up paper in:(2022). Pooling in a predictive model of V1 explains functional and structural diversity across species. PLoS Computational Biology.
- see a follow-up in:
- more about the role of top-down connections:
This is from:
— Yann LeCun (@ylecun) April 21, 2021
Koray Kavukcuoglu, Marc'Aurelio Ranzato, Rob Fergus and Yann LeCun: Learning Invariant Features through Topographic Filter Maps, Proc. International Conference on Computer Vision and Pattern Recognition (CVPR'09), IEEE, 2009 pic.twitter.com/4gH6L3dmaJ
Laurent U Perrinet
Researcher in Computational Neuroscience
My research interests include Machine Learning and computational neuroscience applied to Vision.