Main contributions to the project:

• Alberto Arturo Vergani, Laurent U Perrinet (2021). Simulating anticipatory activity in a 1D Spiking Neural Network Model. Bernstein Conference 2021.

  Cite DOI

• Alberto Arturo Vergani, Laurent U Perrinet (2021). Neural Turing Patterns.

  Project

Project description: Visual computations using Spatio-temporal Diffusion Kernels and Traveling Waves

Biological vision is surprisingly efficient. To take advantage of this efficiency, Deep learning and convolutional neural networks (CNNs) have recently produced great advances in artificial computer vision. However, these algorithms now face multiple challenges: learned architectures are often not interpretable, disproportionally energy greedy, and often lack the integration of contextual information that seems optimized in biological vision and human perception. It is clear from recent advances in system and computational neuroscience that nonlinear, recurrent interactions in visual cortical networks are key to this efficiency (Tang et al., 2018; Kietzmann et al., 2019). We will use inspiration from neurophysiology and brain imaging to resolve this apparent gap between traditional CNNs and biological visual systems.

In this post-doctoral project, I will address these major limitations by focusing on specific dynamical features of cortical circuits: lateral diffusion of sensory-evoked traveling waves (Chavane et al., 2011; Muller et al., 2018) and dynamic neuronal association fields (Frégnac et al., 2012; Frégnac et al., 2016; Gerard-Mercier et al., 2016). Indeed, the architecture of primary visual cortex (V1), the direct target of the feedforward visual flow, contains dense local recurrent connectivity with sparse long-range connections (Voges and Perrinet, 2012). Such connections add to the traditional convolutional kernels representing feedforward and local recurrent amplification a novel lateral interaction kernel within a single layer (across positions and channels). Less studied, but probably decisive in active vision, recurrent cortico-cortical loops add a level of distributed top-down complexity which participates to the lateral integration of sensory input and perceptual context (Keller et al., 2019). Coupled with the continuous time dynamics of cortical circuits, this elaborate multiplexed architecture provides the conditions possible for generating information diffusion through traveling waves. Inspired by recent work in neuroscience uncovering the ubiquity of these waves during visual processing, we aim to design a self-supervised CNN that will exploit these dynamics for new applications in computer vision.

The proposed work will be organized as a collaboration between two labs (INT, Marseille and UNIC, Gif) along three tasks to be integrated in a unified model:

1. The starting point will be to extend results of self-supervised learning that we have obtained on static, natural images (Boutin et al., 2019) showing in a recurrent cortical-like artificial CNN architecture the emergence of interactions which phenomenologically correspond to the “association field” described at the psychophysical (Field et al., 1993), spiking (Li and Gilbert, 2002) and synaptic (Gerard-Mercier et al., 2016) levels.

2. The central aim will be to develop a dynamical version of this feedback/lateral kernel in the context of the ANR Horizontal-V1 project, linking the two labs and confronted to their recent electrophysiological data pointing to different classes of spatio-temporal diffusion and different degree of anisotropies during apparent and continuous motion.

3. The implementation of this kernel inspired by CNN theory will be compared with a biologically realistic models of the early visual system (Antolik et al., 2019), and simulations of the lateral diffusion kernel will be developed in collaboration with Jan Antolik, external collaborator to the ANR grant. In parallel, using tools linking neural activity to VSD imaging (Muller et al., 2014; Chemla et al., 2019), we will analyze at a more mesocopic level the role of observed traveling waves in forming efficient representations of the visual world.

Research context

This project is funded by the French National Research Agency (ANR) under the ANR Horizontal V1 grant (coordinator Y. Frégnac) which aims at understanding the emergence of sensory predictions linking local shape attributes (orientation, contour) to global indices of movement (direction, speed, trajectory) at the earliest stage of cortical processing (primary visual cortex, i.e. V1). The cross-talk between physiological and theoretical approaches is fostered by the close collaboration with the teams of Frederic Chavane at INT and Yves Frégnac at UNIC. The theoretical work is performed in close collaboration with Lyle Muller (Western U) and Jan Antolik (Prague). This project is primarily hosted at the Institut de Neurosciences de la Timone.

References

• Antolik, J, C Monier, Y Frégnac, AP Davison. (2019). “A comprehensive data-driven model of cat primary visual cortex.” BioRxiv, 416156.

• Boutin, Victor, Angelo Franciosini, Frederic Chavane, Franck Ruffier, and Laurent U Perrinet. (2019). “Sparse Deep Predictive Coding captures contour integration capabilities of the early visual system.” arXiv

• Chavane, F., C. Monier, V. Bringuier, P. Baudot, L. Borg-Graham, J. Lorenceau, and Y. Frégnac. 2000. “The Visual Cortical Association Field: A Gestalt Concept or a Psychophysiological Entity?” Frontiers in System Neuroscience 4(5): 1-26.

• Chavane, F., Sharon, D., Jancke, D., Marre, O., Frégnac, Y. and Grinvald, A. (2011). “Lateral spread of orientation selectivity in V1 is controlled by intracortical cooperativity.” Journal of Physiology Paris 94 (5-6): 333–42.

• Chemla, Sandrine, Alexandre Reynaud, Matteo diVolo, Yann Zerlaut, Laurent Perrinet, Alain Destexhe, and Frederic Chavane. (2018). “Suppressive Waves Disambiguate the Representation of Long-Range Apparent Motion in Awake Monkey V1.” Journal of Neuroscience 39 (22) 4282-4298.

• Field, D.J., Hayes, A. and Hess, R.F. (1993). “Contour integration by the human visual system: Evidence for a local “association field”.” Vision Research 33 (2), pp. 173-193.

• Frégnac, Y. (2012) “Reading out the synaptic echoes of low-level perception in V1.” European Conference in Computer Vision 486-495. Springer, Berlin, Heidelberg.

• Frégnac, Y., Fournier, J., Gerard-Mercier, F., Monier, C., Carelli, P., , M., Troncoso, X. (2016). “The Visual Brain: Computing Through Multiscale Complexity.” In Micro-, Meso- and Macro-Dynamics of the Brain pp 43-57.

• Gerard-Mercier, Florian, Pedro V Carelli, Marc Pananceau, Xoana G Troncoso, and Yves Frégnac. (2016). “Synaptic Correlates of Low-Level Perception in V1.” Journal of Neuroscience 36 (14): 3925–42.

• Keller, A., Roth, M.M. and Scanziani, M. (2019). 2019. “The feedback receptive field of neurons in the mammalian primary visual cortex.” American Society for Neuroscience Abstracts, 403.13. Chicago.

• Kietzmann, Tim C., Courtney J. Spoerer, Lynn K. A. Sörensen, Radoslaw M. Cichy, Olaf Hauk, and Nikolaus Kriegeskorte. (2019). “Recurrence Is Required to Capture the Representational Dynamics of the Human Visual System.” Proceedings of the National Academy of Sciences, October, 201905544.

• Li W, Piëch V, Gilbert CD (2006). “Contour saliency in primary visual cortex.” Neuron, 50(6):951–962.

• Muller, Lyle, Alexandre Reynaud, Frederic Chavane, and Alain Destexhe. (2014). “The Stimulus-Evoked Population Response in Visual Cortex of Awake Monkey Is a Propagating Wave.” Nature Communications 5: 3675.

• Muller, Lyle, Frederic Chavane, John Reynolds, and Terrence J Sejnowski. (2018). “Cortical Travelling Waves: Mechanisms and Computational Principles.” Nature Reviews Neuroscience 19 (5): 255.

• Tang, Hanlin, Martin Schrimpf, William Lotter, Charlotte Moerman, Ana Paredes, Josue Ortega Caro, Walter Hardesty, David Cox, and Gabriel Kreiman. (2018). “Recurrent computations for visual pattern completion.” Proceedings of the National Academy of Sciences 115 (35) 8835-8840.

• Voges, Nicole, and Laurent U Perrinet. (2012). “Complex Dynamics in Recurrent Cortical Networks Based on Spatially Realistic Connectivities.” Frontiers in Computational Neuroscience 6.