Primary-Visual-Cortex | Laurent Perrinet

2026-03-05-ue-natural-cognition

Thu, 05 Mar 2026 00:00:00 +0000

Artificial neural networks and machine learning applied to the understanding of biological vision

[2026-03-05] Master 1 Neuroscience, UE Natural Cognition, Artificial Cognition

Contact me @ laurent.perrinet@univ-amu.fr

Principles of Vision

[Marr, 1982]

What is the function of vision?

An Unexpected Visitor (Ilya Repin, 1884)

What is the function of vision?

An Unexpected Visitor (Yarbus, 1965)

What is the function of vision?

An Unexpected Visitor - Age? (Yarbus, 1965)

What is the function of vision?

An Unexpected Visitor - How long? (Yarbus, 1965)

Visual illusions

Ilusions of brightness or lightness Akiyoshi KITAOKA

Visual illusions : Pareidolia

Cydonia Mensae (1976) Viking Orbiter image

Visual illusions : Pareidolia

Cydonia Mensae (2007) Mars Global Surveyor

Visual illusions : Pareidolia

Cydonia Mensae (2007) Mars Global Surveyor

Principles of vision?

[Marr, 1982]

Computational neuroscience of vision

[Marr, 1982]

Computational neuroscience of vision

[Sejnowski, Koch & Churchland (1998)]

Anatomy of the Human Visual system

Human Visual system : the HMAX model

[Serre and Poggio, 2007]

Primary visual cortex

[Hubel & Wiesel, 1962]

Primary visual cortex

[Hubel & Wiesel, 1962]

Hybrid IA models

Using goal-driven deep learning models to understand sensory cortex [Yamins & DiCarlo, 2016]

Principles of vision?

[Marr, 1982]

Convolutional Neural Nets (CNN)

[Marr, 1982]

Convolutional Neural Nets (CNN)

[Boutin et al, 2021]

Convolutional Neural Nets (CNN)

[Jérémie & LP, 2023]

CNN: Mathematics

One-dimensional discrete convolution (eg in time) with a kernel $g$ of radius $K$: $$ (f \ast g)[n]=\sum_{m=-K}^{K} f[n-m] \cdot g[m] $$

CNN: Mathematics

Convolution of an image (two-dimensional) with a kernel $g$ of radius $K\times K$:

$$ (f \ast g)[x, y] = \sum_{i=-K}^{K} \sum_{j=-K}^{K} f[x-i, y-j] \cdot g[i, j] $$

CNN: Mathematics

Cross-correlation of an image (two-dimensional) with a kernel $g$ of radius $K\times K$:

$$ (f \ast \tilde{g})[x, y] = \sum_{i=-K}^{K} \sum_{j=-K}^{K} f[x+i, y+j] \cdot g[i, j] $$

CNN: Mathematics

[Amidi & Amidi]

CNN: Mathematics

Correlation of an image defined on several channels (note the order of the indices):

$$ (f \ast \tilde{g})[x, y] = \sum_{c=1}^{C} \sum_{c,i,j} f[c, x+i, y+j] \cdot g[c, i, j] $$

CNN: Mathematics

Correlation of a multi-channel image for multiple output channels (note the order of the indices):

$$ (f \ast \tilde{g})[k, x, y] = \sum_{c,i,j} f[c, x+i, y+j] \cdot g[k, c, i, j] $$

CNN: the HMAX model

[Serre and Poggio, 2006]

CNN: challenges

[Boutin et al, 2021]

Principles of Vision

[Marr, 1982]

CNN: Predictive processing

[Boutin et al, 2021]

CNN: Predictive processing

[Boutin et al, 2021]

CNN: Topography

[Bosking et al, 1997]

CNN: Topography

[Boutin et al, 2022]

Artificial neural networks applied to the understanding of biological vision

[Marr, 1982]

Artificial neural networks and machine learning applied to the understanding of biological vision

[2026-03-05] Master 1 Neuroscience, UE Natural Cognition, Artificial Cognition

Contact me @ laurent.perrinet@univ-amu.fr

Foveated Retinotopy Improves Classification and Localization in CNNs

Mon, 23 Feb 2026 00:00:00 +0000

Foveated Retinotopy in CNNs. We represent Left an input image and how it is transformed by foveated retinotopy. We show below a representative reconstruction showing that it also acts as a cortical zoom on the image around the point of fixation. The transformed image is then fed to the ResNet deep learning architecture.

From falcons spotting prey to humans recognizing faces, the ability to rapidly process visual information depends on a foveated retinal organization that provides high-acuity central vision while preserving low-resolution peripheral vision. This organization is conserved along early visual pathways, yet remains under-explored in machine learning. Here, we examine the impact of embedding a foveated retinotopic transformation as a preprocessing layer on convolutional neural networks (CNNs) for image classification. By applying a log-polar mapping to off-the-shelf models and retraining them, we achieve comparable accuracy while improving robustness to scale and rotation. We demonstrate that this architecture is highly sensitive to shifts in the fixation point and that this sensitivity provides an effective proxy for defining saliency maps that facilitate object localization. Our results demonstrate that foveated retinotopy encodes prior geometric knowledge, providing a solution for visual searches and a meaningful classification robustness and localization trade-off. These findings provides a proof of concept in order to connect principles of biological vision with artificial networks, suggesting new, robust and efficient approaches for computer vision systems.

Foveated Retinotopy simulated by a log-polar map. We represent Left an input image with some geometrical objects and how it is transformed by the log-polar representation that implements foveated retinotopy. This shows that a rotation amounts to a translation on the polar axis (abscissa) and a zoom to a translation on the ordinates. We show right a representative reconstructionshowing that it also acts as a cortical zoom on the image around the point of fixation.

Ede Rancz

Sat, 03 Jan 2026 09:00:00 +0000

Ede Rancz is a Research Director at the Mediterranean Institute of Neurobiology, Marseille, France. He is interested in the neural circuits underlying visual perception and decision-making. He uses a combination of in vivo and in vitro electrophysiology, optogenetics, and computational modeling to study the function of the visual cortex. He is also interested in the development of new tools and methods for studying neural circuits.

*CENTURI call “Neuromodulatory control of predictive processing in visual cortical circuits” (co-direction with Laurent Perrinet) (POSITION HAS BEEN FILLED)

Using a cloded-loop virtual reality setup, can we trace the role of neuromodulators in active perception in terms of predictive processing?

Thesis director of: Hilde Langengen-Teigen, Mediterranean Institute of Neurobiology, Marseille. This PhD position was made possible thanks to a 3-year contract from AIX-MARSEILLE University awarded by the Turing Centre for Living Systems PhD call (CENTURI).

Population decoding of visual motion direction

Thu, 01 Jan 2026 00:00:00 +0000

🧠 Excited to share our latest research led by Alexandre Lainé and presented this summer at AREADNE 2026!

“Population decoding of visual motion direction”

Our work explores how populations of neurons in the primary visual cortex (V1) of marmoset monkeys encode visual motion direction, with a particular focus on understanding how uncertainty influences this neural decoding process. Key highlights:

Advanced population-level analysis of V1 neural responses to motion stimuli
Novel insights into how the brain handles uncertainty in visual motion processing
Marmoset model providing crucial translational insights for visual neuroscience

This research contributes to our fundamental understanding of how the visual system processes motion information at the earliest stages of cortical processing, with important implications for both basic neuroscience and potential clinical applications. Thank you to the AREADNE organizing committee for hosting such an inspiring conference!

Link to publication: https://laurentperrinet.github.io/publication/laine-26-areadne/

For deeper insights into uncertainty processing mechanisms in the visual cortex, see our Nature Communications Biology study:
Hugo Ladret, Nelson Cortes, Lamyae Ikan, Frédéric Chavane, Christian Casanova, Laurent U Perrinet (2023). Cortical recurrence supports resilience to sensory variance in the primary visual cortex. Nature Communications Biology.
PDF Cite DOI Dataset Code URL bioRxiv HAL

A New Look for Convolutional Deep Networks

Fri, 12 Dec 2025 00:00:00 +0000

🔬 Excited to present our latest research at the #MAIN2025 conference today!

🔗 https://www.main2025.org/

👁️ What if CNNs could see like humans? Our new work shows how foveated vision—concentrating processing at gaze center—makes networks more robust to perturbations & great at localization. Inspired by human vision’s architecture (high-resolution foveal center, low-resolution periphery), we embedded this retinotopic transformation into CNN architectures, allowing to actively scan the image. This gives literally a new look to #ConvNets !

📄 Paper:

Jean-Nicolas Jérémie, Emmanuel Daucé, Laurent U Perrinet (2026). Foveated Retinotopy Improves Classification and Localization in CNNs. Vision.

PDF Cite DOI URL arXiv

#DeepLearning #ComputerVision #AI #Research #NeuralNetworks #NeuroAI #OpenScience I love #Montreal

Media storm - share if you like :

Hugo Ladret

Fri, 29 Aug 2025 00:00:00 +0000

PhD Student (2019-09 / 2024-02): A multiscale cortical model to account for orientation selectivity in natural-like stimulations

Aix-Marseille Université, Institut des Neurosciences de la Timone
Université de Montréal, Laboratoire des Neurosciences de la Vision

Hugo Ladret focuses on predictive coding, an influential brain theory that promises to account for the many seemingly disparate results neuroscientists have gathered over decades of experiments. Using neurobiology with a theory-driven approach, his experimental work deals about vision, and to find theoretical insights for neural network modelling.

Relevant publications

Hugo Ladret (2024). Modélisation multi-échelle de la sélectivité à l'orientation dans les stimulations visuelles naturelles.

Cite URL

Hugo Ladret, Laurent U Perrinet (2023). Convolutional Sparse Coding is improved by heterogeneous uncertainty modeling. ICLR 2023 SNN Workshop.

PDF Cite URL

Hugo Ladret, Nelson Cortes, Lamyae Ikan, Frédéric Chavane, Christian Casanova, Laurent U Perrinet (2023). Cortical recurrence supports resilience to sensory variance in the primary visual cortex. Nature Communications Biology.

PDF Cite DOI Dataset Code URL bioRxiv HAL

Laurent U Perrinet, Hugo Ladret (2024). Le mystère de la Joconde éclairé par les neurosciences. Cerveau et Psycho.

PDF Cite DOI URL

previous experience

master 2B (undergrad, 2019-01-12 / 2019-05-24)

Université de Montréal, Laboratoire des Neurosciences de la Vision

master 2A (undergrad, 2018-09-10 / 2019-01-11) : Learning temporal integrations in a visual spiking neural network

Building upon our previous work, we are investigating how recurrent neural networks learn to integrate temporal information, a dimension which is absent in most deep learning networks but provides a wealth of information in biological neural networks.

To be able to generalize our findings, I created a model of the early visual pathway (retina and thalamus) that generates neural activity from any natural image, based on data gathered in biological systems for the past several decades. The output from this early visual pathway is then processed by a recurrent spiking neural network whose dynamics match that of the primary visual cortex.

We showed that Spike Timing Dependant Plasticity (STDP) and recurrence are key components that allow spiking neural networks to extract patterns from noisy input and build strong internal representations. Such representations not only correctlt predict spatial informations (for example the organization of a visual scene) but also predict temporal structure underlying such informations.

source code : https://github.com/hugoladret/InternshipM2

Neuroscience Specialist for Artistic Creation (2018-07 / 2018-09)

I developed computational neuroscience and computational physics models, in collaboration with well-known contemporary artist Etienne Rey at Friche la Belle de Mai (Marseille) and AI researcher Laurent Perrinet. The idea behind our project was to create works of art by distributing particles in a constrained, semi-stable space, thereby creating discrete illusory perceptions.

To dive into more technical details, my work included the implementation of a Boltzmann lattice for computational fluid dynamics (D2Q9 structure), as well as various electro-magnetic interaction models. On the neuroscience side, I used Deep Convoluted Generative Adverserial Networks (DCGAN), Kohonen maps and Canny edge detectors to generate triangulated graphs with a hidden underlying structure. In order to facilitate collaboration between the three of us, I also developed a GUI and multi-threading support that allowed us to work efficiently and use at best each our respective skill set.

source code : https://github.com/NaturalPatterns/

master 1 (undergrad, 2018-04 / 2018-06): Orientation selectivity in a ring model of the primary visual cortex

I created a ring model that performs orientation discrimination tasks, using an hybrid model of convolutionnal and recurrent networks. This work was, to our knowledge, the first visual ring model based on deep learning techniques.

The recurrence in the network plays a role akin to that of lateral interactions within the primary visual cortex. We have shown in this work that these lateral interactions provide robustness to noisy inputs in the model, which we infer to also be the the case in the brain. To verify this assessment, I designed a 2-outcome discriminative psychophysics task (2AFC) and compared various metrics for human and model trials. The results showed that the lateral interactions allowed human-like performance, which is a strong qualitative argument in favor of the biological plausiblity of this model.

all material is available @ https://github.com/hugoladret/InternshipM1

2025-05-26-master-m-4-nc

Mon, 26 May 2025 00:00:00 +0000

Artificial neural networks and machine learning applied to the understanding of biological vision

Laurent Perrinet

[2025-05-26] Master M4NC de l’institut NeuroMod, cours Prospective Innovation and Research

Contact me @ laurent.perrinet@univ-amu.fr

Principles of Vision

[Marr, 1982]

What is the function of vision?

An Unexpected Visitor (Ilya Repin, 1884)

What is the function of vision?

An Unexpected Visitor (Yarbus, 1965)

What is the function of vision?

An Unexpected Visitor - Age? (Yarbus, 1965)

What is the function of vision?

An Unexpected Visitor - How long? (Yarbus, 1965)

Visual illusions

Rotating Snakes Akiyoshi KITAOKA

Visual illusions

Ilusions of brightness or lightness Akiyoshi KITAOKA

Visual illusions

Hering illusion

Visual illusions

Hering illusion

Visual illusions

Rotating Snakes Akiyoshi KITAOKA

Visual illusions : Pareidolia

Cydonia Mensae (1976) Viking Orbiter image

Visual illusions : Pareidolia

Cydonia Mensae (2007) Mars Global Surveyor

Visual illusions : Pareidolia

Cydonia Mensae (2007) Mars Global Surveyor

Principles of vision?

[Marr, 1982]

Computational neuroscience of vision

[Marr, 1982]

Computational neuroscience of vision

[Sejnowski, Koch & Churchland (1998)]

Anatomy of the Human Visual system

Human Visual system : the HMAX model

[Serre and Poggio, 2007]

Primary visual cortex

[Hubel & Wiesel, 1962]

Primary visual cortex

[Hubel & Wiesel, 1962]

Hybrid IA models

Using goal-driven deep learning models to understand sensory cortex [Yamins & DiCarlo, 2016]

Convolutional Neural Nets (CNN)

[Boutin et al, 2021]

Convolutional Neural Nets (CNN)

[Jérémie & LP, 2023]

CNN: challenges

[Boutin et al, 2021]

CNN: Predictive processing

[Boutin et al, 2021]

CNN: Predictive processing

[Boutin et al, 2021]

CNN: Topography

[Bosking et al, 1997]

CNN: Topography

[Boutin et al, 2022]

Computational neuroscience of vision

[Marr, 1982]

Spiking Neural Networks (SNN)

[Marr, 1982]

SNN: Leaky Integrate-and-Fire Neuron

Review on Precise Spiking Motifs.

SNN in neurobiology

[Mainen & Sejnowski, 1995]

SNN in neurobiology

[Mainen & Sejnowski, 1995]

SNN in neurobiology

[Diesmann et al. 1999]

SNN in neurobiology

[Haimerl et al, 2019]

SNN: Spiking motifs

Review on Precise Spiking Motifs.

SNN: Spiking motifs

Review on Precise Spiking Motifs.

SNN: Spiking motifs

Review on Precise Spiking Motifs.

SNN in neuromorphic engineering

From frame-based to event-based cameras.

SNN in neuromorphic engineering

The HD-SNN neural network.

SNN in neuromorphic engineering

The HD-SNN neural network.

SNN in neuromorphic engineering

The HD-SNN neural network.

Spiking Neural Networks (SNN)

[Marr, 1982]

Artificial neural networks applied to the understanding of biological vision

[Marr, 1982]

Artificial neural networks and machine learning applied to the understanding of biological vision

Laurent Perrinet

[2025-05-26] Master M4NC de l’institut NeuroMod, cours Prospective Innovation and Research

Contact me @ laurent.perrinet@univ-amu.fr

2025-03-11-phd-program-sparse-representations

Tue, 11 Mar 2025 00:00:00 +0000

Sparse representations

Laurent Perrinet

NeuroSchool PhD Program in Neuroscience

[2025-03-11]

Code / Contact me @ laurent.perrinet@univ-amu.fr

Sparse representations?

Paysage catalan (Le Chasseur)

to rephrase the expression “The Unreasonable Effectiveness of Mathematics” by Wigner, the “Unreasonable efficiency of vision” is playfully illustrated in this painting from Joan Miró, which allows us to depict this Catalan landscape with the a few strokes where our imagination will fill the gaps and signify the landscape, allowing us to imagine the hunter, the sardine or the plane.

the whole is the sum of a few parts

Sparse coding is a technique used in signal processing and machine learning to represent data in a more concise and efficient manner. It aims to find a sparse representation of the data, which means representing the data with only a small number of non-zero coefficients or activations. In sparse coding, a set of basis functions or atoms is typically defined, and the goal is to find a linear combination of these atoms that best represents the input data. The coefficients of this linear combination are often constrained to be sparse, meaning that only a few of them are allowed to be non-zero.

Sparse representations in computer vision

[LP et al, 2004]

Sparse representations in computer vision

Code @ A hitchhiker guide to Matching Pursuit

Sparse representations in computer vision

[LP and Bednar, 2015]

Sparse representations in computer vision

[LP, 2021]

Sparse representations in neuromorphic engineering

Ultimately, we get a list of events for each pixel that can be merged to represent the entire image. This list of events includes pixel addresses, times of occurrence, and polarities. Note that since events are generated over time, they are naturally sorted by their time of occurrence. These events are then transmitted in real time to the output bus, often via a USB3 connection. It’s interesting to draw a parallel between this process and the optic nerve that connects our retina to the brain. In fact, the output of the retina consists of a million ganglion cells that emit action potentials, which are the only source of information transmitted by the optic nerve.

https://www.researchgate.net/profile/Guido-Croon/publication/313221316/figure/fig2/AS:668997448134663@1536512829861/Picture-of-the-event-based-camera-employed-in-this-work-the-DVS_W640.jpg

Sparse representations in neuroscience

[Brunel, 2001]

Sparse representations in neuroscience

[Mainen & Sejnowski, 1995]

Sparse representations in neuroscience

[Kremkow et al, 2016]

Sparse representations in neuroscience

[Kremkow et al, 2016]

Sparse representations in neuroscience

[Kremkow et al, 2016]

Sparse representations?

Sparse representations in a nutshell

[LP et al, 2004]

Sparse representations in a nutshell

[Olshausen and Field (1997)]

Sparse representations in a nutshell

Generative model of image synthesis:

$I[x, y] = $ $\sum_{i=1}^{K} a[i] \cdot \phi[i, x, y]$ $ + \varepsilon[x, y]$

Where $\phi$ is a dictionary of $K$ atoms, $a$ is a sparse vector of coefficients, and $\varepsilon$ is a noise term.

[LP (2015)]

Sparse representations in a nutshell

[Olshausen and Field (1997)]

Sparse representations in a nutshell

Given an observation $I$,

$$ \begin{aligned} \mathcal{L}(a) & = - \log Pr( a | I ) \\ \end{aligned} $$

Sparse representations in a nutshell

Given an observation $I$,

$$ \begin{aligned} \mathcal{L}(a) & = - \log Pr( a | I ) \\ & = - \log Pr( I | a ) - \log Pr(a) \\ \end{aligned} $$

Sparse representations in a nutshell

Given an observation $I$,

$$ \begin{aligned} \mathcal{L}(a) & = - \log Pr( a | I ) \\ & = - \log Pr( I | a ) - \log Pr(a) \\ & = \frac{1}{2\sigma_n^2} \sum_{x, y} ( I[x, y] - \sum_{i=1}^{K} a[i] \cdot \phi[i, x, y])^2 - \sum_{i=1}^{K} \log Pr( a[i] ) \end{aligned} $$

Sparse representations in a nutshell

The problem is formalized as an optimization problem $a^\ast = \arg \min_a \mathcal{L}(a)$ with:

$$ \mathcal{L} = \frac{1}{2} \sum_{x, y} ( I[x, y] - \sum_{i=1}^{K} a[i] \cdot \phi[i, x, y])^2 + \lambda \cdot \sum_i ( a[i] \neq 0) $$

[LP (2015)]

Sparse representations in a nutshell

The problem is formalized as an optimization problem $a^\ast = \arg \min_a \mathcal{L}(a)$ with:

$$ \mathcal{L}(a) = \frac{1}{2} \sum_{x, y} ( I[x, y] - \sum_{i=1}^{K} a[i] \cdot \phi[i, x, y])^2 + \lambda \cdot \sum_{i=1}^{K} | a[i] | $$

Sparse representations in a nutshell

[Rentzeperis et al (2023)]

Sparse representations in a nutshell

[Olshausen and Field (1997)]

Neural implementation = gradient descent

LASSO = least absolute shrinkage and selection operator

Orthogonal Matching Pursuit (OMP): OMP is an iterative algorithm used for sparse signal recovery. It starts with an initial sparse solution and iteratively selects the most correlated dictionary atoms with the residual signal. OMP aims to minimize the L2 norm of the residual while maintaining sparsity. It has a greedy nature and can provide a near-optimal sparse solution.

Basis Pursuit (BP): Basis Pursuit is an optimization problem that seeks the sparsest solution to an underdetermined linear system of equations. It involves minimizing the L1 norm of the coefficient vector subject to a linear constraint. BP can be solved using linear programming techniques or convex optimization algorithms.

Iterative Soft Thresholding Algorithm (ISTA): ISTA is an iterative optimization algorithm commonly used in sparse coding. It alternates between a gradient descent step and a soft thresholding step. The gradient descent step minimizes the data fidelity term, and the soft thresholding step enforces sparsity by setting small coefficients to zero. ISTA converges to a sparse solution and can be used for dictionary learning.

FISTA (Fast Iterative Shrinkage-Thresholding Algorithm): FISTA is an accelerated version of ISTA that improves convergence speed. It incorporates momentum into the optimization process and achieves faster convergence rates.

ADMM (Alternating Direction Method of Multipliers): ADMM is an optimization technique that decomposes the original problem into smaller subproblems and solves them iteratively. It is often used for convex optimization problems with L1 regularization. ADMM has been applied to solve sparse coding problems efficiently.

Matching pursuit algorithm

Init : Residual $R = I$, sparse vector $a$ such that $\forall i$, $a[i] = 0$
while $\frac{1}{2} \sum_{x, y} R[x, y]^2 > \vartheta $, do :

Matching pursuit algorithm

Init : $R = I$, $\forall i$, $a[i] = 0$
while $\frac{1}{2} \sum_{x, y} R[x, y]^2 > \vartheta $, do :
- compute $c[i] = \sum_{x, y} (R[x, y] - a[i] \cdot \phi[i, x, y])^2$
- Match: $i^\ast = \arg \min_i c[i]$

Matching pursuit algorithm

Init : $R = I$, $\forall i$, $a[i] = 0$
while $\frac{1}{2} \sum_{x, y} R[x, y]^2 > \vartheta $, do :
- Match : $i^\ast = \arg \max_i \sum_{x, y} R[x, y] \cdot \phi[i, x, y]$

Matching pursuit algorithm

Init : $R = I$, $\forall i$, $a[i] = 0$
while $\frac{1}{2} \sum_{x, y} R[x, y]^2 > \vartheta $, do :
- Match : $i^\ast = \arg \max_i \sum_{x, y} ( I[x, y] \cdot \phi[i, x, y])$
- Assign : $a[i^\ast] = \frac{\sum_{x, y} R[x, y] \cdot \phi[i^\ast, x, y]}{\sum_{x, y} \phi[i^\ast, x, y] \cdot \phi[i^\ast, x, y]}$

Matching pursuit algorithm

Init : $R = I$, $\forall i$, $a[i] = 0$, and normalize $\sum_{x, y} \phi[i, x, y]^2 = 1$
while $\frac{1}{2} \sum_{x, y} R[x, y]^2 > \vartheta $, do :
- Match : $i^\ast = \arg \max_i \sum_{x, y} R[x, y] \cdot \phi[i, x, y]$
- Assign : $a[i^\ast] = \sum_{x, y} R[x, y] \cdot \phi[i^\ast, x, y]$

Matching pursuit algorithm

Init : $R = I$, $\forall i$, $a[i] = 0$, $\sum_{x, y} \phi[i, x, y]^2 = 1$
while $\frac{1}{2} \sum_{x, y} R[x, y]^2 > \vartheta $, do :
- Match : $i^\ast = \arg \max_i \sum_{x, y} R[x, y] \cdot \phi[i, x, y]$
- Assign : $a[i^\ast] = \sum_{x, y} R[x, y] \cdot \phi[i^\ast, x, y]$
- Pursuit : $R[x, y] \leftarrow R[x, y] - a[i^\ast] \cdot \phi[i^\ast, x, y]$

Matching pursuit algorithm

Init : $R = I$, $\forall i$, $a[i] = 0$, $\sum_{x, y} \phi[i, x, y]^2 = 1$
compute $c[i] = \sum_{x, y} R[x, y] \cdot \phi[i, x, y]$
compute $X[i, j] = \sum_{x, y} \phi[i, x, y] \cdot \phi[j, x, y]$
while $\frac{1}{2} \sum_{x, y} R[x, y]^2 > \vartheta $, do :
- Match : $i^\ast = \arg \max_i c[i]$
- Assign : $a[i^\ast] = c[i^\ast]$
- Pursuit : $c[i] \leftarrow c[i] - a[i^\ast] \cdot X[i, i^\ast] $

[LP (2004)]

Matching pursuit algorithm

Code @ A hitchhiker guide to Matching Pursuit

Matching pursuit algorithm

Hebbian learning (once the sparse code is known):

$$ \phi_{i}[x, y] \leftarrow \phi_{i}[x, y] + \eta \cdot a[i] \cdot (I[x, y] - \sum_{i=1}^{K} a[i] \cdot \phi_{i}[x, y] ) $$

[LP (2015)]

Matching pursuit algorithm

Sparse representations in a nutshell

[LP et al, 2004]

Convolutional Sparse Coding

[Boutin et al, 2021]

Convolutional Neural Nets (CNN)

[Boutin et al, 2021]

Convolutional Neural Nets (CNN)

[Jérémie & LP, 2023]

Convolution: Mathematics

One-dimensional discrete convolution (eg in time) with a kernel $g$ of radius $K$: $$ (f \ast g)[n]=\sum_{m=-K}^{K} f[n-m] \cdot g[m] $$

Convolution: Mathematics

Convolution of an image (two-dimensional) with a kernel $g$ of radius $K\times K$:

$$ (f \ast g)[x, y] = \sum_{i=-K}^{K} \sum_{j=-K}^{K} f[x-i, y-j] \cdot g[i, j] $$

Convolution: Mathematics

Cross-correlation of an image (two-dimensional) with a kernel $g$ of radius $K\times K$:

$$ (f \ast \tilde{g})[x, y] = \sum_{i=-K}^{K} \sum_{j=-K}^{K} f[x+i, y+j] \cdot g[i, j] $$

Convolution: Mathematics

[Amidi & Amidi]

Convolution: Mathematics

Correlation of an image defined on several channels (note the order of the indices):

$$ (f \ast \tilde{g})[x, y] = \sum_{c=1}^{C} \sum_{c,i,j} f[c, x+i, y+j] \cdot g[c, i, j] $$

Convolution: Mathematics

Correlation of a multi-channel image for multiple output channels (note the order of the indices):

$$ (f \ast \tilde{g})[k, x, y] = \sum_{c,i,j} f[c, x+i, y+j] \cdot g[k, c, i, j] $$

CNN: the HMAX model

[Serre and Poggio, 2006]

CNN: challenges

[Boutin et al, 2021]

Convolutional Sparse Coding

[Boutin et al, 2021]

Convolutional Sparse Coding

Code @ A hitchhiker guide to Matching Pursuit

Convolutional Sparse Coding

[LP, 2015]

Code @ SparseEdges

Convolutional Sparse Coding

[Ladret et al, 2024]

Convolutional Sparse Coding

[Boutin et al, 2021]

Convolutional Sparse Coding

[Boutin et al, 2021]

CNN: Predictive processing

CNN: Topography

[Bosking et al, 1997]

CNN: Topography

[Boutin et al, 2022]

Sparse representations

Laurent Perrinet

NeuroSchool PhD Program in Neuroscience

[2025-03-11]

Code / Contact me @ laurent.perrinet@univ-amu.fr

2025-02-11-neuromath

Tue, 11 Feb 2025 00:00:00 +0000

[2025-02-11] When Cortical Neurons Talk Sideways: Beyond Feedforward Visual Processing

Laurent Perrinet - https://laurentperrinet.github.io Séminaire Neuromathématiques, Collège de France

Hi, thanks for the introduction! I am Laurent Perrinet, a researcher in computational neuroscience and currently a research director at CNRS at the Institute of Neuroscience of la Timone in Marseille. Thank you for inviting me to participate in this “NeuroMathematics” seminar at the intersection of mathematics and neuroscience.

As an engineer by training, I could have pursued a career in aeronautics rather than becoming a neuroscientist. It is thanks to my mathematics professor Manuel Samuelides that I discovered the beauty of neural networks at the end of my engineering studies. This developped a curiosity, and thanks to him, I was also able to study in a mastere of cognitive sciences (now called CogMaster) in 1998. This is where I particularly want to acknowledge Jean Petitot - for his course I discovered how natural image statistics could link to principles in the central nervous system. This was a vivid revelation, and I’m grateful for his guidance in my academic path. Today’s seminar represents a return to these roots, as I’ll present my research progress since my mastere thesis on this very topic.

Today, I will address our current knowledge about horizontal connectivity rules in V1. Why is this important? As a matter of fact, one main function of sensory systems, such as the pivotal role of the primary visual cortex for vision, is to bind together the different visual features to help ultimately build a global perception.

Paysage catalan (Le Chasseur), Joan Miró (1924)

Trames (Etienne Rey)

Contour detection and the Association Field

[Field et al, 1993]

Contour detection and the Association Field

[Field et al, 1993]

Contour detection and the Association Field

[Field et al, 1993]

Natural Images : Edges are on a common circle

[Sigman et al, 2001]

A significant contribution to understanding the association field came from studying edge co-occurrences in natural images by Sigman et al. (2001). They quantified the probability density function of edge co-occurrences based on their relative positions and orientations. The figure demonstrates this by showing the spatial distribution patterns for edges relative to a reference edge at different orientations. For iso-oriented edges (a), the co-occurrence pattern shows clear structure. As the relative orientation increases through 22.5° (b), 45° (c), 67.5° (d), to 90° (e), distinct spatial patterns emerge.

A key finding was that for any given relative orientation between edges, the angle of maximal interaction occurs at the bisector between the orientations. This suggests that co-occurring edges tend to lie on a common circle - a property known as cocircularity. Panel (f) illustrates this geometrical principle: given two edges at angles w (red, 20°) and c (blue, 40°), the cocircularity solutions (green lines at 30° and 120°) represent the possible orientations of connecting circular arcs. This mathematical relationship provides insights into how the visual system might leverage statistical regularities in natural scenes for contour integration. We will go back into the details of this a bit further in the talk.

This association field concept provided a compelling framework for understanding how the visual system may implement contour integration through neural connectivity patterns. but before going there we should go back to the basic anatomy of the visual cortex.

Contour detection and the Association Field

[Field et al, 1993]

Dynamics of vision

Human Visual system (Grimaldi et al 2022)

Anatomy of the Human Visual system

Human Visual system (Grimaldi et al 2022)

Thalamic, short- & long-range lateral, inter-areal

A key feature of primary visual cortex is its layered organization, which is shared across cortical areas. The main thalamic input arrives in layer 4, which connects to a dense network of vertical connections across layers. These columns can then communicate via horizontal connections within layers.

Hubel and Wiesel also proposed the ice-cube model that every point in the visual field produces a response in a 2 mm x 2 mm area of the cortex. Such an area can contain two complete groups of ocular dominance columns, 16 blobs and interblobs that may contain more than two times all of the orientations possible across 180 degrees. This region of the cortex, which Hubel and Wiesel called a hypercolumn (or, more generally, a cortical module) seems both necessary and sufficient for analyzing the image of a point in visual space. Because the cortex is a continuous cellular layer and because it is very hard to establish the boundaries of these modules physically, their existence from a functional standpoint is still the subject of debate. https://thebrain.mcgill.ca/flash/a/a_02/a_02_cl/a_02_cl_vis/a_02_cl_vis.html

Figure 9.2. Hypercolumn Diagram. Ocular dominance columns are segregated into left and right eye inputs. Orientation columns are neurons that get excited at different orientations and a cluster of these is called a pinwheel. Blobs are color selective and for every pinwheel there is a blob. (Credit: McGill: The Brain from Top to Bottom, Figure of hypercolumns, Copyleft https://copyleft.org/, https://thebrain.mcgill.ca/flash/a/a_02/a_02_cl/a_02_cl_vis/a_02_cl_vis.html. No modifications.)

From: https://pressbooks.umn.edu/sensationandperception/chapter/columns-and-hypercolumns-in-v1/

Thalamic, short- & long-range lateral, inter-areal

[Markov et al 2011]

Anatomy of the Primary Visual Cortex

[Kaschube et al (2010)]

V1 is central to these pathways and shows distinctive anatomical and functional properties along with a complex topographical organization.

This figure from Kaschube et al. (2010) illustrates the organization of orientation preference maps in primary visual cortex (V1). Individual V1 neurons exhibit selective responses to oriented visual stimuli (as denoted by varying hues Colors code preferred ORs as indicated by the bars in (C)), with their spatial arrangement following highly structured patterns across the cortical surface. Panel B shows Synthetic orientation-maps of equal column spacing Λ but widely different pinwheel densities ρ. Left to right: solutions of different models: (13–16).. (C) High (blue frame) and low (orange frame) pinwheel density regions in tree shrew visual cortex. (D to F), Optically recorded orientation-maps in tree shrew (D), galago (E), and ferret (F) visual cortex. Regions shown in (C) are marked in (D). White arrows in (F) mark selected pinwheel centers. Framed regions in (C) and (F) are magnified. In many mammals including cats, monkeys and ferrets, orientation preference is organized in a quasi-periodic manner, forming what are known as orientation preference maps. These maps show remarkable consistency in their geometric properties across species, particularly in the spatial organization of pinwheel centers where orientation preferences converge.

However, this organization shows important species-specific variations. Most notably, while primates and carnivores display orderly orientation maps with smooth transitions between preferred orientations, rodents lack such maps and instead show a “salt-and-pepper” arrangement where neighboring neurons have seemingly random orientation preferences. This organizational diversity raises interesting questions about the computational advantages of these different architectures and their relationship to visual processing requirements and behavioral needs across species.

Horizontal connectivity links different hypercolumns

[Bosking et al, 1997]

Contour detection and the Association Field

[Field et al, 1993]

The like-to-like hypothesis

[Field et al, 2013]

The resemblance between what was shown by Bosking and the structure of the association field that we saw above is such that it is tempting to align both and state that the function of horizontal connections is to bind neurons with a selectivity to similar orientations* over long distances. This like-to-like hypothesis has been influential in understanding horizontal connectivity patterns.

However, we should be cautious about overstating these relationships. While horizontal connections show some orientation specificity, recent evidence indicates the connectivity patterns are more complex and heterogeneous than initially proposed. The functional role of this diverse connectivity remains an active area of investigation.

During the remainder of this talk, I will try to shed light on our current knowledege on horizontal connectivities.

Supplementary: the HMAX model

[Serre and Poggio, 2007]

Supplementary: Convolutional Neural Nets (CNN)

[Boutin et al, 2021]

Supplementary: Orientation selectivity in V1

[Hubel & Wiesel, 1962]

Supplementary: Orientation selectivity in V1

[Hubel & Wiesel, 1962]

Supplementary: Marr’s three levels of analysis

Marr, 1982

cut in different levels: Marr (+ Poggio)
arbitrary, but useful division of labor= computational / algorithm / hardware
here:
anatomy
algorithm / model
function

First: What is the anatomy of horizontal connections?

<!--

[Marr, 1982]]

-->

Challenging the like-to-like hypothesis

[Chavane, LP and Rankin, 2022]

Challenging the like-to-like hypothesis

Revisiting Horizontal Connectivity Rules in V1: From like-to-like towards like-to-All [Chavane, LP and Rankin, 2022]

This figure illustrates different hypothetical connectivity rules for horizontal connections in V1. The target neuron (large circle on left) has a specific orientation preference indicated by its color. Following the classical like-to-like hypothesis (shown in panel A), this neuron would preferentially connect to other neurons with matching orientation preference (similar colors) across multiple hypercolumns, as indicated by the vertical red arrows. The radial spread of connections spans approximately three hypercolumns, consistent with anatomical observations. Each hypercolumn contains a complete set of orientation preferences, represented by the different colored neurons.

This first schematic (noted A) represents one of the like-to-like connectivity rules, where horizontal connections strictly follow orientation similarity.

Challenging the like-to-like hypothesis

Revisiting Horizontal Connectivity Rules in V1: From like-to-like towards like-to-All [Chavane, LP and Rankin, 2022]

Challenging the like-to-like hypothesis

Revisiting Horizontal Connectivity Rules in V1: From like-to-like towards like-to-All [Chavane, LP and Rankin, 2022]

Challenging the like-to-like hypothesis

Revisiting Horizontal Connectivity Rules in V1: From like-to-like towards like-to-All [Chavane, LP and Rankin, 2022]

Challenging the like-to-like hypothesis

Let’s first shows some functional evidence.

This video shows voltage-sensitive dye imaging (VSDI) data from cat primary visual cortex (area 17) in response to a local oriented grating stimulus. The visualization reveals two key aspects:

The broader activation pattern shown by overall fluorescence changes (gray)
The more restricted orientation-selective response pattern (colored regions)

Two contours are overlaid: a red line marking the boundary of significant activation, and a white line delineating regions with statistically significant orientation selectivity. The orientation selectivity is encoded by color hue.

The bottom plots quantify the spatiotemporal dynamics by showing:

Left: The total activated cortical area over time
Right: The extent of orientation-selective regions over time

Together, these measurements demonstrate how orientation-selective signals propagate laterally beyond the classical feedforward input zone through horizontal connections, while maintaining some degree of feature selectivity.

Challenging the like-to-like hypothesis

Revisiting Horizontal Connectivity Rules in V1: From like-to-like towards like-to-All [Chavane, LP and Rankin, 2022]

This figure shows spatial and temporal dynamics of orientation selectivity in cat V1 analyzed from voltage-sensitive dye imaging data. Panel A displays a cortical orientation map averaged over the final 145ms of the response, where hue indicates preferred orientation and brightness shows orientation tuning strength. The dotted red line delineates the expected retinotopic boundary of feedforward input based on Albus (2004).

The inset quantitatively compares the spatial extent of:

Total cortical activation (grey contour)
Orientation-selective activation (black contour)
Theoretical feedforward input boundary (red contour)

This data demonstrates that orientation-selective responses propagate laterally beyond the classical feedforward input zone through horizontal connections, while maintaining some degree of feature selectivity. The systematic comparison between total activation and selective activation provides direct evidence for how horizontal connectivity shapes the spatiotemporal dynamics of orientation processing in V1.

Challenging the like-to-like hypothesis

Revisiting Horizontal Connectivity Rules in V1: From like-to-like towards like-to-All [Chavane, LP and Rankin, 2022]

Challenging the like-to-like hypothesis

Revisiting Horizontal Connectivity Rules in V1: From like-to-like towards like-to-All [Chavane, LP and Rankin, 2022]

Panel C displays intracellular recordings of subthreshold responses visualized as a visuotopic orientation polar map. The color hue represents preferred orientation while brightness indicates the strength of orientation tuning in the membrane potential. White contours outline regions showing statistically significant responses based on both amplitude and orientation selectivity criteria. The middle plots show averaged subthreshold responses to four different oriented stimuli (color-coded) at specific recording locations (marked by circle, triangle and square symbols), with scale bars indicating 50 ms and 1 mV. On the right, normalized orientation tuning curves are shown, computed by integrating responses within a fixed temporal window (shaded region in middle panel). The black circle marks the spontaneous activity level for the depolarizing integral measurement.

These shows a direct functional evidence for a diversity of tuning profile in th horizontal connectivity.

Challenging the like-to-like hypothesis

[Voges and LP, 2012]

Challenging the like-to-like hypothesis

Revisiting Horizontal Connectivity Rules in V1: From like-to-like towards like-to-All [Chavane, LP and Rankin, 2022]

Figure 4 illustrates a neural field model that bridges anatomical structure with functional observations in V1, as developed by Rankin and Chavane (2017).

Panel A depicts radial connectivity profiles with Gaussian-decaying inhibition and distance-dependent excitation that peaks periodically at multiples of distance L. The Ring Width (RW) parameter controls the spread of these excitatory peaks.

Panel B shows how local orientation preference maps influence lateral connectivity patterns under different orientation bias (BR) values in the recurrent connections.

Panel C quantifies the orientation tuning that emerges from these connectivity patterns. While orientations are uniformly represented globally, the local excitatory component shows strong bias around -60°. As BR increases above 0.5, the lateral connection orientation bias strengthens, reaching values around k=1 (consistent with Buzás et al. 2006).

Challenging the like-to-like hypothesis

Revisiting Horizontal Connectivity Rules in V1: From like-to-like towards like-to-All [Chavane, LP and Rankin, 2022]

Challenging the like-to-like hypothesis

Revisiting Horizontal Connectivity Rules in V1: From like-to-like towards like-to-All [Chavane, LP and Rankin, 2022]

Panel F maps the normalized selective area (relative to the feedforward footprint) across Ring Width (RW) and orientation bias (BR) parameters. White contours delineate anatomically plausible ranges where k values fall between 0.7-1.2, consistent with experimental measurements. The green region indicates parameter combinations that additionally satisfy constraints on both orientation preference and the observed radial decay of selectivity.

The neural field model effectively connects anatomical connectivity patterns with functional observations of orientation selectivity propagation in V1. The resulting connectivity structure exhibits similarities with “association field” patterns, suggesting potential optimization for encoding natural image statistics. This framework provides a quantitative basis for investigating computational principles underlying horizontal connectivity in visual cortex.

Challenging the like-to-like hypothesis

Revisiting Horizontal Connectivity Rules in V1: From like-to-like towards like-to-All [Chavane, LP and Rankin, 2022]

Modelling the Association field

[Field et al, 2013]

Edge co-occurences in natural images

Revisiting Horizontal Connectivity Rules in V1: From like-to-like towards like-to-All [Chavane, LP and Rankin, 2022]

Panel A illustrates the groundbreaking approach developed by Geisler et al. (2001) for analyzing edge statistics in natural images. The method involves:

Detecting oriented edge elements in natural images (shown as red segments)
For each edge pair, measuring:
- Their relative orientation difference (𝜃)
- The relative position angle (𝜙)
- Center-to-center distance (d)
- Azimuth difference (φ)
- Co-circularity parameter ψ = φ - θ/2

This quantitative analysis reveals two key distributions:

A strong bias for parallel edge arrangements, evident in the orientation difference histogram
A marked preference for co-circular alignments, shown in the relative position histogram

These statistics vary significantly across image databases. For example, images containing animals exhibit enhanced co-circularity compared to general natural scenes. This suggests that rather than implementing a single fixed association field, the visual system may need to handle diverse statistical regularities present in natural inputs.

The next section will examine how these statistical regularities inform computational models of the association field.

Edge co-occurences in natural images

[Geisler, 2001]

Edge co-occurences in natural images

Edge co-occurrences can account for rapid categorization of natural versus animal images [LP and Bednar, 2015]

The probability distribution function p(ψ,θ) represents the distribution of the different geometrical arrangements of edges’ angles, which we call a “chevron map”. We show here the histogram for non-animal natural images, illustrating the preference for co-linear edge configurations. For each chevron configuration, deeper and deeper red circles indicate configurations that are more and more likely with respect to a uniform prior, with an average maximum of about 3 times more likely, and deeper and deeper blue circles indicate configurations less likely than a flat prior (with a minimum of about 0.8 times as likely). Conveniently, this “chevron map” shows in one graph that non-animal natural images have on average a preference for co-linear and parallel edges, (the horizontal middle axis) and orthogonal angles (the top and bottom rows), along with a slight preference for co-circular configurations (for ψ =0 and ψ = ± π/2, just above and below the central row).

Edge co-occurences in natural images

Edge co-occurrences can account for rapid categorization of natural versus animal images [LP and Bednar, 2015]

Edge co-occurences in natural images

Edge co-occurrences can account for rapid categorization of natural versus animal images [LP and Bednar, 2015]

This figure shows classification performance across image categories using different statistical features. We used an SVM classifier with three feature sets: first-order orientation statistics (FO), the reduced 2D “chevron map” (CM), and full 4D second-order statistics (SO). The classification accuracy (F1 score) was tested for distinguishing between image categories. Results show strong performance in separating man-made from natural images, as expected. More notably, the classifier achieved ~80% accuracy in discriminating animal vs non-animal natural images, matching human performance levels reported by Serre et al. This suggests that relatively simple edge co-occurrence statistics contain sufficient information for basic image categorization tasks, without requiring higher-level semantic processing.

We also found that our model made the same errors as humans do: if an image without an animal contains more co-circular edges, it is more likely to be falsely categorized as containing an animal.

Edge co-occurences in natural images

Edge co-occurrences can account for rapid categorization of natural versus animal images [LP and Bednar, 2015]

Can we explain the diversity ?

[Bosking et al, 1997]

Indeed, this diversity is revealed in the anatomical data: V1 horizontal connectivity exhibits more complexity than suggested by the classical like-to-like hypothesis. While orientation-specific connections exist, they coexist with non-selective connections that link neurons irrespective of their tuning preferences. This diversity likely serves multiple computational functions:

Specific connections could support contour integration and feature binding
Non-selective connections may enable broad contextual modulation
Mixed connectivity patterns could help maintain network stability while preserving functional specificity

This anatomical heterogeneity aligns with V1’s role in both specialized feature detection and broader contextual processing. Understanding how these distinct connectivity patterns interact remains an active area of research in visual neuroscience.

Predictive processing

[Boutin et al, 2021]

Predictive processing

[Boutin et al, 2021]

To bridge the gap between anatomical observations and functional requirements of visual processing, We added two key ingredients in the sparse deep predictive coding (SDPC) model :

Sparse connectivity patterns:
- Enforcing regularization of the activity map using L1 penalty
- Activity computed via recurrent local connectivity
- Similar to biological observations
Feedback from efferent layers:
- Predicts activity of afferent layer
- Only residual prediction error is processed
- Defines long-range inter-areal connectivity
- Specific influence demonstrated in Neural Computation paper

By defining a cost on minimizing the prediction error in each layer, everything stays derivable, such that we can use a classical gradient descent. These additions should allow us to better understand how feedback shapes visual processing in biological neural networks.

Predictive processing

[Boutin et al, 2021]

Predictive processing

[Boutin et al, 2021]

More specifically in the context of our focus today, we can look at the co-occurence

llustration of the procedure to generate interaction map. In this illustrative example we consider a V1 representation with only 4 feature maps (represented in the upper-left box). Step 1 is to extract a neighborhood (of size 3x3 in the illustration only) around the most strongly activated neuron (represented with a red square in the illustration) for a given central preferred orientation (denoted ✓ c ). Step 2 is to normalize the neural activity in the extracted neighborhood using the marginal activity (see Eq.8). Step 3 is to compute the resulting orientation and activity at every position of the neighborhood using a circular mean (see Eq. 11 and Eq. 12 respectively). To keep a concise figure we have illustrated the computation of the central edge of the interaction map only. For simplification, the illustration shows only 1 neighborhood extraction whereas the interaction maps shown in the paper are computed by averaging neighborhoods centered on the 10 most strongly activated neurons

Predictive processing

[Boutin et al, 2021]

Predictive processing

[Boutin et al, 2021]

Predictive processing with pooling

[Boutin et al, 2022]

Challenging the like-to-like hypothesis

[Bosking et al, 1997]

Challenging the like-to-like hypothesis

Revisiting Horizontal Connectivity Rules in V1: From like-to-like towards like-to-All [Chavane, LP and Rankin, 2022]

To conclude, our review of horizontal connectivity in V1 reveals patterns more complex than initially theorized. The classical like-to-like hypothesis, while valuable, doesn’t fully capture the diversity of observed connectivity patterns.

Mathematical modeling has proven essential in bridging theory and biology. Our predictive processing framework shows how simple computational principles can explain the emergence of these complex connectivity patterns. The model demonstrates how feedback influences lateral interactions and reproduces key experimental observations.

However, important questions remain unanswered. We need to better understand how precise timing information is encoded in these circuits, how temporal dynamics shape processing, and whether similar principles apply across other cortical areas.

These fundamental questions will guide future experimental and theoretical work as we continue to unravel the computational principles of cortical processing.

[2025-02-11] When Cortical Neurons Talk Sideways: Beyond Feedforward Visual Processing

Laurent Perrinet - https://laurentperrinet.github.io Séminaire Neuromathématiques, Collège de France

Dynamics of vision

[Marr, 1982]

Dynamics of vision

Visual latencies (see review).

Dynamics of vision

Sensorimotor delays (Perrinet & Friston 2014)

Dynamics of vision

Sensorimotor delays (Perrinet & Friston, 2014)

Dynamics of vision

Diagonal markov model (Khoei et al, 2017).

Dynamics of vision

Flash-lag effect: MBP (Khoei et al, 2017)

Dynamics of vision

[Marr, 1982]

Dynamics of vision: Neural modeling

Spiking Neural Networks: Spiking motifs

[Grimaldi et al, 2023, Precise Spiking Motifs]

Spiking Neural Networks: Spiking motifs

Review on Precise Spiking Motifs.

Spiking Neural Networks: Spiking motifs

Review on Precise Spiking Motifs.

Spiking Neural Networks: HD-SNN

Grimaldi & LP (2023) Biol Cybernetics

Spiking Neural Networks: HD-SNN

Grimaldi & LP (2023) Biol Cybernetics

2 MINUTE

One advantage of this network is that it is differentiable, enabling us to apply classical machine learning methods, notably supervised learning. We then see the emergence of different convolution kernels, and here I represent a subset of its kernels for different directions, as denoted by the red arrows on the left of the graph. It shows the kernels obtained on the spatial representation according to the different columns, and each row represents the different delays from a delay of one on the right to a delay of 12 time steps on the left. Detectors that follow the motion emerge. For example, for the top line from top to bottom. These kernels integrate both positive neurons in red and negative polarity inputs in blue. Such spatio-temporal filtering is observed in neurobiology, but to my knowledge had never been observed in a model of spiking neurons trained under natural conditions.

Spiking Neural Networks: HD-SNN

Grimaldi & LP (2023) Biol Cybernetics

Spiking Neural Networks: HD-SNN

Grimaldi & LP (2023) Biol Cybernetics

Spiking Neural Networks: HD-SNN

Grimaldi & LP (2023) Biol Cybernetics

When Cortical Neurons Talk Sideways: Beyond Feedforward Visual Processing

Tue, 11 Feb 2025 00:00:00 +0000

In this seminar we will challenge the traditional understanding of neuronal connectivity in primary visual cortex. While current theory suggests that neurons connect preferentially to others with similar orientation preferences, I will present evidence for a more complex connectivity pattern based on a distance-dependent rule: short-range connections show a like-to-like bias, while long-range connections connect more widely. This revised model better explains how the visual cortex processes complex stimuli and accounts for observed variations in neuronal interactions at different scales.

When: Wednesday 11th of February, 2025 from 14:30 to 16h30.
Where: room D2.2 of Collège de France
Accompanying paper
Revisiting Horizontal Connectivity Rules in V1: From like-to-like towards like-to-All
Frédéric Chavane, Laurent U Perrinet, James Rankin

PDF Cite DOI URL

How and why foveated retinotopy provides efficient vision

Wed, 08 Jan 2025 00:00:00 +0000

When: Wednesday 9th of January, 2025 at 12 noon.
Where: CRN seminar room, Montreal General Hospital, Livingston Hall, L7-140, with hybrid option.
Read the corresponding paper
Foveated Retinotopy Improves Classification and Localization in CNNs
Jean-Nicolas Jérémie, Emmanuel Daucé, Laurent U Perrinet

PDF Cite DOI URL arXiv

Population decoding of visual motion direction in V1 marmoset monkey : effects of uncertainty

Wed, 01 Jan 2025 00:00:00 +0000

🧠 Excited to share our latest research led by Alexandre Lainé and presented this summer at CNS2025 in beautiful Firenze, Italy!

“Population decoding of visual motion direction in V1 marmoset monkey: effects of uncertainty”

Advanced population-level analysis of V1 neural responses to motion stimuli
Novel insights into how the brain handles uncertainty in visual motion processing
Marmoset model providing crucial translational insights for visual neuroscience

This research contributes to our fundamental understanding of how the visual system processes motion information at the earliest stages of cortical processing, with important implications for both basic neuroscience and potential clinical applications. Thank you to the CNS organizing committee for hosting such an inspiring conference in the stunning venue of Palazzo dei Congressi in Villa Vittoria! 🇮🇹

#ComputationalNeuroscience #VisualNeuroscience #MotionProcessing #CNS2025 #Neuroscience #Research #MarmosetModel #V1 #PopulationDecoding

Link to publication: https://laurentperrinet.github.io/publication/laine-25-cns/

For deeper insights into uncertainty processing mechanisms in the visual cortex, see our Nature Communications Biology study:
Hugo Ladret, Nelson Cortes, Lamyae Ikan, Frédéric Chavane, Christian Casanova, Laurent U Perrinet (2023). Cortical recurrence supports resilience to sensory variance in the primary visual cortex. Nature Communications Biology.
PDF Cite DOI Dataset Code URL bioRxiv HAL
see a follow-up in:
Alexandre Lainé, Nicholas J. Priebe, Guillaume S. Masson, Laurent U Perrinet (2026). Population decoding of visual motion direction. Proceedings of AREADNE.
PDF Cite Poster LinkedIn
https://neuromatch.social/@laurentperrinet/115050564011598328
https://www.linkedin.com/posts/laurent-perrinet-1857b9_population-decoding-of-visual-motion-direction-activity-7363238280143745026-zPkg
https://bsky.app/profile/laurentperrinet.bsky.social/post/3lwowjtpbw22a

Kernel Heterogeneity Improves Sparseness of Natural Images Representations

Tue, 20 Aug 2024 00:00:00 +0000

5 minutes summary: https://hugoladret.github.io/publications/ladret_et_al_sparsecoding/
In a nutshell: We found that sparse coding of images (here extended in a convolutional framework) is improved when using kernels with heterogeneous precision in how they encode orientation information. This was confirmed by learning, but also by comparison with what is observed in the statistics of natural images and in our recordings from neurons in primary visual cortex.

Epistemic uncertainty in a CSC dictionary improves both sparseness and reconstruction performance. (a) Elements from dictionaries with fixed epistemic uncertainty before (green) and after dictionary learning (orange). (b) Elements from a dictionary with heterogeneous epistemic uncertainty before (blue) and after dictionary learning (purple). (c) Elements from a dictionary learned from scratch. (d) Distribution of the sparseness (top) and Peak Signal-to-Noise Ratio (PSNR, right) of the five dictionaries, shown as a scatter plot for each of the 600 images of the dataset (center). Median values are shown as dashed line on the histograms.
open access: https://iopscience.iop.org/article/10.1088/2634-4386/ad5d0f
This work is a followup of
Hugo Ladret, Laurent U Perrinet (2023). Convolutional Sparse Coding is improved by heterogeneous uncertainty modeling. ICLR 2023 SNN Workshop.
PDF Cite URL
This theoretical work accompanies a similar study in neurophysiology:
Hugo Ladret, Nelson Cortes, Lamyae Ikan, Frédéric Chavane, Christian Casanova, Laurent U Perrinet (2023). Cortical recurrence supports resilience to sensory variance in the primary visual cortex. Nature Communications Biology.
PDF Cite DOI Dataset Code URL bioRxiv HAL

2024-05-13-master-m-4-nc

Mon, 13 May 2024 00:00:00 +0000

Artificial neural networks and machine learning applied to the understanding of biological vision

Laurent Perrinet

[2024-05-13]Master M4NC de l’institut NeuroMod, cours Prospective Innovation and Research

Contact me @ laurent.perrinet@univ-amu.fr

Principles of Vision

[Marr, 1982]

What is the function of vision?

An Unexpected Visitor (Ilya Repin, 1884)

What is the function of vision?

An Unexpected Visitor (Yarbus, 1965)

What is the function of vision?

An Unexpected Visitor - Age? (Yarbus, 1965)

What is the function of vision?

An Unexpected Visitor - How long? (Yarbus, 1965)

Visual illusions

Rotating Snakes Akiyoshi KITAOKA

Visual illusions

Ilusions of brightness or lightness Akiyoshi KITAOKA

Visual illusions

Hering illusion

Visual illusions

Hering illusion

Visual illusions

Rotating Snakes Akiyoshi KITAOKA

Visual illusions : Pareidolia

Cydonia Mensae (1976) Viking Orbiter image

Visual illusions : Pareidolia

Cydonia Mensae (2007) Mars Global Surveyor

Visual illusions : Pareidolia

Cydonia Mensae (2007) Mars Global Surveyor

Principles of vision?

[Marr, 1982]

Computational neuroscience of vision

[Marr, 1982]

Computational neuroscience of vision

[Sejnowski, Koch & Churchland (1998)]

Anatomy of the Human Visual system

Human Visual system : the HMAX model

[Serre and Poggio, 2007]

Primary visual cortex

[Hubel & Wiesel, 1962]

Primary visual cortex

[Hubel & Wiesel, 1962]

Convolutional Neural Nets (CNN)

[Boutin et al, 2021]

Convolutional Neural Nets (CNN)

[Jérémie & LP, 2023]

CNN: Mathematics

One-dimensional discrete convolution (eg in time) with a kernel $g$ of radius $K$: $$ (f \ast g)[n]=\sum_{m=-K}^{K} f[n-m] \cdot g[m] $$

CNN: Mathematics

Convolution of an image (two-dimensional) with a kernel $g$ of radius $K\times K$:

$$ (f \ast g)[x, y] = \sum_{i=-K}^{K} \sum_{j=-K}^{K} f[x-i, y-j] \cdot g[i, j] $$

CNN: Mathematics

Cross-correlation of an image (two-dimensional) with a kernel $g$ of radius $K\times K$:

$$ (f \ast \tilde{g})[x, y] = \sum_{i=-K}^{K} \sum_{j=-K}^{K} f[x+i, y+j] \cdot g[i, j] $$

CNN: Mathematics

[Amidi & Amidi]

CNN: Mathematics

Correlation of an image defined on several channels (note the order of the indices):

$$ (f \ast \tilde{g})[x, y] = \sum_{c=1}^{C} \sum_{c,i,j} f[c, x+i, y+j] \cdot g[c, i, j] $$

CNN: Mathematics

Correlation of a multi-channel image for multiple output channels (note the order of the indices):

$$ (f \ast \tilde{g})[k, x, y] = \sum_{c,i,j} f[c, x+i, y+j] \cdot g[k, c, i, j] $$

CNN: the HMAX model

[Serre and Poggio, 2006]

CNN: challenges

[Boutin et al, 2021]

CNN: Predictive processing

[Boutin et al, 2021]

CNN: Predictive processing

[Boutin et al, 2021]

CNN: Topography

[Bosking et al, 1997]

CNN: Topography

[Boutin et al, 2022]

Computational neuroscience of vision

[Marr, 1982]

Dynamics of vision

[Marr, 1982]

Dynamics of vision

Visual latencies (see review).

Dynamics of vision

Visual latencies (see review).

Dynamics of vision

Sensorimotor delays (Perrinet & Friston 2014)

Dynamics of vision

Sensorimotor delays (Perrinet & Friston, 2014)

Dynamics of vision

Diagonal markov model (Khoei et al, 2017).

Dynamics of vision

Flash-lag effect: MBP (Khoei et al, 2017)

Dynamics of vision

[Marr, 1982]

Spiking Neural Networks (SNN)

[Marr, 1982]

SNN: Leaky Integrate-and-Fire Neuron

Review on Precise Spiking Motifs.

SNN in neurobiology

[Mainen & Sejnowski, 1995]

SNN in neurobiology

[Mainen & Sejnowski, 1995]

SNN in neurobiology

[Diesmann et al. 1999]

SNN in neurobiology

[Haimerl et al, 2019]

SNN: Spiking motifs

Review on Precise Spiking Motifs.

SNN: Spiking motifs

Review on Precise Spiking Motifs.

SNN: Spiking motifs

Review on Precise Spiking Motifs.

SNN in neuromorphic engineering

From frame-based to event-based cameras.

SNN in neuromorphic engineering

The HD-SNN neural network.

SNN in neuromorphic engineering

The HD-SNN neural network.

SNN in neuromorphic engineering

The HD-SNN neural network.

Spiking Neural Networks (SNN)

[Marr, 1982]

Artificial neural networks applied to the understanding of biological vision

[Marr, 1982]

Artificial neural networks and machine learning applied to the understanding of biological vision

Laurent Perrinet

[2024-05-13] Master M4NC de l’institut NeuroMod, cours Prospective Innovation and Research

Contact me @ laurent.perrinet@univ-amu.fr

PhD thesis 'Focus of attention: a sensory-motor task for energy reduction in spiking neural networks'

Fri, 03 May 2024 09:00:00 +0000

Dear colleagues,

Applications are welcome for a fully funded PhD position Focus of attention: a sensory-motor task for energy reduction in spiking neural networks. The position will be located at the EDGE Team @ LEAT Laboratory within Université Côte d’Azur and/or at the INT in Marseille, France.

Context

This project takes place in the context of the EMERGENCES project (ANR PEPR IA 2023-2027) which aims to advance the state of the art on machine learning models using inspiration from biology. Indeed, inspiration from brain features promises to show the emergence of unrivalled efficient processing. Among the most promising features studied in the literature of bio-inspired AI are temporal data encoding using spikes, multimodal association, local learning or attention-based processing.

This PhD subject focuses on the association between attention and spiking neural networks for defining new efficient AI models for embedded systems such as drones, robots and more generally autonomous systems.

The thesis will take place between the LEAT research lab in Sophia-Antipolis and the INT institute in Marseille which both develop complementary approaches on bio-inspired AI from neuroscience observation to embedded systems design.

Subject

The volume as well as the diversity of visual information that reaches our eyes at every moment are huge and cannot be fully integrated by the visual system. In other words, the biological system is confronted to the same challenge as the one encountered by artificial systems (especially at the edge) when dealing with the huge amounts of information coming continuously from the real world. Interestingly, the brain has found an original approach to deal with this issue by focusing on a sub-part of the visual information at a time. Indeed, the study of the visual cortex in neuroscience has made it possible to highlight subregions that treat each or all of the multiple properties of information coming from the visual pathways: shapes, colors, movements, etc [1], thus revealing the interaction of attentional processes and the concept of “saliency” used in cognitive science.

Creating a fully autonomous system remains a significant challenge, especially when operating in the dynamic real world. In recent times, machine learning has assumed a prominent role in machine vision, particularly through the implementation of deep learning algorithms. These algorithms have yielded impressive outcomes in tasks such as object detection, recognition, and tracking. However, these systems come with a high computational cost, as they must process entire camera images to generate these results. Additionally, they struggle to dynamically adapt to changes in their environment.

Our focus lies on two integrated bio-inspired approaches that leverage attentional mechanisms. The first approach, known as bottom-up, draws inspiration from the work of the Gestalt theory, the Feature Integration Theory (Triesman, Gelad) [3], and the model of visual attention from Itti & Koch [1]. This approach relies on the saliency of low-level features in the visual field, processed in parallel, including movement, color, and edges. It employs emergent mechanisms to integrate features guided by their saliency in order to detect the consistency of objects, encompassing their form, position, and speed. As shown by the Gestalt theory, only the more salient data are needed in this mechanism. Thus, we can dramatically reduce the amount of needed data by extracting only the more salient regions of interest during bottom-up phase.

The second approach, known as top down, considers that the visual attention is guided by higher level cognitive stages. For instance, in the Guided Search theory [4], Wolfe emphasizes the role of prior knowledges, expectations, and intentions. In this work, Wolfe proposes a guided search mechanism that relies on a “Priority map that represents the system’s best guess as to where to deploy attention next.”. This Priority map is built on multiple sources of information such as the visual system as well as higher-level information such as intention, search history and the actual visual semantics. In this way, higher-level information is used to guide the filtering of the botom-up path, so that only the information required for a given task is selected and processed. Similar systems are proposed by Schöner [5] in which saliency maps, working memories and “priority map”, guided visual search mechanisms are implemented through the Neural Field Theory (NFT). Here, Dynamic Neural Fields are used to implement the saliency of feature maps, as well as scene spatial selection mechanism, working memory, etc.

In a previous work from the LEAT [6], we have proposed a brain inspired attentional process implementing bottom-up and top-down paths based on a dynamic neural fields properties embodied in a sensory-motor loop. In a complementary work, the INT group has developed a dual pathway model of the visual system in which saliency emerges as a property of the perceptual system to perform saccades, that is, rapid shifts of the fixation point [7]. This uses a recognition model which takes as an input a retinotopically transformed input and shows the emergence of saliency maps [8] In the dual-pathway model, the exploration of a visual scene is based on both the saliency of the color feature (bottom-up) and the class of the last selected object recognized by a convolutional neural network (top-down). Both paths are integrated by a dynamic neural field to select the next visual information to be explored or conserved by setting motor orders accordingly.

The main goal of the thesis is to propose a new vision of the integration of attention into machine learning models. The proposed model will draw on the dynamics at play in a sensory-motor approach to perception and will thus reconsider the classical perception tasks in order to better fit with the continuous flow of information coming from the environment.

Work plan

The PhD will be co-supervised between INT in Marseille and LEAT in Nice. According to the preferences of the candidate, a main laboratory of affiliation will be selected. Weekly meetings will be organized remotely and visiting weeks will be planned to work in-person in the other lab along the year.

Year 1

Study the state of the art in both neuroscience and machine learning on the use of attentional properties to make AI models more effective in environmental perception tasks.
Write a synthesis report on this study.
Develop a first neural model integrating attention-based selection in a specific perception task such as visual search.
Define the specific metrics (KPI) dedicated to the evaluation of the performance and efficiency of such a bio-inspired AI model.
Submit a first publication on this preliminary study in an international conference.

Year 2

Analyze of the performances of the preliminary attention-based model
Develop the approach in order to integrate step by step the features related to dual pathway perception, attention, foveation, DNF and make the model compatible with convolutional neural networks
Submit a second publication in a international journal

Year 3

Study the adaptation of the model to spiking neural networks
Evaluation and comparison of the different approaches
Submit publications on the final results of the thesis
Write the thesis report and prepare the defense

Required skills

Master thesis in one of the following domains: neuromorphic systems, spiking neural networks, neurocognition, machine learning.
Background and experience in machine-learning, artificial neural networks, and/or neurosciences.
Strong motivation, team working, fluent in English (spoken and written).
Programming skills in python, keras, pytorch or equivalent

Start: year 2024

Duration: 3 years

Location: Sophia-Antipolis and/or Marseille

Contacts

Benoît Miramond is Full Professor in Electrical Engineering at LEAT laboratory from University Côte d’Azur (UCA). He holds the chair on bio-inspired AI at 3IA Cote d’Azur Institute and leads the eBRAIN research group which develops a interdisciplinary research activity on embedded Bio-inspiRed AI and Neuromorphic architectures, especially based on SNNs. LEAT is a mixt research unit (UMR 72 48) from UCA and CNRS.
Laurent Perrinet is a director of research at Institut des Neurosciences de la Timone (CNRS - Aix-Marseille Université). He is studying the link between brain microstructures and their macroscopic function by implementing realistic models of the primary visual cortex using spiking neural networks.
Laurent Rodriguez is associate professor at LEAT laboratory in the eBRAIN group. He is interested in dynamic neural networks and develop neural models from biological inspiration.
More details on the “Emergences” grant:
Laurent U Perrinet (2023). Emergences (2023 / 2027).
URL

Application

Apply by sending an email directly to the supervisors (Benoit.miramond@univ-cotedazur.fr Laurent.perrinet@univ-amu.fr Laurent.rodriguez@univ-cotedazur.fr). The application will include:

• Letter of recommendation of the master supervisor.

• Curriculum vitæ.

• Motivation Letter.

References

[1] L. Itti et C. Koch, « Computational modelling of visual attention ». Nat Rev Neurosci, vol. 2, 3, 3, mars 2001, doi: 10.1038/35058500.
[2] Gerstner, W., Kistler, W. M., Naud, R., & Paninski, L. (2014). « Neuronal dynamics: From single neurons to networks and models of cognition ». Cambridge University Press

[3] A. M. Treisman et G. Gelade, « A feature-integration theory of attention ». Cognitive Psychology, vol. 12, 1, p. 97‑136, janv. 1980, doi:10.1016/0010-0285(80)90005-5.
[4] Wolfe, J.M. «Guided Search 6.0: An updated model of visual search ». Psychon Bull Rev 28, 1060–1092 (2021). https://doi.org/10.3758/s13423-020-01859-9
[5] Grieben, R., & Schöner, G.. « A neural dynamic process model of combined bottom-up and top-down guidance in triple conjunction visual search». In T. Fitch, Lamm, C., Leder, H., & Teßmar-Raible, K. (Eds.), Proceedings of the 43rd Annual Conference of the Cognitive Science Society
[6] M. Rasamuel, Lyes Khacef, Laurent Rodriguez, et Benoit Miramond, « Specialized visual sensor coupled to a dynamic neural field for embedded attentional process ». IEEE Conference Publication | IEEE Xplore. https://ieeexplore.ieee.org/abstract/document/8705979
[7] Emmanuel Daucé, Pierre Albigès, Laurent U Perrinet (2020). « A dual foveal-peripheral visual processing model implements efficient saccade selection ». Journal of Vision. doi:https://doi.org/10.1167/jov.20.8.22
[8] Jean-Nicolas Jérémie, Emmanuel Daucé, Laurent U Perrinet (2020). « Retinotopic Mapping Enhances the Robustness of Convolutional Neural Networks ». arXiv https://arxiv.org/abs/2402.15480

2024-04-10-ue-neurosciences-computationnelles

Wed, 10 Apr 2024 00:00:00 +0000

Artificial neural networks applied to the understanding of biological vision

Laurent Perrinet

[2024-04-10] Master 1 Neurosciences et Sciences Cognitives.

Contact me @ laurent.perrinet@univ-amu.fr

Principles of Vision

[Marr, 1982]

What is the function of vision?

An Unexpected Visitor (Ilya Repin, 1884)

What is the function of vision?

An Unexpected Visitor (Yarbus, 1965)

What is the function of vision?

An Unexpected Visitor - Age? (Yarbus, 1965)

What is the function of vision?

An Unexpected Visitor - How long? (Yarbus, 1965)

Visual illusions

Rotating Snakes Akiyoshi KITAOKA

Visual illusions

Ilusions of brightness or lightness Akiyoshi KITAOKA

Visual illusions

Hering illusion

Visual illusions

Hering illusion

appear bent

effect of context -> 3D

Visual illusions : Pareidolia

Cydonia Mensae (1976) Viking Orbiter image

Visual illusions : Pareidolia

Cydonia Mensae (2007) Mars Global Surveyor

Visual illusions : Pareidolia

Cydonia Mensae (2007) Mars Global Surveyor

Principles of vision?

[Marr, 1982]

Computational neuroscience of vision

[Marr, 1982]

Computational neuroscience of vision

[Sejnowski, Koch & Churchland (1998)]

Anatomy of the Human Visual system

Human Visual system : the HMAX model

[Serre and Poggio, 2007]

Primary visual cortex

[Hubel & Wiesel, 1962]

Primary visual cortex

[Hubel & Wiesel, 1962]

Convolutional Neural Nets (CNN)

[Boutin et al, 2021]

Convolutional Neural Nets (CNN)

[Jérémie & LP, 2023]

CNN: Mathematics

One-dimensional discrete convolution (eg in time) with a kernel $g$ of radius $K$: $$ (f \ast g)[n]=\sum_{m=-K}^{K} f[n-m] \cdot g[m] $$

CNN: Mathematics

Convolution of an image (two-dimensional) with a kernel $g$ of radius $K\times K$:

$$ (f \ast g)[x, y] = \sum_{i=-K}^{K} \sum_{j=-K}^{K} f[x-i, y-j] \cdot g[i, j] $$

CNN: Mathematics

Cross-correlation of an image (two-dimensional) with a kernel $g$ of radius $K\times K$:

$$ (f \ast \tilde{g})[x, y] = \sum_{i=-K}^{K} \sum_{j=-K}^{K} f[x+i, y+j] \cdot g[i, j] $$

CNN: Mathematics

[Amidi & Amidi]

CNN: Mathematics

Correlation of an image defined on several channels (note the order of the indices):

$$ (f \ast \tilde{g})[x, y] = \sum_{c=1}^{C} \sum_{c,i,j} f[c, x+i, y+j] \cdot g[c, i, j] $$

CNN: Mathematics

Correlation of a multi-channel image for multiple output channels (note the order of the indices):

$$ (f \ast \tilde{g})[k, x, y] = \sum_{c,i,j} f[c, x+i, y+j] \cdot g[k, c, i, j] $$

CNN: the HMAX model

[Serre and Poggio, 2006]

CNN: challenges

[Boutin et al, 2021]

CNN: Predictive processing

[Boutin et al, 2021]

CNN: Predictive processing

[Boutin et al, 2021]

CNN: Topography

[Bosking et al, 1997]

CNN: Topography

[Boutin et al, 2022]

Computational neuroscience of vision

[Marr, 1982]

Dynamics of vision

[Marr, 1982]

Dynamics of vision

Visual latencies (see review).

Dynamics of vision

Visual latencies (see review).

Dynamics of vision

Sensorimotor delays (Perrinet & Friston 2014)

Dynamics of vision

Sensorimotor delays (Perrinet & Friston, 2014)

Dynamics of vision

Diagonal markov model (Khoei et al, 2017).

Dynamics of vision

Flash-lag effect: MBP (Khoei et al, 2017)

Dynamics of vision

[Marr, 1982]

Spiking Neural Networks (SNN)

[Marr, 1982]

SNN: Leaky Integrate-and-Fire Neuron

Review on Precise Spiking Motifs.

SNN in neurobiology

[Mainen & Sejnowski, 1995]

SNN in neurobiology

[Mainen & Sejnowski, 1995]

SNN in neurobiology

[Diesmann et al. 1999]

SNN in neurobiology

[Haimerl et al, 2019]

SNN: Spiking motifs

Review on Precise Spiking Motifs.

SNN: Spiking motifs

Review on Precise Spiking Motifs.

SNN: Spiking motifs

Review on Precise Spiking Motifs.

SNN in neuromorphic engineering

From frame-based to event-based cameras.

SNN in neuromorphic engineering

The HD-SNN neural network.

SNN in neuromorphic engineering

The HD-SNN neural network.

SNN in neuromorphic engineering

The HD-SNN neural network.

Spiking Neural Networks (SNN)

[Marr, 1982]

Artificial neural networks applied to the understanding of biological vision

[Marr, 1982]

Artificial neural networks applied to the understanding of biological vision

Laurent Perrinet

[2024-04-10] Master 1 Neurosciences et Sciences Cognitives.

Contact me @ laurent.perrinet@univ-amu.fr

2024-04-17-phd-program-sparse-representations

Wed, 10 Apr 2024 00:00:00 +0000

Sparse representations

Laurent Perrinet

NeuroSchool PhD Program in Neuroscience

[2024-04-17]

Code / Contact me @ laurent.perrinet@univ-amu.fr

Sparse representations?

Sparse representations in computer vision

[LP et al, 2004]

Sparse representations in computer vision

[LP and Bednar, 2015]

Sparse representations in computer vision

[LP, 2021]

Sparse representations in neuromorphic engineering

https://www.researchgate.net/profile/Guido-Croon/publication/313221316/figure/fig2/AS:668997448134663@1536512829861/Picture-of-the-event-based-camera-employed-in-this-work-the-DVS_W640.jpg

Sparse representations in neuromorphic engineering

The HD-SNN neural network.

Sparse representations in neuromorphic engineering

The HD-SNN neural network.

Sparse representations in neuroscience

[Brunel, 2001]

Sparse representations in neuroscience

[Mainen & Sejnowski, 1995]

Sparse representations in neuroscience

[Kremkow et al, 2016]

Sparse representations in neuroscience

[Kremkow et al, 2016]

Sparse representations in neuroscience

[Kremkow et al, 2016]

Sparse representations?

Sparse representations in a nutshell

[LP et al, 2004]

Sparse representations in a nutshell

[Olshausen and Field (1997)]

Sparse representations in a nutshell

Generative model of image synthesis:

$I[x, y] = $ $\sum_{i=1}^{K} a[i] \cdot \phi[i, x, y]$ $ + \varepsilon[x, y]$

Where $\phi$ is a dictionary of $K$ atoms, $a$ is a sparse vector of coefficients, and $\varepsilon$ is a noise term.

[LP (2015)]

Sparse representations in a nutshell

[Olshausen and Field (1997)]

Sparse representations in a nutshell

Given an observation $I$,

$$ \begin{aligned} \mathcal{L}(a) & = - \log Pr( a | I ) \\ \end{aligned} $$

Sparse representations in a nutshell

Given an observation $I$,

$$ \begin{aligned} \mathcal{L}(a) & = - \log Pr( a | I ) \\ & = - \log Pr( I | a ) - \log Pr(a) \\ \end{aligned} $$

Sparse representations in a nutshell

Given an observation $I$,

Sparse representations in a nutshell

The problem is formalized as an optimization problem $a^\ast = \arg \min_a \mathcal{L}(a)$ with:

$$ \mathcal{L} = \frac{1}{2} \sum_{x, y} ( I[x, y] - \sum_{i=1}^{K} a[i] \cdot \phi[i, x, y])^2 + \lambda \cdot \sum_i ( a[i] \neq 0) $$

[LP (2015)]

Sparse representations in a nutshell

The problem is formalized as an optimization problem $a^\ast = \arg \min_a \mathcal{L}(a)$ with:

$$ \mathcal{L}(a) = \frac{1}{2} \sum_{x, y} ( I[x, y] - \sum_{i=1}^{K} a[i] \cdot \phi[i, x, y])^2 + \lambda \cdot \sum_{i=1}^{K} | a[i] | $$

Sparse representations in a nutshell

[Rentzeperis et al (2023)]

Sparse representations in a nutshell

[Olshausen and Field (1997)]

Neural implementation = gradient descent

LASSO = least absolute shrinkage and selection operator

Matching pursuit algorithm

Init : Residual $R = I$, sparse vector $a$ such that $\forall i$, $a[i] = 0$
while $\frac{1}{2} \sum_{x, y} R[x, y]^2 > \vartheta $, do :

Matching pursuit algorithm

Init : $R = I$, $\forall i$, $a[i] = 0$
while $\frac{1}{2} \sum_{x, y} R[x, y]^2 > \vartheta $, do :
- compute $c[i] = \sum_{x, y} (R[x, y] - a[i] \cdot \phi[i, x, y])^2$
- Match: $i^\ast = \arg \min_i c[i]$

Matching pursuit algorithm

Init : $R = I$, $\forall i$, $a[i] = 0$
while $\frac{1}{2} \sum_{x, y} R[x, y]^2 > \vartheta $, do :
- Match : $i^\ast = \arg \max_i \sum_{x, y} R[x, y] \cdot \phi[i, x, y]$

Matching pursuit algorithm

Init : $R = I$, $\forall i$, $a[i] = 0$
while $\frac{1}{2} \sum_{x, y} R[x, y]^2 > \vartheta $, do :
- Match : $i^\ast = \arg \max_i \sum_{x, y} ( I[x, y] \cdot \phi[i, x, y])$
- Assign : $a[i^\ast] = \frac{\sum_{x, y} R[x, y] \cdot \phi[i^\ast, x, y]}{\sum_{x, y} \phi[i^\ast, x, y] \cdot \phi[i^\ast, x, y]}$

Matching pursuit algorithm

Init : $R = I$, $\forall i$, $a[i] = 0$, and normalize $\sum_{x, y} \phi[i, x, y]^2 = 1$
while $\frac{1}{2} \sum_{x, y} R[x, y]^2 > \vartheta $, do :
- Match : $i^\ast = \arg \max_i \sum_{x, y} R[x, y] \cdot \phi[i, x, y]$
- Assign : $a[i^\ast] = \sum_{x, y} R[x, y] \cdot \phi[i^\ast, x, y]$

Matching pursuit algorithm

Init : $R = I$, $\forall i$, $a[i] = 0$, $\sum_{x, y} \phi[i, x, y]^2 = 1$
while $\frac{1}{2} \sum_{x, y} R[x, y]^2 > \vartheta $, do :
- Match : $i^\ast = \arg \max_i \sum_{x, y} R[x, y] \cdot \phi[i, x, y]$
- Assign : $a[i^\ast] = \sum_{x, y} R[x, y] \cdot \phi[i^\ast, x, y]$
- Pursuit : $R[x, y] \leftarrow R[x, y] - a[i^\ast] \cdot \phi[i^\ast, x, y]$

Matching pursuit algorithm

Init : $R = I$, $\forall i$, $a[i] = 0$, $\sum_{x, y} \phi[i, x, y]^2 = 1$
compute $c[i] = \sum_{x, y} R[x, y] \cdot \phi[i, x, y]$
compute $X[i, j] = \sum_{x, y} \phi[i, x, y] \cdot \phi[j, x, y]$
while $\frac{1}{2} \sum_{x, y} R[x, y]^2 > \vartheta $, do :
- Match : $i^\ast = \arg \max_i c[i]$
- Assign : $a[i^\ast] = c[i^\ast]$
- Pursuit : $c[i] \leftarrow c[i] - a[i^\ast] \cdot X[i, i^\ast] $

[LP (2004)]

Matching pursuit algorithm

Code @ A hitchhiker guide to Matching Pursuit

Matching pursuit algorithm

Hebbian learning (once the sparse code is known):

$$ \phi_{i}[x, y] \leftarrow \phi_{i}[x, y] + \eta \cdot a[i] \cdot (I[x, y] - \sum_{i=1}^{K} a[i] \cdot \phi_{i}[x, y] ) $$

[LP (2015)]

Matching pursuit algorithm

Sparse representations in a nutshell

[LP et al, 2004]

Convolutional Sparse Coding

[Boutin et al, 2021]

Convolutional Neural Nets (CNN)

[Boutin et al, 2021]

Convolutional Neural Nets (CNN)

[Jérémie & LP, 2023]

Convolution: Mathematics

One-dimensional discrete convolution (eg in time) with a kernel $g$ of radius $K$: $$ (f \ast g)[n]=\sum_{m=-K}^{K} f[n-m] \cdot g[m] $$

Convolution: Mathematics

Convolution of an image (two-dimensional) with a kernel $g$ of radius $K\times K$:

$$ (f \ast g)[x, y] = \sum_{i=-K}^{K} \sum_{j=-K}^{K} f[x-i, y-j] \cdot g[i, j] $$

Convolution: Mathematics

Cross-correlation of an image (two-dimensional) with a kernel $g$ of radius $K\times K$:

$$ (f \ast \tilde{g})[x, y] = \sum_{i=-K}^{K} \sum_{j=-K}^{K} f[x+i, y+j] \cdot g[i, j] $$

Convolution: Mathematics

[Amidi & Amidi]

Convolution: Mathematics

Correlation of an image defined on several channels (note the order of the indices):

$$ (f \ast \tilde{g})[x, y] = \sum_{c=1}^{C} \sum_{c,i,j} f[c, x+i, y+j] \cdot g[c, i, j] $$

Convolution: Mathematics

Correlation of a multi-channel image for multiple output channels (note the order of the indices):

$$ (f \ast \tilde{g})[k, x, y] = \sum_{c,i,j} f[c, x+i, y+j] \cdot g[k, c, i, j] $$

CNN: the HMAX model

[Serre and Poggio, 2006]

CNN: challenges

[Boutin et al, 2021]

Convolutional Sparse Coding

[Boutin et al, 2021]

Convolutional Sparse Coding

Code @ A hitchhiker guide to Matching Pursuit

Convolutional Sparse Coding

[LP, 2015]

Code @ SparseEdges

Convolutional Sparse Coding

[Ladret et al, 2024]

Convolutional Sparse Coding

[Boutin et al, 2021]

Convolutional Sparse Coding

[Boutin et al, 2021]

CNN: Predictive processing

[Boutin et al, 2021]

CNN: Predictive processing

[Boutin et al, 2021]

CNN: Predictive processing

[Boutin et al, 2021]

CNN: Predictive processing

[Boutin et al, 2021]

CNN: Predictive processing

CNN: Topography

[Bosking et al, 1997]

CNN: Topography

[Boutin et al, 2022]

Sparse representations

Laurent Perrinet

NeuroSchool PhD Program in Neuroscience

[2024-04-17]

Code / Contact me @ laurent.perrinet@univ-amu.fr

2024-03-27-emergences.md

Wed, 27 Mar 2024 00:00:00 +0000

Analyser de larges volumes de données neurobiologiques

Laurent Perrinet

[2024-03-27] Emergences workshop, Autrans, France

laurent.perrinet@univ-amu.fr

Hello, can you hear me in the back? First of all, I’d like to thank the organizers for this opportunity and all of you for coming.

I’m Laurent Perrinet from the Institut des Neurosciences de la Timone, a joint AMU / CNRS unit, and I’m a computational neuroscientist interested in large-scale models of vision.

Alors que ce projet vient juste de commencer, je voudrais déjà parler de quelques idées pour l’avenir. En effet, la question peut se poser quant aux applications futures des puces neuromorphiques qui vont être développées dans le cadre du projet “Emergences”. pour ce développement technologique, on va souvent penser à des applications technologiques, comme les voitures autonome ou la vision robotique. Mais il y a aussi des applications qui peuvent viser à la compréhension du fonctionnement du cerveau et de la cognition en général. Et ceci passe par une meilleure connaissance de la façon dont celle-ci est contenues dans l’activité neurale.

If you wish to go further, these slides along with a number of references and useful links are available on my website.

Techniques d’enregistrement de données neurobiologiques

Enregistrement extracellulaire

[Hubel & Wiesel, 1962]

Même si ce ne sont pas les premiers à avoir enregistré l’activité électrique de neurones (ce sont physiologistes allemands Emil du Bois-Reymond et Hermann von Helmholtz au milieu du 19e siècle), David Hubel et Torsten Wiesel ont marqué leur époque. En 1962, ils ont mené des expériences révolutionnaires qui ont permis de comprendre les mécanismes de base de la perception visuelle et ont jeté les bases de la compréhension de l’organisation fonctionnelle du cortex visuel. Leur travail a valu à Hubel et Wiesel le prix Nobel de physiologie ou médecine en 1981.

La technique principale utilisée par Hubel et Wiesel dans leurs expériences était la microélectrode d’enregistrement extracellulaire. Ils ont inséré de fines électrodes dans le cortex visuel primaire (aussi appelé cortex strié) de chats et de singes anesthésiés. Ces électrodes leur ont permis d’enregistrer l’activité électrique des neurones individuels lors de la présentation de stimuli visuels.

Aire visuelle primaire

Enregistrement extracellulaire

[Hubel & Wiesel, 1962]

Hubel et Wiesel ont utilisé une variété de stimuli visuels, tels que des lignes, des barres, des points lumineux et des motifs en mouvement, qu’ils ont présentés à des animaux dans des conditions contrôlées. En enregistrant les réponses des neurones visuels, ils ont pu observer des motifs caractéristiques d’activité neuronale en fonction des propriétés visuelles des stimuli.

Leur travail a révélé l’existence de neurones spécifiques, appelés neurones simples et neurones complexes, qui répondent de manière sélective à des caractéristiques visuelles spécifiques, telles que l’orientation, la direction du mouvement et la taille des stimuli. Ils ont également découvert que ces neurones étaient organisés de manière hiérarchique, avec des neurones simples détectant des caractéristiques visuelles élémentaires et des neurones complexes intégrant ces informations pour former des représentations plus complexes.

mais aussi: sharp electrodes, patch-clamp

Multi-électrodes

[Microelectrode array (MEAs)]

Différentes échelles

[Chemla et al, 2017]

Vers des données massives

[Stevenson and Kording, 2011]

Vers des données massives

[Steinmetz et al, 2017]

Techniques d’analyse des données neurobiologiques

Méthodes statistiques

[Ladret et al, 2023]

Méthodes statistiques

[Ladret et al, 2023]

Méthodes statistiques

[Ladret et al, 2023]

… et au-delà!

[Churchland & Cunningham et al. (2012)]

… et au-delà: le décodage

[Ladret et al, 2023]

… et au-delà: le décodage

[Ladret et al, 2023]

… et au-delà: le décodage

[Ladret et al, 2023]

Brain-Computer Interface (BCI)

[Interface neuronale directe (BCI)]

Perspectives et opportunités du neuromorphique

Exploitation d’un timing précis

[Mainen & Sejnowski, 1995]

Exploitation d’un timing précis

[Mainen & Sejnowski, 1995]

Exploitation d’un timing précis

[Kremkow et al, 2016]

Exploitation d’un timing précis

[Diesmann et al. 1999]

Codage par latence

[Haimerl et al, 2019]

Codage par latence

[Thorpe (2001)]

Latences et rapidité

Visual latencies (see review).

Algorithmes neuromorphiques

Always-on classification using HOTS

[Grimaldi, Boutin, Sio-Ieng, Benosman & LP, 2023]

Always-on classification using HOTS

[Grimaldi, Boutin, Sio-Ieng, Benosman & LP, 2023]

Always-on classification using HOTS

[Grimaldi, Boutin, Sio-Ieng, Benosman & LP, 2023]

Spiking motifs in vision

[Grimaldi et al, 2023, Precise Spiking Motifs]

Spiking motifs in vision

Grimaldi & LP (2023) Biol Cybernetics

Spiking motifs in vision

Grimaldi & LP (2023) Biol Cybernetics

Spiking motifs in vision

Grimaldi & LP (2023) Biol Cybernetics

Spiking motifs in vision

Grimaldi & LP (2023) Biol Cybernetics

Spiking motifs in vision

Grimaldi & LP (2023) Biol Cybernetics

Spiking motifs pour la bio (HD-SNN)

LP (2023)

Future steps

unsupervised

high-throughput

real-time

Analyser de larges volumes de données neurobiologiques

Laurent Perrinet

[2024-03-27] Emergences workshop, Autrans, France

laurent.perrinet@univ-amu.fr

Beyond $\ell_1$ sparse coding in V1

Tue, 12 Sep 2023 00:00:00 +0000

Follows
Ilias Rentzeperis, Luca Calatroni, Laurent U Perrinet, Dario Prandi (2022). Which sparsity problem does the brain solve?. Proceedings of AREADNE.
PDF Cite URL

Ce que le paranormal dit de notre cerveau

Wed, 26 Jul 2023 00:00:00 +0000

Participation à un article de dissémination pour l’excellent magazine Epsiloon, écrit par Alexandra Pihen: qu’est-ce que qe l’étrange et le paranormal peut révéler sur notre cerveau

Sortir de son corps, entendre des voix, voir des fantômes… Depuis quelques années, les chercheurs commencent à prendre ces phénomènes très au sérieux. Et si ces expériences permettaient d’ouvrir de nouvelles fenêtres sur notre cerveau ?

Lire l’article sur:

https://www.epsiloon.com/tous-les-numeros/n26/ce_que_le_paranormal_dit_de_notre_cerveau/

Cortical recurrence supports resilience to sensory variance in the primary visual cortex

Tue, 06 Jun 2023 00:00:00 +0000

open access: https://www.nature.com/articles/s42003-023-05042-3
5 minutes summary: https://hugoladret.github.io/publications/ladret_et_al_variance_v1/
This neurophysiological work accompanies a similar study in theoretical neuroscience :
Hugo Ladret, Laurent U Perrinet (2023). Convolutional Sparse Coding is improved by heterogeneous uncertainty modeling. ICLR 2023 SNN Workshop.
PDF Cite URL

Communiqué de presse: Comment le cerveau fait face à l’incertitude ?

[Introduction :] Nous vivons dans un monde fait d’incertitudes, qui pourtant ne nous empêchepas d’effectuer nos tâches quotidiennes. Vous ne traverseriez pas la route avant d’être certain que le conducteur de la voiture passante vous a vu, pas d’avantage que vous ne vous approcheriez pas d’un buisson avant d’être sûr qu’il est occupé par un oiseau plutôt que par un lion. Malgré la nécessité fondamentale de résoudre ces incertitudes au quotidien, nous savons relativement peu sur la manière dont notre cerveau procède pour ce faire. Dans cet article publié dans Nature Communications Biology, les scientifiques présentent des enregistrements des neurones du cerveau, et mettent en évidence un nouveau type de neurone qui encode cette incertitude. Cette recherche est clé pour avancer la compréhension de notre cerveau et construire des modèles artificiels qui peuvent prendre en compte leurs certitudes. Imaginez que vous vous promeniez dans une forêt. Le vent bruisse dans les feuilles, quand soudain un bruit étrange attire votre attention. S’agit-il d’un écureuil qui se précipite sur le sentier à la vue de tous ? Ou peut-être d’un oiseau niché derrière les buissons, caché dans le feuillage ? Dans ce dernier cas, prenez-vous le temps de voir l’oiseau, ou en déduirez-vous que le bruissement est plutôt celui d’un lion, et vous enfuirez-vous le plus vite possible ? Ce simple scénario illustre un problème quotidien auquel nous sommes confrontés : comment notre cerveau peut-il donner un sens au monde, alors que nos sens sont bombardés d’informations peu fiables ? Ce manque de précision - l’inverse de la variance d’une information - est marquant dans le domaine de la vision. En effet, une image peut être décomposée en de nombreuses lignes ou “bords” qui forment sa structure, à l’instar d’un puzzle composé de nombreuses pièces différentes.

Figure 1. Les images naturelles (ici, une vue des calanques de Marseille), sont décomposées en éléments orientés (en bas à gauche) par des réseaux de neurones, dont les interactions sont contraintes par l’incertitude locale qui décrit des parties du champ visuel.

Cependant, toutes les pièces du puzzle ne sont pas coupées de la même manière, et certaines ont des bords plus variables que d’autres. C’est un problème pour la toute première zone de notre cerveau qui commence à donner un sens à ces “pièces de puzzle” visuelles, le cortex visuel primaire. Jusqu’à récemment, notre compréhension de la manière dont le cerveau traite ces données visuelles complexes reposait en grande partie sur l’observation du comportement humain [1,2]. Ces dernières années, cependant, les chercheurs ont commencé à sonder le cortex visuel primaire des macaques et ont découvert que cette zone du cerveau présente des comportements complexes qui reflètent les processus de prise de décision complexes que nous entreprenons en tant qu’êtres humains [3]. En effectuant des enregistrements dans le cortex visuel primaire, la zone responsable du traitement de l’information visuelle dans le cerveau, les chercheurs ont découvert un phénomène remarquable : les neurones de notre cortex visuel primaire ont des réponses distinctes à la complexité des images. Deux types principaux de neurones ont été identifiés sur la base de leurs réponses : certains sont relativement indifférents à l’augmentation de la variance, tandis que d’autres montrent une décroissance rapide de leur capacité d’encodage face à cette variance (non linéaires).

Figure 2 La variation du code des neurones face a une augmentation d’incertitude dépend de leur position dans le cortex (a, b), un phénomène expliqué par une activité récurrente plus intense pour les neurones encodant l’incertitude.

Globalement, la récurrence peut expliquer comment différents neurones encodent (ou non) la variance de leur entrée. Ces résultats vont dans le sens d’une compréhension plus complète du cerveau, qui ne se contente pas d’encoder des caractéristiques moyennes, comme le suggéraient les modèles précédents, mais prend également en compte la complexité des entrées, grâce à la connectivité entre les neurones. Il s’agit d’une étape cruciale pour comprendre comment notre cortex gère les “puzzles visuels” que nous rencontrons tous les jours, permettant au cerveau d’effectuer des calculs complexes sur des distributions probabilistes - un modèle qui gagne en popularité dans les neurosciences [5].

Références

[1] Von Helmholtz, H. (1925). Helmholtz’s treatise on physiological optics (Vol. 3). Optical Society of America. [2] Barthelmé, S., & Mamassian, P. (2009). Evaluation of objective uncertainty in the visual system. PLoS computational biology, 5(9), e1000504. [3] Hénaff, O. J., Boundy-Singer, Z. M., Meding, K., Ziemba, C. M., & Goris, R. L. (2020). Representation of visual uncertainty through neural gain variability. Nature communications, 11(1), 2513. [4] Leon, P. S., Vanzetta, I., Masson, G. S., & Perrinet, L. U. (2012). Motion clouds: model-based stimulus synthesis of natural-like random textures for the study of motion perception. Journal of neurophysiology, 107(11), 3217-3226. [5] Spratling, M. W. (2016). A neural implementation of Bayesian inference based on predictive coding. Connection Science, 28(4), 346-383.

2023-05-10-phd-program_neurosciences-computationnelles.md

Wed, 10 May 2023 00:00:00 +0000

Interactions between machine learning, artificial neural networks and our understanding of biological vision

Laurent Perrinet

[2023-05-10] NeuroSchool PhD Program in Neuroscience: Computation Neuroscience

Contact me @ laurent.perrinet@univ-amu.fr

Principles of Vision

[Marr, 1982]

What is the function of vision?

An Unexpected Visitor (Ilya Repin, 1884)

What is the function of vision?

An Unexpected Visitor (Yarbus, 1965)

What is the function of vision?

An Unexpected Visitor - Age? (Yarbus, 1965)

What is the function of vision?

An Unexpected Visitor - How long? (Yarbus, 1965)

Visual illusions

Rotating Snakes Akiyoshi KITAOKA

Visual illusions : Pareidolia

Cydonia Mensae, 1976, Viking Orbiter image

Visual illusions : Pareidolia

Cydonia Mensae, 2007, Mars Global Surveyor

Visual illusions : Pareidolia

Cydonia Mensae, 2007, Mars Global Surveyor

Visual illusions: Context

Ilusions of brightness or lightness Akiyoshi KITAOKA

Visual illusions: Context

Hering illusion

Visual illusions: Context

Hering illusion

Principles of vision?

[Marr, 1982]

Computational neuroscience of vision

[Marr, 1982]

Computational neuroscience of vision

[Sejnowski, Koch & Churchland, 1998]

Anatomy of the Human Visual system

Human Visual system : the HMAX model

[Serre and Poggio, 2006]

Convolutional Neural Networks : Hierarchy

[Boutin et al, 2021]

Convolutional Neural Networks (CNNs)

[Jérémie & LP, 2023]

Primary visual cortex: Hubel & Wiesel

[Hubel & Wiesel, 1962]

Primary visual cortex: Hubel & Wiesel

[Hubel & Wiesel, 1962] - from @Neuroslicer

Convolutional Neural Networks : hierarchy

[Boutin et al, 2021]

Convolutional Neural Networks : Mathematics

One-dimensional discrete convolution (eg in time) with a kernel $g$ of radius $K$: $$ (f \ast g)[n]=\sum_{m=-K}^{K} f[n-m] \cdot g[m] $$

Convolutional Neural Networks : Mathematics

Convolution of an image (two-dimensional) with a kernel $g$ of radius $K\times K$:

$$ (f \ast g)[x, y] = \sum_{i=-K}^{K} \sum_{j=-K}^{K} f[x-i, y-j] \cdot g[i, j] $$

Convolutional Neural Networks : Mathematics

Cross-correlation of an image (two-dimensional) with a kernel $g$ of radius $K\times K$:

$$ (f \ast \tilde{g})[x, y] = \sum_{i=-K}^{K} \sum_{j=-K}^{K} f[x+i, y+j] \cdot g[i, j] $$

Convolutional Neural Networks : Mathematics

[Amidi & Amidi]

Convolutional Neural Networks : Mathematics

Correlation of an image defined on several channels (note the order of the indices):

$$ (f \ast \tilde{g})[x, y] = \sum_{c=1}^{C} \sum_{i,j} f[c, x+i, y+j] \cdot g[c, i, j] $$

Convolutional Neural Networks : Mathematics

Correlation of a multi-channel image for multiple output channels (note the order of the indices):

$$ (f \ast \tilde{g})[k, x, y] = \sum_{c=1}^{C} \sum_{i,j} f[c, x+i, y+j] \cdot g[k, c, i, j] $$

Convolutional Neural Networks : Predictive coding

[Boutin et al, 2021]

Convolutional Neural Networks : Predictive coding

[Boutin et al, 2021]

Convolutional Neural Networks : Topography

[Bosking et al, 1997]

Convolutional Neural Networks : Topography

[Boutin et al, 2022]

Computational neuroscience of vision

[Marr, 1982]

Dynamics of vision

[Marr, 1982]

Dynamics of vision

Visual latencies (see review).

Dynamics of vision

Visual latencies (see review).

Dynamics of vision

Sensorimotor delays (Perrinet & Friston 2014)

Dynamics of vision

Sensorimotor delays (Perrinet & Friston, 2014)

Dynamics of vision

Diagonal markov model (Khoei et al, 2017).

Dynamics of vision

Flash-lag effect: MBP (Khoei et al, 2017)

Dynamics of vision

[Marr, 1982]

Spiking Neural Networks

[Marr, 1982]

Spiking Neural Networks: Leaky Integrate-and-Fire Neuron

[Grimaldi et al, 2023, Precise Spiking Motifs]

Spiking Neural Networks in neurobiology

[Mainen & Sejnowski, 1995]

Spiking Neural Networks in neurobiology

[Mainen & Sejnowski, 1995]

Spiking Neural Networks in neurobiology

[Diesmann et al. 1999]

Spiking Neural Networks in neurobiology

[Haimerl et al, 2019]

Spiking Neural Networks: Spiking motifs

[Grimaldi et al, 2023, Precise Spiking Motifs]

Spiking Neural Networks: Spiking motifs

Review on Precise Spiking Motifs.

Spiking Neural Networks: Spiking motifs

Review on Precise Spiking Motifs.

Spiking Neural Networks in neuromorphic engineering

From frame-based to event-based cameras.

Spiking Neural Networks in neuromorphic engineering

The HD-SNN neural network.

Spiking Neural Networks in neuromorphic engineering

The HD-SNN neural network.

Spiking Neural Networks in neuromorphic engineering

The HD-SNN neural network.

Artificial neural networks and machine learning applied to the understanding of biological vision

[Marr, 1982]

Interactions between machine learning, artificial neural networks and our understanding of biological vision

Laurent Perrinet

[2023-05-10] NeuroSchool PhD Program in Neuroscience: Computation Neuroscience

Contact me @ laurent.perrinet@univ-amu.fr

Convolutional Sparse Coding is improved by heterogeneous uncertainty modeling

Fri, 05 May 2023 00:00:00 +0000

Accepted paper (poster) at the ICLR 2023 Workshop on Sparsity in Neural Networks:
the focus of the WS is on “On practical limitations and tradeoffs between sustainability and efficiency” in Kigali, Rwanda / May 5th 2023
reviews will be made public on https://openreview.net/forum?id=tgr8FEcl28M
In a nutshell: We found that sparse coding of images (here extended in a convolutional framework) is improved when using kernels with heterogeneous precision in how they encode orientation information. This was confirmed by learning, but also by comparison with what is observed in the statistics of natural images and in our recordings from neurons in primary visual cortex.

Epistemic uncertainty in a CSC dictionary improves both sparseness and reconstruction performance. (a) Elements from dictionaries with fixed epistemic uncertainty before (green) and after dictionary learning (orange). (b) Elements from a dictionary with heterogeneous epistemic uncertainty before (blue) and after dictionary learning (purple). (c) Elements from a dictionary learned from scratch. (d) Distribution of the sparseness (top) and Peak Signal-to-Noise Ratio (PSNR, right) of the five dictionaries, shown as a scatter plot for each of the 600 images of the dataset (center). Median values are shown as dashed line on the histograms.
This theoretical work accompanies a similar study in neurophysiology:
Hugo Ladret, Nelson Cortes, Lamyae Ikan, Frédéric Chavane, Christian Casanova, Laurent U Perrinet (2023). Cortical recurrence supports resilience to sensory variance in the primary visual cortex. Nature Communications Biology.
PDF Cite DOI Dataset Code URL bioRxiv HAL
This work was extended in
Hugo Ladret, Christian Casanova, Laurent U Perrinet (2024). Kernel Heterogeneity Improves Sparseness of Natural Images Representations. Neuromorphic Computing and Engineering.
Cite Dataset DOI URL HAL

Postdoc position "Accurate detection of precise spiking motifs in neurobiological data"

Mon, 01 May 2023 09:00:00 +0000

THE POSITION HAS BEEN FILLED.

Dear colleagues,

Applications are welcome for a fully funded 18-month postdoctoral position for the development of an algorithm for the accurate detection of precise spiking motifs in neurobiological data. The position will be located at the INT in Marseille, France. The project is funded by the polychronies grant (AMX-21-RID-025) and coordinated by Laurent Perrinet together with Thomas Schatz (theory) and Rosa Cossart (neurobiology).

Candidates should have experience in computational neuroscience, physics, engineering, or related fields, and a strong background in machine learning. The candidate must have good computer science skills (programming skills, git versioning, …) and Python programming experience is required. A multidisciplinary background would be highly appreciated, especially an advanced knowledge of mathematics. The candidate must have a strong interest in neuroscience. The candidate must be fluent in English and willing to proactively interact with partners in different communities, including theoretical neuroscience, machine learning, or neurobiology. The preferred candidate should have the ability to work independently and be flexible to adapt to the working methods of the supervisors.

More details on the “polychronies” grant:
Rosa Cossart, Thomas Schatz, Laurent U Perrinet (2022). Polychronies (2022 / 2025).
Our recent review on Precise spiking motifs in neurobiological and neuromorphic data:
Antoine Grimaldi, Amélie Gruel, Camille Besnainou, Jean-Nicolas Jérémie, Jean Martinet, Laurent U Perrinet (2022). Precise spiking motifs in neurobiological and neuromorphic data.
Cite DOI Hal Code URL arXiv
Application of detecting spiking motifs in neuromorphic data:
Antoine Grimaldi, Laurent U Perrinet (2023). Learning heterogeneous delays in a layer of spiking neurons for fast motion detection. Biological Cybernetics.
PDF Cite DOI URL Code Supplementary Material
A theoretical framework on the accurate (supervised) detection of spiking motifs in (synthetic) multi-unit raster plots
Laurent U Perrinet (2023). Accurate Detection of Spiking Motifs by Learning Heterogeneous Delays of a Spiking Neural Network. ICANN Special Session on Recent Advances in Spiking Neural Networks.
PDF Cite Slides DOI Code arXiv
job offer posted on: Euraxess - jobrXiv - academic positions

Research context

The position will be carried out in the team “NEuronal OPerations in visual TOpographic maps” (NeOpTo) within the Institut de Neurosciences de la Timone in Marseille, a welcoming and lively town by the Mediterranean Sea in the south of France. The research team is led by F. Chavane (DR, CNRS) and currently hosts 4 permanent staff, 3 post-docs and 4 PhD students. The research themes of the team are focused on neuronal operations within visual cortical maps. Indeed, along the cortical hierarchy, low-level features such as the position and orientation of the visual stimulus (but also auditory tone, somatosensory touch, etc…) but also higher-level features (such as faces, viewpoints of objects, etc…) are represented topographically on the cortical surface.

2023-04-05-ue-neurosciences-computationnelles

Wed, 05 Apr 2023 00:00:00 +0000

Artificial neural networks and machine learning applied to the understanding of biological vision

Laurent Perrinet

[2023-04-05] Master 1 Neurosciences et Sciences Cognitives.

Contact me @ laurent.perrinet@univ-amu.fr

Principles of Vision

[Marr, 1982]

What is the function of vision?

An Unexpected Visitor (Ilya Repin, 1884)

What is the function of vision?

An Unexpected Visitor (Yarbus, 1965)

What is the function of vision?

An Unexpected Visitor - Age? (Yarbus, 1965)

What is the function of vision?

An Unexpected Visitor - How long? (Yarbus, 1965)

Visual illusions

Ilusions of brightness or lightness Akiyoshi KITAOKA

Visual illusions

Hering illusion

Visual illusions

Hering illusion

Visual illusions

Rotating Snakes Akiyoshi KITAOKA

Visual illusions : Pareidolia

Cydonia Mensae (1976) Viking Orbiter image

Visual illusions : Pareidolia

Cydonia Mensae (2007) Mars Global Surveyor

Visual illusions : Pareidolia

Cydonia Mensae (2007) Mars Global Surveyor

Principles of vision?

[Marr, 1982]

Computational neuroscience of vision

[Marr, 1982]

Computational neuroscience of vision

[Sejnowski, Koch & Churchland (1998)]

Anatomy of the Human Visual system

Human Visual system : the HMAX model

[Serre and Poggio, 2007]

Primary visual cortex: Hubel & Wiesel

[Hubel & Wiesel, 1962]

Primary visual cortex: Hubel & Wiesel

[Hubel & Wiesel, 1962]

Convolutional Neural Networks : Hierarchy

[Boutin et al, 2021]

Convolutional Neural Networks : Mathematics

One-dimensional discrete convolution (eg in time) with a kernel $g$ of radius $K$: $$ (f \ast g)[n]=\sum_{m=-K}^{K} f[n-m] \cdot g[m] $$

Convolutional Neural Networks : Mathematics

Convolution of an image (two-dimensional) with a kernel $g$ of radius $K\times K$:

$$ (f \ast g)[x, y] = \sum_{i=-K}^{K} \sum_{j=-K}^{K} f[x-i, y-j] \cdot g[i, j] $$

Convolutional Neural Networks : Mathematics

Cross-correlation of an image (two-dimensional) with a kernel $g$ of radius $K\times K$:

$$ (f \ast \tilde{g})[x, y] = \sum_{i=-K}^{K} \sum_{j=-K}^{K} f[x+i, y+j] \cdot g[i, j] $$

Convolutional Neural Networks : Mathematics

[Amidi & Amidi]

Convolutional Neural Networks : Mathematics

Correlation of an image defined on several channels (note the order of the indices):

$$ (f \ast \tilde{g})[x, y] = \sum_{c=1}^{C} \sum_{i,j} f[c, x+i, y+j] \cdot g[c, i, j] $$

Convolutional Neural Networks : Mathematics

Correlation of a multi-channel image for multiple output channels (note the order of the indices):

$$ (f \ast \tilde{g})[k, x, y] = \sum_{c=1}^{C} \sum_{i,j} f[c, x+i, y+j] \cdot g[k, c, i, j] $$

Convolutional Neural Networks : the HMAX model

[Serre and Poggio, 2006]

Convolutional Neural Networks (CNNs)

[Jérémie & LP, 2023]

Convolutional Neural Networks : hierarchy

[Boutin et al, 2021]

Convolutional Neural Networks : Predictive coding

[Boutin et al, 2021]

Convolutional Neural Networks : Predictive coding

[Boutin et al, 2021]

Convolutional Neural Networks : Topography

[Bosking et al, 1997]

Convolutional Neural Networks : Topography

[Boutin et al, 2022]

Computational neuroscience of vision

[Marr, 1982]

Dynamics of vision

[Marr, 1982]

Dynamics of vision

Visual latencies (see review).

Dynamics of vision

Visual latencies (see review).

Dynamics of vision

Sensorimotor delays (Perrinet & Friston 2014)

Dynamics of vision

Sensorimotor delays (Perrinet & Friston, 2014)

Dynamics of vision

Diagonal markov model (Khoei et al, 2017).

Dynamics of vision

Flash-lag effect: MBP (Khoei et al, 2017)

Dynamics of vision

[Marr, 1982]

Spiking Neural Networks

[Marr, 1982]

Spiking Neural Networks: Leaky Integrate-and-Fire Neuron

Review on Precise Spiking Motifs.

Spiking Neural Networks in neurobiology

[Mainen & Sejnowski, 1995]

Spiking Neural Networks in neurobiology

[Mainen & Sejnowski, 1995]

Spiking Neural Networks in neurobiology

[Diesmann et al. 1999]

Spiking Neural Networks in neurobiology

[Haimerl et al, 2019]

Spiking Neural Networks: Spiking motifs

Review on Precise Spiking Motifs.

Spiking Neural Networks: Spiking motifs

Review on Precise Spiking Motifs.

Spiking Neural Networks: Spiking motifs

Review on Precise Spiking Motifs.

Spiking Neural Networks in neuromorphic engineering

From frame-based to event-based cameras.

Spiking Neural Networks in neuromorphic engineering

The HD-SNN neural network.

Spiking Neural Networks in neuromorphic engineering

The HD-SNN neural network.

Spiking Neural Networks in neuromorphic engineering

The HD-SNN neural network.

Artificial neural networks and machine learning applied to the understanding of biological vision

[Marr, 1982]

Artificial neural networks and machine learning applied to the understanding of biological vision

Laurent Perrinet

[2023-04-05] Master 1 Neurosciences et Sciences Cognitives.

Contact me @ laurent.perrinet@univ-amu.fr

2023-04-03-master-m-4-nc

Mon, 03 Apr 2023 00:00:00 +0000

Artificial neural networks and machine learning applied to the understanding of biological vision

Laurent Perrinet

[2023-04-03] Master M4NC de l’institut NeuroMod, cours Prospective Innovation and Research.

Contact me @ laurent.perrinet@univ-amu.fr

Principles of Vision

[Marr, 1982]

What is the function of vision?

An Unexpected Visitor (Ilya Repin, 1884)

What is the function of vision?

An Unexpected Visitor (Yarbus, 1965)

What is the function of vision?

An Unexpected Visitor - Age? (Yarbus, 1965)

What is the function of vision?

An Unexpected Visitor - How long? (Yarbus, 1965)

Visual illusions

Ilusions of brightness or lightness Akiyoshi KITAOKA

Visual illusions

Hering illusion

Visual illusions

Hering illusion

Visual illusions

Rotating Snakes Akiyoshi KITAOKA

Visual illusions : Pareidolia

Cydonia Mensae (1976) Viking Orbiter image

Visual illusions : Pareidolia

Cydonia Mensae (2007) Mars Global Surveyor

Visual illusions : Pareidolia

Cydonia Mensae (2007) Mars Global Surveyor

Principles of vision?

[Marr, 1982]

Computational neuroscience of vision

[Marr, 1982]

Computational neuroscience of vision

[Sejnowski, Koch & Churchland (1998)]

Anatomy of the Human Visual system

Human Visual system : the HMAX model

[Serre and Poggio, 2007]

Primary visual cortex: Hubel & Wiesel

[Hubel & Wiesel, 1962]

Primary visual cortex: Hubel & Wiesel

[Hubel & Wiesel, 1962]

Convolutional Neural Networks : Hierarchy

[Boutin et al, 2021]

Convolutional Neural Networks : Mathematics

One-dimensional discrete convolution (eg in time) with a kernel $g$ of radius $K$: $$ (f \ast g)[n]=\sum_{m=-K}^{K} f[n-m] \cdot g[m] $$

Convolutional Neural Networks : Mathematics

Convolution of an image (two-dimensional) with a kernel $g$ of radius $K\times K$:

$$ (f \ast g)[x, y] = \sum_{i=-K}^{K} \sum_{j=-K}^{K} f[x-i, y-j] \cdot g[i, j] $$

Convolutional Neural Networks : Mathematics

Cross-correlation of an image (two-dimensional) with a kernel $g$ of radius $K\times K$:

$$ (f \ast \tilde{g})[x, y] = \sum_{i=-K}^{K} \sum_{j=-K}^{K} f[x+i, y+j] \cdot g[i, j] $$

Convolutional Neural Networks : Mathematics

[Amidi & Amidi]

Convolutional Neural Networks : Mathematics

Correlation of an image defined on several channels (note the order of the indices):

$$ (f \ast \tilde{g})[x, y] = \sum_{c=1}^{C} \sum_{i,j} f[c, x+i, y+j] \cdot g[c, i, j] $$

Convolutional Neural Networks : Mathematics

Correlation of a multi-channel image for multiple output channels (note the order of the indices):

$$ (f \ast \tilde{g})[k, x, y] = \sum_{c=1}^{C} \sum_{i,j} f[c, x+i, y+j] \cdot g[k, c, i, j] $$

Convolutional Neural Networks : the HMAX model

[Serre and Poggio, 2006]

Convolutional Neural Networks (CNNs)

[Jérémie & LP, 2023]

Convolutional Neural Networks : hierarchy

[Boutin et al, 2021]

Convolutional Neural Networks : Predictive coding

[Boutin et al, 2021]

Convolutional Neural Networks : Predictive coding

[Boutin et al, 2021]

Convolutional Neural Networks : Topography

[Bosking et al, 1997]

Convolutional Neural Networks : Topography

[Boutin et al, 2022]

Computational neuroscience of vision

[Marr, 1982]

Dynamics of vision

[Marr, 1982]

Dynamics of vision

Visual latencies (see review).

Dynamics of vision

Visual latencies (see review).

Dynamics of vision

Sensorimotor delays (Perrinet & Friston 2014)

Dynamics of vision

Sensorimotor delays (Perrinet & Friston, 2014)

Dynamics of vision

Diagonal markov model (Khoei et al, 2017).

Dynamics of vision

Flash-lag effect: MBP (Khoei et al, 2017)

Dynamics of vision

[Marr, 1982]

Spiking Neural Networks

[Marr, 1982]

Spiking Neural Networks: Leaky Integrate-and-Fire Neuron

Review on Precise Spiking Motifs.

Spiking Neural Networks in neurobiology

[Mainen & Sejnowski, 1995]

Spiking Neural Networks in neurobiology

[Mainen & Sejnowski, 1995]

Spiking Neural Networks in neurobiology

[Diesmann et al. 1999]

Spiking Neural Networks in neurobiology

[Haimerl et al, 2019]

Spiking Neural Networks: Spiking motifs

Review on Precise Spiking Motifs.

Spiking Neural Networks: Spiking motifs

Review on Precise Spiking Motifs.

Spiking Neural Networks: Spiking motifs

Review on Precise Spiking Motifs.

Spiking Neural Networks in neuromorphic engineering

From frame-based to event-based cameras.

Spiking Neural Networks in neuromorphic engineering

The HD-SNN neural network.

Spiking Neural Networks in neuromorphic engineering

The HD-SNN neural network.

Spiking Neural Networks in neuromorphic engineering

The HD-SNN neural network.

Artificial neural networks and machine learning applied to the understanding of biological vision

[Marr, 1982]

Artificial neural networks and machine learning applied to the understanding of biological vision

Laurent Perrinet

[2023-04-03] Master M4NC de l’institut NeuroMod, cours Prospective Innovation and Research.

Contact me @ laurent.perrinet@univ-amu.fr

Resilience to sensory uncertainty in the primary visual cortex

Thu, 09 Mar 2023 00:00:00 +0000

This poster is presented in the following paper (published in Nature Comm Biology):
Hugo Ladret, Nelson Cortes, Lamyae Ikan, Frédéric Chavane, Christian Casanova, Laurent U Perrinet (2023). Cortical recurrence supports resilience to sensory variance in the primary visual cortex. Nature Communications Biology.
PDF Cite DOI Dataset Code URL bioRxiv HAL

Precise spiking motifs in neurobiological and neuromorphic data

Fri, 23 Dec 2022 00:00:00 +0000

read the paper online or in PDF
Video Abstract
join the Zotero group to add and discuss more items
code for paper (including revisions): https://github.com/SpikeAI/2022_polychronies-review

Core mechanism of polychrony detection. (Left) In this example, three presynaptic neurons denoted b, c and d are fully connected to two post-synaptic neurons a and e, with different delays of respectively 1, 5, and 9 ms for a and 8, 5, and 1 ms for e. (Middle) If three synchronous pulses are emitted from presynaptic neurons, this will generate post-synaptic potentials that will reach a and e asynchronously because of the heterogeneous delays, and they may not be sufficient to reach the membrane threshold in either of the post-synaptic neurons; therefore, no spike will be emitted, as this is not sufficient to reach the membrane threshold of the post synaptic neuron, so no output spike is emitted. (Right) If the pulses are emitted from presynaptic neurons such that, taking into account the delays, they reach the post-synaptic neuron a at the same time (here, at t = 10 ms), the post-synaptic potentials evoked by the three pre-synaptic neurons sum up, causing the voltage threshold to be crossed and thus to the emission of an output spike (red color), while none is emitted from post-synaptic neuron e.

more posts on reddit, RG, or HAL
see follow-up paper:
Laurent U Perrinet (2023). Accurate Detection of Spiking Motifs by Learning Heterogeneous Delays of a Spiking Neural Network. ICANN Special Session on Recent Advances in Spiking Neural Networks.
PDF Cite Slides DOI Code arXiv

Pooling in a predictive model of V1 explains functional and structural diversity across species

Mon, 18 Jul 2022 00:00:00 +0000

this paper follows this COSYNE presentation :
Angelo Franciosini, Victor Boutin, Laurent U Perrinet (2020). Modelling Complex-cells and topological structure in the visual cortex of mammals using Sparse Predictive Coding. Computational and Systems Neuroscience (Cosyne) 2020.
Cite URL
see a related work describing SDPC in:
Victor Boutin, Angelo Franciosini, Frédéric Chavane, Franck Ruffier, Laurent U Perrinet (2021). Sparse Deep Predictive Coding captures contour integration capabilities of the early visual system. PLoS Computational Biology.
PDF Cite DOI Code URL arXiv

more about the role of top-down connections:
Victor Boutin, Angelo Franciosini, Franck Ruffier, Laurent U Perrinet (2020). Effect of top-down connections in Hierarchical Sparse Coding. Neural Computation.
PDF Cite DOI URL arXiv

Recurrent cortical connectivity in the primary visual cortex supports robust encoding of natural sensory inputs

Mon, 11 Jul 2022 00:00:00 +0000

This poster is presented in the following paper (published in Nature Comm Biology):
Hugo Ladret, Nelson Cortes, Lamyae Ikan, Frédéric Chavane, Christian Casanova, Laurent U Perrinet (2023). Cortical recurrence supports resilience to sensory variance in the primary visual cortex. Nature Communications Biology.
PDF Cite DOI Dataset Code URL bioRxiv HAL

Ultra-rapid visual search in natural images using active deep learning

Sun, 10 Jul 2022 00:00:00 +0000

This work extends to natural scenes a previous work on visual search on a simplified task formulated in
Emmanuel Daucé, Pierre Albigès, Laurent U Perrinet (2020). A dual foveal-peripheral visual processing model implements efficient saccade selection. Journal of Vision.
Cite DOI Pdf Code URL bioRxiv
follows
Jean-Nicolas Jérémie, Emmanuel Daucé, Laurent U Perrinet (2022). Ultra-rapid visual search in natural images using active deep learning. Proceedings of AREADNE.
PDF Cite Venue
It is based on a first work on transfer learning and its application to a natural task :
Jean-Nicolas Jérémie, Laurent U Perrinet (2023). Ultra-Fast Image Categorization in biology and in neural models. Vision.
Cite DOI arXiv
in particular, we found retinotopic mapping to be adapted to that extension :
Jean-Nicolas Jérémie, Emmanuel Daucé, Laurent U Perrinet (2022). Retinotopic mapping improves the reliability of image classification. NeuroVision Workshop in conjunction with CVPR 2022.
PDF Cite URL

A resilient neural code in V1 to process natural images

Wed, 29 Jun 2022 00:00:00 +0000

for a follow-up, check out
Hugo Ladret, Laurent U Perrinet (2022). Recurrent cortical connectivity in the primary visual cortex supports robust encoding of natural sensory inputs. Proceedings of the FENS Forum 2022.
Cite URL
This poster is presented in the following paper (published in Nature Comm Biology):
Hugo Ladret, Nelson Cortes, Lamyae Ikan, Frédéric Chavane, Christian Casanova, Laurent U Perrinet (2023). Cortical recurrence supports resilience to sensory variance in the primary visual cortex. Nature Communications Biology.
PDF Cite DOI Dataset Code URL bioRxiv HAL

Ultra-rapid visual search in natural images using active deep learning

Wed, 29 Jun 2022 00:00:00 +0000

This work extends to natural scenes a previous work on visual search on a simplified task formulated in
Emmanuel Daucé, Pierre Albigès, Laurent U Perrinet (2020). A dual foveal-peripheral visual processing model implements efficient saccade selection. Journal of Vision.
Cite DOI Pdf Code URL bioRxiv
It is based on a first work on transfer learning and its application to a natural task :
Jean-Nicolas Jérémie, Laurent U Perrinet (2023). Ultra-Fast Image Categorization in biology and in neural models. Vision.
Cite DOI arXiv
in particular, we found retinotopic mapping to be adapted to that extension :
Jean-Nicolas Jérémie, Emmanuel Daucé, Laurent U Perrinet (2022). Retinotopic mapping improves the reliability of image classification. NeuroVision Workshop in conjunction with CVPR 2022.
PDF Cite URL
Read the corresponding paper
Jean-Nicolas Jérémie, Emmanuel Daucé, Laurent U Perrinet (2026). Foveated Retinotopy Improves Classification and Localization in CNNs. Vision.
PDF Cite DOI URL arXiv

Which sparsity problem does the brain solve?

Wed, 29 Jun 2022 00:00:00 +0000

Follow-up paper
Ilias Rentzeperis, Luca Calatroni, Laurent U Perrinet, Dario Prandi (2023). Beyond $\ell_1$ sparse coding in V1. PLOS Computational Biology.
Cite DOI URL

Contributions of neuroscience to the detection and localization of objects in visual inputs

Tue, 14 Jun 2022 15:00:00 +0000

for visual search see:
A dual foveal-peripheral visual processing model implements efficient saccade selection
Emmanuel Daucé, Pierre Albigès, Laurent U Perrinet

Cite DOI Pdf Code URL bioRxiv
for retinotopy, see:
Retinotopic mapping improves the reliability of image classification
PDF Cite URL
for event-based computations, see:
Learning heterogeneous delays of Spiking Neurons for motion detection
PDF Cite URL
for event-based motion detection, see:
Learning heterogeneous delays of spiking neurons for motion detection
Antoine Grimaldi, Camille Besnainou, Hugo Ladret, Laurent U Perrinet

PDF Cite DOI Venue URL

2022-03-23_UE-neurosciences-computationnelles

Wed, 23 Mar 2022 09:00:00 +0000

Réseaux de neurones artificiels et apprentissage machine appliqués à la compréhension de la vision

Laurent Perrinet

[2022-03-23] Master 1 Neurosciences et Sciences Cognitives

Principes de la Vision

À quoi sert la vision?

An Unexpected Visitor (Ilya Repin, 1884)

À quoi sert la vision?

An Unexpected Visitor (Yarbus, 1965)

À quoi sert la vision?

An Unexpected Visitor - Age? (Yarbus, 1965)

À quoi sert la vision?

An Unexpected Visitor - How long? (Yarbus, 1965)

Les illusions visuelles

Hering illusion

Les illusions visuelles

Hering illusion

Les illusions visuelles

Ilusions of brightness or lightness Akiyoshi KITAOKA

Les illusions visuelles

Rotating Snakes Akiyoshi KITAOKA

Les illusions visuelles : Paréidolie

Cydonia Mensae (1976) Viking Orbiter image

Les illusions visuelles : Paréidolie

Cydonia Mensae (2007) Mars Global Surveyor

Les illusions visuelles : Paréidolie

Cydonia Mensae (2007) Mars Global Surveyor

Les neurosciences computationnelles

[Sejnowski, Koch & Churchland (1998)]

De V1 aux réseaux convolutionnels

Le système visuel

Système visuel humain (Wikipedia)

Le cortex visuel primaire

[Hubel & Wiesel, 1962]

Hubel & Wiesel

[Hubel & Wiesel, 1962]

Réseaux convolutionnels : hiérarchie

[Boutin et al, 2021]

Réseaux convolutionnels : Math

Convolution discrète uni-dimensionnelle (eg dans le temps) avec un noyau f de rayon $K$: $$ (f \ast g)[n]=\sum_{m=-K}^{K} f[m] g[n-m] $$

Réseaux convolutionnels : Math

Convolution discrète d’une image (bi-dimensionnelle):

$$ (f \ast g)[x, y] = \sum_{i=-K}^{K} \sum_{j=-K}^{K} f[i, j] g[i-x, j-y] $$

Réseaux convolutionnels : l’opération de convolution

[Amidi & Amidi]

Réseaux convolutionnels : Math

Convolution discrète d’une image sur plusieurs canaux de sortie:

$$ (f \ast g)[x, y, k] = \sum_{i=-K}^{K} \sum_{j=-K}^{K} f[k, i, j, k] g[i-x, j-y] $$

Réseaux convolutionnels : Math

Convolution discrète d’une image multi-canaux (eg. RGB) sur plusieurs canaux de sortie (noter l’ordre des indices):

$$ (f \ast g)[x, y, k] = \ \sum_{i=-K}^{K} \sum_{j=-K}^{K} \sum_{c=1}^{C} f[k, c, i, j] g[i-x, j-y, c] $$

Réseaux convolutionnels : CNN

[Amidi & Amidi]

Mise en pratique: détecter & apprendre

Perspectives

Réseaux convolutionnels : hiérarchie

[Boutin et al, 2021]

Réseaux prédictifs

[Boutin et al, 2021]

Topographie dans V1

[Bosking et al, 1997]

Spiking Neural Networks

From frame-based to event-based cameras.

Recurrent processing

[Amidi & Amidi]

Dynamique de la vision

[Thorpe (2001)]

Applications robotiques

Our system is divided into 3 units to process visual inputs communicating by event-driven, feed-forward and feed-back communications.

Questions?

Ask info @ laurent.perrinet@univ-amu.fr

More info @ web-site

Statistics of the sparse representations of natural images

Tue, 22 Mar 2022 00:00:00 +0000

see previous work: https://laurentperrinet.github.io/sciblog/posts/2018-11-05-statistics-of-the-natural-input-to-a-ring-model.html

Mini-Symposium “Learning from vision: Efficient representation, sparse coding, and modelling”

Although recent years have seen a striking improvement in imaging techniques, there are many tasks for which human interaction is still essential, as color gamut correction in the cinema industry. This suggests that a better understanding of the mechanisms underlying the visual system is instrumental to advances in imaging techniques. Along these lines, various ideas from computational neurosciences have found application in imaging, from pattern recognition to image inpainting. A promising line of investigation is built on methods based on models of the primary visual cortex and on neural coding, in particular via the efficient representation principle. These methods have recently allowed to define new artificial neural networks paradigms and to reproduce complex visual illusions. In this mini-symposium we aim to gather together experts working in the field of mathematical neuroscience and imaging, with a focus on these methods. In particular, the speakers will present recent results based on sparse coding and models of the visual system.

Organizer: Dario Prandi

12:40-1:05 The intrinsically nonlinear nature of receptive fields in vision: implications for imaging, vision science and artificial neural networks Marcelo Bertalmío, Spanish National Research Council, Spain
1:10-1:35 ChebLieNet: Invariant Spectral Graph Nns Turned Equivariant by Sub-Riemannian Geometry on Lie Groups Erik Bekkers, University of Amsterdam, Netherlands
1:40-2:05 Deep Predictive Coding for More Robust and Human-Like Vision Rufin VanRullen, Centre de Recherche Cerveau et Cognition (CerCo), France
2:10-2:35 Statistics of the Sparse Representations of Natural Images Hugo Ladret and Laurent U. Perrinet, CNRS & Aix-Marseille Université, Marseille, France More on https://meetings.siam.org/sess/dsp_programsess.cfm?sessioncode=73028

Revisiting Horizontal Connectivity Rules in V1: From like-to-like towards like-to-All

Sat, 05 Feb 2022 00:00:00 +0000

Check-out this presentation of the paper:

When Cortical Neurons Talk Sideways: Beyond Feedforward Visual Processing

In this seminar we will challenge the traditional understanding of neuronal connectivity in primary visual cortex. While current theory suggests that neurons connect preferentially to others with similar orientation preferences, I will present evidence for a more complex connectivity pattern based on a distance-dependent rule: short-range connections show a like-to-like bias, while long-range connections connect more widely. This revised model better explains how the visual cortex processes complex stimuli and accounts for observed variations in neuronal interactions at different scales.

Laurent U Perrinet
2025-02-11 NeuroAI & Machine Learning

Cite Slides URL

Decoding orientation distributions from noisy observations in V1

Fri, 15 Oct 2021 00:00:00 +0000

This poster is presented in the following paper (published in Nature Comm Biology):
Hugo Ladret, Nelson Cortes, Lamyae Ikan, Frédéric Chavane, Christian Casanova, Laurent U Perrinet (2023). Cortical recurrence supports resilience to sensory variance in the primary visual cortex. Nature Communications Biology.
PDF Cite DOI Dataset Code URL bioRxiv HAL

SDPC : A Sparse and Predictive Model of the Early Visual System

Tue, 28 Sep 2021 00:00:00 +0000

Simulating anticipatory activity in a 1D Spiking Neural Network Model

Wed, 22 Sep 2021 00:00:00 +0000

poster number: 94
scheduled on Wednesday, Sep 22, 18:00 CEST.
https://abstracts.g-node.org/conference/BC21/abstracts#/uuid/05f81f30-d5d5-4467-b977-f28e9bed65f0

Soutenance de thèse Angelo Franciosini

Thu, 09 Sep 2021 09:00:00 +0000

“SDPC : a sparse and predictive model of the early visual system” Soutenance de thèse Angelo Franciosini

Date : Mardi 28 septembre 2021 à 13h (CEST)
Lieu: en virtuel et salle Henri Gastaut, au rez de chaussée de l’INT (how to get there). La thèse était suivie d’un pot au R+4 de l’Institut de Neurosciences de la Timone (how to get there)
Quoi: le manuscrit sera disponible après la soutenance.

Jury

Anthony Burkitt, University of Melbourne, Rapporteur
Thomas Serre, Brown University, Rapporteur
Laura Dugué, Integrative Neuroscience & Cognition Center, Examinateur
Emmanuel Daucé, CNRS, Examinateur
Stéphane Viollet, CNRS, Examinateur
Laurent Perrinet, CNRS, Directeur de thèse

Abstract

One goal of visual neuroscience is to understand how the brain interprets sensory information and to describe cortical representations according to a specific computational model. In this thesis, we describe how a successful model for visual perception, Predictive Coding (PC), can be extended to account for highly nonlinear operations in the primary visual cortex of mammals (V1). In this thesis, we generalize PC in a convolutional network and propose an algorithm called Sparse Deep Predictive Coding (SDPC), which models the properties of the early visual cortex. We present the SDPC framework in two scientific articles: in the first, we use our network to model local interactions in the early visual system (V1/V2) and we show how feedback connectivity allows the visual system to adapt to the statistics of natural images. In a second article, we show that the SDPC can predict the emergence of nonlinear responses in V1 (complex cells) and explain the link between complex cells and higher-level structures like cortical orientation maps, across species. Finally, we will propose some extensions that will allow the SDPC to serve as a general model of the visual system.

Résumé

Un des objectifs des neurosciences visuelles est de comprendre comment le cerveau interprète les informations sensorielles et de décrire les représentations corticales grâce à un modèle computationnel. Dans cette thèse, nous décrivons comment un modèle de perception visuelle, le Codage Prédictif, peut être étendu pour rendre compte des opérations non linéaires dans le cortex visuel primaire des mammifères (V1). Dans cette thèse, nous généralisons le Codage Prédictif dans un réseau convolutif pour créer un modèle appelé Sparse Deep Predictive Coding (SDPC). Nous présentons le SDPC dans deux articles scientifiques : dans le premier, nous utilisons notre réseau pour modéliser les interactions locales dans le système visuel précoce (V1/V2) et nous montrons comment la connectivité de rétroaction permet au système visuel de s’adapter aux statistiques des images naturelles. Dans un second article, nous montrons que le SDPC peut prédire l’émergence de réponses non linéaires dans V1 (cellules complexes) et expliquer le lien entre cellules complexes et des structures de plus haut niveau comme les cartes d’orientation corticales, et ceci pour différentes espèces. Enfin, nous proposerons quelques extensions qui permettront au SDPC de servir de modèle général du système visuel.

Dynamical processing of orientation precision in the primary visual cortex

Fri, 27 Aug 2021 00:00:00 +0000

This is 40th edition of Dynamicsdays
Nice, 23-27 August 2021 - https://dynamicsdays2021.univ-cotedazur.fr
check out the book of abstracts
In this talk, we will present the following paper :
Preliminary Program:
- Bruno Cessac, The Retina as a Dynamical System
- Hugo Ladret & Laurent Perrinet, Dynamics of the processing of orientation precision in the primary visual cortex
- Gianluigi Mongillo, Glassy phase in dynamically balanced networks
- Romain Veltz, Spatial and color hallucinations in a mathematical model of primary visual cortex

Pooling in a predictive model of V1 explains functional and structural diversity across species

Tue, 15 Jun 2021 11:15:00 +0000

In this talk, I will present the following paper :
Pooling in a predictive model of V1 explains functional and structural diversity across species
Victor Boutin, Angelo Franciosini, Frédéric Chavane, Laurent U Perrinet

Cite DOI URL bioRxiv
see a follow-up in:
Sparse Deep Predictive Coding captures contour integration capabilities of the early visual system
Victor Boutin, Angelo Franciosini, Frédéric Chavane, Franck Ruffier, Laurent U Perrinet

PDF Cite DOI Code URL arXiv
more about the role of top-down connections:
Effect of top-down connections in Hierarchical Sparse Coding
Victor Boutin, Angelo Franciosini, Franck Ruffier, Laurent U Perrinet

PDF Cite DOI URL arXiv

Dynamical processing of orientation precision in the primary visual cortex

Thu, 20 May 2021 14:00:00 +0000

As presented during the NeuroFrance 2021 meeting
get the abstract book
In this talk, we will present the following paper :

Sparse Deep Predictive Coding captures contour integration capabilities of the early visual system

Tue, 26 Jan 2021 00:00:00 +0000

Fig 1. Architecture of a 2-layered SDPC model.

One often compares biological vision to a camera-like system where an image would be processed according to a sequence of successive transformations. In particular, this “feedforward” view is prevalent in models of visual processing such as deep learning. However, neuroscientists have long stressed that more complex information flow is necessary to reach natural vision efficiency. In particular, recurrent and feedback connections in the visual cortex allow to integrate contextual information in our representation of visual stimuli. These modulations have been observed both at the low-level of neural activity and at the higher level of perception.

Fig 2. Results of training SDPC on the natural images (left column) and on the face database (right column) with a feedback strength kFB = 1.

Fig 14. Illustration of the hierarchical generative model learned by the SDPC model on the face database.

In this study, we present an architecture that describes biological vision at both levels of analysis. It suggests that the brain uses feedforward and feedback connections to compare the sensory stimulus with its own internal representation. In contrast to classical deep learning approaches, we show that our model learns interpretable features.

Fig 5. Example of a 9 × 9 interaction map of a V1 area centered on neurons strongly responding to a central preferred orientation of 30°.

Fig 7. Example of a 9 × 9 interaction map of a V1 area centered on neurons strongly responding to a central preferred orientation of 45°, and colored with the relative response w.r.t. no feedback.

Moreover, we demonstrate that feedback signals modulate neural activity to promote good continuity of contours. Finally, the same model can disambiguate images corrupted by noise. To the best of our knowledge, this is the first time that the same model describes the effect of recurrent and feedback modulations at both neural and representational levels.

Fig 10. Effect of the feedback strength on noisy images from natural images database.

more about the role of top-down connections:
Victor Boutin, Angelo Franciosini, Franck Ruffier, Laurent U Perrinet (2020). Effect of top-down connections in Hierarchical Sparse Coding. Neural Computation.
PDF Cite DOI URL arXiv
presented during this talk:
Victor Boutin, Angelo Franciosini, Laurent U Perrinet (2019). From the retina to action: Predictive processing in the visual system.
Cite Slides Code URL

Modulation of orientation selectivity by orientation precision

Fri, 01 Jan 2021 00:00:00 +0000

This poster is presented in the following paper (published in Nature Comm Biology):
Hugo Ladret, Nelson Cortes, Lamyae Ikan, Frédéric Chavane, Christian Casanova, Laurent U Perrinet (2023). Cortical recurrence supports resilience to sensory variance in the primary visual cortex. Nature Communications Biology.
PDF Cite DOI Dataset Code URL bioRxiv HAL

Modelling Complex-cells and topological structure in the visual cortex of mammals using Sparse Predictive Coding

Sun, 27 Sep 2020 00:00:00 +0000

see the follow-up paper in:
Victor Boutin, Angelo Franciosini, Frédéric Chavane, Laurent U Perrinet (2022). Pooling in a predictive model of V1 explains functional and structural diversity across species. PLoS Computational Biology.
Cite DOI URL bioRxiv
see a follow-up in:
Victor Boutin, Angelo Franciosini, Frédéric Chavane, Franck Ruffier, Laurent U Perrinet (2021). Sparse Deep Predictive Coding captures contour integration capabilities of the early visual system. PLoS Computational Biology.
PDF Cite DOI Code URL arXiv
more about the role of top-down connections:
Victor Boutin, Angelo Franciosini, Franck Ruffier, Laurent U Perrinet (2020). Effect of top-down connections in Hierarchical Sparse Coding. Neural Computation.
PDF Cite DOI URL arXiv

PhD offer "Ultra-fast vision using Spiking Neural Networks"

Tue, 30 Jun 2020 09:00:00 +0000

THE POSITION HAS BEEN FILLED.

Dear colleagues,

Applications are welcome for a fully funded doctoral position at INT in Marseille, France. Your mission will be to build ultra-fast vision algorithms using event-based cameras and spiking neural networks. The project is funded by the APROVIS3D grant (ANR-19-CHR3-0008-03) and will be coordinated by Laurent Perrinet. The work will be carried out in collaboration with a leading computer science institute at Université Côte d’Azur (Sophia Antipolis, France), the Laboratoire d’Informatique, Signaux et Systèmes de Sophia-Antipolis (I3S, UMR7271 - UNS CNRS), that will be part of the supervision team. We are seeking candidates with a strong background in machine learning, computer vision and computational neuroscience.

To obtain further information, please visit https://laurentperrinet.github.io/post/2020-06-30_phd-position or contact me @ Laurent.Perrinet@univ-amu.fr. To candidate, follow instructions on the dedicated server from the CNRS.

The starting date is set to October 1st, 2020 and the appointment is for 36 months. Applications are welcome immediately.

Thanks for distributing this announcement to potential candidates!

Detailed description: “Ultra-fast vision using Spiking Neural Networks”

Biological vision is surprisingly efficient. To take advantage of this efficiency, Deep learning and convolutional neural networks (CNNs) have recently produced great advances in artificial computer vision. However, these algorithms now face multiple challenges: learned architectures are often not interpretable, disproportionally energy greedy, and often lack the integration of contextual information that seems optimized in biological vision and human perception. Crucially, given an equal constraint on energy consumption, these algorithms are relatively slow compared to biological vision. It is believed that one major factor of this rapidity is the fact that visual information is represented by short pulses (spikes) at analog – not discrete – times (Paugam and Bohte, 2012). However, most classical computer vision algorithms rely on such frame-based approaches. One solution to overcome their limitations is to use event-based representations, but these still lack in practice, and their high potential is largely underexploited. Inspired by biology, the project addresses the scientific question of developing a low-power sensing architecture for the processing of visual scenes, able to function on analog devices without a central clock and aimed at being validated in real-life situations. More specifically, the project will develop new paradigms for biologically inspired computer vision (Cristobal, Keil and Perrinet, 2015), from sensing to processing, in order to help machines such as Unmanned Autonomous Vehicles (UAV), autonomous vehicles, or robots gain high-level understanding from visual scenes.

In this doctoral project, we propose to address major limitations of classical computer vision by implementing specific dynamical features of cortical circuits: spiking neural networks (Perrinet, Thorpe and Samuelides, 2004; Lagorce et al., 2018), lateral diffusion of neural information (Chavane et al., 2011; Muller et al., 2018) and dynamic neuronal association fields (Frégnac et al., 2012; Frégnac et al., 2016; Gerard-Mercier et al., 2016). One starting point is to use event-based cameras (Dupeyroux et al., 2018) and to extend results of self-supervised learning that we have obtained on static, natural images (Boutin et al., 2020) showing in a recurrent cortical-like artificial CNN architecture the emergence of interactions which phenomenologically correspond to the “association field” described at the psychophysical (Field et al., 1993), spiking (Li and Gilbert, 2002) and synaptic (Gerard-Mercier et al., 2016) levels. Indeed, the architecture of primary visual cortex (V1), the direct target of the feedforward visual flow, contains dense local recurrent connectivity with sparse long-range connections (Voges and Perrinet, 2012). Such connections add to the traditional convolutional kernels representing feedforward and local recurrent amplification a novel lateral interaction kernel within a single layer (across positions and channels). It is not well understood, but probably decisive for ultra-fast vision, how recurrent cortico-cortical loops add a level of distributed top-down complexity in the feed-forward stream of information which participates to the ultra-fast integration of sensory input and perceptual context (Keller et al., 2019). Coupled with the dynamics of cortical circuits, this elaborate multiplexed architecture provides the conditions possible for defining ultra-fast vision algorithms.

Expected profile of the candidate

Candidates should have experience in the domain of computational neuroscience, physics, engineering or related, and a solid training in machine learning and computer vision.

The candidate has to show good skills in computer science (programming skills, architecture understanding, git versioning, …), and in image processing methods. Good command of programming tools (Python scripting) is required. Multidisciplinary background would be strongly appreciated and in particular an advanced knowledge in mathematics, for a deep understanding of signal processing methods, along with strong computational skills. The candidate needs to show a keen interest in neuroscience. It is a bonus if the candidate is curious about neuroscience and visual perception.

The candidate has to fluently speak English to understand publications and to attend international conferences and workshops and pro-actively interact with partners in France, Switzerland, Spain and Greece. The preferred candidate will have the ability to work autonomously, and needs to be flexible to comply with the working method of the supervisors.

Research context

The thesis will be carried out in the team “NEuronal OPerations in visual TOpographic maps” (NeOpTo) within the Institut de Neurosciences de la Timone in Marseille, a welcoming and lively town by the Mediterranean sea in the south of France. The research team is led by F. Chavane (DR2, CNRS) and currently hosts 4 permanent staff, 3 post-docs and 4 PhD students. The research themes of the team are focused on neuronal operations within visual cortical maps. Indeed, along the cortical hierarchy, low-level features such as the position and orientation of the visual stimulus (but also auditory tone, somatosensory touch, etc…) but also higher-level features (such as faces, viewpoints of objects, etc…) are represented topographically on the cortical surface.

This work will be conducted in direct collaboration with Jean Martinet who will co-supervise the thesis. We will develop these algorithms in collaboration with Ryad Benosman (Université Pierre et Marie Curie) and Stéphane Viollet (équipe biorobotique, Institut des Sciences du Mouvement).

FR: Description du sujet de thèse

La vision biologique est étonnamment efficace. Pour tirer parti de cette efficacité, l’apprentissage profond et les réseaux neuronaux convolutionnels (CNN) ont récemment permis de réaliser de grandes avancées en matière de vision artificielle par ordinateur. Cependant, ces algorithmes sont aujourd’hui confrontés à de multiples défis : les architectures apprises sont souvent peu interprétables, sont démesurément gourmandes en énergie, n’intègrent généralement pas les informations contextuelles qui semblent parfaitement adaptées à la vision biologique et à la perception humaine. Aussi ces algorithmes sont relativement lents -à consommation énergétique égale- par rapport à la vision biologique. On pense qu’un facteur majeur de cette rapidité est le fait que l’information est représentée par de courtes impulsions à des moments analogiques - et non discrets. Toutefois, les algorithmes de vision par ordinateur utilisant une telle représentation dans des réseaux de neurones impulsionnels font encore défaut dans la pratique, et son important potentiel est largement sous-exploité. Ce projet, qui est inspiré de la biologie, aborde la question scientifique du développement d’une architecture ultra-rapide de détection et de traitement de scènes visuelles, fonctionnant sur des appareils sans horloge centrale, et visant à valider ce genre d’algorithmes événementiels dans des situations réelles. Plus spécifiquement, le projet développera de nouveaux paradigmes pour une vision d’inspiration biologique, de la détection au traitement, afin d’aider des machines telles que les robots aériens autonomes (UAV), les véhicules autonomes ou les robots à acquérir une compréhension de haut niveau des scènes visuelles.

FR: Contexte de travail

La thèse sera effectuée dans l’équipe “NEuronal OPerations in visual TOpographic maps” (NeOpTo) au sein de l’Institut de Neurosciences de la Timone (INT). L’équipe de recherche est dirigée par F. Chavane (DR2, CNRS) et accueille actuellement 4 personnels permanents, 3 post-doctorants et 4 doctorants. Les thématiques de recherche de l’équipe sont centrées sur les opérations neuronales au sein de cartes corticales visuelles. En effet, le long de la hiérarchie corticale, les caractéristiques de bas niveau telles que la position, l’orientation du stimulus visuel (mais aussi la tonalité auditive, le toucher somatosensoriel, etc…) mais aussi les caractéristiques de niveau supérieur (telles que les visages, les points de vue d’objets, etc…) sont représentées topographiquement sur la surface corticale.

Cette thèse sera menée en collaboration directe avec Jean Martinet qui co-supervisera cette thèse. Nous développerons ces algorithmes en collaboration avec Ryad Benosman (Université Pierre et Marie Curie) et Stéphane Viollet (équipe biorobotique, Institut des Sciences du Mouvement).

References

Boutin, Victor, Angelo Franciosini, Frédéric Chavane, Franck Ruffier, and Laurent U Perrinet. (2019). “Sparse Deep Predictive Coding captures contour integration capabilities of the early visual system.” arXiv
Julien Dupeyroux, Victor Boutin, Julien R Serres, Laurent U Perrinet, Stéphane Viollet. (2018). “M2APix: a bio-inspired auto-adaptive visual sensor for robust ground height estimation.” ISCAS
Chavane, F., Sharon, D., Jancke, D., Marre, O., Frégnac, Y. and Grinvald, A. (2011). “Lateral spread of orientation selectivity in V1 is controlled by intracortical cooperativity.” Journal of Physiology Paris 94 (5-6): 333–42.
Gabriel Cristóbal, Laurent U Perrinet, Matthias S Keil (2015). “Biologically Inspired Computer Vision.” Wiley.
Field, D.J., Hayes, A. and Hess, R.F. (1993). “Contour integration by the human visual system: Evidence for a local “association field”.” Vision Research 33 (2), pp. 173-193.
Gerard-Mercier, Florian, Pedro V Carelli, Marc Pananceau, Xoana G Troncoso, and Yves Frégnac. (2016). “Synaptic Correlates of Low-Level Perception in V1.” Journal of Neuroscience 36 (14): 3925–42.
Keller, A., Roth, M.M. and Scanziani, M. (2019). 2019. “The feedback receptive field of neurons in the mammalian primary visual cortex.” American Society for Neuroscience Abstracts, 403.13. Chicago.
Lagorce, X., Orchard, G., Galluppi, F., Shi, B. E., & Benosman, R. B. (2016). “HOTS: a hierarchy of event-based time-surfaces for pattern recognition.” IEEE transactions on pattern analysis and machine intelligence.
Li W, Piëch V, Gilbert CD (2006). “Contour saliency in primary visual cortex.” Neuron, 50(6):951–962.
Muller, Lyle, Frédéric Chavane, John Reynolds, and Terrence J Sejnowski. (2018). “Cortical Travelling Waves: Mechanisms and Computational Principles.” Nature Reviews Neuroscience 19 (5): 255.
Hélène Paugam-Moisy, Sander M. Bohte. (2012). “Computing with Spiking Neuron Networks.” Handbook of Natural Computing, Springer-Verlag, pp.335-376, 2012
Laurent U Perrinet, Manuel Samuelides, Simon J Thorpe. (2004). “Coding static natural images using spiking event times: do neurons cooperate?” IEEE Transactions on Neural Networks.
Tang, Hanlin, Martin Schrimpf, William Lotter, Charlotte Moerman, Ana Paredes, Josue Ortega Caro, Walter Hardesty, David Cox, and Gabriel Kreiman. (2018). “Recurrent computations for visual pattern completion.” Proceedings of the National Academy of Sciences 115 (35) 8835-8840.
Voges, Nicole, and Laurent U Perrinet. (2012). “Complex Dynamics in Recurrent Cortical Networks Based on Spatially Realistic Connectivities.” Frontiers in Computational Neuroscience 6.

A dual foveal-peripheral visual processing model implements efficient saccade selection

Fri, 05 Jun 2020 00:00:00 +0000

for a more mathematical treatment, see
Emmanuel Daucé, Laurent U Perrinet (2020). Visual search as active inference. IWAI 2020.
PDF Cite DOI Slides Pdf Code URL

ANR ShootingStar (2021/2024)

Mon, 27 Apr 2020 00:00:00 +0000

The natural visual environments in which we have evolved have shaped and constrained the neural mechanisms of vision. Rapid progress has been made in recent years in understanding how the retina and visual cortex are specifically adapted to processing natural scenes.1–3 However, studies in this research tradition have mainly addressed the processing of natural images in the spatial domain. Although the processing of temporal properties of visual stimuli is just as important as spatial properties, stimuli with naturalistically valid temporal dynamics have not been sufficiently investigated. Although objects and creatures we view undergo a variety of intrinsic movements, probably the most common motions on the retina are image shifts due to our own eye movements: in free viewing in humans, ocular saccades occur about three times every second, shifting the retinal image at speeds of 100-500 degrees of visual angle per second.4 How these very fast shifts are suppressed, leading to clear, accurate and stable representations of the visual scene is an fundamental unsolved problem in visual neuroscience known as saccadic suppression. One reason why this problem is difficult is technological: to make progress we need to visually simulate these fast retinal shifts, but computer displays have been too slow to produce adequate simulations.

In this project we propose a unique convergence between neurophysiology, modeling and psychophysics, aided by recent technological developments. Some of the partners have been at the forefront of recent developments that have led to a realization that moving stimuli lead to traveling waves of activity in primary visual cortex, propagating at speeds similar to those produced by saccades. Other partners have developed detailed models of the retina and primary visual cortex based on multielectrode recordings from the retina and optical imaging of the cortex that have been able to account for these wave phenomena. Finally, another partner recently made psychophysical observations—aided by new, ultrafast computer displays that allow us to realistically simulate saccadic dynamics on a static retina—that show how image dynamics alone can account for saccadic suppression phenomena.

We expect that the convergence of these three research currents and methodologies will lead to rapid progress in understanding how the visual system is adapted to naturalistic dynamics. The psychophysical observations will provide new leads and targets for the neurophysiology and modeling, which in turn may provide detailed neural explanations for the psychophysics. Our main hypothesis is that the neural architectures that have been uncovered in the retina and the primary visual cortex will be revealed as most effective when processing naturalistic, fast stimuli that arise as the consequence of eye movements.

carte d’identité du projet

Durée: 4 ans, à partir du 1er avril 2021
Budget total (partenaire français): 665 k€
Coordinateur Scientifique : Mark WEXLER (CNRS‐INCC)
Partenaire(s) : AGENCE NATIONALE DE LA RECHERCHE
Responsable Scientifique INT : Frédéric Chavane (UMR7289)

Acknowledgement

This work was supported by ANR project “ShootingStar” N° ANR-XX-XXX-XXXX.

From the retina to action: Understanding visual processing

Fri, 03 Apr 2020 16:00:00 +0000

2020-04_UE-neurosciences-computationnelles, matériel pour le cours de modélisation

Où: Marseille (France)
Quoi: Master Neurosciences et Sciences Cognitives
But de ce travail: lire un article scientifique, pouvoir le reproduire avec des simulations d’un neurone et afin d’améliorer sa compréhension.
Modalités: les étudiants s’organisent seuls, en binome ou en trinome pour fournir un mémoire sous forme de notebook complété à partir du modèle qui est fourni. Suivez les balises TODO dans le notebook pour vous guider dans cette rédaction. Les commentaires doivent être fait en français (ou en anglais si nécessaire) dans le notebook (n’oubliez-pas de sauver vos changements) et envoyé par e-mail à mailto:laurent.perrinet@univ-amu.fr une fois votre travail fini (de préférence avant le 31 avri).
Outils nécessaires: Jupyter, avec numpy et matplotlib. Ce sont des outils standard et qui sont facilement installables sur toute plateforme. Si vous avez des problèmes, me joindre par e-mail 👇

Modelling Complex-cells and topological structure in the visual cortex of mammals using Sparse Predictive Coding

Mon, 30 Mar 2020 00:00:00 +0000

see a follow-up in:
Victor Boutin, Angelo Franciosini, Frédéric Chavane, Franck Ruffier, Laurent U Perrinet (2021). Sparse Deep Predictive Coding captures contour integration capabilities of the early visual system. PLoS Computational Biology.
PDF Cite DOI Code URL arXiv
more about the role of top-down connections:
Victor Boutin, Angelo Franciosini, Franck Ruffier, Laurent U Perrinet (2020). Effect of top-down connections in Hierarchical Sparse Coding. Neural Computation.
PDF Cite DOI URL arXiv

Etude d’un Algorithme Hiérarchique de Codage Épars et Prédictif : Vers Un Modèle Bio-Inspiré de La Perception Visuelle

Fri, 13 Mar 2020 00:00:00 +0000

2020-03-13: Soutenance de thèse Victor Boutin

Wed, 04 Mar 2020 14:00:00 +0000

Date : Vendredi 13 mars à 14h
Lieu: salle Henri Gastaut, au rez de chaussée de l’INT (how to get there). La thèse était suivie d’un pot au R+4 de l’Institut de Neurosciences de la Timone (how to get there)
Quoi: le manuscrit est disponible sur http://www.theses.fr/2020AIXM0028

Jury

Ryad Benosman, Université Pierre et Marie Curie, Rapporteur
Simon Thorpe, CNRS, Rapporteur
Sandrine Anthoine, CNRS, Examinateur
Yves Frégnac, CNRS, Examinateur
Sid Kouider, CNRS, Examinateur
Laurent Perrinet, CNRS, Directeur de thèse
Franck Ruffier, CNRS, Co-directeur de thèse
Mossadek Talby, AMU, Jury invité

Abstract

Building models to efficiently represent images is a central and difficult problem in the machine learning community. The neuroscientific study of the early visual cortical areas is a great source of inspiration to find economical and robust solutions. For instance, Sparse Coding (SC) is one of the most successful frameworks to model neural computation at the local scale in the visual cortex. It directly derives from the efficient coding hypothesis and could be thought of as a competitive mechanism that describes visual stimulus using the activity of a small fraction of neurons. At the structural scale of the ventral visual pathways, feedforward models of vision have accounted for neurophysiological evidence and provide the most successful frameworks for object recognition tasks. Nevertheless, these models do not leverage the high density of feedback and lateral interactions observed in the visual cortex. In particular, these connections are known to integrate contextual and attentional modulations to feedforward signals. The Predictive Coding (PC) theory has been proposed to model top-down and bottom-up interaction between cortical regions. The presented thesis introduces a model combining Sparse Coding and Predictive Coding in a hierarchical and convolutional architecture. Our model, called Sparse Deep Predictive Coding (SDPC), was trained on several different databases including faces and natural images. We analyze the SPDC from a computational and a biological perspective. In terms of computation, the recurrent connectivity introduced by the PC framework allows the SDPC to converge to lower prediction errors with a higher convergence rate. In addition, we combine neuroscientific evidence with machine learning methods to analyze the impact of recurrent processing at both the neural organization and representational level. At the neural organization level, the feedback signal of the model accounted for a reorganization of the V1 association fields that promotes contour integration. At the representational level, the SDPC exhibited significant denoising ability which is highly correlated with the strength of the feedback from V2 to V1. These results from the SDPC model demonstrate that neuro-inspiration might be the right methodology to design more powerful and more robust computer vision algorithms.

Résumé

La représentation concise et efficace de l’information est un problème qui occupe une place centrale dans l’apprentissage machine. Le cerveau, et plus particulièrement le cortex visuel, ont depuis longtemps trouvé des solutions performantes et robustes afin de résoudre un tel problème. A l’échelle locale, le codage épars est l’un des mécanismes les plus prometteurs pour modéliser le traitement de l’information au sein des populations de neurones dans le cortex visuel. Le codage épars introduit une compétition entre les neurones afin de décrire un stimulus visuel en limitant le nombre de neurones actifs. A l’échelle structurelle, les modèles dits ascendants décrivent le cortex visuel comme une succession d’unités de traitement dans lesquelles l’information se propage de la rétine vers les couches profondes du cortex. Ces modèles ont expliqué avec succès un grand nombre de phénomènes neuro-physiologiques et ont servi d’inspiration afin de construire des algorithmes de reconnaissance d’objets extrêmement performants. Néanmoins, les modèles ascendants n’expliquent pas le grand nombre de connections récurrentes et descendantes que l’on trouve dans le cortex visuel. Ces connections sont connues pour moduler l’activité des neurones en incluant des details contextuels au flux d’information ascendant. La théorie du codage prédictif a été suggérée pour modéliser les connections ascendantes, récurrentes, et descendantes que l’on retrouve entre les différentes régions corticales. Cette thèse propose de combiner codage épars et codage prédictif au sein d’un modèle hiérarchique et convolutif. Nous avons entrainé ce modèle sur différentes bases de données afin de l’analyser avec une perspective à la fois computationnelle et biologique. D’un point de vue computationnel, nous démontrons que les connections descendantes, introduites par le codage prédictif, permettent une convergence meilleure et plus rapide du modèle. De plus, nous analysons les effets des connections descendantes sur l’organisation des populations de neurones, ainsi que leurs conséquences sur la manière dont notre algorithme se représente les images. Nous montrons que les connections descendantes réorganisent les champs d’association de neurones dans V1 afin de permettre une meilleure intégration des contours. En outre, nous observons que ces connections permettent une meilleure reconstruction des images bruitées. Nos résultats suggèrent que l’inspiration des neurosciences fournit un cadre prometteur afin de développer des algorithmes de vision artificielles plus performants et plus robustes.

Anticipatory Responses along Motion Trajectories in Awake Monkey Area V1

Wed, 01 Jan 2020 00:00:00 +0000

From the retina to action: Dynamics of predictive processing in the visual system

Wed, 01 Jan 2020 00:00:00 +0000

Find the text at https://laurentperrinet.github.io/Perrinet20PredictiveProcessing/
The source code of the text is available at https://github.com/laurentperrinet/Perrinet20PredictiveProcessing This chapter is available as part of the book “The Philosophy and Science of Predictive Processing” : List of Contributors :
Preface: The Brain as a Prediction Machine, Anil Seth
Introduction, Dina Mendonça, Manuel Curado & Steven S. Gouveia
Part I: Predictive Processing: Philosophical Approaches

Predictive Processing and Representation: How Less Can Be More, Erik Myin and Thomas van Es
A Humean Challenge to Predictive Coding, Colin Klein
Are Markov Blankets Real and Does it Matter?, Richard Menary and Alexander J. Gillett
Predictive Processing and Metaphysical Views of the Self, Robert Clowes and Klaus Gärtner

Part II: Predictive Processing: Cognitive Science and Neuroscientific Approaches

From the Retina to Action: Dynamics of Predictive Processing in the Visual System, Laurent Perrinet
Predictive Processing and Consciousness: Prediction Fallacy and its Spatiotemporal Resolution, Steven S. Gouveia
The Many Faces of Attention: Why Precision Optimization is not Attention, Sina Fazelpour and Madeleine Ransom
Predictive Processing: Does it Compute?, Chris Thornton

Part III: Predictive Processing: Mental Health

The Predictive Brain, Conscious Experience and Brain-related Conditions, Lisa Feldman Barrett and Lorena Chanes
Disconnection and Diaschisis: Active Inference in Neuropsychology, Thomas Parr and Karl Friston
The Phenomenology and Predictive Processing of Time in Depression, Zachariah Neemeh and Shaun Gallagher
Why Use Predictive Processing to Explain Psychopathology? The Case of Anorexia Nervosa, Jakob Hohwy and Stephen Gadsby

Afterword, Manuel Curado

Learning dynamics in a neural network model of the primary visual cortex

Wed, 01 Jan 2020 00:00:00 +0000

See also Ladret and Perrinet, 2019

Postdoc position on Visual computations using Spatio-temporal Diffusion Kernels and Traveling Waves

Mon, 21 Oct 2019 09:00:00 +0000

THE POSITION HAS BEEN FILLED.

Dear colleagues,

Applications are welcome for a post-doctoral position at INT in Marseille, France. Your mission will be to explore novel visual computations using spatio-temporal diffusion kernels and traveling waves. The project is funded by the ANR Horizontal V1 grant (ANR-17-CE37-0006) from the French National Research Agency (ANR) and will be coordinated by Laurent Perrinet, in collaboration with Lyle Muller and Frédéric Chavane at INT and Yves Frégnac and Jan Antolik at UNIC-NeuroPSI, Gif. We are seeking candidates with a strong background in machine learning, computer vision and computational neuroscience.

For more information, visit https://laurentperrinet.github.io/post/2019-10-28_postdoc-position.

The starting date is set to January 6th, 2020 but can be flexibly extended. To obtain further information or send applications (including a full CV, a letter of motivation, 2 reference names), please contact: Laurent.Perrinet@univ-amu.fr. The appointment is for 18 months. Applications are welcome immediately and until the end of year 2019.

Thanks for distributing this announcement to potential candidates!

Detailed description: Visual computations using Spatio-temporal Diffusion Kernels and Traveling Waves

In this post-doctoral project, we propose to address these major limitations by focusing on specific dynamical features of cortical circuits: lateral diffusion of sensory-evoked traveling waves (Chavane et al., 2011; Muller et al., 2018) and dynamic neuronal association fields (Frégnac et al., 2012; Frégnac et al., 2016; Gerard-Mercier et al., 2016). Indeed, the architecture of primary visual cortex (V1), the direct target of the feedforward visual flow, contains dense local recurrent connectivity with sparse long-range connections (Voges and Perrinet, 2012). Such connections add to the traditional convolutional kernels representing feedforward and local recurrent amplification a novel lateral interaction kernel within a single layer (across positions and channels). Less studied, but probably decisive in active vision, recurrent cortico-cortical loops add a level of distributed top-down complexity which participates to the lateral integration of sensory input and perceptual context (Keller et al., 2019). Coupled with the continuous time dynamics of cortical circuits, this elaborate multiplexed architecture provides the conditions possible for generating information diffusion through traveling waves. Inspired by recent work in neuroscience uncovering the ubiquity of these waves during visual processing, we aim to design a self-supervised CNN that will exploit these dynamics for new applications in computer vision.

The proposed work will be organized as a collaboration between two labs (INT, Marseille and UNIC, Gif) along three tasks to be integrated in a unified model:

The starting point will be to extend results of self-supervised learning that we have obtained on static, natural images (Boutin et al., 2019) showing in a recurrent cortical-like artificial CNN architecture the emergence of interactions which phenomenologically correspond to the “association field” described at the psychophysical (Field et al., 1993), spiking (Li and Gilbert, 2002) and synaptic (Gerard-Mercier et al., 2016) levels.
The central aim will be to develop a dynamical version of this feedback/lateral kernel in the context of the ANR Horizontal-V1 project, linking the two labs and confronted to their recent electrophysiological data pointing to different classes of spatio-temporal diffusion and different degree of anisotropies during apparent and continuous motion.
The implementation of this kernel inspired by CNN theory will be compared with a biologically realistic models of the early visual system (Antolik et al., 2019), and simulations of the lateral diffusion kernel will be developed in collaboration with Jan Antolik, external collaborator to the ANR grant. In parallel, using tools linking neural activity to VSD imaging (Muller et al., 2014; Chemla et al., 2019), we will analyze at a more mesocopic level the role of observed traveling waves in forming efficient representations of the visual world.

Expected profile of the candidate

Candidates should have at least a PhD degree in the domain of computational neuroscience, physics, engineering or related, and a solid training in machine learning and computer vision.

The candidate has to fluently speak English to understand publications and to attend international conferences and workshops. The preferred candidate will have the ability to work autonomously, and needs to be flexible to comply with the working method of the supervisors.

Research context

This project is funded by the French National Research Agency (ANR) under the ANR Horizontal V1 grant (coordinator Y. Frégnac) which aims at understanding the emergence of sensory predictions linking local shape attributes (orientation, contour) to global indices of movement (direction, speed, trajectory) at the earliest stage of cortical processing (primary visual cortex, i.e. V1). The cross-talk between physiological and theoretical approaches will be fostered by the close collaboration with the teams of Frédéric Chavane at INT and Yves Frégnac at UNIC. The theoretical work will be performed in close collaboration with Lyle Muller (Western U) and Jan Antolik (Prague). The project will be primarily hosted at the Institut de Neurosciences de la Timone in Marseille, a lively town by the Mediterranean sea in the south of France, but the applicant will be asked also to show mobility to visit the other partner lab when needed.

References

Antolik, J, C Monier, Y Frégnac, AP Davison. (2019). “A comprehensive data-driven model of cat primary visual cortex.” BioRxiv, 416156.
Boutin, Victor, Angelo Franciosini, Frédéric Chavane, Franck Ruffier, and Laurent U Perrinet. (2019). “Sparse Deep Predictive Coding captures contour integration capabilities of the early visual system.” arXiv
Chavane, F., C. Monier, V. Bringuier, P. Baudot, L. Borg-Graham, J. Lorenceau, and Y. Frégnac. 2000. “The Visual Cortical Association Field: A Gestalt Concept or a Psychophysiological Entity?” Frontiers in System Neuroscience 4(5): 1-26.
Chavane, F., Sharon, D., Jancke, D., Marre, O., Frégnac, Y. and Grinvald, A. (2011). “Lateral spread of orientation selectivity in V1 is controlled by intracortical cooperativity.” Journal of Physiology Paris 94 (5-6): 333–42.
Chemla, Sandrine, Alexandre Reynaud, Matteo diVolo, Yann Zerlaut, Laurent Perrinet, Alain Destexhe, and Frédéric Chavane. (2018). “Suppressive Waves Disambiguate the Representation of Long-Range Apparent Motion in Awake Monkey V1.” Journal of Neuroscience 39 (22) 4282-4298.
Field, D.J., Hayes, A. and Hess, R.F. (1993). “Contour integration by the human visual system: Evidence for a local “association field”.” Vision Research 33 (2), pp. 173-193.
Frégnac, Y. (2012) “Reading out the synaptic echoes of low-level perception in V1.” European Conference in Computer Vision 486-495. Springer, Berlin, Heidelberg.
Frégnac, Y., Fournier, J., Gerard-Mercier, F., Monier, C., Carelli, P., , M., Troncoso, X. (2016). “The Visual Brain: Computing Through Multiscale Complexity.” In Micro-, Meso- and Macro-Dynamics of the Brain pp 43-57.
Gerard-Mercier, Florian, Pedro V Carelli, Marc Pananceau, Xoana G Troncoso, and Yves Frégnac. (2016). “Synaptic Correlates of Low-Level Perception in V1.” Journal of Neuroscience 36 (14): 3925–42.
Keller, A., Roth, M.M. and Scanziani, M. (2019). 2019. “The feedback receptive field of neurons in the mammalian primary visual cortex.” American Society for Neuroscience Abstracts, 403.13. Chicago.
Kietzmann, Tim C., Courtney J. Spoerer, Lynn K. A. Sörensen, Radoslaw M. Cichy, Olaf Hauk, and Nikolaus Kriegeskorte. (2019). “Recurrence Is Required to Capture the Representational Dynamics of the Human Visual System.” Proceedings of the National Academy of Sciences, October, 201905544.
Li W, Piëch V, Gilbert CD (2006). “Contour saliency in primary visual cortex.” Neuron, 50(6):951–962.
Muller, Lyle, Alexandre Reynaud, Frédéric Chavane, and Alain Destexhe. (2014). “The Stimulus-Evoked Population Response in Visual Cortex of Awake Monkey Is a Propagating Wave.” Nature Communications 5: 3675.
Muller, Lyle, Frédéric Chavane, John Reynolds, and Terrence J Sejnowski. (2018). “Cortical Travelling Waves: Mechanisms and Computational Principles.” Nature Reviews Neuroscience 19 (5): 255.
Tang, Hanlin, Martin Schrimpf, William Lotter, Charlotte Moerman, Ana Paredes, Josue Ortega Caro, Walter Hardesty, David Cox, and Gabriel Kreiman. (2018). “Recurrent computations for visual pattern completion.” Proceedings of the National Academy of Sciences 115 (35) 8835-8840.
Voges, Nicole, and Laurent U Perrinet. (2012). “Complex Dynamics in Recurrent Cortical Networks Based on Spatially Realistic Connectivities.” Frontiers in Computational Neuroscience 6.

A dynamic model for decoding direction and orientation in macaque primary visual cortex

Mon, 23 Sep 2019 00:00:00 +0000

Wahiba Taouali

Mon, 23 Sep 2019 00:00:00 +0000

Motion Integration By V1 Population (Post-Doc, 2013-03 / 2015-01)

Description

Wahiba hold the postdoctoral position at the “Institut de Neurosciences de la Timone”, CNRS, Marseille (France) to study object motion integration and representation at the level of V1 populations:

The objective is in modeling, with Laurent Perrinet, anisotropic diffusive processes, such as observed in V1, at the functional and neural levels.
The work was done in collaboration with a post-doc in physiology, with Frédéric Chavane, that focused on the role of propagation and diffusion of activity at the level of neuronal population in V1 of awake monkeys (using Voltage-sensitive dye imaging and UTAH array recording).

Wahiba is now scientific software developper at Enthought.

Main publications:

Wahiba Taouali, Giacomo Benvenuti, Pascal Wallisch, Frédéric Chavane, Laurent U Perrinet (2016). Testing the odds of inherent vs. observed overdispersion in neural spike counts. Journal of Neurophysiology.
Cite DOI URL HAL

Context

This grant was funded by a large European integrated project called BrainScales whose aim is to understand brain information processing at multiple spatial and temporal scales. The successful applicants will have the opportunity to interact with a large and exciting consortium composed of 18 europeans teams working in biology, modeling and hardware.

References::

Reynaud A., Masson G. S. and Chavane F. Dynamics of Local Input Normalization Result from Balanced Short- and Long-Range Intracortical Interactions in Area V1 Journal of Neuroscience, 2012, 32(36): 12558-12569
Reynaud A., Takerkart S, Masson G. S. and Chavane F. Linear model decomposition for voltage-sensitive dye imaging signals: Application in awake behaving monkey. Neuroimage, 2011, 54(2), 1196–1210
Perrinet, L. and Masson G. Motion-based prediction is sufficient to solve the aperture problem Neural Computation, 2012

Learning where to look: a foveated visuomotor control model

Mon, 15 Jul 2019 12:20:00 +0000

download a preliminary PDF

Problem setting: In generic, ecological settings, the visual system faces a tricky problem when searching for one target (from a class of targets) in a cluttered environment. A) It is synthesized in the following experiment: After a fixation period of 200 ms, an observer is presented with a luminous display showing a single target from a known class (here digits) and at a random position. The display is presented for a short period of 500 ms (light shaded area in B), that is enough to perform at most one saccade (here, successful) on the potential target. Finally, the observer has to identify the digit by a keypress. B) Prototypical trace of a saccadic eye movement to the target position. In particular, we show the fixation window and the temporal window during which a saccade is possible (green shaded area). C) Simulated reconstruction of the visual information from the (interoceptive) retinotopic map at the onset of the display and after a saccade, the dashed red box indicating the visual area of the ``what’’ pathway. In contrast to an exteroceptive representation (see A), this demonstrates that the position of the target has to be inferred from a degraded (sampled) image. In particular, the configuration of the display is such that by adding clutter and reducing the size of the digit, it may become necessary to perform a saccade to be able to identify the digit. The computational pathway mediating the action has to infer the location of the target \emph{before seeing it}, that is, before being able to actually identify the target’s category from a central fixation.

Results: success

Results: failure to classify

Results: failure to locate

Des illusions aux hallucinations visuelles: une porte sur la perception

Thu, 18 Apr 2019 13:00:00 +0000

Le texte de cette présentation est reprise dans cet article de The Conversation (lien direct).
Voir la @ présentation au NeuroStories

From the retina to action: Understanding visual processing

Wed, 03 Apr 2019 16:00:00 +0000

Cours de Licence Sciences & Humanité, 3/4/2019

From the retina to action: Predictive processing in the visual system

Mon, 25 Mar 2019 14:30:00 +0000

see a follow-up in:
Sparse Deep Predictive Coding captures contour integration capabilities of the early visual system
Victor Boutin, Angelo Franciosini, Frédéric Chavane, Franck Ruffier, Laurent U Perrinet

PDF Cite DOI Code URL arXiv
more about the role of top-down connections:
Effect of top-down connections in Hierarchical Sparse Coding
Victor Boutin, Angelo Franciosini, Franck Ruffier, Laurent U Perrinet

PDF Cite DOI URL arXiv

Suppressive waves disambiguate the representation of long-range apparent motion in awake monkey V1

Mon, 18 Mar 2019 00:00:00 +0000

Sparse Deep Predictive Coding captures contour integration capabilities of the early visual system

Sun, 03 Mar 2019 00:00:00 +0000

presented during this talk
see a follow-up in:
Victor Boutin, Angelo Franciosini, Frédéric Chavane, Franck Ruffier, Laurent U Perrinet (2021). Sparse Deep Predictive Coding captures contour integration capabilities of the early visual system. PLoS Computational Biology.
PDF Cite DOI Code URL arXiv
more about the role of top-down connections:
Victor Boutin, Angelo Franciosini, Franck Ruffier, Laurent U Perrinet (2020). Effect of top-down connections in Hierarchical Sparse Coding. Neural Computation.
PDF Cite DOI URL arXiv

Should I stay or should I go? Adaption of human observers to the volatility of visual inputs

Fri, 18 Jan 2019 10:45:00 +0000

See the final publication @
Humans adapt their anticipatory eye movements to the volatility of visual motion properties
Chloé Pasturel, Anna Montagnini, Laurent U Perrinet

Cite DOI Press Pdf Code URL bioRxiv
previous talk @ LAW, Lyon
previous talk @ INVIBE FEST, Paris
previous talk @ Brain workshop, Marseille
next talk @ CAUSAL Kick-off, Marseille

Role of dynamics in neural computations underlying visual processing

Thu, 17 Jan 2019 10:45:00 +0000

Efficient coding of visual information in neural computations

Wed, 16 Jan 2019 10:45:00 +0000

Modelling spiking neural networks using Brian, Nest and pyNN

Mon, 14 Jan 2019 11:00:00 +0000

A hierarchical, multi-layer convolutional sparse coding algorithm based on predictive coding

Tue, 01 Jan 2019 00:00:00 +0000

see a follow-up in:
Victor Boutin, Angelo Franciosini, Frédéric Chavane, Franck Ruffier, Laurent U Perrinet (2021). Sparse Deep Predictive Coding captures contour integration capabilities of the early visual system. PLoS Computational Biology.
PDF Cite DOI Code URL arXiv
more about the role of top-down connections:
Victor Boutin, Angelo Franciosini, Franck Ruffier, Laurent U Perrinet (2020). Effect of top-down connections in Hierarchical Sparse Coding. Neural Computation.
PDF Cite DOI URL arXiv

An adaptive homeostatic algorithm for the unsupervised learning of visual features

Tue, 01 Jan 2019 00:00:00 +0000

“An adaptive algorithm for unsupervised learning”

supplementary info : https://spikeai.github.io/HULK/
Abstract
HTML
PDF
code for paper: https://github.com/SpikeAI/HULK
code for framework: https://github.com/bicv/SparseHebbianLearning/
code for figures https://github.com/SpikeAI/HULK/blob/master/Annex.ipynb (which is rendered @ https://spikeai.github.io/HULK/ )
video abstract (and the code for generating it)
previous publication :
Laurent U Perrinet (2010). Role of homeostasis in learning sparse representations. Neural Computation.
PDF Cite DOI Hal Code URL arXiv

$$f(k;p_{0}^{*}) = \begin{cases}p_{0}^{*} & \text{if }k=1, \\ 1-p_{0}^{*} & \text{if }k=0.\end{cases}$$

Illusions et hallucinations visuelles : une porte sur la perception

Tue, 01 Jan 2019 00:00:00 +0000

Ce texte est disponible dans cet article de The Conversation.
Voir la @ présentation au NeuroStories

Orientation selectivity to synthetic natural patterns in a cortical-like model of the cat primary visual cortex

Tue, 01 Jan 2019 00:00:00 +0000

See a followup in Ladret and Perrinet, 2020

Temps et cerveau : comment notre perception nous fait voyager dans le temps

Tue, 01 Jan 2019 00:00:00 +0000

Un article dans The Conversation dont l’objectif est d’être accessible et réutilisable (dans des cours d’introduction aux neurosciences, sciences cognitives, vision, réseaux de neurones, intelligence artificielle).
Le flash-lag effect original:
la même chose avec un arrêt:
pour illustrer la fleche du temps (" Or dans tout système, d’après le second principe de la thermodynamique, le désordre mesuré par l’entropie se doit d’augmenter. Voilà pourquoi il existe une asymétrie dans l’écoulement du temps, c’est-à-dire une flèche du temps. Résultat, si l’on filme une partie de billard, on trouvera incongru cette séquence si on la projette dans le sens inverse du temps. “), on peut aussi utiliser cette video d’un bocal qui se brise qu’il est aisé de lire dans le sens inverse du temps:
Neurostories: d’autres videos du flash-lag effect
Laurent Perrinet a reçu des financements de l’Agence Nationale de la Recherche (ANR HOR-V1 ANR-17-CE37-0006) et du CNRS (SpikeAI). Cet article n’aurait pas vu le jour sans la journée des Neurostories de la NeuroSchool d’Aix-Marseille Université, ceux qui l’ont fait vivre et parmi eux: François Féron, Alexia Belleville, Jean-Marc Michelangeli, Camille Grasso, Daniele Schön, Anne-Marie François-Bellan, Jennifer Coull, Corine Sombrun et Francis Taulelle.

Reinforcement effects in anticipatory smooth eye movements

Mon, 01 Oct 2018 00:00:00 +0000

From biological vision to unsupervised hierarchical sparse coding

Mon, 01 Jan 2018 00:00:00 +0000

accepted submission @ iTWIST: international Traveling Workshop on Interactions between low-complexity data models and Sensing Techniques, 21 - 23 November, 2018
poster session scheduled on Thursday, November 22th, from 10h30 till 12h00.
CIRM, Marseille, France.
get the full proceedings
Poster as PDF
see a follow-up in:
Victor Boutin, Angelo Franciosini, Frédéric Chavane, Franck Ruffier, Laurent U Perrinet (2021). Sparse Deep Predictive Coding captures contour integration capabilities of the early visual system. PLoS Computational Biology.
PDF Cite DOI Code URL arXiv
more about the role of top-down connections:
Victor Boutin, Angelo Franciosini, Franck Ruffier, Laurent U Perrinet (2020). Effect of top-down connections in Hierarchical Sparse Coding. Neural Computation.
PDF Cite DOI URL arXiv

On the Origins of Hierarchy in Visual Processing

Mon, 01 Jan 2018 00:00:00 +0000

see a follow-up in:
Victor Boutin, Angelo Franciosini, Frédéric Chavane, Franck Ruffier, Laurent U Perrinet (2021). Sparse Deep Predictive Coding captures contour integration capabilities of the early visual system. PLoS Computational Biology.
PDF Cite DOI Code URL arXiv
more about the role of top-down connections:
Victor Boutin, Angelo Franciosini, Franck Ruffier, Laurent U Perrinet (2020). Effect of top-down connections in Hierarchical Sparse Coding. Neural Computation.
PDF Cite DOI URL arXiv

Selectivity to oriented patterns of different precisions

Mon, 01 Jan 2018 00:00:00 +0000

poster présenté au GDR vision, Paris.
program : https://gdrvision2018.sciencesconf.org/data/pages/posters_GDRVision2018.pdf
Poster (pdf)
code : https://github.com/hugoladret/InternshipM1

Participation au jury

Fri, 17 Nov 2017 18:30:00 +0000

FESTIVAL INTERFÉRENCES

Cinéma Documentaire et Débat Public

FESTIVAL INTERFÉRENCES

Le collectif Scènes Publiques composé de citoyens, chercheurs et cinéastes, organise la deuxième édition du Festival Interférences du 8 au 18 novembre 2017 à Lyon. J’ai eu la chance de pouvoir participer au jury autour de documentaires avec un regard scientifiques. Une occasion aussi de parler du métier de chercheur.

Date
17 et 18 Novembre 2017
Location
Lyon
Programmation
http://www.lacitedoc.com/interferences-programmation

Mina A Khoei

Thu, 26 Jan 2017 00:00:00 +0000

Emerging properties in a neural field model implementing probabilistic prediction (PhD, 2011-2014)

In the early visual system, information about the visual world as represented by neural activity is dynamically building up from sensory input but also by contextual information coming from neighboring cells and re-entrant signal from other cortical areas. Low-level sensory areas are therefore an excellent model for exploring how neural computations solve the problem of selecting a single, coherent and global representation from the dispersed information collected locally and in parallel by neurons. Our goal in this program is to study the dynamics of neural fields implementing probabilistic computations for early sensory processing. Emphasis will be put onto the role of anisotropic diffusion, in particular within a cortical area through lateral interactions.

We have previously elaborated probabilistic (Perrinet & Masson, 2010) or dynamical (Tlapale et al., 2010) models of motion information diffusion along cortical retinotopic trajectories. Probabilistic models give a complete representation of the information that is represented by populations of neurons. In such a dynamical system, prediction acts as a prior, filtering possible future states knowing the current one. An approximation using particle filtering methods will be used to investigate how this propagation can solve low-level computational problems such as integration, extrapolation or prediction in visual (Mason & Ilg, 2010) or somatosensory (Shulz et al. 2006) cortices.

Using this architecture, we will explore the consequences of such context-dependent propagation in terms of coding and of learning. First at the time scale of coding, knowing the prior, we will study the emergent properties of the system like its ability to track objects independently of their shape or to segment parts of the scene that are moving coherently. We will study of this motion information may help shape the selectivity of neurons in a given area, for instance orientation selectivity on the priamry visual cortex. At the time scale of learning, we will build models exploring the emergence of maps of cortical receptive fields optimally tuned to elaborate sparse, multi-scale representations of the visual or tactile world. In fact, a simple functional model allows to understand emergence in a model of a simple macro-column of the primary visual cortex (Perrinet, 2010). One challenging question is whether these functional models of self-organization can be translated to large-scale networks of the early sensory system. Using the probabilistic model, we will investigate how spatio-temporal receptive fields can emerge through learning of statistical regularities in the images and study how hierarchic structures can arise as a self-organized property.

Main publications:

Mina A Khoei, Guillaume S Masson, Laurent U Perrinet (2013). Motion-based prediction explains the role of tracking in motion extrapolation. Journal of Physiology-Paris.
PDF Cite DOI URL
Bernhard a Kaplan, Mina A Khoei, Anders Lansner, Laurent U Perrinet (2014). Signature of an anticipatory response in area V1 as modeled by a probabilistic model and a spiking neural network. IEEE International Joint Conference on Neural Networks (IJCNN) 2014 Beijing, China.
Cite DOI URL
Mina Aliakbari Khoei (2014). Une Approche Computationnelle de La Dépendance Au Mouvement Du Codage de La Position Dans La Système Visuel.
Cite URL
Mina A Khoei, Guillaume S Masson, Laurent U Perrinet (2017). The flash-lag effect as a motion-based predictive shift. PLoS Computational Biology.
Cite Slides DOI Press Code URL HAL

Propriétés émergentes d’un modèle de prédiction probabiliste utilisant un champ neural

Dans le système visuel de bas niveau, des informations sur le monde visuel tel que celles représentées par l’activité neuronale est dynamiquement causée par l’entrée sensorielle, mais aussi par des informations contextuelles provenant de cellules voisines et par le signal réentrant d’autres aires corticales. Les aires sensorielles primaires sont donc un excellent modèle pour étudier comment les neurones peuvent résoudre le problème de la sélection d’une seul représentation globale et cohérente depuis l’information collectée localement et en parallèle par les neurones. Notre objectif dans ce programme est d’étudier la dynamique de champs neuronaux mettant en œuvre des calculs probabilistes pour le traitement sensoriel précoce.

L’accent sera mis sur le rôle de la diffusion anisotrope, en particulier celle implémentée par les interactions latérales dans une aire corticale. Nous avons déjà élaboré des modèles probabiliste (Perrinet & Masson, 2010) ou dynamique (Tlapale et al., 2010) de diffusion de l’information de mouvement le long de trajectoires. Les modèles probabilistes donnent une représentation complète de l’information qui est représentée par des populations de neurones. Dans un tel système dynamique, la prédiction agit comme un prior, qui permet un filtrage des états futurs possibles en sachant la distribution de probabilité de l’état actuel. Une approximation à l’aide des méthodes de filtrage particulaires seront utilisées pour étudier comment cette propagation peut résoudre des problèmes de calcul de bas niveau telles que l’intégration, l’extrapolation ou la prédiction dans les système visuel (Mason & Ilg, 2010) ou somatosensoriel (Shulz et al. 2006).

En utilisant cette architecture, nous allons explorer les conséquences de la propagation dépendant du contexte tant en termes de codage que d’apprentissage. Premièrement, à l’échelle de temps de codage, connaissant l’architecture du réseau, nous allons étudier les propriétés émergentes du système, comme sa capacité à suivre les objets indépendamment de leur forme ou à segmenter des parties de la scène qui se déplacent de façon cohérente. Nous allons étudier le mouvement de cette information peut aider à façonner la sélectivité des neurones, par exemple la sélectivité à l’orientation sur le cortex visuel primaire. À l’échelle de temps d’apprentissage, nous allons construire des modèles d’émergence de cartes de champs récepteurs corticaux optimisées pour élaborer des représentations multi-échelles efficaces de l’univers visuel ou tactile. En fait, un modèle fonctionnel simple permet de comprendre l’émergence d’un modèle d’une simple macro-colonne du cortex visuel primaire (Perrinet, 2010). Une question difficile est de savoir si ces modèles fonctionnels d’auto-organisation peuvent être traduits à des réseaux à grande échelle du système sensoriel primaire. En utilisant ce modèle probabiliste, nous allons étudier comment des champs récepteurs spatio-temporels peuvent émerger à travers l’apprentissage des régularités statistiques dans les images et comment des structures hiérarchiques peuvent apparaitre comme la solution d’une propriété d’efficacité fonctionnelle.

The flash-lag effect as a motion-based predictive shift

Thu, 26 Jan 2017 00:00:00 +0000

Press release

Visual illusions: their origin lies in prediction

Flash-Lag Effect. When a visual stimulus moves along a continuous trajectory, it may be seen ahead of its veridical position with respect to an unpredictable event such as a punctuate flash. This illusion tells us something important about the visual system: contrary to classical computers, neural activity travels at a relatively slow speed. It is largely accepted that the resulting delays cause this perceived spatial lag of the flash. Still, after several decades of debates, there is no consensus regarding the underlying mechanisms.

Researchers from the Timone Institute of Neurosciences bring a new theoretical hypothesis on a visual illusion discovered at the beginning of the 20th century. This illusion remained misunderstood while it poses fundamental questions about how our brains represent events in space and time. This study published on January 26, 2017 in the journal PLOS Computational Biology, shows that the solution lies in the predictive mechanisms intrinsic to the neural processing of information.

Visual illusions are still popular: in a quasi-magical way, they can make objects appear where they are not expected… They are also excellent opportunities to question the constraints of our perceptual system. Many illusions are based on motion, such as the flash-lag effect. Observe a luminous dot that moves along a rectilinear trajectory. If a second light dot is flashed very briefly just above the first, the moving point will always be perceived in front of the flash while they are vertically aligned.

Fig 2. Diagonal Markov chain. In the current study, the estimated state vector z = {x, y, u, v} is composed of the 2D position (x and y) and velocity (u and v) of a (moving) stimulus. (A) First, we extend a classical Markov chain using Nijhawan’s diagonal model in order to take into account the known neural delay τ: At time t, information is integrated until time t − τ, using a Markov chain and a model of state transitions p(zt|zt−δt) such that one can infer the state until the last accessible information p(zt−τ|I0:t−τ). This information can then be “pushed” forward in time by predicting its trajectory from t − τ to t. In particular p(zt|I0:t−τ) can be predicted by the same internal model by using the state transition at the time scale of the delay, that is, p(zt|zt−τ). This is virtually equivalent to a motion extrapolation model but without sensory measurements during the time window between t − τ and t. Note that both predictions in this model are based on the same model of state transitions. (B) One can write a second, equivalent “pull” mode for the diagonal model. Now, the current state is directly estimated based on a Markov chain on the sequence of delayed estimations. While being equivalent to the push-mode described above, such a direct computation allows to more easily combine information from areas with different delays. Such a model implements Nijhawan’s “diagonal model”, but now motion information is probabilistic and therefore, inferred motion may be modulated by the respective precisions of the sensory and internal representations. (C) Such a diagonal delay compensation can be demonstrated in a two-layered neural network including a source (input) and a target (predictive) layer [44]. The source layer receives the delayed sensory information and encodes both position and velocity topographically within the different retinotopic maps of each layer. For the sake of simplicity, we illustrate only one 2D map of the motions (x, v). The integration of coherent information can either be done in the source layer (push mode) or in the target layer (pull mode). Crucially, to implement a delay compensation in this motion-based prediction model, one may simply connect each source neuron to a predictive neuron corresponding to the corrected position of stimulus (x + v ⋅ τ, v) in the target layer. The precision of this anisotropic connectivity map can be tuned by the width of convergence from the source to the target populations. Using such a simple mapping, we have previously shown that the neuronal population activity can infer the current position along the trajectory despite the existence of neural delays.

Processing visual information takes time and even if these delays are remarkably short, they are not negligible and the nervous system must compensate them. For an object that moves predictably, the neural network can infer its most probable position taking into account this processing time. For the flash, however, this prediction can not be established because its appearance is unpredictable. Thus, while the two targets are aligned on the retina at the time of the flash, the position of the moving object is anticipated by the brain to compensate for the processing time: it is this differentiated treatment that causes the flash-lag effect. The researchers show that this hypothesis also makes it possible to explain the cases where this illusion does not work: for example if the flash appears at the end of the moving dot’s trajectory or if the target reverses its path in an unexpected way. In this work, the major innovation is to use the accuracy of information in the dynamics of the model. Thus, the corrected position of the moving target is calculated by combining the sensory flux with the internal representation of the trajectory, both of which exist in the form of probability distributions. To manipulate the trajectory is to change the precision and therefore the relative weight of these two information when they are optimally combined in order to know where an object is at the present time. The researchers propose to call parodiction (from the ancient Greek paron, the present) this new theory that joins Bayesian inference with taking into account neuronal delays.

Fig 5. Histogram of the estimated positions as a function of time for the dMBP model. Histograms of the inferred horizontal positions (blueish bottom panel) and horizontal velocity (reddish top panel), as a function of time frame, from the dMBP model. Darker levels correspond to higher probabilities, while a light color corresponds to an unlikely estimation. We highlight three successive epochs along the trajectory, corresponding to the flash initiated, standard (mid-point) and flash terminated cycles. The timing of the flashes are respectively indicated by the dashed vertical lines. In dark, the physical time and in green the delayed input knowing τ = 100 ms. Histograms are plotted at two different levels of our model in the push mode. The left-hand column illustrates the source layer that corresponds to the integration of delayed sensory information, including the prior on motion. The right-hand illustrates the target layer corresponding to the same information but after the occurrence of some motion extrapolation compensating for the known neural delay τ.

Despite the simplicity of this solution, parodiction has elements that may seem counter-intuitive. Indeed, in this model, the physical world is considered “hidden”, that is to say, it can only be guessed by our sensations and our experience. The role of visual perception is then to deliver to our central nervous system the most likely information despite the different sources of noise, ambiguity and time delays. According to the authors of this publication, the visual treatment would consist in a “simulation” of the visual world projected at the present time, even before the visual information can actually modulate, confirm or cancel this simulation. This hypothesis, which seems to belong to “science fiction”, is being tested with more detailed and biologically plausible hierarchical neural network models that should allow us to better understand the mysteries underlying our perception. Visual illusions have still the power to amaze us!

check_out further results on introducing anisotropies in the FLE

Biologically-inspired characterization of sparseness in natural images

Sat, 01 Oct 2016 00:00:00 +0000

Modelling the dynamics of cognitive processes: from the Bayesian brain to particles

Thu, 07 Jul 2016 13:00:00 +0000

See a followup in Perrinet et al, 2012

ANR BalaV1 (2013/2016)

Wed, 27 Apr 2016 00:00:00 +0000

ANR BalaV1: Balanced states in area V1 (2013/2016)

Official website

In carnivores and primates the orientation selectivity (OS) of the cells in the primary visual cortex (V1) is organized in maps in which preferred orientations (POs) of the cells change gradually except near “pin- wheels”, around which all orientations are present. Over the last half-century the mechanism for OS has been hotly debated. However the theories that purport to explain OS have almost all considered cortical networks in which the neurons receive input preferentially from cells with similar PO. Such theories certainly capture the connectivity for neurons in orientation domains where neurons are surrounded by other cells with similar PO. However this does not necessarily hold near pinwheels: because of the discontinuous change in orientation preference at the pinwheel, neurons in this area are surrounded by cells of all preferred orientations. Thus if the probability of connection is solely dependent on anatomical distance, the inputs that these neurons receive should represent all orientations by roughly the same amount. Thus one may expect that the response of the cells near pinwheels should hardly vary with orientation, in contrast to experimental data. As a result, the common belief is that, at least near pinwheels, the connectivity depends also on the differences between preferred orientation. The situation near pinwheels in V1 of carnivores and primates is similar to that in the whole of V1 of rodents. In these species, neurons in V1 are OS but the network does not exhibit an orientation map and the surround of the cells represents all orientations roughly equally. In a recent theoretical paper (Hansel and van Vreeswijk 2012) we have demonstrated that in this situation, the response of the cells can still be orientation selective provided that the network operates in the balanced regime. Here we hypothesize that V1 with an orientation map operates in the balanced regime and therefore neurons can exhibit OS near pinwheels even in the absence of functional specific connectivity. The goal of this interdisciplinary project is to investigate whether the “balance hypothesis” holds for layer 2/3 in V1 of primate and carnivore and whether the functional organization observed in that layer can be accounted for without feature specific connectivity. We will combine modeling and experiments to investigate how the response of the neurons – the mean firing, the mean voltage, the inhibitory and excitatory conductances and importantly, the power spectrum of their fluctuations – vary with the location in the map, and also how a population of neurons – LFP, voltage-sensitive dye imaging or 2 photons – is affected by the various para- meters used to test the system. Whether V1 indeed operates in the balanced regime in more realistic conditions will be further investigated by determining how the local network responds to visual stimuli beyond the classical receptive field. We will investigate this issue in models of layer 2/3 representing multiple hyper- columns to characterize center-surround interactions and their dependence on the long-range connectivity. This will provide us with predictions for center-surround interactions for cells near pinwheels and in orientation domains. These predictions will be tested experimentally.

The proposed project is new and ambitious. It aims at building a comprehensive and coherent understand- ing of the physiology of V1 layer 2/3 on several spatial scales from single cells to several hypercolumns and to account for this in mechanistic models. To accomplish these ambitious aims, we propose a combination of experimental and computational studies that take advantage of the unique strengths and the complementarity of expertise of 3 research teams. The Paris team has extensive experience in large-scale modeling of V1. The Toulouse and Marseille teams master both intra- and extracellular electrophysiology. In addition, the Marseille team is expert in microscopic and mesoscopic imaging techniques in V1.

Acknowledgement

This work was supported by ANR project "BalaV1" N° ANR-13-BSV4-0014-02.

ANR Horizontal-V1 (2017/2021)

Wed, 27 Apr 2016 00:00:00 +0000

Description on the official website of the ANR

The Horizontal-V1 project aimed at understanding the emergence of sensory predictions linking local shape attributes (orientation, contour) to global indices of movement (direction, speed, trajectory) at the earliest stage of cortical processing (primary visual cortex, i.e. V1). We studied how the long-distance “horizontal” connectivity, intrinsic to V1 and the feedback from higher cortical areas contribute to a dynamic processing of local-to-global features as a function of the context (eg displacement along a trajectory; during reafference change induced by eye-movements…). We characterized the dynamic processes based on lateral propagation intra-V1, through which spatio-temporal inferences (continuous movement or apparent motion sequences) facilitating spatial (“filling-in”) or positional (“flash-lag”) future expected responses may be generated.

Our main contributions to the project:

Victor Boutin, Angelo Franciosini, Frédéric Chavane, Franck Ruffier, Laurent U Perrinet (2021). Sparse Deep Predictive Coding captures contour integration capabilities of the early visual system. PLoS Computational Biology.
PDF Cite DOI Code URL arXiv
Victor Boutin, Angelo Franciosini, Franck Ruffier, Laurent U Perrinet (2020). Effect of top-down connections in Hierarchical Sparse Coding. Neural Computation.
PDF Cite DOI URL arXiv
Victor Boutin, Angelo Franciosini, Frédéric Chavane, Laurent U Perrinet (2022). Pooling in a predictive model of V1 explains functional and structural diversity across species. PLoS Computational Biology.
Cite DOI URL bioRxiv
Alberto Arturo Vergani, Laurent U Perrinet (2021). Simulating anticipatory activity in a 1D Spiking Neural Network Model. Bernstein Conference 2021.
Cite DOI
Alberto Arturo Vergani, Laurent U Perrinet (2021). Neural Turing Patterns.
Project

WP3 - Design of novel visual paradigms, probabilistic model of V1 and data-driven simulations - co lead UNIC-INT.

Objectives : This WP will have two primary goals. The first one is theoretically driven, and for sake of simplicity will ignore the dynamic features of neural integration (as expected from a statistical model of image analysis). Binding the different features of visual objects at the local scale (contours) as well as a more global level involves understanding the statistical regularities of the sensory inflow. In particular, titrating the predictions that can be done at the statistical level can be seen as a first pass to better search for critical parameters constraining the network behaviour. From these, we will build probabilistic predictive models optimized for edge co-occurrence classification and generate novel visual statistics 1) which obey rules imposed by the functional horizontal connectivity anisotropies, such as co- circularity, and 2) which facilitate binding in the orientation domain, such as log-polar planforms. These statistics generated in the first half of the grant will be implemented and tested experimentally in the second half of the grant. The second one is more data-driven (as well as phenomenological for feedback from higher cortical areas, since it will not be explored in the grant). Since model fitting will depend on close interactions with WP1 and WP2 measurements, it will be done in the second half of the grant.

WP3-Task 1: Theoretically oriented workplan – Lead INT (Laurent Perrinet)

WP3-Task 1.1 - theory : we will exploit our current expertise in integrating these statistics in the form of probabilistic models to make predictions both at the physiological and modelling levels. First, we will take advantage of our previous work on the quantification of the association field in different classes of natural images (Perrinet & Bednar, 2015). Using an existing library (https://github.com/bicv/SparseEdges), we will use the sparse representation of static natural images to compute histograms of edge co-occurrences. Using an existing algorithm for unsupervised learning (https://github.com/bicv/SparseHebbianLearning), we will learn the different independent components of edge co-occurrences. Such an algorithm fits well a traditional deep-learning convolutional neural network, but, in addition, will include constraints imposed by intra-layer horizontal connectivity. We expect that relevant features will be co-linear or co-circular pairs of edges, but also T-junctions or end-stopping features.
WP3-Task 1.2 - image/film synthesis : We have previously found that random synthetic textures, coined “Motion Clouds”, can be used to quantify V1 implication in visual motion perception (Leon et al, 2012; Simoncini et al, 2012). Recently, the INT and UNIC, partners proved mathematically that these stimuli were optimal with respect to some common geometrical transformations, such as translation, zoom or rotations (Vacher et al, 2015). A main characteristic of these textures is to be generated with a maximally entropic arrangement of elementary textures (so-called textons).

** Informed by the generative model of edge co-occurrences studied in subtask 1, we will be able to extend the family of motion cloud stimuli (Leon et al, 2012; Simoncini et al, 2012) to include joint dependencies between different elements in position or orientation. An exact solution to this problem is hard to achieve as it involves a combinatorial search of all possible combinations of pairs of edges. However, numerous variational approaches are possible and fit well with our probabilistic framework. We will use the convolutional neural network described above but using a back-propagating stream to generate different images. Such a representation will then be optimized using an unsupervised learning method. This is similar to the process used in Generative Adversarial Networks in deep-learning architectures (Radford et al, Archives). ** Finally, the regularities observed in static images will be extended to dynamical scenes by observing that a co-occurrence can be implemented by simple geometrical operations as they are operated in time. For instance a co-circularity is easily described as the set of smooth roto-translational transformations of an edge in time using the group of Galilean transformations (Sarti and Citti, 2006). This theory calls for a first prediction to understand the set of whole possible spatio-temporal co-occurrences of edges as geodesics in the lifted space of all possible trajectories. We predict that such decomposition should allow us to better understand the different classes of features that emerged in the first task.

WP3-Task 1.3 - Feedback of theory on experimentation : An essential aspect of this work would be to apply these stimuli in neurophysiological experiments and in the modelling. In particular, the ability to select different types of dependencies from the different classes learned above (for instance, co-circularities of a certain curvature range) will make it possible to evaluate the relative contribution of different components of the contextual information. This justifies the fact that the WP3 post-doctoral fellow should have the mobility (between INT and UNIC) and multi-disciplinar profile (theoretical and experimental) to perform this task.
WP3-Task 1.4 - Generic modelling : These various subtasks will allow us to determine the hierarchy of critical features relevant to describe the full statistics of the space of spatio-temporal edge co-occurrences. Indeed, in static images, we will be able to find independent component in the histograms of edge co-occurrences between metric aspect (distance or scale between edge) from configurational aspects (difference of angle or co-circularity angle).

Similarly, we expect to see that the different independent features should decompose at various scales both in space and in time. For instance, we expect configurational aspects to be more local while aspects related to a motion (Perrinet and Masson, 2012; Khoei et al, 2016) or global shape (form) should be more global. This translates into a probabilistic hierarchical model that would combine dependencies from different cues. In particular, through the emergence of differential pathways for form and motion. These quantitative predictions should finally be confronted at the modelling and neurophysiological levels.

WP3-Task 2 : Data-driven comprehensive model of V1 – Co-lead UNIC and INT

The second task is more data-driven (as well as phenomenological for the feedback circuit part, since largely unknown). Since simulations will depend on close interactions with WP1 and WP2 measurements, it will be developed by the WP3-Post-Doc in the second half of the grant. It will benefit from existing structuro-functional models addressing separately two distinct levels of neural integration, microscopic (conductance-based in Kremkow et al, 2016; Antolik et al, submitted, Chariker et al, 2016) and mesoscopic (VSD-like mean field in Rankin and Chavane, 2017). Efforts will be made to merge these models to fit - in a unified multiscale biologically realistic model - the cellular and VSD data targeting critically horizontal propagation. The parameterization should be flexible enough to produce a generic cortical architecture accounting possibly for species-specificity (Antolik for cat; Chaliker for monkey)

Acknowledgement

This work was supported by ANR project “Horizontal-V1” N° ANR-17-CE37-0006.

ANR TRAJECTORY (2016/2019)

Wed, 27 Apr 2016 00:00:00 +0000

Global motion processing is a major computational task of biological visual systems. When an object moves across the visual field, the sequence of visited positions is strongly correlated in space and time, forming a trajectory. These correlated images generate a sequence of local activation of the feed-forward stream. Local properties such as position, direction and orientation can be extracted at each time step by a feed-forward cascade of linear filters and static non-linearities. However such local, piecewise, analysis ignores the recent history of motion and faces several difficulties, such as systematic delays, ambiguous information processing (e.g., aperture and correspondence problems61) high sensitivity to noise and segmentation problems when several objects are present. Indeed, two main aspects of visual processing have been largely ignored by the dominant, classical feed-forward scheme. First, natural inputs are often ambiguous, dynamic and non-stationary as, e.g., objects moving along complex trajectories. To process them, the visual system must segment them from the scene, estimate their position and direction over time and predict their future location and velocity. Second, each of these processing steps, from the retina to the highest cortical areas, is implemented by an intricate interplay of feed-forward, feedback and horizontal interactions1. Thus, at each stage, a moving object will not only be processed locally, but also generate a lateral propagation of information. Despite decades of motion processing research, it is still unclear how the early visual system processes motion trajectories. We, among others, have proposed that anisotropic diffusion of motion information in retinotopic maps can contribute resolving many of these difficulties25 13. Under this perspective, motion integration, anticipation and prediction would be jointly achieved through the interactions between feed-forward, lateral and feedback propagations within a common spatial reference frame, the retinotopic maps.

Addressing this question is particularly challenging, as it requires to probe these sequences of events at multiple scales (from individual cells to large networks) and multiple stages (retina, primary visual cortex (V1)). “TRAJECTORY” proposes such an integrated approach. Using state-of-the-art micro- and mesoscopic recording techniques combined with modeling approaches, we aim at dissecting, for the first time, the population responses at two key stages of visual motion encoding: the retina and V1. Preliminary experiments and previous computational studies demonstrate the feasibility of our work. We plan three coordinated physiology and modeling work-packages aimed to explore two crucial early visual stages in order to answer the following questions: How is a translating bar represented and encoded within a hierarchy of visual networks and for which condition does it elicit anticipatory responses? How is visual processing shaped by the recent history of motion along a more or less predictable trajectory? How much processing happens in V1 as opposed to simply reflecting transformations occurring already in the retina?

The project is timely because partners master new tools such as multi-electrode arrays and voltage-sensitive dye imaging for investigating the dynamics of neuronal populations covering a large segment of the motion trajectory, both in retina and V1. Second, it is strategic: motion trajectories are a fundamental aspect of visual processing that is also a technological obstacle in computer vision and neuroprostheses design. Third, this project is unique by proposing to jointly investigate retinal and V1 levels within a single experimental and theoretical framework. Lastly, it is mature being grounded on (i) preliminary data paving the way of the three different aims and (ii) a history of strong interactions between the different groups that have decided to join their efforts.

The Marseille team

Frédéric Chavane (DR, CNRS, NEOPTO team) is working in the field of vision research for about 20 years with a special interest in the role of lateral interactions in the integration of sensory input in the primary visual cortex. His recent work suggest that lateral interactions mediated by horizontal intracortical connectivity participates actively in the input normalization that controls a wide range of function, from the contrast-response gain to the representation of illusory or real motion. His expertise range from microscopic (intracellular recordings) to mesoscopic (optical imaging, multi-electrode array) recording scales in the primary visual cortex of anesthetized and awake behaving animals.
Laurent Perrinet (CR, CNRS, NEOPTO team). His scientific interests focus on bridging computational understanding of neural dynamics and low-level sensory processing by focusing on motion perception. He is the author of papers in machine learning, computational neuroscience and behavioral psychology. One key concept is the use of statistical regularities from natural scenes as a main drive to integrate local neural information into a global understanding of the scene. In a recent paper that he coauthored (in Nature Neuroscience), he developed a method to use synthesized stimuli targeted to analyze physiological data in a system-identification approach.
Ivo Vanzetta (CR, CNRS, NEOPTO team). His scientific interests focus on how to optimally use photonics-based imaging methods to investigate visual information processing in low-level visual areas, in the anesthetized and awake animal (rodent & primate). As can be seen from his bibliographic record, these methods include optical imaging of intrinsic signals and voltage sensitive dyes and, recently, 2 photon microscopy. Finally I. Vanzetta has an ongoing collaboration with L. Perrinet on the utilization of well-controlled, synthesized nature-like visual stimuli to probe the response characteristics of the primate’s visual system (Sanz-Leon & al. 2012).

Progress meeting ANR TRAJECTORY

Time January 15th, 2018
Location INT
General presentation of the grant, see Anr TRAJECTORY
Overview of my current projects https://laurentperrinet.github.io/sciblog/files/2017-11-15_ColloqueMaster.html
MotionClouds with trajectories https://laurentperrinet.github.io/sciblog/posts/2018-01-16-testing-more-complex-trajectories.html or https://laurentperrinet.github.io/sciblog/posts/2018-11-13-testing-more-complex-trajectories.html

A predictive sequence is essential in resolving the coherence problem. The sequence in which a set of local motion is shown is essential for the detection of global motion. we replicate here the experiments by Scott Watamaniuk and colleagues. They have shown behaviourally that a dot in noise is much more detectable when it follows a coherent trajectory, up to an order of magnitude of 10 times what would be predicted by the local components of the trajectory. In this movie we observe white noise and at first sight, no information is detectable. In fact, there is a dot moving along some smooth linear trajectory. Since this is compatible with a predictive sequence, it is much easier to see the dot (from left to right in the top of the image, a smooth pursuit helps to catch it). This simple experiment shows that, even if local motion is similar in both movies, a coherent trajectory is more easy to track. Obviously, we may thus conclude that the whole trajectory is more that its individual parts, and that the independence hypothesis does not hold if we want to account for the predictive information in input sequences such as seems to be crucial for the AP.

Acknowledgement

This work was supported by ANR project “TRAJECTORY” N° ANR-15-CE37-0011.

Yves Fregnac

Wed, 27 Apr 2016 00:00:00 +0000

Yves Frégnac is Emeritus Research Director (DRCE2, Exceptional Class) at the Centre National de la Recherche Scientifique. He has been the Head of the CNRS interdisciplinary department that he founded in 1999 (UNIC : Unit of Integrative and Computational Neuroscience, which became Unit of Information and Complexity in 2011), and co-director of the CNRS Federative Institute Alfred Fessard of Neurobiology (INAF) in Gif sur Yvette. He is now Emeritus Research Director (since January 2017). He has been also Full Professor in the Department of Humanities and Social Sciences at the Ecole Polytechnique, near Paris, for the past 8 years, and continues teaching a course on « Brain and Cognition » at the Ecole Centrale-Supelec.

Collaborative publications

I had the chance to collaborate with Yves Frégnac as part of numerous consortiums (FACETS, BrainScales, ANR HORV1). This allowed us to establish a direct collaboration with Jens Kremkow which led to modelling work on feed-forward inhibition Kremkow et al, 2008 and a model accounting for the observed sparseness of neural activity when presenting natural images Kremkow et al, 2008.

Jens Kremkow, Laurent U Perrinet, Pierre Baudot, Manu Levy, Olivier Marre, Cyril Monier, Yves Frégnac, Guillaume S Masson, Ad M Aertsen (2008). Control of the temporal interplay between excitation and inhibition by the statistics of visual input: a V1 network modelling study. Proceedings of the Society for Neuroscience conference.
Cite
Jens Kremkow, Laurent U Perrinet, Cyril Monier, Jose-Manuel Alonso, Ad M Aertsen, Yves Frégnac, Guillaume S Masson (2016). Push-Pull Receptive Field Organization and Synaptic Depression: Mechanisms for Reliably Encoding Naturalistic Stimuli in V1. Frontiers in Neural Circuits.
Cite DOI URL HAL

Testing the odds of inherent vs. observed overdispersion in neural spike counts

Fri, 22 Jan 2016 00:00:00 +0000

A dynamic model for decoding direction and orientation in macaque primary visual cortex

Fri, 01 Jan 2016 00:00:00 +0000

see a follow-up in this poster
This is a followup in Perrinet et al, 2012

A dynamic model for decoding direction and orientation in macaque primary visual cortex

Fri, 01 Jan 2016 00:00:00 +0000

Jens Kremkow

Fri, 01 Jan 2016 00:00:00 +0000

Correlating Excitation and Inhibition in Visual Cortical Circuits: Functional Consequences and Biological Feasibility (PhD, 2006-01 / 2009-05)

The goal of the FACETS (Fast Analog Computing with Emergent Transient States) project was to create a theoretical and experimental foundation for the realisation of novel computing paradigms which exploit the concepts experimentally observed in biological nervous systems. The continuous interaction and scientific exchange between biological experiments, computer modelling and hardware emulations within the project provides a unique research infrastructure that will in turn provide an improved insight into the computing principles of the brain. This insight may potentially contribute to an improved understanding of mental disorders in the human brain and help to develop remedies.

Venue: Thèse de Doctorat de l’Université d’Aix-Marseille II, Ecole Doctorale des Sciences de la Vie et de la Santé Marseille, France en Cotutelle avec la Fakultät für Biologie Albert-Ludwigs-Universität Freiburg im Breisgau, Allemagne
Thesis director: Guillaume MASSON and Dr. Laurent PERRINET

Main publications:

Jens Kremkow, Laurent U Perrinet, Cyril Monier, Jose-Manuel Alonso, Ad M Aertsen, Yves Frégnac, Guillaume S Masson (2016). Push-Pull Receptive Field Organization and Synaptic Depression: Mechanisms for Reliably Encoding Naturalistic Stimuli in V1. Frontiers in Neural Circuits.
Cite DOI URL HAL
Jens Kremkow, Laurent U Perrinet, Guillaume S Masson, Ad M Aertsen (2010). Functional consequences of correlated excitatory and inhibitory conductances in cortical networks. Journal of Computational Neuroscience.
PDF Cite DOI URL

Description of the PHD thesis project

The primary visual cortex (V1) is one of the most studied cortical area in neuroscience. Together with the retina and the lateral geniculate nucleus (LGN), it forms the early visual system, which has become a common model for studying computational principles in the sensory systems. Simple artificial stimuli (such as drifting gratings (DG)) have given precious insights into the neural basis of visual processing. However, recently more researchers have used more complex natural images (NI) visual stimuli, arguing that the low dimensional artificial stimuli are not sufficient for a complete understanding of the visual system. For example, whereas the responses of V1 neurons to DG are dense but with variable spike timings, the neurons are activated with only few and precise spikes to NI. Furthermore, if linear receptive field models provide a good fit to responses during simple stimuli, they often fail during NI.

To investigate the mechanisms behind the stimulus dependent responses of cortical neurons we have built a biophysical, yet simple and comprehensible, model of the early visual system. We show how the spatial and temporal stimulus properties interact with the model architecture to give rise to differential response behaviour. Our results show in particular that during NI, the LGN afferents show epochs of correlated activity. These temporal correlations are necessary to induce transient excitatory synaptic inputs, and result in precise spike timings in V1. Furthermore, the sparseness of the responses to NI can be explained by a hardwired, correlated and lagging inhibitory conductance, or conductance temporal window, which is induced by the interactions of the thalamocortical circuit with the spatiotemporal correlations in the stimulus.

We continue by investigating the origin of nonlinear responses during NI in the temporal window, by comparing models of different complexity. Our results suggest first that adaptive processes shape the responses, depending on the temporal properties of the stimuli. The different spatial properties can result in nonlinear inputs through the recurrent cortical network. We then study the functional consequences of correlated excitatory and inhibitory condutances in more details in general models. These results show that: (1) spiking of individual neurons becomes sparse and precise, (2) the selectivity of signal propagation increases and the detailed delay allows to gate the propagation through feed-forward structures (3) and recurrent cortical networks are more stable and more likely to elicit in vivo type activity states. Lastly our work illustrates new advances in methods of constructing and exchanging models of neuronal systems by the means of a simulator independent description language (called PyNN). We use this new tool to investigate the feasibility of comparing software simulations with neuromorphic hardware emulations. The presented work give new perspectives on the way conductances can be used for computations and it opens the door for more elaborated models of visual system’s mechanisms.

Push-Pull Receptive Field Organization and Synaptic Depression: Mechanisms for Reliably Encoding Naturalistic Stimuli in V1

Fri, 01 Jan 2016 00:00:00 +0000

Motion-based prediction with neuromorphic hardware

Thu, 05 Nov 2015 13:00:00 +0000

Sparse Models for Computer Vision

Sun, 01 Nov 2015 00:00:00 +0000

Appeared in this book:
Gabriel Cristóbal, Laurent U Perrinet, Matthias S Keil (2015). Biologically Inspired Computer Vision.
Cite DOI Code URL

Sparse Coding Of Natural Images Using A Prior On Edge Co-Occurences

Sat, 01 Aug 2015 00:00:00 +0000

Edge co-occurrences can account for rapid categorization of natural versus animal images

Thu, 01 Jan 2015 00:00:00 +0000

A study of how people can quickly spot animals by sight is helping uncover the workings of the human brain.

Scientists examined why volunteers who were shown hundreds of pictures - some with animals and some without - were able to detect animals in as little as one-tenth of a second. They found that one of the first parts of the brain to process visual information - the primary visual cortex - can control this fast response. More complex parts of the brain are not required at this stage, contrary to what was previously thought.

The probability distribution function $p(\psi, \theta)$ represents the distribution of the different geometrical arrangements of edges’ angles, which we call a chevron map. We show here the histogram for non-animal natural images, illustrating the preference for co-linear edge configurations. For each chevron configuration, deeper and deeper red circles indicate configurations that are more and more likely with respect to a uniform prior, with an average maximum of about $3$ times more likely, and deeper and deeper blue circles indicate configurations less likely than a flat prior (with a minimum of about $0.8$ times as likely). Conveniently, this chevron map shows in one graph that non-animal natural images have on average a preference for co-linear and parallel edges, (the horizontal middle axis) and orthogonal angles (the top and bottom rows),along with a slight preference for co-circular configurations (for $\psi=0$ and $\psi=\pm \frac \pi 2$, just above and below the central row). We compare chevron maps in different image categories in Figure~3.

As for Figure 2, we show the probability of edge configurations as chevron maps for two databases (man-made, animal). Here, we show the ratio of histogram counts relative to that of the non-animal natural image dataset. Deeper and deeper red circles indicate configurations that are more and more likely (and blue respectively less likely) with respect to the histogram computed for non-animal images. In the left plot, the animal images exhibit relatively more circular continuations and converging angles (red chevrons in the central vertical axis) relative to non-animal natural images, at the expense of co-linear, parallel, and orthogonal configurations (blue circles along the middle horizontal axis). The man-made images have strikingly more co-linear features (central circle), which reflects the prevalence of long, straight lines in the cage images in that dataset. We use this representation to categorize images from these different categories in Figure~4.

Classification results. To quantify the difference in low-level feature statistics across categories (see Figure3, we used a standard Support Vector Machine (SVM) classifier to measure how each representation affected the classifier’s reliability for identifying the image category. For each individual image, we constructed a vector of features as either (FO) the histogram of first-order statistics as the histogram of edges’ orientations, (CM) the chevron map subset of the second-order statistics, (i.e., the two-dimensional histogram of relative orientation and azimuth; see Figure 2 ), or (SO) the full, four-dimensional histogram of second-order statistics (i.e., all parameters of the edge co-occurrences). We gathered these vectors for each different class of images and report here the results of the SVM classifier using an F1 score (50% represents chance level). While it was expected that differences would be clear between non-animal natural images versus laboratory (man-made) images, results are still quite high for classifying animal images versus non-animal natural images, and are in the range reported by\citet{Serre07} (F1 score of 80% for human observers and 82% for their model), even using the CM features alone. We further extend this results to the psychophysical results of Serre et al. (2007) in Figure 5.

To see whether the patterns of errors made by humans are consistent with our model, we studied the second-order statistics of the 50 non-animal images that human subjects in Serre et al. (2007) most commonly falsely reported as having an animal. We call this set of images the false-alarm image dataset. (Left) This chevron map plot shows the ratio between the second-order statistics of the false-alarm images and the full non-animal natural image dataset, computed as in Figure 3 (left). Just as for the images that actually do contain animals (Figure~3, left), the images falsely reported as having animals have more co-circular and converging (red chevrons) and fewer collinear and orthogonal configurations (blue chevrons). (Right) To quantify this similarity, we computed the Kullback-Leibler distance between the histogram of each of these images from the false-alarm image dataset, and the average histogram of each class. The difference between these two distances gives a quantitative measure of how close each image is to the average histograms for each class. Consistent with the idea that humans are using edge co-occurences to do rapid image categorization, the 50 non-animal images that were worst classified are biased toward the animal histogram ($d’ = 1.04$), while the 550 best classified non-animal images are closer to the non-animal histogram.

Communiqué de presse INSB : Comment nait la première impression d’une scène visuelle

communiqué de presse

En modélisant notre capacité à distinguer un animal dans une scène visuelle, des chercheurs de l’Institut de Neurosciences de la Timone et de l’Université d’Edinburgh lèvent le voile sur certains des mystères de la perception visuelle. Ils démontrent que la classification très rapide par le cerveau d’une image contenant ou non un animal, est possible à un niveau de représentation relativement primitif à partir de régularités statistiques simples, et non, comme cela est généralement admis, après une longue série d’analyses visuelles de plus en plus abstraites. Cette étude est publiée dans la revue Scientific Reports.

Classifier une image, par exemple en décidant si elle contient ou non un animal, est une des fonctions de base du cerveau. Dans le royaume animal, on comprend aisément qu’elle constitue une fonction vitale aussi bien pour des prédateurs que pour leurs proies. Les mécanismes sous-jacents sont de plus en plus étudiés aussi bien dans le domaine des systèmes d’intelligence artificielle que dans celui des Neurosciences, mais ils restent encore bien mystérieux pour les chercheurs. En effet, si les réseaux d’ordinateurs les plus avancés peuvent aujourd’hui aisément calculer numériquement des quantités phénoménales de données à partir de bases de données pharaoniques, même les systèmes les plus avancés de classification d’images n’égalent pas encore les capacités d’un jeune enfant!

Laurent Perrinet de l’Institut de Neurosciences de la Timone à Marseille et James Bednar de l’université d’Edinburgh en Écosse, ont modélisé la façon dont nous pouvons classer différentes catégories d’images. Leur l’objectif initial était de différencier des scènes visuelles naturelles de scènes d’intérieur, mais ils ont pu montrer que ce système simple de classification permettait aussi de détecter en une fraction de seconde des animaux dans une image. En effet, ils ont mis en évidence qu’un niveau de performance comparable à celui d’observateurs humains est atteignable tout en utilisant un niveau de représentation très primitif, et non, comme cela est généralement admis, après une longue série d’analyses visuelles de plus en plus abstraites (détection des yeux et des membres, puis de la tête et du corps, etc…).

Cette représentation primitive se base sur les modèles existants de représentation des images dans les aires visuelles de bas niveau des primates. On estime en effet que dans le cortex visuel primaire les images visuelles sont représentées dans l’activité neurale comme l’organisation de contours élémentaires, à la manière d’un peintre qui dessine une silhouette en une série de coups de pinceau. Une des innovations majeures dans cette étude consiste à simplement utiliser la fréquence des configurations entre des paires de contours élémentaires comme représentation d’entrée utilisée pour le classificateur.

Pour arriver à ce résultat, les chercheurs ont utilisé des modèles mathématiques de la représentation des images dans le cortex visuel primaire et en particulier les inter-relations entre des éléments de contours voisins. En étudiant les résultats de l’analyse, on note que dans les images naturelles, des contours parallèles sont observés majoritairement, signe que les contours et textures présents dans les images contiennent en majorité des alignements. C’est encore plus vrai dans les environnements artificiels comme dans une scène d’intérieur (par exemple un bureau) où les bords francs dominent. On montre aussi que les objets co-circulaires (c’est-à-dire des configurations symétriques) sont aussi relativement plus présents que des configurations aléatoires.

La principale nouveauté de cette étude est de montrer que les images contenant un animal (quelle que soit son espèce ou sa position dans l’image) contiennent sensiblement plus de configurations symétriques. Cette différence suffit pour expliquer le niveau de performance de classification chez les humains quand on leur présente de telles scènes de façon très brève.

Pour valider cette hypothèse, les chercheurs ont alors utilisé des données précédemment enregistrées dans lesquelles des volontaires regardaient et classifiaient des centaines d’images. En utilisant cette représentation primitive, ils ont mis en évidence qu’un programme très simple pouvait facilement classifier les images comme contenant ou non un animal, sans avoir besoin d’une connaissance plus élaborée sur les caractéristiques de l’animal comme sa position, sa taille ou son orientation sur l’image.

Cette découverte peut accélérer le développement de requêtes via des images dans les moteurs de recherche, comme Google et Facebook, car elle permet une classification simple et robuste grâce à des caractéristiques statistiques de bas niveau basées sur la géométrie des objets. Elle pourrait ainsi améliorer l’efficacité de tels algorithmes. Toutefois, et comme cela a été mis en évidence dans la psychophysique humaine, les catégories visuelles doivent être visuellement assez distinctes: ce traitement rapide ne permet pas, par exemple, de distinguer une scène de montagne d’une scène de mer. De manière surprenante, les chercheurs ont montré que lorsque les humains se trompent en classifiant de manière erronée une image comme contenant un animal, le programme a tendance à se tromper de la même façon! En utilisant des modèles mathématiques, on peut donc imaginer synthétiser des images d’animaux qui en fait, n’en contiendraient pas. Ces “chimères” seront sûrement très utiles pour percer encore plus les mystères du système visuel.

Dans le futur, l’extension de cette représentation calculée sur l’ensemble de l’image pourrait être améliorée en la couplant à des processus de classification locaux permettant de déterminer par exemple la position de l’objet à classifier et de segmenter progressivement la figure du fond afin de diminuer ainsi les distractions.

TÀ partir d’une image naturelle (en haut à gauche), les chercheurs ont déterminé la façon la plus efficace de la représenter comme une succession de contours élémentaires orientés. Sur cet exemple, l’image est décomposée en contours élémentaires (marqués en rouge) et l’image correspond à sa reconstruction à partir de cette représentation, gage d’une représentation correcte de l’image. Le schéma (en bas à gauche) décrit alors les relations géométriques pour chaque paire de contours élémentaires (dénotés ici A et B) et en particulier la différence entre leurs orientations (cette différence est nulle pour des contours parallèles) ainsi que leur différence d’azimuth. Une valeur nulle de cette dernière indiquant une symétrie, c’est-à-dire que ces contours sont co-circulaires. On peut alors compiler les statistiques des différentes configurations possibles sur des bases de données de 600 images contenant ou ne contenant pas d’animal. On voit alors que les images contenant un animal présentent relativement moins de configurations parallèles (disques bleus, jusqu’à 50% de moins) et plus de configurations co-circulaires, c’est à dire le long de l’axe vertical médian (disques rouges, jusqu’à 20% d’occurences en plus). Cette différence, aussi tenue soit elle, permet alors de classifier une image pour permettre de deviner si elle contient ou non un animal.

On overdispersion in neuronal evoked activity

Thu, 01 Jan 2015 00:00:00 +0000

see a follow-up in this publication

Une Approche Computationnelle de La Dépendance Au Mouvement Du Codage de La Position Dans La Système Visuel

Mon, 06 Oct 2014 00:00:00 +0000

Edge co-occurrences are sufficient to categorize natural versus animal images

Fri, 22 Aug 2014 00:00:00 +0000

see a follow-up:
Laurent U Perrinet, James A Bednar (2015). Edge co-occurrences can account for rapid categorization of natural versus animal images. Scientific Reports.
Cite DOI Code URL HAL

Motion-based prediction model for flash lag effect

Fri, 22 Aug 2014 00:00:00 +0000

Based on
Laurent U Perrinet, Guillaume S Masson (2012). Motion-based prediction is sufficient to solve the aperture problem. Neural Computation.
PDF Cite Pdf arXiv Doi
see follow-up on motion extrapolation:
Mina A Khoei, Guillaume S Masson, Laurent U Perrinet (2013). Motion-based prediction explains the role of tracking in motion extrapolation. Journal of Physiology-Paris.
PDF Cite DOI URL
see follow-up on the flash-lag effect:
Mina A Khoei, Guillaume S Masson, Laurent U Perrinet (2017). The flash-lag effect as a motion-based predictive shift. PLoS Computational Biology.
Cite Slides DOI Press Code URL HAL

Signature of an anticipatory response in area V1 as modeled by a probabilistic model and a spiking neural network

Fri, 25 Apr 2014 00:00:00 +0000

see Kaplan and al, 2014

WP5 - Demo 1.3 : Spiking model of motion-based prediction

Thu, 20 Mar 2014 13:00:00 +0000

A Simple Model of Orientation Encoding Accounting For Multivariate Neural Noise

Wed, 01 Jan 2014 00:00:00 +0000

see a follow-up in this publication

A Simple Model of Orientation Encoding Accounting For Multivariate Neural Noise

Wed, 01 Jan 2014 00:00:00 +0000

see a follow-up in this publication

Relationship between natural image statistics and lateral connectivity in the primary visual cortex

Wed, 01 Jan 2014 00:00:00 +0000

see a follow-up:
Laurent U Perrinet, James A Bednar (2015). Edge co-occurrences can account for rapid categorization of natural versus animal images. Scientific Reports.
Cite DOI Code URL HAL

Demo 1, Task4: Implementation of models showing emergence of cortical fields and maps

Tue, 26 Nov 2013 00:00:00 +0000

Together with Bernhard Kaplan, we talked about how we aim at “compiling” a predictive motion-based approach as a spiking neural networks and then as a parallel wafer systems in the BrainscaleS project (Demo 1, Task4).
(private to the consortium: https://brainscales.kip.uni-heidelberg.de/internal/jss/AttendMeeting?m=showMeetingInfoPage&meetingID=52 https://brainscales.kip.uni-heidelberg.de/internal/jss/AttendMeeting?m=showAgenda&meetingID=52 including copies of the slides)

Anisotropic connectivity implements motion-based prediction in a spiking neural network

Tue, 17 Sep 2013 00:00:00 +0000

Based on
Laurent U Perrinet, Guillaume S Masson (2012). Motion-based prediction is sufficient to solve the aperture problem. Neural Computation.
PDF Cite Pdf arXiv Doi
see follow-up on motion extrapolation:
Mina A Khoei, Guillaume S Masson, Laurent U Perrinet (2013). Motion-based prediction explains the role of tracking in motion extrapolation. Journal of Physiology-Paris.
PDF Cite DOI URL
see follow-up on the flash-lag effect:
Mina A Khoei, Guillaume S Masson, Laurent U Perrinet (2017). The flash-lag effect as a motion-based predictive shift. PLoS Computational Biology.
Cite Slides DOI Press Code URL HAL

lication/khoei-13-jpp" view=“4” >}}

Edge statistics in natural images versus laboratory animal environments: implications for understanding lateral connectivity in V1

Thu, 10 May 2012 13:00:00 +0000

See a followup in
Edge co-occurrences can account for rapid categorization of natural versus animal images
Laurent U Perrinet, James A Bednar

Cite DOI Code URL HAL

Apparent motion in V1 - Probabilistic approaches

Fri, 23 Mar 2012 13:00:00 +0000

Edge statistics in natural images versus laboratory animal environments: implications for understanding lateral connectivity in V1

Tue, 24 Jan 2012 13:00:00 +0000

See a followup in
Edge co-occurrences can account for rapid categorization of natural versus animal images
Laurent U Perrinet, James A Bednar

Cite DOI Code URL HAL

Nicole Voges

Sun, 01 Jan 2012 00:00:00 +0000

Complex dynamics in recurrent cortical networks based on spatially realistic connectivities (Post-Doc, 2008 / 2010)

Description

Most studies on the dynamics of recurrent cortical networks are either based on purely random wiring or neighborhood couplings. Neuronal cortical connectivity, however, shows a complex spatial pattern composed of local and remote patchy connections. We ask to what extent such geometric traits influence the “idle” dynamics of two-dimensional (2d) cortical network models composed of conductance-based integrate-and-fire (iaf) neurons. In contrast to the typical 1 mm2 used in most studies, we employ an enlarged spatial set-up of 25 mm2 to provide for long-range connections. Our models range from purely random to distance-dependent connectivities including patchy projections, i.e., spatially clustered synapses. Analyzing the characteristic measures for synchronicity and regularity in neuronal spiking, we explore and compare the phase spaces and activity patterns of our simulation results. Depending on the input parameters, different dynamical states appear, similar to the known synchronous regular (SR) or asynchronous irregular (AI) firing in random networks. Our structured networks, however, exhibit shifted and sharper transitions, as well as more complex activity patterns. Distance-dependent connectivity structures induce a spatio-temporal spread of activity, e.g., propagating waves, that random networks cannot account for. Spatially and temporally restricted activity injections reveal that a high amount of local coupling induces rather unstable AI dynamics. We find that the amount of local versus long-range connections is an important parameter, whereas the structurally advantageous wiring cost optimization of patchy networks has little bearing on the phase space.

Main publications:

Nicole Voges, Laurent U Perrinet (2010). Phase space analysis of networks based on biologically realistic parameters. Journal of Physiology-Paris.
PDF Cite DOI URL
Nicole Voges, Laurent U Perrinet (2012). Complex dynamics in recurrent cortical networks based on spatially realistic connectivities. Frontiers in Computational Neuroscience.
PDF Cite DOI URL

Context

References

Reynaud A., Masson G. S. and Chavane F. Dynamics of Local Input Normalization Result from Balanced Short- and Long-Range Intracortical Interactions in Area V1 Journal of Neuroscience, 2012, 32(36): 12558-12569
Reynaud A., Takerkart S, Masson G. S. and Chavane F. Linear model decomposition for voltage-sensitive dye imaging signals: Application in awake behaving monkey. Neuroimage, 2011, 54(2), 1196–1210
Perrinet, L. and Masson G. Motion-based prediction is sufficient to solve the aperture problem Neural Computation, 2012

Edge statistics in natural images versus laboratory animal environments: implications for understanding lateral connectivity in V1

Tue, 15 Nov 2011 08:45:00 +0000

Abstract Control Number: 17671
Presentation Number: 530.04
Presentation Time: 8:45am - 9:00am
session:
Session Type: Nanosymposium
Session Number: 530
Session Title: Development of Motor and Sensory Systems
See a followup in
Edge co-occurrences can account for rapid categorization of natural versus animal images
Laurent U Perrinet, James A Bednar

Cite DOI Code URL HAL

Edge statistics in natural images versus laboratory animal environments: implications for understanding lateral connectivity in V1

Wed, 28 Sep 2011 13:00:00 +0000

See a followup in
Edge co-occurrences can account for rapid categorization of natural versus animal images
Laurent U Perrinet, James A Bednar

Cite DOI Code URL HAL

Edge statistics in natural images versus laboratory animal environments: implications for understanding lateral connectivity in V1

Sat, 01 Jan 2011 00:00:00 +0000

See a followup in
Laurent U Perrinet, James A Bednar (2015). Edge co-occurrences can account for rapid categorization of natural versus animal images. Scientific Reports.
Cite DOI Code URL HAL

Role of homeostasis in learning sparse representations

Sat, 17 Jul 2010 00:00:00 +0000

related publication :
Laurent U Perrinet (2019). An adaptive homeostatic algorithm for the unsupervised learning of visual features. Vision.
PDF Cite DOI Code URL

Reading out the dynamics of lateral interactions in the primary visual cortex from VSD data

Mon, 30 Nov 2009 00:00:00 +0000

see this more recent poster @ VSS

Correlating Excitation and Inhibition in Visual Cortical Circuits : Functional Consequences and Biological Feasibility

Thu, 01 Jan 2009 00:00:00 +0000

Decoding center-surround interactions in population of neurons for the ocular following response

Thu, 01 Jan 2009 00:00:00 +0000

Inferring monkey ocular following responses from V1 population dynamics using a probabilistic model of motion integration

Thu, 01 Jan 2009 00:00:00 +0000

From neural activity to behavior: computational neuroscience as a synthetic approach for understanding the neural code.

Tue, 01 Apr 2008 00:00:00 +0000

Modeling of spikes, sparseness and adaptation in the primary visual cortex: applications to imaging

Fri, 01 Feb 2008 00:00:00 +0000

related publications @ FENS 2006, @ NeuroComp 2008 and @ AREADNE 2008

Adaptive Sparse Spike Coding : applications of Neuroscience to the compression of natural images

Tue, 01 Jan 2008 00:00:00 +0000

Control of the temporal interplay between excitation and inhibition by the statistics of visual input: a V1 network modelling study

Tue, 01 Jan 2008 00:00:00 +0000

see this subsequent paper in the Journal of Computational Neuroscience

Neural Codes for Adaptive Sparse Representations of Natural Images

Sat, 01 Sep 2007 00:00:00 +0000

Self-Invertible 2D Log-Gabor Wavelets

Sat, 13 Jan 2007 00:00:00 +0000

This library defines the set of LogGabor kernels. These are generic edge-like filters at different scales, phases and orientations. The library develops a simple method to construct a simple multi-scale linear transform.

Web-site
Source code
logGabor filters are used in numerous computer vision applications and reaches 177 citations on Google Scholar (last updated 22/10/2021).

Figure 1 Multiresolution schemes. (a) Schematic contours of the log-Gabor filters in the Fourier domain with 5 scales and 8 orientations (only the contours at 78% of the filter maximum are drawn). (b) The real part of the corresponding filters is drawn in the spatial domain. The two first scales are drawn at the bottom magnified by a factor of 4 for a better visualization. The different scales are arranged in rows and the orientations in columns. The low-pass filter is drawn in the upper-left part. (c) The corresponding imaginary parts of the filters are shown in the same arrangement. Note that the low-pass filter does not have imaginary part. Insets (b) and (c) show the final filters built through all the processes described in Section 2. (d) In the proposed scheme the elongation of log-Gabor wavelets increases with the number of orientations nt . Here the real parts (left column) and imaginary parts (right column) are drawn for the 3, 4, 6, 8, 10, 12 and 16 orientation schemes. (e) As a comparison orthogonal wavelet filters ‘Db4’ are shown. Horizontal, vertical and diagonal wavelets are arranged on columns (low-pass on top). (f) As a second comparison, steerable pyramid filters (Portilla et al., 2003) are shown. The arrangement over scales and orientations is the same as for the log-Gabor scheme.

Neural Codes for Adaptive Sparse Representations of Natural Images

Mon, 01 Jan 2007 00:00:00 +0000

On efficient sparse spike coding schemes for learning natural scenes in the primary visual cortex

Mon, 01 Jan 2007 00:00:00 +0000

Sparse Approximation of Images Inspired from the Functional Architecture of the Primary Visual Areas

Mon, 01 Jan 2007 00:00:00 +0000

relies on log-Gabor filters:
Sylvain Fischer, Filip Šroubek, Laurent U Perrinet, Rafael Redondo, Gabriel Cristóbal (2007). Self-Invertible 2D Log-Gabor Wavelets. International Journal of Computer Vision.
PDF Cite DOI Code URL

Schematic structure of the primary visual cortex implemented in the present study. Simple cortical cells are modeled through log-Gabor functions. They are organized in pairs in quadrature of phase (dark-gray circles). For each position the set of different orientations compose a pinwheel (large light-gray circles). The retinotopic organization induces that adjacent spatial positions are arranged in adjacent pinwheels. Inhibition interactions occur towards the closest adjacent positions which are in the direc-tions perpendicular to the cell preferred orientation and toward adjacent orientations (light-red connections). Facilitation occurs to-wards co-aligned cells up to a larger distance (dark-blue connections).

An efficiency razor for model selection and adaptation in the primary visual cortex

Sun, 01 Jan 2006 00:00:00 +0000

Dynamical contrast gain control mechanisms in a layer 2/3 model of the primary visual cortex

Sun, 01 Jan 2006 00:00:00 +0000

Modeling of simple cells through a sparse overcomplete gabor wavelet representation based on local inhibition and facilitation

Mon, 01 Aug 2005 00:00:00 +0000

relies on log-Gabor filters:
Sylvain Fischer, Filip Šroubek, Laurent U Perrinet, Rafael Redondo, Gabriel Cristóbal (2007). Self-Invertible 2D Log-Gabor Wavelets. International Journal of Computer Vision.
PDF Cite DOI Code URL

Schematic structure of the primary visual cortex implemented in the present study. Simple cortical cells are modeled through log-Gabor functions. They are organized in pairs in quadrature of phase (dark-gray circles). For each position the set of different orientations compose a pinwheel (large light-gray circles). The retinotopic organization induces that adjacent spatial positions are arranged in adjacent pinwheels. Inhibition interactions occur towards the closest adjacent positions which are in the direc-tions perpendicular to the cell preferred orientation and toward adjacent orientations (light-red connections). Facilitation occurs to-wards co-aligned cells up to a larger distance (dark-blue connections).

Sparse Gabor wavelets by local operations

Wed, 29 Jun 2005 00:00:00 +0000

relies on log-Gabor filters:
Sylvain Fischer, Filip Šroubek, Laurent U Perrinet, Rafael Redondo, Gabriel Cristóbal (2007). Self-Invertible 2D Log-Gabor Wavelets. International Journal of Computer Vision.
PDF Cite DOI Code URL

Schematic structure of the primary visual cortex implemented in the present study. Simple cortical cells are modeled through log-Gabor functions. They are organized in pairs in quadrature of phase (dark-gray circles). For each position the set of different orientations compose a pinwheel (large light-gray circles). The retinotopic organization induces that adjacent spatial positions are arranged in adjacent pinwheels. Inhibition interactions occur towards the closest adjacent positions which are in the direc-tions perpendicular to the cell preferred orientation and toward adjacent orientations (light-red connections). Facilitation occurs to-wards co-aligned cells up to a larger distance (dark-blue connections).

Efficient representation of natural images using local cooperation

Sat, 01 Jan 2005 00:00:00 +0000

relies on log-Gabor filters:
Sylvain Fischer, Filip Šroubek, Laurent U Perrinet, Rafael Redondo, Gabriel Cristóbal (2007). Self-Invertible 2D Log-Gabor Wavelets. International Journal of Computer Vision.
PDF Cite DOI Code URL

Schematic structure of the primary visual cortex implemented in the present study. Simple cortical cells are modeled through log-Gabor functions. They are organized in pairs in quadrature of phase (dark-gray circles). For each position the set of different orientations compose a pinwheel (large light-gray circles). The retinotopic organization induces that adjacent spatial positions are arranged in adjacent pinwheels. Inhibition interactions occur towards the closest adjacent positions which are in the direc-tions perpendicular to the cell preferred orientation and toward adjacent orientations (light-red connections). Facilitation occurs to-wards co-aligned cells up to a larger distance (dark-blue connections).

Coding static natural images using spiking event times: do neurons cooperate?

Wed, 01 Sep 2004 00:00:00 +0000

Progressive reconstruction of a static image using spikes in a multi-scale oriented representation.

Feature detection using spikes : the greedy approach

Thu, 01 Jul 2004 00:00:00 +0000

Network of integrate-and-fire neurons using Rank Order Coding B: spike timing dependant plasticity and emergence of orientation selectivity

Mon, 01 Jan 2001 00:00:00 +0000