Visual Neuroscience | Laurent Perrinet

Victor Vasarely (1971) Gare Montparnasse

Illusions visuelles

Hering illusion

Mais si nous les plaçons devant ce faisceau de lignes, alors elles apparaissent légèrement tordues.

L’explication se trouverait dans le fait que nous interprétons l’image en 3D et que les distorsions que nous attendons rendent plus plausibles des lignes courbées.

La vision montre là toute sa créativité à créer elle-même des illusions.

La perception comme processus émergent

La perception comme processus émergent

François Morrelet (1962) Mönchengladbach, Sphère - trames

La perception comme processus émergent

Étienne Rey, Trames

La perception comme processus émergent

Étienne Rey (2025) Variable Density, série Delaunay

# Neurosciences computationnelles

Neurosciences computationnelles

Étienne Rey (2010) Spectre audiographique

Anatomie du système visuel humain

Système visuel humain : le modèle HMAX

[Serre and Poggio, 2007]

Cortex visuel primaire

[Hubel & Wiesel, 1962]

Cortex visuel primaire

[Hubel & Wiesel, 1962]

Modèles hybrides d’IA

Utiliser des modèles d’apprentissage profond pour comprendre le cortex sensoriel [Yamins & DiCarlo, 2016]

Modèles de langage

Transformer: Attention is All You Need [Vaswani et al., 2017]

Les Neurosciences Computationnelles c’est un métier ?

Ingénieur, chercheur, journaliste, …

Neurosciences Computationnelles pour l’IA

Neurosciences Computationnelles pour la biologie

Topo Neurosciences Computationnelles

Laurent Perrinet

Computational Neuroscience Center

[2026-06-18]

Tout public / Me contacter : laurent.perrinet@univ-amu.fr

---

2026-04-11-intelligence-du-regard

Sat, 11 Apr 2026 00:00:00 +0000

L’intelligence du regard

Laurent Perrinet

Forum des Sciences Cognitives 2026

[2026-04-11]

Art-Sciences / Contact me @ laurent.perrinet@univ-amu.fr

Bonjour, je me présente : Laurent Perrinet. Je suis très heureux de participer au Forum des Sciences Cognitives et je remercie les organisateurs pour cette invitation. Je suis d’autant plus ravi d’y participer que je ne vais pas parler de mes recherches habituelles, mais plutôt exposer la collaboration que j’ai avec un artiste plasticien à Marseille. Mon objectif : vous convaincre des bénéfices que l’on peut tirer à s’ouvrir au monde artistique pour mieux percer les mystères de la cognition dans toute sa diversité.

Les objectifs de cet exposé seront multiples :

Vous faire découvrir certains artistes contemporains qui questionnent notre rapport aux nombres visuels
Montrer la diversité de la vision à travers les interactions entre art et science
Dévoiler certains mystères de la vision au travers de l’expérience artistique et les applications que cela peut avoir sur notre compréhension de la cognition

Art & Sciences révèlent la diversité de notre vision

Étienne Rey

## Art & Sciences révèlent la diversité de notre vision

Notre collaboration a commencé quand il m’a invité à présenter mon travail sur la perception visuelle au vernissage de cette œuvre qui représente une visualisation spatio-temporelle du spectre audiographique du son d’une cloche. Il est composé de multiples plaques semi-transparentes et dichroïques, c’est-à-dire ayant la capacité de présenter différentes couleurs selon l’angle de vue. Ce volume sculptural donne, de façon furtive, toute la profondeur de cette expérience sensorielle.

C’est là que se révèle « L’irraisonnable efficacité de la vision » — je reprends les mots de Wigner à propos de la capacité des mathématiques à sonder le monde. Car nous nous retrouvons devant un constat similaire : comment est-il possible avec aussi peu de moyens d’obtenir une perception si vivante du monde qui nous entoure ?

“L’irraisonnable efficacité de la vision”

Nage de la raie, 1894 [Étienne-Jules Marey]

L’enjeu est d’abord scientifique : comprendre le mouvement. Il a d’ailleurs donné son nom à l’ISM, l’Institute for Scientific Motion.

Et cette capacité n’est pas nouvelle.

“L’irraisonnable efficacité de la vision”

Panneau Des Lions [Grotte chauvet, -30 kA]

“L’irraisonnable efficacité de la vision”

Comment la vision a évolué… [LP, 2024, The Conversation]

L’organe de notre vision, ce sont nos yeux, dont l’anatomie est la suivante.

Premier miracle : de l’énergie lumineuse est transformée en un signal electro-chimique, la magie peut commencer.

Je ne vais pas rentrer dans les détails — il faudrait des heures — mais explorons plutôt ce que nous appelons…

Illusions visuelles

Ilusions of brightness or lightness Akiyoshi KITAOKA

Illusions visuelles

Illusions visuelles

Illusions visuelles : Paréidolie

Illusions visuelles : Paréidolie

Illusions visuelles : Paréidolie

Victor Vasarely (1971) Gare Montparnasse

La perception comme processus émergent

La perception comme processus émergent

François Morrelet (1962) Mönchengladbach, Sphère - trames

La perception comme processus émergent

Étienne Rey, Trames

C’est dans ce cadre que nous avons expérimenté avec Étienne Rey sur des trames, qui crée ces interférences. Émergence de nouvelles formes — hexagones comme utilisés à l’Alhambra — espaces tridimensionnels.

Je reviendrai sur le fait que vous pouvez transformer l’image en bougeant les yeux.

La perception comme processus émergent

Étienne Rey (2025) Variable Density, série Delaunay

---

La vibration des apparences

Paul Cézanne, Montagne Sainte-Victoire, 1904

le muset Granet est le musée de Cézanne

le titre de l’exposition fait référence…

La vibration des apparences

Merleau-Ponty, Sens et non-sens

Étienne Rey (2019) Horizon faille - Densité flou - Sans gravité - une poétique de l’air à Ardenome Avignon https://www.enrevenantdelexpo.com

La vibration des apparences

Étienne Rey (2025) Polaire

La vibration des apparences


def retino_grid(cr, N_rho=34, N_phi=233, N_H, N_V, offset,
 size_mag, ecc_max, alpha, c1, c2, power):
 # https://laurentperrinet.github.io/sciblog/posts/2020-04-16-creating-an-hexagonal-grid.html
 phi_v, rho_v = np.meshgrid(np.linspace(0, 2*np.pi, N_phi, endpoint=False),
 np.linspace(0, ecc_max, N_rho+1, endpoint=True)[1:], sparse=False, indexing='xy')
 phi_v[::2, :] += np.pi/N_phi

 offsets, colors = [-offset, offset], [c1, c2]
 for offset_, color in zip(offsets, colors):
 # convert to cartesian coordinates
 X, Y = (rho_v * np.sin(phi_v) + offset_+1)/2, (rho_v * np.cos(phi_v)+1)/2
 R = size_mag * rho_v**power / N_rho

 for x, y, r in zip(X.ravel(), Y.ravel(), R.ravel()):
 circle(cr, x, y, r)
 cr.set_source_rgba(*hue_to_rgba(color, alpha))
 cr.fill()
 return cr
c_blue, dc = 240, 60
opts = dict(N_rho=N_rho, N_phi=N_phi, N_H=N_H, N_V=N_V,
 offset=0.07, size_mag=0.3, ecc_max=0.8, alpha=0.80,
 c1=c_blue-dc, c2=c_blue+dc)
@disp
def draw(cr, N_H=N_H, N_V=N_V): cr = retino_grid(cr, **opts)

## À quoi sert la vision ?

Ilya Repin (1884) An Unexpected Visitor

Nous avons vu que la vision est un processus qui essaie de faire du sens — même s’il n’y en a pas forcément. Ce processus assemble différentes parties ensemble. On connaît le processus « comment », mais on peut se poser la question « pourquoi » : à quoi ça sert, la vision ?

C’est là qu’intervient Yarbus et cette peinture d’Ilya Repin : un soldat rentrant à la maison, tension liée à la surprise évoquée par le titre de la peinture.

À quoi sert la vision ?

Yarbus (1965) An Unexpected Visitor

À quoi sert la vision ?

Yarbus (1965) An Unexpected Visitor How long?

À quoi sert la vision ?

Yarbus (1965) An Unexpected Visitor - Age?

À quoi sert la vision : rétinotopie fovéale

À quoi sert la vision : rétinotopie fovéale

À quoi sert la vision : rétinotopie fovéale

E Rey et LP (2026) Formes & perception

À quoi sert la vision : rétinotopie fovéale

À quoi sert la vision : rétinotopie fovéale

Illusions visuelles

Étienne Rey, Spectre audiographique

Art & Sciences révèlent la vision en action

Art & Sciences révèlent la vision en action

Carlos Cruz-Diez (2013) Chromosaturation

Felice Varini.

Art & Sciences révèlent la vision en action : Tropique

Étienne Rey, Tropique

Art & Sciences révèlent la vision en action : Tropique

---

Art & Sciences révèlent la vision en action : Tropique

Étienne Rey, TRAME ÉLASTICITÉ

Autre collaboration de taille:

DIMENSIONS : 3 M DE HAUT 5 M DE LARGE INOX POLI MIROIR / ALUMINIUM / ACIER / MOTEURS / PROGRAMME TEMPS RÉEL À la Fondation Vasarely à Aix-en-Provence, Étienne Rey a choisi d’installer dans la salle des Intégrations architectoniques un ballet visuel hypnotique. Composé d’une succession de lames de miroirs, verticales et rotatives, l’installation Trame se joue des reflets et de la démultiplication de l’espace, offrant au spectateur une multiplicité de points de vue dans lesquels il peut se perdre à loisir. Par un effet de « porosité » recherché par l’artiste, le dispositif dialogue intensément avec les Intégrations.

Art & Sciences révèlent la vision en action : TRAME ÉLASTICITÉ

Art & Sciences révèlent la vision en action

Ede Rancz, Role of neuromodulators in active perception

Art & Sciences révèlent la vision en action

Ede Rancz, Role of neuromodulators in active perception

L’intelligence du regard

Laurent Perrinet

Forum des Sciences Cognitives 2026

[2026-04-11]

Art-Sciences / Contact me @ laurent.perrinet@univ-amu.fr

---

Diapositives supplémentaires

Victor Vasarely

Victor Vasarely (1962) Mönchengladbach, Sphère - trames

Victor Vasarely

Victor Vasarely (1977)Outdoor Vasarely artwork at the church of Pálos in Pécs.

Victor Vasarely

Victor Vasarely (1962) Supernovae

Victor Vasarely

Victor Vasarely (1976) Fondation Vasarely, Aix-en-Provence

Étienne Rey, Cristal n2

La vibration des apparences

2026-03-05-ue-natural-cognition

Thu, 05 Mar 2026 00:00:00 +0000

Artificial neural networks and machine learning applied to the understanding of biological vision

Laurent Perrinet

[2026-03-05] Master 1 Neuroscience, UE Natural Cognition, Artificial Cognition

Contact me @ laurent.perrinet@univ-amu.fr

Principles of Vision

What is the function of vision?

An Unexpected Visitor (Ilya Repin, 1884)

seeing= interacting with the visual world

social animals: looking at emotions

What is the function of vision?

active: the eye is always moving

https://fr.wikipedia.org/wiki/Alfred_Iarbous

consistency of eye traces

What is the function of vision?

An Unexpected Visitor - Age? (Yarbus, 1965)

social animals: looking at emotions

What is the function of vision?

Visual illusions

Ilusions of brightness or lightness Akiyoshi KITAOKA

Visual illusions

Visual illusions

Visual illusions

Visual illusions : Pareidolia

Visual illusions : Pareidolia

Visual illusions : Pareidolia

Principles of vision?

Computational neuroscience of vision

Computational neuroscience of vision

Anatomy of the Human Visual system

Human Visual system : the HMAX model

[Serre and Poggio, 2007]

Primary visual cortex

[Hubel & Wiesel, 1962]

Primary visual cortex

[Hubel & Wiesel, 1962]

Hybrid IA models

Using goal-driven deep learning models to understand sensory cortex [Yamins & DiCarlo, 2016]

a single neuron is selective to some visual features…

Principles of vision?

# Convolutional Neural Nets (CNN)

Convolutional Neural Nets (CNN)

Convolutional Neural Nets (CNN)

sota…

CNN: Mathematics

One-dimensional discrete convolution (eg in time) with a kernel $g$ of radius $K$: $$ (f \ast g)[n]=\sum_{m=-K}^{K} f[n-m] \cdot g[m] $$

CNN: Mathematics

Convolution of an image (two-dimensional) with a kernel $g$ of radius $K\times K$:

$$ (f \ast g)[x, y] = \sum_{i=-K}^{K} \sum_{j=-K}^{K} f[x-i, y-j] \cdot g[i, j] $$

CNN: Mathematics

Cross-correlation of an image (two-dimensional) with a kernel $g$ of radius $K\times K$:

$$ (f \ast \tilde{g})[x, y] = \sum_{i=-K}^{K} \sum_{j=-K}^{K} f[x+i, y+j] \cdot g[i, j] $$

CNN: Mathematics

CNN: Mathematics

Correlation of an image defined on several channels (note the order of the indices):

$$ (f \ast \tilde{g})[x, y] = \sum_{c=1}^{C} \sum_{c,i,j} f[c, x+i, y+j] \cdot g[c, i, j] $$

CNN: Mathematics

Correlation of a multi-channel image for multiple output channels (note the order of the indices):

$$ (f \ast \tilde{g})[k, x, y] = \sum_{c,i,j} f[c, x+i, y+j] \cdot g[k, c, i, j] $$

CNN: the HMAX model

CNN: challenges

# Principles of Vision

CNN: Predictive processing

modifications= adding sparse coding + feedback

CNN: Predictive processing

CNN: Topography

[Bosking et al, 1997]

CNN: Topography

# Artificial neural networks applied to the understanding of biological vision

Artificial neural networks and machine learning applied to the understanding of biological vision

Laurent Perrinet

[2026-03-05] Master 1 Neuroscience, UE Natural Cognition, Artificial Cognition

Contact me @ laurent.perrinet@univ-amu.fr

Foveated Retinotopy Improves Classification and Localization in CNNs

Mon, 23 Feb 2026 00:00:00 +0000

Foveated Retinotopy in CNNs. We represent Left an input image and how it is transformed by foveated retinotopy. We show below a representative reconstruction showing that it also acts as a cortical zoom on the image around the point of fixation. The transformed image is then fed to the ResNet deep learning architecture.

From falcons spotting prey to humans recognizing faces, the ability to rapidly process visual information depends on a foveated retinal organization that provides high-acuity central vision while preserving low-resolution peripheral vision. This organization is conserved along early visual pathways, yet remains under-explored in machine learning. Here, we examine the impact of embedding a foveated retinotopic transformation as a preprocessing layer on convolutional neural networks (CNNs) for image classification. By applying a log-polar mapping to off-the-shelf models and retraining them, we achieve comparable accuracy while improving robustness to scale and rotation. We demonstrate that this architecture is highly sensitive to shifts in the fixation point and that this sensitivity provides an effective proxy for defining saliency maps that facilitate object localization. Our results demonstrate that foveated retinotopy encodes prior geometric knowledge, providing a solution for visual searches and a meaningful classification robustness and localization trade-off. These findings provides a proof of concept in order to connect principles of biological vision with artificial networks, suggesting new, robust and efficient approaches for computer vision systems.

Foveated Retinotopy simulated by a log-polar map. We represent Left an input image with some geometrical objects and how it is transformed by the log-polar representation that implements foveated retinotopy. This shows that a rotation amounts to a translation on the polar axis (abscissa) and a zoom to a translation on the ordinates. We show right a representative reconstructionshowing that it also acts as a cortical zoom on the image around the point of fixation.

Population decoding of visual motion direction

Thu, 01 Jan 2026 00:00:00 +0000

🧠 Excited to share our latest research led by Alexandre Lainé and presented this summer at AREADNE 2026!

“Population decoding of visual motion direction”

Our work explores how populations of neurons in the primary visual cortex (V1) of marmoset monkeys encode visual motion direction, with a particular focus on understanding how uncertainty influences this neural decoding process. Key highlights:

Advanced population-level analysis of V1 neural responses to motion stimuli
Novel insights into how the brain handles uncertainty in visual motion processing
Marmoset model providing crucial translational insights for visual neuroscience

This research contributes to our fundamental understanding of how the visual system processes motion information at the earliest stages of cortical processing, with important implications for both basic neuroscience and potential clinical applications. Thank you to the AREADNE organizing committee for hosting such an inspiring conference!

Link to publication: https://laurentperrinet.github.io/publication/laine-26-areadne/

For deeper insights into uncertainty processing mechanisms in the visual cortex, see our Nature Communications Biology study:
Hugo Ladret, Nelson Cortes, Lamyae Ikan, Frédéric Chavane, Christian Casanova, Laurent U Perrinet (2023). Cortical recurrence supports resilience to sensory variance in the primary visual cortex. Nature Communications Biology.
PDF Cite DOI Dataset Code URL bioRxiv HAL

A New Look for Convolutional Deep Networks

Fri, 12 Dec 2025 00:00:00 +0000

🔬 Excited to present our latest research at the #MAIN2025 conference today!

🔗 https://www.main2025.org/

👁️ What if CNNs could see like humans? Our new work shows how foveated vision—concentrating processing at gaze center—makes networks more robust to perturbations & great at localization. Inspired by human vision’s architecture (high-resolution foveal center, low-resolution periphery), we embedded this retinotopic transformation into CNN architectures, allowing to actively scan the image. This gives literally a new look to #ConvNets !

📄 Paper:

Jean-Nicolas Jérémie, Emmanuel Daucé, Laurent U Perrinet (2026). Foveated Retinotopy Improves Classification and Localization in CNNs. Vision.

PDF Cite DOI URL arXiv

#DeepLearning #ComputerVision #AI #Research #NeuralNetworks #NeuroAI #OpenScience I love #Montreal

Media storm - share if you like :

2025-05-26-master-m-4-nc

Mon, 26 May 2025 00:00:00 +0000

Artificial neural networks and machine learning applied to the understanding of biological vision

Laurent Perrinet

[2025-05-26] Master M4NC de l’institut NeuroMod, cours Prospective Innovation and Research

Contact me @ laurent.perrinet@univ-amu.fr

Principles of Vision

What is the function of vision?

An Unexpected Visitor (Ilya Repin, 1884)

seeing= interacting with the visual world

social animals: looking at emotions

What is the function of vision?

active: the eye is always moving

https://fr.wikipedia.org/wiki/Alfred_Iarbous

consistency of eye traces

What is the function of vision?

An Unexpected Visitor - Age? (Yarbus, 1965)

social animals: looking at emotions

What is the function of vision?

Visual illusions

Ilusions of brightness or lightness Akiyoshi KITAOKA

Visual illusions

Visual illusions

Visual illusions

Visual illusions

Visual illusions : Pareidolia

Visual illusions : Pareidolia

Visual illusions : Pareidolia

Principles of vision?

Computational neuroscience of vision

Computational neuroscience of vision

Anatomy of the Human Visual system

Human Visual system : the HMAX model

[Serre and Poggio, 2007]

Primary visual cortex

[Hubel & Wiesel, 1962]

Primary visual cortex

[Hubel & Wiesel, 1962]

Hybrid IA models

Using goal-driven deep learning models to understand sensory cortex [Yamins & DiCarlo, 2016]

a single neuron is selective to some visual features…

Convolutional Neural Nets (CNN)

Convolutional Neural Nets (CNN)

CNN: challenges

CNN: Predictive processing

modifications= adding sparse coding + feedback

CNN: Predictive processing

CNN: Topography

[Bosking et al, 1997]

CNN: Topography

Computational neuroscience of vision

Spiking Neural Networks (SNN)

SNN: Leaky Integrate-and-Fire Neuron

Review on Precise Spiking Motifs.

SNN in neurobiology

SNN in neurobiology

SNN in neurobiology

SNN in neurobiology

SNN: Spiking motifs

Review on Precise Spiking Motifs.

SNN: Spiking motifs

Review on Precise Spiking Motifs.

SNN: Spiking motifs

Review on Precise Spiking Motifs.

2 MINUTE

A nice HSD neuron

SNN in neuromorphic engineering

From frame-based to event-based cameras.

2 MINUTE

event-based cameras

SNN in neuromorphic engineering

The HD-SNN neural network.

2 MINUTE

For instance, we show how precise spike times may be used to detect the direction of motion from such a stream of events in an ultrafast fashion.

SNN in neuromorphic engineering

2 MINUTE

A nice HSD neuron

For instance, we show how precise spike times may be used to detect the direction of motion from such a stream of events in an ultrafast fashion.

SNN in neuromorphic engineering

The HD-SNN neural network.

2 MINUTE

nice kernels

SNN in neuromorphic engineering

The HD-SNN neural network.

Spiking Neural Networks (SNN)

# Artificial neural networks applied to the understanding of biological vision

Artificial neural networks and machine learning applied to the understanding of biological vision

Laurent Perrinet

[2025-05-26] Master M4NC de l’institut NeuroMod, cours Prospective Innovation and Research

Contact me @ laurent.perrinet@univ-amu.fr

2025-04-18-vibration-apparences

Fri, 18 Apr 2025 00:00:00 +0000

La vibration des apparences

Laurent Perrinet

Journées d’Ouverture Scientifique (JOS)

[2025-04-18]

Art-Sciences / Contact me @ laurent.perrinet@univ-amu.fr

## "L'irraisonnable efficacité de la vision"

Comment la vision a évolué… [Perrinet, 2024]

Illusions visuelles

Ilusions of brightness or lightness Akiyoshi KITAOKA

Illusions visuelles

Illusions visuelles

Illusions visuelles

Illusions visuelles : Paréidolie

Illusions visuelles : Paréidolie

Illusions visuelles : Paréidolie

## Neurosciences computationnelles de la vision

Étienne Rey, SPECTRE AUDIOGRAPHIQUE – DIFFRACTION

Anatomie du système visuel humain

Cortex visuel primaire

[Hubel & Wiesel, 1962]

Cortex visuel primaire

[Hubel & Wiesel, 1962]

Modèles hybrides d’IA

Using goal-driven deep learning models to understand sensory cortex [Yamins & DiCarlo, 2016]

## Art & Sciences

Étienne Rey

Tropique

Étienne Rey, Tropique

Tropique

Étienne Rey, Cristal n2

Étienne Rey, TRAME ÉLASTICITÉ

TRAME ÉLASTICITÉ

## De la nature des choses

Phyllotaxie

Par Cmglee — Travail personnel, CC BY-SA 4.0, Lien

Étienne Rey, Densité flou

Étienne Rey, Horizon Faille

Caustiques

La vibration des apparences

Paul Cézanne, Montagne Sainte-Victoire, 1904

La vibration des apparences

Merleau-Ponty, Sens et non-sens

Étienne Rey, Trames

La vibration des apparences

N_rho, N_phi = 34, 233

def retino_grid(cr, N_rho, N_phi, N_H, N_V, offset, size_mag,
 ecc_max, alpha, c1, c2, power, operator,
 channel='both'):

 cr.scale(N_H, N_V)
 cr.set_operator(operator)

 # https://laurentperrinet.github.io/sciblog/posts/2020-04-16-creating-an-hexagonal-grid.html
 phi_v, rho_v = np.meshgrid(np.linspace(0, 2*np.pi, N_phi, endpoint=False),
 np.linspace(0, ecc_max, N_rho+1, endpoint=True)[1:], sparse=False, indexing='xy')
 phi_v[::2, :] += np.pi/N_phi

 offsets = [-offset, offset]
 colors = [c1, c2]

 for offset_, color in zip(offsets, colors):
 # convert to cartesian coordinates
 X = rho_v * np.sin(phi_v) + offset_
 Y = rho_v * np.cos(phi_v)
 X = (X+1)/2
 Y = (Y+1)/2
 R = size_mag * rho_v**power / N_rho

 # draw 
 for x, y, r in zip(X.ravel(), Y.ravel(), R.ravel()):
 circle(cr, x, y, r)
 cr.set_source_rgba(*hue_to_rgba(color, alpha))
 cr.fill()

 return cr

c_blue = 240
dc = 60
opts = dict(N_rho=N_rho, N_phi=N_phi, N_H=N_H, N_V=N_V,
 offset=0.07, size_mag=0.3, ecc_max=0.8, alpha=0.80, c1=c_blue-dc, c2=c_blue+dc, power=.5, operator=cairo.OPERATOR_MULTIPLY)

@disp
def draw(cr, N_H=N_H, N_V=N_V): cr = retino_grid(cr, **opts)

Étienne Rey, La vibration des apparences

La vibration des apparences

# [La vibration des apparences](https://laurentperrinet.github.io/slides/2025-04-18-vibration-apparences/?transition=fade) ## *[Laurent Perrinet](https://laurentperrinet.github.io/talk/2025-04-18-vibration-apparences/)* ### [Journées d’Ouverture Scientifique (JOS)](https://jos.lis-lab.fr/) ### [2025-04-18] ![logo](https://github.com/laurentperrinet/perrinet_curriculum-vitae.tex/raw/master/logotypes/troislogos.jpg) [Art-Sciences](https://laurentperrinet.github.io/project/art-science/) / Contact me @ [laurent.perrinet@univ-amu.fr](mailto:laurent.perrinet@univ-amu.fr)

2025-02-11-neuromath

Tue, 11 Feb 2025 00:00:00 +0000

[2025-02-11] When Cortical Neurons Talk Sideways: Beyond Feedforward Visual Processing

Laurent Perrinet - https://laurentperrinet.github.io Séminaire Neuromathématiques, Collège de France

Hi, thanks for the introduction! I am Laurent Perrinet, a researcher in computational neuroscience and currently a research director at CNRS at the Institute of Neuroscience of la Timone in Marseille. Thank you for inviting me to participate in this “NeuroMathematics” seminar at the intersection of mathematics and neuroscience.

As an engineer by training, I could have pursued a career in aeronautics rather than becoming a neuroscientist. It is thanks to my mathematics professor Manuel Samuelides that I discovered the beauty of neural networks at the end of my engineering studies. This developped a curiosity, and thanks to him, I was also able to study in a mastere of cognitive sciences (now called CogMaster) in 1998. This is where I particularly want to acknowledge Jean Petitot - for his course I discovered how natural image statistics could link to principles in the central nervous system. This was a vivid revelation, and I’m grateful for his guidance in my academic path. Today’s seminar represents a return to these roots, as I’ll present my research progress since my mastere thesis on this very topic.

Today, I will address our current knowledge about horizontal connectivity rules in V1. Why is this important? As a matter of fact, one main function of sensory systems, such as the pivotal role of the primary visual cortex for vision, is to bind together the different visual features to help ultimately build a global perception.

Paysage catalan (Le Chasseur), Joan Miró (1924)

Trames (Étienne Rey)

Contour detection and the Association Field

[Field et al, 1993]

Contour detection and the Association Field

[Field et al, 1993]

Contour detection and the Association Field

[Field et al, 1993]

Natural Images : Edges are on a common circle

[Sigman et al, 2001]

A significant contribution to understanding the association field came from studying edge co-occurrences in natural images by Sigman et al. (2001). They quantified the probability density function of edge co-occurrences based on their relative positions and orientations. The figure demonstrates this by showing the spatial distribution patterns for edges relative to a reference edge at different orientations. For iso-oriented edges (a), the co-occurrence pattern shows clear structure. As the relative orientation increases through 22.5° (b), 45° (c), 67.5° (d), to 90° (e), distinct spatial patterns emerge.

A key finding was that for any given relative orientation between edges, the angle of maximal interaction occurs at the bisector between the orientations. This suggests that co-occurring edges tend to lie on a common circle - a property known as cocircularity. Panel (f) illustrates this geometrical principle: given two edges at angles w (red, 20°) and c (blue, 40°), the cocircularity solutions (green lines at 30° and 120°) represent the possible orientations of connecting circular arcs. This mathematical relationship provides insights into how the visual system might leverage statistical regularities in natural scenes for contour integration. We will go back into the details of this a bit further in the talk.

This association field concept provided a compelling framework for understanding how the visual system may implement contour integration through neural connectivity patterns. but before going there we should go back to the basic anatomy of the visual cortex.

Contour detection and the Association Field

[Field et al, 1993]

Dynamics of vision

Human Visual system (Grimaldi et al 2022)

Anatomy of the Human Visual system

Human Visual system (Grimaldi et al 2022)

Thalamic, short- & long-range lateral, inter-areal

A key feature of primary visual cortex is its layered organization, which is shared across cortical areas. The main thalamic input arrives in layer 4, which connects to a dense network of vertical connections across layers. These columns can then communicate via horizontal connections within layers. Hubel and Wiesel also proposed the ice-cube model that every point in the visual field produces a response in a 2 mm x 2 mm area of the cortex. Such an area can contain two complete groups of ocular dominance columns, 16 blobs and interblobs that may contain more than two times all of the orientations possible across 180 degrees. This region of the cortex, which Hubel and Wiesel called a hypercolumn (or, more generally, a cortical module) seems both necessary and sufficient for analyzing the image of a point in visual space. Because the cortex is a continuous cellular layer and because it is very hard to establish the boundaries of these modules physically, their existence from a functional standpoint is still the subject of debate. https://thebrain.mcgill.ca/flash/a/a_02/a_02_cl/a_02_cl_vis/a_02_cl_vis.html

Figure 9.2. Hypercolumn Diagram. Ocular dominance columns are segregated into left and right eye inputs. Orientation columns are neurons that get excited at different orientations and a cluster of these is called a pinwheel. Blobs are color selective and for every pinwheel there is a blob. (Credit: McGill: The Brain from Top to Bottom, Figure of hypercolumns, Copyleft https://copyleft.org/, https://thebrain.mcgill.ca/flash/a/a_02/a_02_cl/a_02_cl_vis/a_02_cl_vis.html. No modifications.)

From: https://pressbooks.umn.edu/sensationandperception/chapter/columns-and-hypercolumns-in-v1/

Thalamic, short- & long-range lateral, inter-areal

[Markov et al 2011]

Anatomy of the Primary Visual Cortex

[Kaschube et al (2010)]

V1 is central to these pathways and shows distinctive anatomical and functional properties along with a complex topographical organization.

This figure from Kaschube et al. (2010) illustrates the organization of orientation preference maps in primary visual cortex (V1). Individual V1 neurons exhibit selective responses to oriented visual stimuli (as denoted by varying hues Colors code preferred ORs as indicated by the bars in (C)), with their spatial arrangement following highly structured patterns across the cortical surface. Panel B shows Synthetic orientation-maps of equal column spacing Λ but widely different pinwheel densities ρ. Left to right: solutions of different models: (13–16).. (C) High (blue frame) and low (orange frame) pinwheel density regions in tree shrew visual cortex. (D to F), Optically recorded orientation-maps in tree shrew (D), galago (E), and ferret (F) visual cortex. Regions shown in (C) are marked in (D). White arrows in (F) mark selected pinwheel centers. Framed regions in (C) and (F) are magnified. In many mammals including cats, monkeys and ferrets, orientation preference is organized in a quasi-periodic manner, forming what are known as orientation preference maps. These maps show remarkable consistency in their geometric properties across species, particularly in the spatial organization of pinwheel centers where orientation preferences converge.

However, this organization shows important species-specific variations. Most notably, while primates and carnivores display orderly orientation maps with smooth transitions between preferred orientations, rodents lack such maps and instead show a “salt-and-pepper” arrangement where neighboring neurons have seemingly random orientation preferences. This organizational diversity raises interesting questions about the computational advantages of these different architectures and their relationship to visual processing requirements and behavioral needs across species.

Horizontal connectivity links different hypercolumns

[Bosking et al, 1997]

This figure shows landmark results by Bosking et al. (1997) combining orientation preference maps with retrograde tracers. After injecting tracers (white arrow), they found labeled synapses (black dots) primarily connecting neurons of similar orientation preference, leading to the influential “like-to-like” connectivity hypothesis. However, later studies by Hunt, Goodhill and others revealed significant diversity in these connection patterns across cortical regions and species, suggesting more complex connectivity rules than initially proposed. This nuanced understanding has important implications for how we think about the functional organization of horizontal connections in V1.

Contour detection and the Association Field

[Field et al, 1993]

The like-to-like hypothesis

[Field et al, 2013]

The resemblance between what was shown by Bosking and the structure of the association field that we saw above is such that it is tempting to align both and state that the function of horizontal connections is to bind neurons with a selectivity to *similar orientations over long distances. This like-to-like hypothesis has been influential in understanding horizontal connectivity patterns.

However, we should be cautious about overstating these relationships. While horizontal connections show some orientation specificity, recent evidence indicates the connectivity patterns are more complex and heterogeneous than initially proposed. The functional role of this diverse connectivity remains an active area of investigation.

During the remainder of this talk**, I will try to shed light on our current knowledege on horizontal connectivities.

Supplementary: the HMAX model

[Serre and Poggio, 2007]

Supplementary: Convolutional Neural Nets (CNN)

[Chavane, LP and Rankin, 2022]

Supplementary: Orientation selectivity in V1

[Hubel & Wiesel, 1962]

Supplementary: Orientation selectivity in V1

[Hubel & Wiesel, 1962]

Supplementary: Marr’s three levels of analysis

Marr, 1982

cut in different levels: Marr (+ Poggio)
arbitrary, but useful division of labor= computational / algorithm / hardware
here:
anatomy
algorithm / model
function

First: What is the anatomy of horizontal connections?

<!--

[Marr, 1982]]

-->

Challenging the like-to-like hypothesis

Challenging the like-to-like hypothesis

Revisiting Horizontal Connectivity Rules in V1: From like-to-like towards like-to-All [Chavane, LP and Rankin, 2022]

This figure illustrates different hypothetical connectivity rules for horizontal connections in V1. The target neuron (large circle on left) has a specific orientation preference indicated by its color. Following the classical like-to-like hypothesis (shown in panel A), this neuron would preferentially connect to other neurons with matching orientation preference (similar colors) across multiple hypercolumns, as indicated by the vertical red arrows. The radial spread of connections spans approximately three hypercolumns, consistent with anatomical observations. Each hypercolumn contains a complete set of orientation preferences, represented by the different colored neurons.

This first schematic (noted A) represents one of the like-to-like connectivity rules, where horizontal connections strictly follow orientation similarity.

Challenging the like-to-like hypothesis

Revisiting Horizontal Connectivity Rules in V1: From like-to-like towards like-to-All [Chavane, LP and Rankin, 2022]

Challenging the like-to-like hypothesis

Revisiting Horizontal Connectivity Rules in V1: From like-to-like towards like-to-All [Chavane, LP and Rankin, 2022]

Challenging the like-to-like hypothesis

Revisiting Horizontal Connectivity Rules in V1: From like-to-like towards like-to-All [Chavane, LP and Rankin, 2022]

Challenging the like-to-like hypothesis

Let’s first shows some functional evidence.

This video shows voltage-sensitive dye imaging (VSDI) data from cat primary visual cortex (area 17) in response to a local oriented grating stimulus. The visualization reveals two key aspects:

The broader activation pattern shown by overall fluorescence changes (gray)
The more restricted orientation-selective response pattern (colored regions)

Two contours are overlaid: a red line marking the boundary of significant activation, and a white line delineating regions with statistically significant orientation selectivity. The orientation selectivity is encoded by color hue.

The bottom plots quantify the spatiotemporal dynamics by showing:

Left: The total activated cortical area over time
Right: The extent of orientation-selective regions over time

Together, these measurements demonstrate how orientation-selective signals propagate laterally beyond the classical feedforward input zone through horizontal connections, while maintaining some degree of feature selectivity.

Challenging the like-to-like hypothesis

Revisiting Horizontal Connectivity Rules in V1: From like-to-like towards like-to-All [Chavane, LP and Rankin, 2022]

This figure shows spatial and temporal dynamics of orientation selectivity in cat V1 analyzed from voltage-sensitive dye imaging data. Panel A displays a cortical orientation map averaged over the final 145ms of the response, where hue indicates preferred orientation and brightness shows orientation tuning strength. The dotted red line delineates the expected retinotopic boundary of feedforward input based on Albus (2004).

The inset quantitatively compares the spatial extent of:

Total cortical activation (grey contour)
Orientation-selective activation (black contour)
Theoretical feedforward input boundary (red contour)

This data demonstrates that orientation-selective responses propagate laterally beyond the classical feedforward input zone through horizontal connections, while maintaining some degree of feature selectivity. The systematic comparison between total activation and selective activation provides direct evidence for how horizontal connectivity shapes the spatiotemporal dynamics of orientation processing in V1.

Challenging the like-to-like hypothesis

Revisiting Horizontal Connectivity Rules in V1: From like-to-like towards like-to-All [Chavane, LP and Rankin, 2022]

Challenging the like-to-like hypothesis

Revisiting Horizontal Connectivity Rules in V1: From like-to-like towards like-to-All [Chavane, LP and Rankin, 2022]

Panel C displays intracellular recordings of subthreshold responses visualized as a visuotopic orientation polar map. The color hue represents preferred orientation while brightness indicates the strength of orientation tuning in the membrane potential. White contours outline regions showing statistically significant responses based on both amplitude and orientation selectivity criteria. The middle plots show averaged subthreshold responses to four different oriented stimuli (color-coded) at specific recording locations (marked by circle, triangle and square symbols), with scale bars indicating 50 ms and 1 mV. On the right, normalized orientation tuning curves are shown, computed by integrating responses within a fixed temporal window (shaded region in middle panel). The black circle marks the spontaneous activity level for the depolarizing integral measurement.

These shows a direct functional evidence for a diversity of tuning profile in th horizontal connectivity.

Challenging the like-to-like hypothesis

[Voges and LP, 2012]

Challenging the like-to-like hypothesis

Revisiting Horizontal Connectivity Rules in V1: From like-to-like towards like-to-All [Chavane, LP and Rankin, 2022]

Figure 4 illustrates a neural field model that bridges anatomical structure with functional observations in V1, as developed by Rankin and Chavane (2017).

Panel A depicts radial connectivity profiles with Gaussian-decaying inhibition and distance-dependent excitation that peaks periodically at multiples of distance L. The Ring Width (RW) parameter controls the spread of these excitatory peaks.

Panel B shows how local orientation preference maps influence lateral connectivity patterns under different orientation bias (BR) values in the recurrent connections.

Panel C quantifies the orientation tuning that emerges from these connectivity patterns. While orientations are uniformly represented globally, the local excitatory component shows strong bias around -60°. As BR increases above 0.5, the lateral connection orientation bias strengthens, reaching values around k=1 (consistent with Buzás et al. 2006).

Challenging the like-to-like hypothesis

Revisiting Horizontal Connectivity Rules in V1: From like-to-like towards like-to-All [Chavane, LP and Rankin, 2022]

Challenging the like-to-like hypothesis

Revisiting Horizontal Connectivity Rules in V1: From like-to-like towards like-to-All [Chavane, LP and Rankin, 2022]

Panel F maps the normalized selective area (relative to the feedforward footprint) across Ring Width (RW) and orientation bias (BR) parameters. White contours delineate anatomically plausible ranges where k values fall between 0.7-1.2, consistent with experimental measurements. The green region indicates parameter combinations that additionally satisfy constraints on both orientation preference and the observed radial decay of selectivity.

The neural field model effectively connects anatomical connectivity patterns with functional observations of orientation selectivity propagation in V1. The resulting connectivity structure exhibits similarities with “association field” patterns, suggesting potential optimization for encoding natural image statistics. This framework provides a quantitative basis for investigating computational principles underlying horizontal connectivity in visual cortex.

Challenging the like-to-like hypothesis

Revisiting Horizontal Connectivity Rules in V1: From like-to-like towards like-to-All [Chavane, LP and Rankin, 2022]

Modelling the Association field

[Field et al, 2013]

Edge co-occurences in natural images

Revisiting Horizontal Connectivity Rules in V1: From like-to-like towards like-to-All [Chavane, LP and Rankin, 2022]

Panel A illustrates the groundbreaking approach developed by Geisler et al. (2001) for analyzing edge statistics in natural images. The method involves:

Detecting oriented edge elements in natural images (shown as red segments)
For each edge pair, measuring:
- Their relative orientation difference (𝜃)
- The relative position angle (𝜙)
- Center-to-center distance (d)
- Azimuth difference (φ)
- Co-circularity parameter ψ = φ - θ/2

This quantitative analysis reveals two key distributions:

A strong bias for parallel edge arrangements, evident in the orientation difference histogram
A marked preference for co-circular alignments, shown in the relative position histogram

These statistics vary significantly across image databases. For example, images containing animals exhibit enhanced co-circularity compared to general natural scenes. This suggests that rather than implementing a single fixed association field, the visual system may need to handle diverse statistical regularities present in natural inputs.

The next section will examine how these statistical regularities inform computational models of the association field.

Edge co-occurences in natural images

[Geisler, 2001]

Edge co-occurences in natural images

Edge co-occurrences can account for rapid categorization of natural versus animal images [LP and Bednar, 2015]

The probability distribution function p(ψ,θ) represents the distribution of the different geometrical arrangements of edges’ angles, which we call a “chevron map”. We show here the histogram for non-animal natural images, illustrating the preference for co-linear edge configurations. For each chevron configuration, deeper and deeper red circles indicate configurations that are more and more likely with respect to a uniform prior, with an average maximum of about 3 times more likely, and deeper and deeper blue circles indicate configurations less likely than a flat prior (with a minimum of about 0.8 times as likely). Conveniently, this “chevron map” shows in one graph that non-animal natural images have on average a preference for co-linear and parallel edges, (the horizontal middle axis) and orthogonal angles (the top and bottom rows), along with a slight preference for co-circular configurations (for ψ =0 and ψ = ± π/2, just above and below the central row).

Edge co-occurences in natural images

Edge co-occurrences can account for rapid categorization of natural versus animal images [LP and Bednar, 2015]

Edge co-occurences in natural images

Edge co-occurrences can account for rapid categorization of natural versus animal images [LP and Bednar, 2015]

This figure shows classification performance across image categories using different statistical features. We used an SVM classifier with three feature sets: first-order orientation statistics (FO), the reduced 2D “chevron map” (CM), and full 4D second-order statistics (SO). The classification accuracy (F1 score) was tested for distinguishing between image categories. Results show strong performance in separating man-made from natural images, as expected. More notably, the classifier achieved ~80% accuracy in discriminating animal vs non-animal natural images, matching human performance levels reported by Serre et al. This suggests that relatively simple edge co-occurrence statistics contain sufficient information for basic image categorization tasks, without requiring higher-level semantic processing.

We also found that our model made the same errors as humans do: if an image without an animal contains more co-circular edges, it is more likely to be falsely categorized as containing an animal.

Edge co-occurences in natural images

Edge co-occurrences can account for rapid categorization of natural versus animal images [LP and Bednar, 2015]

Can we explain the diversity ?

[Bosking et al, 1997]

Indeed, this diversity is revealed in the anatomical data: V1 horizontal connectivity exhibits more complexity than suggested by the classical like-to-like hypothesis. While orientation-specific connections exist, they coexist with non-selective connections that link neurons irrespective of their tuning preferences. This diversity likely serves multiple computational functions:

Specific connections could support contour integration and feature binding
Non-selective connections may enable broad contextual modulation
Mixed connectivity patterns could help maintain network stability while preserving functional specificity

This anatomical heterogeneity aligns with V1’s role in both specialized feature detection and broader contextual processing. Understanding how these distinct connectivity patterns interact remains an active area of research in visual neuroscience.

Predictive processing

To understand the diversity in horizontal connectivity patterns, we developed a biologically plausible hierarchical model based on Convolutional Neural Networks (CNNs) backbone. The model processes natural images through multiple convolutional layers organized in a hierarchical structure:.

Natural image as input

Local receptive fields via convolution operations

Hierarchical processing through multiple layers

Predictive processing

To bridge the gap between anatomical observations and functional requirements of visual processing, We added two key ingredients in the sparse deep predictive coding (SDPC) model :

Sparse connectivity patterns:

Enforcing regularization of the activity map using L1 penalty

Activity computed via recurrent local connectivity

Similar to biological observations

Feedback from efferent layers:

Predicts activity of afferent layer

Only residual prediction error is processed

Defines long-range inter-areal connectivity

Specific influence demonstrated in Neural Computation paper

By defining a cost on minimizing the prediction error in each layer, everything stays derivable, such that we can use a classical gradient descent. These additions should allow us to better understand how feedback shapes visual processing in biological neural networks.

Predictive processing

Predictive processing

More specifically in the context of our focus today, we can look at the co-occurence

llustration of the procedure to generate interaction map. In this illustrative example we consider a V1 representation with only 4 feature maps (represented in the upper-left box). Step 1 is to extract a neighborhood (of size 3x3 in the illustration only) around the most strongly activated neuron (represented with a red square in the illustration) for a given central preferred orientation (denoted ✓ c ). Step 2 is to normalize the neural activity in the extracted neighborhood using the marginal activity (see Eq.8). Step 3 is to compute the resulting orientation and activity at every position of the neighborhood using a circular mean (see Eq. 11 and Eq. 12 respectively). To keep a concise figure we have illustrated the computation of the central edge of the interaction map only. For simplification, the illustration shows only 1 neighborhood extraction whereas the interaction maps shown in the paper are computed by averaging neighborhoods centered on the 10 most strongly activated neurons

Predictive processing

Predictive processing

Predictive processing with pooling

## Challenging the like-to-like hypothesis

[Bosking et al, 1997]

Challenging the like-to-like hypothesis

Revisiting Horizontal Connectivity Rules in V1: From like-to-like towards like-to-All [Chavane, LP and Rankin, 2022]

To conclude, our review of horizontal connectivity in V1 reveals patterns more complex than initially theorized. The classical like-to-like hypothesis, while valuable, doesn’t fully capture the diversity of observed connectivity patterns.

Mathematical modeling has proven essential in bridging theory and biology. Our predictive processing framework shows how simple computational principles can explain the emergence of these complex connectivity patterns. The model demonstrates how feedback influences lateral interactions and reproduces key experimental observations.

However, important questions remain unanswered. We need to better understand how precise timing information is encoded in these circuits, how temporal dynamics shape processing, and whether similar principles apply across other cortical areas.

These fundamental questions will guide future experimental and theoretical work as we continue to unravel the computational principles of cortical processing.

[2025-02-11] When Cortical Neurons Talk Sideways: Beyond Feedforward Visual Processing

Laurent Perrinet - https://laurentperrinet.github.io Séminaire Neuromathématiques, Collège de France

Dynamics of vision

Dynamics of vision

Visual latencies (see review).

Dynamics of vision

Sensorimotor delays (Perrinet & Friston 2014)

Dynamics of vision

Sensorimotor delays (Perrinet & Friston, 2014)

Dynamics of vision

Diagonal markov model (Khoei et al, 2017).

Dynamics of vision

Flash-lag effect: MBP (Khoei et al, 2017)

Dynamics of vision

Dynamics of vision: Neural modeling

Spiking Neural Networks: Spiking motifs

[Grimaldi et al, 2023, Precise Spiking Motifs]

Spiking Neural Networks: Spiking motifs

Review on Precise Spiking Motifs.

Spiking Neural Networks: Spiking motifs

Review on Precise Spiking Motifs.

Spiking Neural Networks: HD-SNN

2 MINUTE

We then used a neural network with a classical architecture, which we enhanced by using an impulse representation that takes into account different possible synaptic delays. In this figure, we have represented the input in the left grid, which represents the occurrence of spikes of positive or negative polarity. Then we have represented different processing channels denoted by the colors green and orange, which are applied to this input to produce membrane activity. As illustrated above, this activity will produce output pulses, notably in synaptic connection nuclei, with heterogeneous delays corresponding to the detection of precise spatio-temporal patterns.

Spiking Neural Networks: HD-SNN

2 MINUTE

One advantage of this network is that it is differentiable, enabling us to apply classical machine learning methods, notably supervised learning. We then see the emergence of different convolution kernels, and here I represent a subset of its kernels for different directions, as denoted by the red arrows on the left of the graph. It shows the kernels obtained on the spatial representation according to the different columns, and each row represents the different delays from a delay of one on the right to a delay of 12 time steps on the left. Detectors that follow the motion emerge. For example, for the top line from top to bottom. These kernels integrate both positive neurons in red and negative polarity inputs in blue. Such spatio-temporal filtering is observed in neurobiology, but to my knowledge had never been observed in a model of spiking neurons trained under natural conditions.

Spiking Neural Networks: HD-SNN

Spiking Neural Networks: HD-SNN

Spiking Neural Networks: HD-SNN

When Cortical Neurons Talk Sideways: Beyond Feedforward Visual Processing

Tue, 11 Feb 2025 00:00:00 +0000

In this seminar we will challenge the traditional understanding of neuronal connectivity in primary visual cortex. While current theory suggests that neurons connect preferentially to others with similar orientation preferences, I will present evidence for a more complex connectivity pattern based on a distance-dependent rule: short-range connections show a like-to-like bias, while long-range connections connect more widely. This revised model better explains how the visual cortex processes complex stimuli and accounts for observed variations in neuronal interactions at different scales.

When: Wednesday 11th of February, 2025 from 14:30 to 16h30.
Where: room D2.2 of Collège de France
Accompanying paper
Revisiting Horizontal Connectivity Rules in V1: From like-to-like towards like-to-All
Frédéric Chavane, Laurent U Perrinet, James Rankin

PDF Cite DOI URL

How and why foveated retinotopy provides efficient vision

Wed, 08 Jan 2025 00:00:00 +0000

When: Wednesday 9th of January, 2025 at 12 noon.
Where: CRN seminar room, Montreal General Hospital, Livingston Hall, L7-140, with hybrid option.
Read the corresponding paper
Foveated Retinotopy Improves Classification and Localization in CNNs
Jean-Nicolas Jérémie, Emmanuel Daucé, Laurent U Perrinet

PDF Cite DOI URL arXiv

Population decoding of visual motion direction in V1 marmoset monkey : effects of uncertainty

Wed, 01 Jan 2025 00:00:00 +0000

🧠 Excited to share our latest research led by Alexandre Lainé and presented this summer at CNS2025 in beautiful Firenze, Italy!

“Population decoding of visual motion direction in V1 marmoset monkey: effects of uncertainty”

Advanced population-level analysis of V1 neural responses to motion stimuli
Novel insights into how the brain handles uncertainty in visual motion processing
Marmoset model providing crucial translational insights for visual neuroscience

This research contributes to our fundamental understanding of how the visual system processes motion information at the earliest stages of cortical processing, with important implications for both basic neuroscience and potential clinical applications. Thank you to the CNS organizing committee for hosting such an inspiring conference in the stunning venue of Palazzo dei Congressi in Villa Vittoria! 🇮🇹

#ComputationalNeuroscience #VisualNeuroscience #MotionProcessing #CNS2025 #Neuroscience #Research #MarmosetModel #V1 #PopulationDecoding

Link to publication: https://laurentperrinet.github.io/publication/laine-25-cns/

For deeper insights into uncertainty processing mechanisms in the visual cortex, see our Nature Communications Biology study:
Hugo Ladret, Nelson Cortes, Lamyae Ikan, Frédéric Chavane, Christian Casanova, Laurent U Perrinet (2023). Cortical recurrence supports resilience to sensory variance in the primary visual cortex. Nature Communications Biology.
PDF Cite DOI Dataset Code URL bioRxiv HAL
see a follow-up in:
Alexandre Lainé, Nicholas J. Priebe, Guillaume S. Masson, Laurent U Perrinet (2026). Population decoding of visual motion direction. Proceedings of AREADNE.
PDF Cite Poster LinkedIn
https://neuromatch.social/@laurentperrinet/115050564011598328
https://www.linkedin.com/posts/laurent-perrinet-1857b9_population-decoding-of-visual-motion-direction-activity-7363238280143745026-zPkg
https://bsky.app/profile/laurentperrinet.bsky.social/post/3lwowjtpbw22a

Kernel Heterogeneity Improves Sparseness of Natural Images Representations

Tue, 20 Aug 2024 00:00:00 +0000

5 minutes summary: https://hugoladret.github.io/publications/ladret_et_al_sparsecoding/
In a nutshell: We found that sparse coding of images (here extended in a convolutional framework) is improved when using kernels with heterogeneous precision in how they encode orientation information. This was confirmed by learning, but also by comparison with what is observed in the statistics of natural images and in our recordings from neurons in primary visual cortex.

Epistemic uncertainty in a CSC dictionary improves both sparseness and reconstruction performance. (a) Elements from dictionaries with fixed epistemic uncertainty before (green) and after dictionary learning (orange). (b) Elements from a dictionary with heterogeneous epistemic uncertainty before (blue) and after dictionary learning (purple). (c) Elements from a dictionary learned from scratch. (d) Distribution of the sparseness (top) and Peak Signal-to-Noise Ratio (PSNR, right) of the five dictionaries, shown as a scatter plot for each of the 600 images of the dataset (center). Median values are shown as dashed line on the histograms.
open access: https://iopscience.iop.org/article/10.1088/2634-4386/ad5d0f
This work is a followup of
Hugo Ladret, Laurent U Perrinet (2023). Convolutional Sparse Coding is improved by heterogeneous uncertainty modeling. ICLR 2023 SNN Workshop.
PDF Cite URL
This theoretical work accompanies a similar study in neurophysiology:
Hugo Ladret, Nelson Cortes, Lamyae Ikan, Frédéric Chavane, Christian Casanova, Laurent U Perrinet (2023). Cortical recurrence supports resilience to sensory variance in the primary visual cortex. Nature Communications Biology.
PDF Cite DOI Dataset Code URL bioRxiv HAL

2024-05-13-master-m-4-nc

Mon, 13 May 2024 00:00:00 +0000

Artificial neural networks and machine learning applied to the understanding of biological vision

Laurent Perrinet

[2024-05-13]Master M4NC de l’institut NeuroMod, cours Prospective Innovation and Research

Contact me @ laurent.perrinet@univ-amu.fr

Principles of Vision

What is the function of vision?

An Unexpected Visitor (Ilya Repin, 1884)

seeing= interacting with the visual world

social animals: looking at emotions

What is the function of vision?

active: the eye is always moving

https://fr.wikipedia.org/wiki/Alfred_Iarbous

consistency of eye traces

What is the function of vision?

An Unexpected Visitor - Age? (Yarbus, 1965)

social animals: looking at emotions

What is the function of vision?

Visual illusions

Ilusions of brightness or lightness Akiyoshi KITAOKA

Visual illusions

Visual illusions

Visual illusions

Visual illusions

Visual illusions : Pareidolia

Visual illusions : Pareidolia

Visual illusions : Pareidolia

Principles of vision?

Computational neuroscience of vision

Computational neuroscience of vision

Anatomy of the Human Visual system

Human Visual system : the HMAX model

Primary visual cortex

[Hubel & Wiesel, 1962]

Primary visual cortex

[Hubel & Wiesel, 1962]

Convolutional Neural Nets (CNN)

Convolutional Neural Nets (CNN)

CNN: Mathematics

One-dimensional discrete convolution (eg in time) with a kernel $g$ of radius $K$: $$ (f \ast g)[n]=\sum_{m=-K}^{K} f[n-m] \cdot g[m] $$

CNN: Mathematics

Convolution of an image (two-dimensional) with a kernel $g$ of radius $K\times K$:

$$ (f \ast g)[x, y] = \sum_{i=-K}^{K} \sum_{j=-K}^{K} f[x-i, y-j] \cdot g[i, j] $$

CNN: Mathematics

Cross-correlation of an image (two-dimensional) with a kernel $g$ of radius $K\times K$:

$$ (f \ast \tilde{g})[x, y] = \sum_{i=-K}^{K} \sum_{j=-K}^{K} f[x+i, y+j] \cdot g[i, j] $$

CNN: Mathematics

CNN: Mathematics

Correlation of an image defined on several channels (note the order of the indices):

$$ (f \ast \tilde{g})[x, y] = \sum_{c=1}^{C} \sum_{c,i,j} f[c, x+i, y+j] \cdot g[c, i, j] $$

CNN: Mathematics

Correlation of a multi-channel image for multiple output channels (note the order of the indices):

$$ (f \ast \tilde{g})[k, x, y] = \sum_{c,i,j} f[c, x+i, y+j] \cdot g[k, c, i, j] $$

CNN: the HMAX model

sota

CNN: challenges

CNN: Predictive processing

modifications= adding sparse coding + feedback

CNN: Predictive processing

CNN: Topography

[Bosking et al, 1997]

CNN: Topography

Computational neuroscience of vision

Dynamics of vision

Dynamics of vision

Visual latencies (see review).

Dynamics of vision

Visual latencies (see review).

Dynamics of vision

Sensorimotor delays (Perrinet & Friston 2014)

Dynamics of vision

Sensorimotor delays (Perrinet & Friston, 2014)

Dynamics of vision

Diagonal markov model (Khoei et al, 2017).

Dynamics of vision

Flash-lag effect: MBP (Khoei et al, 2017)

Dynamics of vision

Spiking Neural Networks (SNN)

SNN: Leaky Integrate-and-Fire Neuron

Review on Precise Spiking Motifs.

SNN in neurobiology

SNN in neurobiology

SNN in neurobiology

SNN in neurobiology

SNN: Spiking motifs

Review on Precise Spiking Motifs.

SNN: Spiking motifs

Review on Precise Spiking Motifs.

SNN: Spiking motifs

Review on Precise Spiking Motifs.

2 MINUTE

A nice HSD neuron

SNN in neuromorphic engineering

From frame-based to event-based cameras.

2 MINUTE

event-based cameras

SNN in neuromorphic engineering

The HD-SNN neural network.

2 MINUTE

For instance, we show how precise spike times may be used to detect the direction of motion from such a stream of events in an ultrafast fashion.

SNN in neuromorphic engineering

2 MINUTE

A nice HSD neuron

For instance, we show how precise spike times may be used to detect the direction of motion from such a stream of events in an ultrafast fashion.

SNN in neuromorphic engineering

The HD-SNN neural network.

2 MINUTE

nice kernels

SNN in neuromorphic engineering

The HD-SNN neural network.

Spiking Neural Networks (SNN)

# Artificial neural networks applied to the understanding of biological vision

Artificial neural networks and machine learning applied to the understanding of biological vision

Laurent Perrinet

[2024-05-13] Master M4NC de l’institut NeuroMod, cours Prospective Innovation and Research

Contact me @ laurent.perrinet@univ-amu.fr

PhD thesis 'Focus of attention: a sensory-motor task for energy reduction in spiking neural networks'

Fri, 03 May 2024 09:00:00 +0000

Dear colleagues,

Applications are welcome for a fully funded PhD position Focus of attention: a sensory-motor task for energy reduction in spiking neural networks. The position will be located at the EDGE Team @ LEAT Laboratory within Université Côte d’Azur and/or at the INT in Marseille, France.

Context

This project takes place in the context of the EMERGENCES project (ANR PEPR IA 2023-2027) which aims to advance the state of the art on machine learning models using inspiration from biology. Indeed, inspiration from brain features promises to show the emergence of unrivalled efficient processing. Among the most promising features studied in the literature of bio-inspired AI are temporal data encoding using spikes, multimodal association, local learning or attention-based processing.

This PhD subject focuses on the association between attention and spiking neural networks for defining new efficient AI models for embedded systems such as drones, robots and more generally autonomous systems.

The thesis will take place between the LEAT research lab in Sophia-Antipolis and the INT institute in Marseille which both develop complementary approaches on bio-inspired AI from neuroscience observation to embedded systems design.

Subject

The volume as well as the diversity of visual information that reaches our eyes at every moment are huge and cannot be fully integrated by the visual system. In other words, the biological system is confronted to the same challenge as the one encountered by artificial systems (especially at the edge) when dealing with the huge amounts of information coming continuously from the real world. Interestingly, the brain has found an original approach to deal with this issue by focusing on a sub-part of the visual information at a time. Indeed, the study of the visual cortex in neuroscience has made it possible to highlight subregions that treat each or all of the multiple properties of information coming from the visual pathways: shapes, colors, movements, etc [1], thus revealing the interaction of attentional processes and the concept of “saliency” used in cognitive science.

Creating a fully autonomous system remains a significant challenge, especially when operating in the dynamic real world. In recent times, machine learning has assumed a prominent role in machine vision, particularly through the implementation of deep learning algorithms. These algorithms have yielded impressive outcomes in tasks such as object detection, recognition, and tracking. However, these systems come with a high computational cost, as they must process entire camera images to generate these results. Additionally, they struggle to dynamically adapt to changes in their environment.

Our focus lies on two integrated bio-inspired approaches that leverage attentional mechanisms. The first approach, known as bottom-up, draws inspiration from the work of the Gestalt theory, the Feature Integration Theory (Triesman, Gelad) [3], and the model of visual attention from Itti & Koch [1]. This approach relies on the saliency of low-level features in the visual field, processed in parallel, including movement, color, and edges. It employs emergent mechanisms to integrate features guided by their saliency in order to detect the consistency of objects, encompassing their form, position, and speed. As shown by the Gestalt theory, only the more salient data are needed in this mechanism. Thus, we can dramatically reduce the amount of needed data by extracting only the more salient regions of interest during bottom-up phase.

The second approach, known as top down, considers that the visual attention is guided by higher level cognitive stages. For instance, in the Guided Search theory [4], Wolfe emphasizes the role of prior knowledges, expectations, and intentions. In this work, Wolfe proposes a guided search mechanism that relies on a “Priority map that represents the system’s best guess as to where to deploy attention next.”. This Priority map is built on multiple sources of information such as the visual system as well as higher-level information such as intention, search history and the actual visual semantics. In this way, higher-level information is used to guide the filtering of the botom-up path, so that only the information required for a given task is selected and processed. Similar systems are proposed by Schöner [5] in which saliency maps, working memories and “priority map”, guided visual search mechanisms are implemented through the Neural Field Theory (NFT). Here, Dynamic Neural Fields are used to implement the saliency of feature maps, as well as scene spatial selection mechanism, working memory, etc.

In a previous work from the LEAT [6], we have proposed a brain inspired attentional process implementing bottom-up and top-down paths based on a dynamic neural fields properties embodied in a sensory-motor loop. In a complementary work, the INT group has developed a dual pathway model of the visual system in which saliency emerges as a property of the perceptual system to perform saccades, that is, rapid shifts of the fixation point [7]. This uses a recognition model which takes as an input a retinotopically transformed input and shows the emergence of saliency maps [8] In the dual-pathway model, the exploration of a visual scene is based on both the saliency of the color feature (bottom-up) and the class of the last selected object recognized by a convolutional neural network (top-down). Both paths are integrated by a dynamic neural field to select the next visual information to be explored or conserved by setting motor orders accordingly.

The main goal of the thesis is to propose a new vision of the integration of attention into machine learning models. The proposed model will draw on the dynamics at play in a sensory-motor approach to perception and will thus reconsider the classical perception tasks in order to better fit with the continuous flow of information coming from the environment.

Work plan

The PhD will be co-supervised between INT in Marseille and LEAT in Nice. According to the preferences of the candidate, a main laboratory of affiliation will be selected. Weekly meetings will be organized remotely and visiting weeks will be planned to work in-person in the other lab along the year.

Year 1

Study the state of the art in both neuroscience and machine learning on the use of attentional properties to make AI models more effective in environmental perception tasks.
Write a synthesis report on this study.
Develop a first neural model integrating attention-based selection in a specific perception task such as visual search.
Define the specific metrics (KPI) dedicated to the evaluation of the performance and efficiency of such a bio-inspired AI model.
Submit a first publication on this preliminary study in an international conference.

Year 2

Analyze of the performances of the preliminary attention-based model
Develop the approach in order to integrate step by step the features related to dual pathway perception, attention, foveation, DNF and make the model compatible with convolutional neural networks
Submit a second publication in a international journal

Year 3

Study the adaptation of the model to spiking neural networks
Evaluation and comparison of the different approaches
Submit publications on the final results of the thesis
Write the thesis report and prepare the defense

Required skills

Master thesis in one of the following domains: neuromorphic systems, spiking neural networks, neurocognition, machine learning.
Background and experience in machine-learning, artificial neural networks, and/or neurosciences.
Strong motivation, team working, fluent in English (spoken and written).
Programming skills in python, keras, pytorch or equivalent

Start: year 2024

Duration: 3 years

Location: Sophia-Antipolis and/or Marseille

Contacts

Benoît Miramond is Full Professor in Electrical Engineering at LEAT laboratory from University Côte d’Azur (UCA). He holds the chair on bio-inspired AI at 3IA Cote d’Azur Institute and leads the eBRAIN research group which develops a interdisciplinary research activity on embedded Bio-inspiRed AI and Neuromorphic architectures, especially based on SNNs. LEAT is a mixt research unit (UMR 72 48) from UCA and CNRS.
Laurent Perrinet is a director of research at Institut des Neurosciences de la Timone (CNRS - Aix-Marseille Université). He is studying the link between brain microstructures and their macroscopic function by implementing realistic models of the primary visual cortex using spiking neural networks.
Laurent Rodriguez is associate professor at LEAT laboratory in the eBRAIN group. He is interested in dynamic neural networks and develop neural models from biological inspiration.
More details on the “Emergences” grant:
Laurent U Perrinet (2023). Emergences (2023 / 2027).

Application

Apply by sending an email directly to the supervisors (Benoit.miramond@univ-cotedazur.fr Laurent.perrinet@univ-amu.fr Laurent.rodriguez@univ-cotedazur.fr). The application will include:

• Letter of recommendation of the master supervisor.

• Curriculum vitæ.

• Motivation Letter.

References

[1] L. Itti et C. Koch, « Computational modelling of visual attention ». Nat Rev Neurosci, vol. 2, 3, 3, mars 2001, doi: 10.1038/35058500.
[2] Gerstner, W., Kistler, W. M., Naud, R., & Paninski, L. (2014). « Neuronal dynamics: From single neurons to networks and models of cognition ». Cambridge University Press

[3] A. M. Treisman et G. Gelade, « A feature-integration theory of attention ». Cognitive Psychology, vol. 12, 1, p. 97‑136, janv. 1980, doi:10.1016/0010-0285(80)90005-5.
[4] Wolfe, J.M. «Guided Search 6.0: An updated model of visual search ». Psychon Bull Rev 28, 1060–1092 (2021). https://doi.org/10.3758/s13423-020-01859-9
[5] Grieben, R., & Schöner, G.. « A neural dynamic process model of combined bottom-up and top-down guidance in triple conjunction visual search». In T. Fitch, Lamm, C., Leder, H., & Teßmar-Raible, K. (Eds.), Proceedings of the 43rd Annual Conference of the Cognitive Science Society
[6] M. Rasamuel, Lyes Khacef, Laurent Rodriguez, et Benoit Miramond, « Specialized visual sensor coupled to a dynamic neural field for embedded attentional process ». IEEE Conference Publication | IEEE Xplore. https://ieeexplore.ieee.org/abstract/document/8705979
[7] Emmanuel Daucé, Pierre Albigès, Laurent U Perrinet (2020). « A dual foveal-peripheral visual processing model implements efficient saccade selection ». Journal of Vision. doi:https://doi.org/10.1167/jov.20.8.22
[8] Jean-Nicolas Jérémie, Emmanuel Daucé, Laurent U Perrinet (2020). « Retinotopic Mapping Enhances the Robustness of Convolutional Neural Networks ». arXiv https://arxiv.org/abs/2402.15480

2024-04-10-ue-neurosciences-computationnelles

Wed, 10 Apr 2024 00:00:00 +0000

Artificial neural networks applied to the understanding of biological vision

Laurent Perrinet

[2024-04-10] Master 1 Neurosciences et Sciences Cognitives.

Contact me @ laurent.perrinet@univ-amu.fr

Principles of Vision

An Unexpected Visitor (Ilya Repin, 1884)

What is the function of vision?

What is the function of vision?

What is the function of vision?

An Unexpected Visitor - Age? (Yarbus, 1965)

social animals: looking at emotions

What is the function of vision?

Visual illusions

Ilusions of brightness or lightness Akiyoshi KITAOKA

Visual illusions

Visual illusions

Visual illusions

appear bent

effect of context -> 3D

Visual illusions : Pareidolia

Visual illusions : Pareidolia

Visual illusions : Pareidolia

Principles of vision?

Computational neuroscience of vision

Computational neuroscience of vision

Anatomy of the Human Visual system

Human Visual system : the HMAX model

Primary visual cortex

[Hubel & Wiesel, 1962]

Primary visual cortex

[Hubel & Wiesel, 1962]

Convolutional Neural Nets (CNN)

Convolutional Neural Nets (CNN)

CNN: Mathematics

One-dimensional discrete convolution (eg in time) with a kernel $g$ of radius $K$: $$ (f \ast g)[n]=\sum_{m=-K}^{K} f[n-m] \cdot g[m] $$

CNN: Mathematics

Convolution of an image (two-dimensional) with a kernel $g$ of radius $K\times K$:

$$ (f \ast g)[x, y] = \sum_{i=-K}^{K} \sum_{j=-K}^{K} f[x-i, y-j] \cdot g[i, j] $$

CNN: Mathematics

Cross-correlation of an image (two-dimensional) with a kernel $g$ of radius $K\times K$:

$$ (f \ast \tilde{g})[x, y] = \sum_{i=-K}^{K} \sum_{j=-K}^{K} f[x+i, y+j] \cdot g[i, j] $$

CNN: Mathematics

CNN: Mathematics

Correlation of an image defined on several channels (note the order of the indices):

$$ (f \ast \tilde{g})[x, y] = \sum_{c=1}^{C} \sum_{c,i,j} f[c, x+i, y+j] \cdot g[c, i, j] $$

CNN: Mathematics

Correlation of a multi-channel image for multiple output channels (note the order of the indices):

$$ (f \ast \tilde{g})[k, x, y] = \sum_{c,i,j} f[c, x+i, y+j] \cdot g[k, c, i, j] $$

CNN: the HMAX model

sota

CNN: challenges

CNN: Predictive processing

modifications= adding sparse coding + feedback

CNN: Predictive processing

CNN: Topography

[Bosking et al, 1997]

CNN: Topography

Computational neuroscience of vision

Dynamics of vision

Dynamics of vision

Visual latencies (see review).

Dynamics of vision

Visual latencies (see review).

Dynamics of vision

Sensorimotor delays (Perrinet & Friston 2014)

Dynamics of vision

Sensorimotor delays (Perrinet & Friston, 2014)

Dynamics of vision

Diagonal markov model (Khoei et al, 2017).

Dynamics of vision

Flash-lag effect: MBP (Khoei et al, 2017)

Dynamics of vision

Spiking Neural Networks (SNN)

SNN: Leaky Integrate-and-Fire Neuron

Review on Precise Spiking Motifs.

SNN in neurobiology

SNN in neurobiology

SNN in neurobiology

SNN in neurobiology

SNN: Spiking motifs

Review on Precise Spiking Motifs.

SNN: Spiking motifs

Review on Precise Spiking Motifs.

SNN: Spiking motifs

Review on Precise Spiking Motifs.

2 MINUTE

A nice HSD neuron

SNN in neuromorphic engineering

From frame-based to event-based cameras.

2 MINUTE

event-based cameras

SNN in neuromorphic engineering

The HD-SNN neural network.

2 MINUTE

For instance, we show how precise spike times may be used to detect the direction of motion from such a stream of events in an ultrafast fashion.

SNN in neuromorphic engineering

2 MINUTE

A nice HSD neuron

For instance, we show how precise spike times may be used to detect the direction of motion from such a stream of events in an ultrafast fashion.

SNN in neuromorphic engineering

The HD-SNN neural network.

2 MINUTE

nice kernels

SNN in neuromorphic engineering

The HD-SNN neural network.

Spiking Neural Networks (SNN)

# Artificial neural networks applied to the understanding of biological vision

Artificial neural networks applied to the understanding of biological vision

Laurent Perrinet

[2024-04-10] Master 1 Neurosciences et Sciences Cognitives.

Contact me @ laurent.perrinet@univ-amu.fr

Retinotopy in CNN's implements Efficient Visual Search

Tue, 08 Aug 2023 00:00:00 +0000

Read the corresponding paper
Jean-Nicolas Jérémie, Emmanuel Daucé, Laurent U Perrinet (2026). Foveated Retinotopy Improves Classification and Localization in CNNs. Vision.
PDF Cite DOI URL arXiv

2023-05-10-phd-program_neurosciences-computationnelles.md

Wed, 10 May 2023 00:00:00 +0000

Interactions between machine learning, artificial neural networks and our understanding of biological vision

Laurent Perrinet

[2023-05-10] NeuroSchool PhD Program in Neuroscience: Computation Neuroscience

Contact me @ [laurent.perrinet@univ-amu.fr](mailto:laurent.perrinet@univ-amu.fr)

Principles of Vision

What is the function of vision?

An Unexpected Visitor (Ilya Repin, 1884)

seeing= interacting with the visual world

social animals: looking at emotions

What is the function of vision?

active: the eye is always moving

https://fr.wikipedia.org/wiki/Alfred_Iarbous

“1) examine the painting freely”

consistency of eye traces / interindividual differences

What is the function of vision?

An Unexpected Visitor - Age? (Yarbus, 1965)

active: depends on task:

“3) assess the ages of the characters”

What is the function of vision?

Visual illusions

Cydonia Mensae, 1976, Viking Orbiter image

Visual illusions : Pareidolia

Visual illusions : Pareidolia

Cydonia Mensae, 2007, Mars Global Surveyor

Visual illusions : Pareidolia

Cydonia Mensae, 2007, Mars Global Surveyor

Visual illusions: Context

Ilusions of brightness or lightness Akiyoshi KITAOKA

Visual illusions: Context

Visual illusions: Context

of showing an effect of context -> 3D

Principles of vision?

Computational neuroscience of vision

[Marr, 1982]

Computational neuroscience of vision

[Sejnowski, Koch & Churchland, 1998]

Anatomy of the Human Visual system

Human Visual system : the HMAX model

Convolutional Neural Networks : Hierarchy

Convolutional Neural Networks (CNNs)

Primary visual cortex: Hubel & Wiesel

[Hubel & Wiesel, 1962]

Primary visual cortex: Hubel & Wiesel

[Hubel & Wiesel, 1962] - from @Neuroslicer

Convolutional Neural Networks : hierarchy

backpropagation is not bioplausible

modification

Convolutional Neural Networks : Mathematics

One-dimensional discrete convolution (eg in time) with a kernel $g$ of radius $K$: $$ (f \ast g)[n]=\sum_{m=-K}^{K} f[n-m] \cdot g[m] $$

Convolutional Neural Networks : Mathematics

Convolution of an image (two-dimensional) with a kernel $g$ of radius $K\times K$:

$$ (f \ast g)[x, y] = \sum_{i=-K}^{K} \sum_{j=-K}^{K} f[x-i, y-j] \cdot g[i, j] $$

Convolutional Neural Networks : Mathematics

Cross-correlation of an image (two-dimensional) with a kernel $g$ of radius $K\times K$:

$$ (f \ast \tilde{g})[x, y] = \sum_{i=-K}^{K} \sum_{j=-K}^{K} f[x+i, y+j] \cdot g[i, j] $$

Convolutional Neural Networks : Mathematics

Convolutional Neural Networks : Mathematics

Correlation of an image defined on several channels (note the order of the indices):

$$ (f \ast \tilde{g})[x, y] = \sum_{c=1}^{C} \sum_{i,j} f[c, x+i, y+j] \cdot g[c, i, j] $$

Convolutional Neural Networks : Mathematics

Correlation of a multi-channel image for multiple output channels (note the order of the indices):

$$ (f \ast \tilde{g})[k, x, y] = \sum_{c=1}^{C} \sum_{i,j} f[c, x+i, y+j] \cdot g[k, c, i, j] $$

Convolutional Neural Networks : Predictive coding

adding sparse coding + feedback

Convolutional Neural Networks : Predictive coding

Convolutional Neural Networks : Topography

[Bosking et al, 1997]

Convolutional Neural Networks : Topography

Computational neuroscience of vision

Dynamics of vision

Dynamics of vision

Visual latencies (see review).

Dynamics of vision

Visual latencies (see review).

Dynamics of vision

Sensorimotor delays (Perrinet & Friston 2014)

Dynamics of vision

Sensorimotor delays (Perrinet & Friston, 2014)

Dynamics of vision

Diagonal markov model (Khoei et al, 2017).

Dynamics of vision

Flash-lag effect: MBP (Khoei et al, 2017)

Dynamics of vision

Spiking Neural Networks

Spiking Neural Networks: Leaky Integrate-and-Fire Neuron

[Grimaldi et al, 2023, Precise Spiking Motifs]

Spiking Neural Networks in neurobiology

Spiking Neural Networks in neurobiology

Spiking Neural Networks in neurobiology

Spiking Neural Networks in neurobiology

Spiking Neural Networks: Spiking motifs

[Grimaldi et al, 2023, Precise Spiking Motifs]

Spiking Neural Networks: Spiking motifs

Review on Precise Spiking Motifs.

Spiking Neural Networks: Spiking motifs

Review on Precise Spiking Motifs.

2 MINUTE

A nice HSD neuron

Spiking Neural Networks in neuromorphic engineering

From frame-based to event-based cameras.

2 MINUTE

event-based cameras

Spiking Neural Networks in neuromorphic engineering

The HD-SNN neural network.

2 MINUTE

For instance, we show how precise spike times may be used to detect the direction of motion from such a stream of events in an ultrafast fashion.

Spiking Neural Networks in neuromorphic engineering

2 MINUTE

A nice HSD neuron

For instance, we show how precise spike times may be used to detect the direction of motion from such a stream of events in an ultrafast fashion.

Spiking Neural Networks in neuromorphic engineering

The HD-SNN neural network.

2 MINUTE

nice kernels

Spiking Neural Networks in neuromorphic engineering

The HD-SNN neural network.

## Artificial neural networks and machine learning applied to the understanding of biological vision

Interactions between machine learning, artificial neural networks and our understanding of biological vision

Laurent Perrinet

[2023-05-10] NeuroSchool PhD Program in Neuroscience: Computation Neuroscience

Contact me @ [laurent.perrinet@univ-amu.fr](mailto:laurent.perrinet@univ-amu.fr)

Convolutional Sparse Coding is improved by heterogeneous uncertainty modeling

Fri, 05 May 2023 00:00:00 +0000

Accepted paper (poster) at the ICLR 2023 Workshop on Sparsity in Neural Networks:
the focus of the WS is on “On practical limitations and tradeoffs between sustainability and efficiency” in Kigali, Rwanda / May 5th 2023
reviews will be made public on https://openreview.net/forum?id=tgr8FEcl28M
In a nutshell: We found that sparse coding of images (here extended in a convolutional framework) is improved when using kernels with heterogeneous precision in how they encode orientation information. This was confirmed by learning, but also by comparison with what is observed in the statistics of natural images and in our recordings from neurons in primary visual cortex.

Epistemic uncertainty in a CSC dictionary improves both sparseness and reconstruction performance. (a) Elements from dictionaries with fixed epistemic uncertainty before (green) and after dictionary learning (orange). (b) Elements from a dictionary with heterogeneous epistemic uncertainty before (blue) and after dictionary learning (purple). (c) Elements from a dictionary learned from scratch. (d) Distribution of the sparseness (top) and Peak Signal-to-Noise Ratio (PSNR, right) of the five dictionaries, shown as a scatter plot for each of the 600 images of the dataset (center). Median values are shown as dashed line on the histograms.
This theoretical work accompanies a similar study in neurophysiology:
Hugo Ladret, Nelson Cortes, Lamyae Ikan, Frédéric Chavane, Christian Casanova, Laurent U Perrinet (2023). Cortical recurrence supports resilience to sensory variance in the primary visual cortex. Nature Communications Biology.
PDF Cite DOI Dataset Code URL bioRxiv HAL
This work was extended in
Hugo Ladret, Christian Casanova, Laurent U Perrinet (2024). Kernel Heterogeneity Improves Sparseness of Natural Images Representations. Neuromorphic Computing and Engineering.
Cite Dataset DOI URL HAL

2023-04-05-ue-neurosciences-computationnelles

Wed, 05 Apr 2023 00:00:00 +0000

Artificial neural networks and machine learning applied to the understanding of biological vision

Laurent Perrinet

[2023-04-05] Master 1 Neurosciences et Sciences Cognitives.

Contact me @ laurent.perrinet@univ-amu.fr

Principles of Vision

What is the function of vision?

An Unexpected Visitor (Ilya Repin, 1884)

seeing= interacting with the visual world

social animals: looking at emotions

What is the function of vision?

Ilusions of brightness or lightness Akiyoshi KITAOKA

active: the eye is always moving

https://fr.wikipedia.org/wiki/Alfred_Iarbous

What is the function of vision?

An Unexpected Visitor - Age? (Yarbus, 1965)

active: depends on task

What is the function of vision?

An Unexpected Visitor - How long? (Yarbus, 1965)

consistency of eye traces

Visual illusions

Visual illusions

Visual illusions

Visual illusions

Visual illusions : Pareidolia

Visual illusions : Pareidolia

Visual illusions : Pareidolia

Cydonia Mensae (2007) Mars Global Surveyor

Principles of vision?

Computational neuroscience of vision

[Marr, 1982]

Computational neuroscience of vision

[Sejnowski, Koch & Churchland (1998)]

Anatomy of the Human Visual system

Human Visual system : the HMAX model

Primary visual cortex: Hubel & Wiesel

[Hubel & Wiesel, 1962]

Primary visual cortex: Hubel & Wiesel

[Hubel & Wiesel, 1962]

Convolutional Neural Networks : Hierarchy

Convolutional Neural Networks : Mathematics

One-dimensional discrete convolution (eg in time) with a kernel $g$ of radius $K$: $$ (f \ast g)[n]=\sum_{m=-K}^{K} f[n-m] \cdot g[m] $$

Convolutional Neural Networks : Mathematics

Convolution of an image (two-dimensional) with a kernel $g$ of radius $K\times K$:

$$ (f \ast g)[x, y] = \sum_{i=-K}^{K} \sum_{j=-K}^{K} f[x-i, y-j] \cdot g[i, j] $$

Convolutional Neural Networks : Mathematics

Cross-correlation of an image (two-dimensional) with a kernel $g$ of radius $K\times K$:

$$ (f \ast \tilde{g})[x, y] = \sum_{i=-K}^{K} \sum_{j=-K}^{K} f[x+i, y+j] \cdot g[i, j] $$

Convolutional Neural Networks : Mathematics

Convolutional Neural Networks : Mathematics

Correlation of an image defined on several channels (note the order of the indices):

$$ (f \ast \tilde{g})[x, y] = \sum_{c=1}^{C} \sum_{i,j} f[c, x+i, y+j] \cdot g[c, i, j] $$

Convolutional Neural Networks : Mathematics

Correlation of a multi-channel image for multiple output channels (note the order of the indices):

$$ (f \ast \tilde{g})[k, x, y] = \sum_{c=1}^{C} \sum_{i,j} f[c, x+i, y+j] \cdot g[k, c, i, j] $$

Convolutional Neural Networks : the HMAX model

sota

Convolutional Neural Networks (CNNs)

Convolutional Neural Networks : hierarchy

Convolutional Neural Networks : Predictive coding

adding sparse coding + feedback

Convolutional Neural Networks : Predictive coding

Convolutional Neural Networks : Topography

[Bosking et al, 1997]

Convolutional Neural Networks : Topography

Computational neuroscience of vision

Dynamics of vision

Dynamics of vision

Visual latencies (see review).

Dynamics of vision

Visual latencies (see review).

Dynamics of vision

Sensorimotor delays (Perrinet & Friston 2014)

Dynamics of vision

Sensorimotor delays (Perrinet & Friston, 2014)

Dynamics of vision

Diagonal markov model (Khoei et al, 2017).

Dynamics of vision

Flash-lag effect: MBP (Khoei et al, 2017)

Dynamics of vision

Spiking Neural Networks

Spiking Neural Networks: Leaky Integrate-and-Fire Neuron

Review on Precise Spiking Motifs.

Spiking Neural Networks in neurobiology

Spiking Neural Networks in neurobiology

Spiking Neural Networks in neurobiology

Spiking Neural Networks in neurobiology

Spiking Neural Networks: Spiking motifs

Review on Precise Spiking Motifs.

Spiking Neural Networks: Spiking motifs

Review on Precise Spiking Motifs.

Spiking Neural Networks: Spiking motifs

Review on Precise Spiking Motifs.

2 MINUTE

A nice HSD neuron

Spiking Neural Networks in neuromorphic engineering

From frame-based to event-based cameras.

2 MINUTE

event-based cameras

Spiking Neural Networks in neuromorphic engineering

The HD-SNN neural network.

2 MINUTE

For instance, we show how precise spike times may be used to detect the direction of motion from such a stream of events in an ultrafast fashion.

Spiking Neural Networks in neuromorphic engineering

2 MINUTE

A nice HSD neuron

For instance, we show how precise spike times may be used to detect the direction of motion from such a stream of events in an ultrafast fashion.

Spiking Neural Networks in neuromorphic engineering

The HD-SNN neural network.

2 MINUTE

nice kernels

Spiking Neural Networks in neuromorphic engineering

The HD-SNN neural network.

## Artificial neural networks and machine learning applied to the understanding of biological vision

Artificial neural networks and machine learning applied to the understanding of biological vision

Laurent Perrinet

[2023-04-05] Master 1 Neurosciences et Sciences Cognitives.

Contact me @ laurent.perrinet@univ-amu.fr

2023-04-03-master-m-4-nc

Mon, 03 Apr 2023 00:00:00 +0000

Artificial neural networks and machine learning applied to the understanding of biological vision

Laurent Perrinet

[2023-04-03] Master M4NC de l’institut NeuroMod, cours Prospective Innovation and Research.

Contact me @ laurent.perrinet@univ-amu.fr

Principles of Vision

What is the function of vision?

An Unexpected Visitor (Ilya Repin, 1884)

seeing= interacting with the visual world

social animals: looking at emotions

What is the function of vision?

Ilusions of brightness or lightness Akiyoshi KITAOKA

active: the eye is always moving

https://fr.wikipedia.org/wiki/Alfred_Iarbous

What is the function of vision?

An Unexpected Visitor - Age? (Yarbus, 1965)

active: depends on task

What is the function of vision?

An Unexpected Visitor - How long? (Yarbus, 1965)

consistency of eye traces

Visual illusions

Visual illusions

Visual illusions

Visual illusions

Visual illusions : Pareidolia

Visual illusions : Pareidolia

Visual illusions : Pareidolia

Cydonia Mensae (2007) Mars Global Surveyor

Principles of vision?

Computational neuroscience of vision

[Marr, 1982]

Computational neuroscience of vision

[Sejnowski, Koch & Churchland (1998)]

Anatomy of the Human Visual system

Human Visual system : the HMAX model

Primary visual cortex: Hubel & Wiesel

[Hubel & Wiesel, 1962]

Primary visual cortex: Hubel & Wiesel

[Hubel & Wiesel, 1962]

Convolutional Neural Networks : Hierarchy

Convolutional Neural Networks : Mathematics

One-dimensional discrete convolution (eg in time) with a kernel $g$ of radius $K$: $$ (f \ast g)[n]=\sum_{m=-K}^{K} f[n-m] \cdot g[m] $$

Convolutional Neural Networks : Mathematics

Convolution of an image (two-dimensional) with a kernel $g$ of radius $K\times K$:

$$ (f \ast g)[x, y] = \sum_{i=-K}^{K} \sum_{j=-K}^{K} f[x-i, y-j] \cdot g[i, j] $$

Convolutional Neural Networks : Mathematics

Cross-correlation of an image (two-dimensional) with a kernel $g$ of radius $K\times K$:

$$ (f \ast \tilde{g})[x, y] = \sum_{i=-K}^{K} \sum_{j=-K}^{K} f[x+i, y+j] \cdot g[i, j] $$

Convolutional Neural Networks : Mathematics

Convolutional Neural Networks : Mathematics

Correlation of an image defined on several channels (note the order of the indices):

$$ (f \ast \tilde{g})[x, y] = \sum_{c=1}^{C} \sum_{i,j} f[c, x+i, y+j] \cdot g[c, i, j] $$

Convolutional Neural Networks : Mathematics

Correlation of a multi-channel image for multiple output channels (note the order of the indices):

$$ (f \ast \tilde{g})[k, x, y] = \sum_{c=1}^{C} \sum_{i,j} f[c, x+i, y+j] \cdot g[k, c, i, j] $$

Convolutional Neural Networks : the HMAX model

sota

Convolutional Neural Networks (CNNs)

Convolutional Neural Networks : hierarchy

Convolutional Neural Networks : Predictive coding

adding sparse coding + feedback

Convolutional Neural Networks : Predictive coding

Convolutional Neural Networks : Topography

[Bosking et al, 1997]

Convolutional Neural Networks : Topography

Computational neuroscience of vision

Dynamics of vision

Dynamics of vision

Visual latencies (see review).

Dynamics of vision

Visual latencies (see review).

Dynamics of vision

Sensorimotor delays (Perrinet & Friston 2014)

Dynamics of vision

Sensorimotor delays (Perrinet & Friston, 2014)

Dynamics of vision

Diagonal markov model (Khoei et al, 2017).

Dynamics of vision

Flash-lag effect: MBP (Khoei et al, 2017)

Dynamics of vision

Spiking Neural Networks

Spiking Neural Networks: Leaky Integrate-and-Fire Neuron

Review on Precise Spiking Motifs.

Spiking Neural Networks in neurobiology

Spiking Neural Networks in neurobiology

Spiking Neural Networks in neurobiology

Spiking Neural Networks in neurobiology

Spiking Neural Networks: Spiking motifs

Review on Precise Spiking Motifs.

Spiking Neural Networks: Spiking motifs

Review on Precise Spiking Motifs.

Spiking Neural Networks: Spiking motifs

Review on Precise Spiking Motifs.

2 MINUTE

A nice HSD neuron

Spiking Neural Networks in neuromorphic engineering

From frame-based to event-based cameras.

2 MINUTE

event-based cameras

Spiking Neural Networks in neuromorphic engineering

The HD-SNN neural network.

2 MINUTE

For instance, we show how precise spike times may be used to detect the direction of motion from such a stream of events in an ultrafast fashion.

Spiking Neural Networks in neuromorphic engineering

2 MINUTE

A nice HSD neuron

For instance, we show how precise spike times may be used to detect the direction of motion from such a stream of events in an ultrafast fashion.

Spiking Neural Networks in neuromorphic engineering

The HD-SNN neural network.

2 MINUTE

nice kernels

Spiking Neural Networks in neuromorphic engineering

The HD-SNN neural network.

## Artificial neural networks and machine learning applied to the understanding of biological vision

https://laurentperrinet.github.io/talk/2023-01-23-game-theory-and-the-brain

Artificial neural networks and machine learning applied to the understanding of biological vision

Laurent Perrinet

[2023-04-03] Master M4NC de l’institut NeuroMod, cours Prospective Innovation and Research.

Contact me @ laurent.perrinet@univ-amu.fr

Resilience to sensory uncertainty in the primary visual cortex

Thu, 09 Mar 2023 00:00:00 +0000

This poster is presented in the following paper (published in Nature Comm Biology):
Hugo Ladret, Nelson Cortes, Lamyae Ikan, Frédéric Chavane, Christian Casanova, Laurent U Perrinet (2023). Cortical recurrence supports resilience to sensory variance in the primary visual cortex. Nature Communications Biology.
PDF Cite DOI Dataset Code URL bioRxiv HAL

2023-01-23_game-theory-and-the-brain

Mon, 23 Jan 2023 00:00:00 +0000

Game theory and brain strategies

[2023-01-23] Atelier jeu et cerveau

Game theory and brain strategies

Le jeu du cerveau et du hasard, The Conversation

Aleatoric noise

A Huynh, generating Poisson disk noise

what is noise? it exists at quantum level, but if I were to ask you to draw random points how would it look like?
Aleatoric comes from alea, the Latin word for “dice.” Aleatoric uncertainty is the uncertainty introduced by the randomness of an event. For example, the result of flipping a coin is an aleatoric event.
In your opinion, which of the two is the most random pattern?
from your responses …
the answer is that … When it comes to true randomness, one of its stranger aspects is that it often behaves differently to people’s expectations. Take the two diagrams below – which one do you think is a random distribution, and which has been deliberately created/adjusted?

randomized dots Only one of these panels shows a random distribution of dots | Source: Bully for Brontosaurus – Stephen Jay Gould

If you said the right panel, you are in good company, as this is most people’s expectation of what randomness looks like. However, this relatively uniform distribution has been adjusted to ensure the dots are evenly spread. In fact, it is the left panel, with its clumps and voids, that reflects a true random distribution. It is also this tendency for randomness to produce clumps and voids that leads to some unintuitive outcomes.

https://theconversation.com/daniel-kahneman-on-noise-the-flaw-in-human-judgement-harder-to-detect-than-cognitive-bias-160525

Instabilité, Étienne Rey.

Visual illusions : Pareidolia

Visual illusions : Pareidolia

Visual illusions : Pareidolia