Artificial neural networks and machine learning applied to the understanding of biological vision

Laurent Perrinet

[2024-05-13]Master M4NC de l’institut NeuroMod, cours Prospective Innovation and Research

logo
Contact me @ laurent.perrinet@univ-amu.fr

Principles of Vision

[[Marr, 1982](https://outde.xyz/2020-01-12/overappreciated-arguments-marrs-three-levels.html)]
[Marr, 1982]

What is the function of vision?

[An Unexpected Visitor (Ilya Repin, 1884)](https://www.cabinetmagazine.org/issues/30/archibald.php)
An Unexpected Visitor (Ilya Repin, 1884)

What is the function of vision?

[An Unexpected Visitor (Yarbus, 1965)](https://www.cabinetmagazine.org/issues/30/archibald.php)
An Unexpected Visitor (Yarbus, 1965)

What is the function of vision?

[An Unexpected Visitor - *Age?* (Yarbus, 1965)](https://www.cabinetmagazine.org/issues/30/archibald.php)
An Unexpected Visitor - Age? (Yarbus, 1965)

What is the function of vision?

[An Unexpected Visitor - *How long?* (Yarbus, 1965)](https://www.cabinetmagazine.org/issues/30/archibald.php)
An Unexpected Visitor - How long? (Yarbus, 1965)

Visual illusions

[Rotating Snakes *Akiyoshi KITAOKA*](http://www.ritsumei.ac.jp/~akitaoka/index-e.html)
Rotating Snakes Akiyoshi KITAOKA

Visual illusions

Ilusions of brightness or lightness Akiyoshi KITAOKA

Visual illusions

[Hering illusion](https://en.wikipedia.org/wiki/Hering_illusion)
Hering illusion

Visual illusions

[Hering illusion](https://en.wikipedia.org/wiki/Hering_illusion)
Hering illusion

Visual illusions

[Rotating Snakes *Akiyoshi KITAOKA*](http://www.ritsumei.ac.jp/~akitaoka/index-e.html)
Rotating Snakes Akiyoshi KITAOKA

Visual illusions : Pareidolia

[Cydonia Mensae (1976) *Viking Orbiter image*](https://en.wikipedia.org/wiki/Cydonia_(Mars))
Cydonia Mensae (1976) Viking Orbiter image

Visual illusions : Pareidolia

[Cydonia Mensae (2007) *Mars Global Surveyor*](https://en.wikipedia.org/wiki/Cydonia_(Mars))
Cydonia Mensae (2007) Mars Global Surveyor

Visual illusions : Pareidolia

[Cydonia Mensae (2007) *Mars Global Surveyor*](https://en.wikipedia.org/wiki/Cydonia_(Mars))
Cydonia Mensae (2007) Mars Global Surveyor

Principles of vision?

[[Marr, 1982](https://outde.xyz/2020-01-12/overappreciated-arguments-marrs-three-levels.html)]
[Marr, 1982]

Computational neuroscience of vision

[[Marr, 1982](https://outde.xyz/2020-01-12/overappreciated-arguments-marrs-three-levels.html)]
[Marr, 1982]

Computational neuroscience of vision

[[Sejnowski, Koch & Churchland (1998)](http://www.hms.harvard.edu/bss/neuro/bornlab/nb204/papers/sejnowski-koch-churchland-science1988.pdf)]
[Sejnowski, Koch & Churchland (1998)]

Anatomy of the Human Visual system

Human Visual system : the HMAX model

[[Serre and Poggio, 2007](https://biology.stackexchange.com/questions/10955/ventral-stream-pathway-and-architecture-proposed-by-poggios-group)]
[Serre and Poggio, 2007]

Primary visual cortex

[Hubel & Wiesel, 1962]
[Hubel & Wiesel, 1962]

Primary visual cortex

[Hubel & Wiesel, 1962]

Convolutional Neural Nets (CNN)

[[Boutin *et al*, 2021](https://laurentperrinet.github.io/publication/boutin-franciosini-chavane-ruffier-perrinet-20/)]
[Boutin et al, 2021]

Convolutional Neural Nets (CNN)

[[Jérémie & LP, 2023](https://laurentperrinet.github.io/publication/jeremie-23-ultra-fast-cat/)]
[Jérémie & LP, 2023]

CNN: Mathematics

  • One-dimensional discrete convolution (eg in time) with a kernel $g$ of radius $K$: $$ (f \ast g)[n]=\sum_{m=-K}^{K} f[n-m] \cdot g[m] $$

CNN: Mathematics

  • Convolution of an image (two-dimensional) with a kernel $g$ of radius $K\times K$:

$$ (f \ast g)[x, y] = \sum_{i=-K}^{K} \sum_{j=-K}^{K} f[x-i, y-j] \cdot g[i, j] $$

CNN: Mathematics

  • Cross-correlation of an image (two-dimensional) with a kernel $g$ of radius $K\times K$:

$$ (f \ast \tilde{g})[x, y] = \sum_{i=-K}^{K} \sum_{j=-K}^{K} f[x+i, y+j] \cdot g[i, j] $$

CNN: Mathematics

[[Amidi & Amidi](https://stanford.edu/~shervine/teaching/cs-230/cheatsheet-convolutional-neural-networks)]
[Amidi & Amidi]

CNN: Mathematics

$$ (f \ast \tilde{g})[x, y] = \sum_{c=1}^{C} \sum_{c,i,j} f[c, x+i, y+j] \cdot g[c, i, j] $$

CNN: Mathematics

$$ (f \ast \tilde{g})[k, x, y] = \sum_{c,i,j} f[c, x+i, y+j] \cdot g[k, c, i, j] $$

CNN: the HMAX model

[[Serre and Poggio, 2006]](https://biology.stackexchange.com/questions/10955/ventral-stream-pathway-and-architecture-proposed-by-poggios-group)
[Serre and Poggio, 2006]

CNN: challenges

[[Boutin *et al*, 2021](https://laurentperrinet.github.io/publication/boutin-franciosini-chavane-ruffier-perrinet-20/)]
[Boutin et al, 2021]

CNN: Predictive processing

[[Boutin *et al*, 2021](https://laurentperrinet.github.io/publication/boutin-franciosini-chavane-ruffier-perrinet-20/)]
[Boutin et al, 2021]

CNN: Predictive processing

[[Boutin *et al*, 2021](https://laurentperrinet.github.io/publication/boutin-franciosini-chavane-ruffier-perrinet-20/)]
[Boutin et al, 2021]

CNN: Topography

[Bosking *et al*, 1997]
[Bosking et al, 1997]

CNN: Topography

[[Boutin *et al*, 2022](https://laurentperrinet.github.io/publication/franciosini-21/)]
[Boutin et al, 2022]

Computational neuroscience of vision

[[Marr, 1982](https://outde.xyz/2020-01-12/overappreciated-arguments-marrs-three-levels.html)]
[Marr, 1982]

Dynamics of vision

[[Marr, 1982](https://outde.xyz/2020-01-12/overappreciated-arguments-marrs-three-levels.html)]
[Marr, 1982]

Dynamics of vision

Visual latencies ([see review](https://laurentperrinet.github.io/publication/grimaldi-22-polychronies/)).
Visual latencies (see review).

Dynamics of vision

Visual latencies ([see review](https://laurentperrinet.github.io/publication/grimaldi-22-polychronies/)).
Visual latencies (see review).

Dynamics of vision

Sensorimotor delays ([Perrinet & Friston 2014](https://laurentperrinet.github.io/publication/perrinet-adams-friston-14/))
Sensorimotor delays (Perrinet & Friston 2014)

Dynamics of vision

Sensorimotor delays ([Perrinet & Friston, 2014](https://laurentperrinet.github.io/publication/perrinet-adams-friston-14/))
Sensorimotor delays (Perrinet & Friston, 2014)

Dynamics of vision

Dynamics of vision

Diagonal markov model ([Khoei *et al*, 2017](https://laurentperrinet.github.io/publication/khoei-masson-perrinet-17/)).
Diagonal markov model (Khoei et al, 2017).

Dynamics of vision

Flash-lag effect: MBP (Khoei et al, 2017)

Dynamics of vision

[[Marr, 1982](https://outde.xyz/2020-01-12/overappreciated-arguments-marrs-three-levels.html)]
[Marr, 1982]

Spiking Neural Networks (SNN)

[[Marr, 1982](https://outde.xyz/2020-01-12/overappreciated-arguments-marrs-three-levels.html)]
[Marr, 1982]

SNN: Leaky Integrate-and-Fire Neuron

Review on [Precise Spiking Motifs](https://laurentperrinet.github.io/publication/grimaldi-22-polychronies/).
Review on Precise Spiking Motifs.

SNN in neurobiology

[[Mainen & Sejnowski, 1995](https://github.com/SpikeAI/2022_polychronies-review/blob/main/src/Figure_2_MainenSejnowski1995.ipynb)]
[Mainen & Sejnowski, 1995]

SNN in neurobiology

[[Mainen & Sejnowski, 1995](https://github.com/SpikeAI/2022_polychronies-review/blob/main/src/Figure_2_MainenSejnowski1995.ipynb)]
[Mainen & Sejnowski, 1995]

SNN in neurobiology

[[Diesmann et al. 1999](https://github.com/SpikeAI/2022_polychronies-review/blob/main/src/Figure_3_Diesmann_et_al_1999.py)]
[Diesmann et al. 1999]

SNN in neurobiology

[[Haimerl et al, 2019](https://laurentperrinet.github.io/publication/grimaldi-22-polychronies/)]
[Haimerl et al, 2019]

SNN: Spiking motifs

Review on [Precise Spiking Motifs](https://laurentperrinet.github.io/publication/grimaldi-22-polychronies/).
Review on Precise Spiking Motifs.

SNN: Spiking motifs

Review on [Precise Spiking Motifs](https://laurentperrinet.github.io/publication/grimaldi-22-polychronies/).
Review on Precise Spiking Motifs.

SNN: Spiking motifs

Review on [Precise Spiking Motifs](https://laurentperrinet.github.io/publication/grimaldi-22-polychronies/).
Review on Precise Spiking Motifs.

SNN in neuromorphic engineering

From frame-based to event-based cameras.
From frame-based to event-based cameras.

SNN in neuromorphic engineering

The HD-SNN neural network.
The HD-SNN neural network.

SNN in neuromorphic engineering

SNN in neuromorphic engineering

The HD-SNN neural network.
The HD-SNN neural network.

SNN in neuromorphic engineering

The HD-SNN neural network.
The HD-SNN neural network.

Spiking Neural Networks (SNN)

[[Marr, 1982](https://outde.xyz/2020-01-12/overappreciated-arguments-marrs-three-levels.html)]
[Marr, 1982]

Artificial neural networks applied to the understanding of biological vision

[[Marr, 1982](https://outde.xyz/2020-01-12/overappreciated-arguments-marrs-three-levels.html)]
[Marr, 1982]

Artificial neural networks and machine learning applied to the understanding of biological vision

Laurent Perrinet

[2024-05-13] Master M4NC de l’institut NeuroMod, cours Prospective Innovation and Research

logo
Contact me @ laurent.perrinet@univ-amu.fr