Interactions between machine learning, artificial neural networks and our understanding of biological vision

Laurent Perrinet

[2023-05-10] NeuroSchool PhD Program in Neuroscience: Computation Neuroscience

logos qrcode

Contact me @ laurent.perrinet@univ-amu.fr

Principles of Vision

[[Marr, 1982](https://outde.xyz/2020-01-12/overappreciated-arguments-marrs-three-levels.html)]
[Marr, 1982]

What is the function of vision?

[An Unexpected Visitor (Ilya Repin, 1884)](https://www.cabinetmagazine.org/issues/30/archibald.php)
An Unexpected Visitor (Ilya Repin, 1884)

What is the function of vision?

[An Unexpected Visitor (Yarbus, 1965)](https://www.cabinetmagazine.org/issues/30/archibald.php)
An Unexpected Visitor (Yarbus, 1965)

What is the function of vision?

[An Unexpected Visitor - *Age?* (Yarbus, 1965)](https://www.cabinetmagazine.org/issues/30/archibald.php)
An Unexpected Visitor - Age? (Yarbus, 1965)

What is the function of vision?

[An Unexpected Visitor - *How long?* (Yarbus, 1965)](https://www.cabinetmagazine.org/issues/30/archibald.php)
An Unexpected Visitor - How long? (Yarbus, 1965)

Visual illusions

[Rotating Snakes *Akiyoshi KITAOKA*](http://www.ritsumei.ac.jp/~akitaoka/index-e.html)
Rotating Snakes Akiyoshi KITAOKA

Visual illusions : Pareidolia

[Cydonia Mensae, 1976, *Viking Orbiter image*](https://en.wikipedia.org/wiki/Cydonia_(Mars))
Cydonia Mensae, 1976, Viking Orbiter image

Visual illusions : Pareidolia

[Cydonia Mensae, 2007, *Mars Global Surveyor*](https://en.wikipedia.org/wiki/Cydonia_(Mars))
Cydonia Mensae, 2007, Mars Global Surveyor

Visual illusions : Pareidolia

[Cydonia Mensae, 2007, *Mars Global Surveyor*](https://en.wikipedia.org/wiki/Cydonia_(Mars))
Cydonia Mensae, 2007, Mars Global Surveyor

Visual illusions: Context

Ilusions of brightness or lightness Akiyoshi KITAOKA

Visual illusions: Context

[Hering illusion](https://en.wikipedia.org/wiki/Hering_illusion)
Hering illusion

Visual illusions: Context

[Hering illusion](https://en.wikipedia.org/wiki/Hering_illusion)
Hering illusion

Principles of vision?

[[Marr, 1982](https://outde.xyz/2020-01-12/overappreciated-arguments-marrs-three-levels.html)]
[Marr, 1982]

Computational neuroscience of vision

[[Marr, 1982](https://outde.xyz/2020-01-12/overappreciated-arguments-marrs-three-levels.html)]
[Marr, 1982]

Computational neuroscience of vision

[[Sejnowski, Koch & Churchland, 1998](http://www.hms.harvard.edu/bss/neuro/bornlab/nb204/papers/sejnowski-koch-churchland-science1988.pdf)]
[Sejnowski, Koch & Churchland, 1998]

Anatomy of the Human Visual system

Human Visual system : the HMAX model

[[Serre and Poggio, 2006]](https://biology.stackexchange.com/questions/10955/ventral-stream-pathway-and-architecture-proposed-by-poggios-group)
[Serre and Poggio, 2006]

Convolutional Neural Networks : Hierarchy

[[Boutin *et al*, 2021](https://laurentperrinet.github.io/publication/boutin-franciosini-chavane-ruffier-perrinet-20/)]
[Boutin et al, 2021]

Convolutional Neural Networks (CNNs)

[[Jérémie & LP, 2023](https://laurentperrinet.github.io/publication/jeremie-23-ultra-fast-cat/)]
[Jérémie & LP, 2023]

Primary visual cortex: Hubel & Wiesel

[Hubel & Wiesel, 1962]
[Hubel & Wiesel, 1962]

Primary visual cortex: Hubel & Wiesel

[Hubel & Wiesel, 1962] - from @Neuroslicer

Convolutional Neural Networks : hierarchy

[[Boutin *et al*, 2021](https://laurentperrinet.github.io/publication/boutin-franciosini-chavane-ruffier-perrinet-20/)]
[Boutin et al, 2021]

Convolutional Neural Networks : Mathematics

  • One-dimensional discrete convolution (eg in time) with a kernel $g$ of radius $K$: $$ (f \ast g)[n]=\sum_{m=-K}^{K} f[n-m] \cdot g[m] $$

Convolutional Neural Networks : Mathematics

  • Convolution of an image (two-dimensional) with a kernel $g$ of radius $K\times K$:

$$ (f \ast g)[x, y] = \sum_{i=-K}^{K} \sum_{j=-K}^{K} f[x-i, y-j] \cdot g[i, j] $$

Convolutional Neural Networks : Mathematics

  • Cross-correlation of an image (two-dimensional) with a kernel $g$ of radius $K\times K$:

$$ (f \ast \tilde{g})[x, y] = \sum_{i=-K}^{K} \sum_{j=-K}^{K} f[x+i, y+j] \cdot g[i, j] $$

Convolutional Neural Networks : Mathematics

[[Amidi & Amidi](https://stanford.edu/~shervine/teaching/cs-230/cheatsheet-convolutional-neural-networks)]
[Amidi & Amidi]

Convolutional Neural Networks : Mathematics

$$ (f \ast \tilde{g})[x, y] = \sum_{c=1}^{C} \sum_{i,j} f[c, x+i, y+j] \cdot g[c, i, j] $$

Convolutional Neural Networks : Mathematics

$$ (f \ast \tilde{g})[k, x, y] = \sum_{c=1}^{C} \sum_{i,j} f[c, x+i, y+j] \cdot g[k, c, i, j] $$

Convolutional Neural Networks : Predictive coding

[[Boutin *et al*, 2021](https://laurentperrinet.github.io/publication/boutin-franciosini-chavane-ruffier-perrinet-20/)]
[Boutin et al, 2021]

Convolutional Neural Networks : Predictive coding

[[Boutin *et al*, 2021](https://laurentperrinet.github.io/publication/boutin-franciosini-chavane-ruffier-perrinet-20/)]
[Boutin et al, 2021]

Convolutional Neural Networks : Topography

[Bosking *et al*, 1997]
[Bosking et al, 1997]

Convolutional Neural Networks : Topography

[[Boutin *et al*, 2022](https://laurentperrinet.github.io/publication/franciosini-21/)]
[Boutin et al, 2022]

Computational neuroscience of vision

[[Marr, 1982](https://outde.xyz/2020-01-12/overappreciated-arguments-marrs-three-levels.html)]
[Marr, 1982]

Dynamics of vision

[[Marr, 1982](https://outde.xyz/2020-01-12/overappreciated-arguments-marrs-three-levels.html)]
[Marr, 1982]

Dynamics of vision

Visual latencies ([see review](https://laurentperrinet.github.io/publication/grimaldi-22-polychronies/)).
Visual latencies (see review).

Dynamics of vision

Visual latencies ([see review](https://laurentperrinet.github.io/publication/grimaldi-22-polychronies/)).
Visual latencies (see review).

Dynamics of vision

Sensorimotor delays ([Perrinet & Friston 2014](https://laurentperrinet.github.io/publication/perrinet-adams-friston-14/))
Sensorimotor delays (Perrinet & Friston 2014)

Dynamics of vision

Sensorimotor delays ([Perrinet & Friston, 2014](https://laurentperrinet.github.io/publication/perrinet-adams-friston-14/))
Sensorimotor delays (Perrinet & Friston, 2014)

Dynamics of vision

Dynamics of vision

Diagonal markov model ([Khoei *et al*, 2017](https://laurentperrinet.github.io/publication/khoei-masson-perrinet-17/)).
Diagonal markov model (Khoei et al, 2017).

Dynamics of vision

Flash-lag effect: MBP (Khoei et al, 2017)

Dynamics of vision

[[Marr, 1982](https://outde.xyz/2020-01-12/overappreciated-arguments-marrs-three-levels.html)]
[Marr, 1982]

Spiking Neural Networks

[[Marr, 1982](https://outde.xyz/2020-01-12/overappreciated-arguments-marrs-three-levels.html)]
[Marr, 1982]

Spiking Neural Networks: Leaky Integrate-and-Fire Neuron

[Grimaldi *et al*, 2023, [Precise Spiking Motifs](https://laurentperrinet.github.io/publication/grimaldi-22-polychronies/)]
[Grimaldi et al, 2023, Precise Spiking Motifs]

Spiking Neural Networks in neurobiology

[[Mainen & Sejnowski, 1995](https://github.com/SpikeAI/2022_polychronies-review/blob/main/src/Figure_2_MainenSejnowski1995.ipynb)]
[Mainen & Sejnowski, 1995]

Spiking Neural Networks in neurobiology

[[Mainen & Sejnowski, 1995](https://github.com/SpikeAI/2022_polychronies-review/blob/main/src/Figure_2_MainenSejnowski1995.ipynb)]
[Mainen & Sejnowski, 1995]

Spiking Neural Networks in neurobiology

[[Diesmann et al. 1999](https://github.com/SpikeAI/2022_polychronies-review/blob/main/src/Figure_3_Diesmann_et_al_1999.py)]
[Diesmann et al. 1999]

Spiking Neural Networks in neurobiology

[[Haimerl et al, 2019](https://laurentperrinet.github.io/publication/grimaldi-22-polychronies/)]
[Haimerl et al, 2019]

Spiking Neural Networks: Spiking motifs

[Grimaldi *et al*, 2023, [Precise Spiking Motifs](https://laurentperrinet.github.io/publication/grimaldi-22-polychronies/)]
[Grimaldi et al, 2023, Precise Spiking Motifs]

Spiking Neural Networks: Spiking motifs

Review on [Precise Spiking Motifs](https://laurentperrinet.github.io/publication/grimaldi-22-polychronies/).
Review on Precise Spiking Motifs.

Spiking Neural Networks: Spiking motifs

Review on [Precise Spiking Motifs](https://laurentperrinet.github.io/publication/grimaldi-22-polychronies/).
Review on Precise Spiking Motifs.

Spiking Neural Networks in neuromorphic engineering

From frame-based to event-based cameras.
From frame-based to event-based cameras.

Spiking Neural Networks in neuromorphic engineering

The HD-SNN neural network.
The HD-SNN neural network.

Spiking Neural Networks in neuromorphic engineering

Spiking Neural Networks in neuromorphic engineering

The HD-SNN neural network.
The HD-SNN neural network.

Spiking Neural Networks in neuromorphic engineering

The HD-SNN neural network.
The HD-SNN neural network.

Artificial neural networks and machine learning applied to the understanding of biological vision

[[Marr, 1982](https://outde.xyz/2020-01-12/overappreciated-arguments-marrs-three-levels.html)]
[Marr, 1982]

Interactions between machine learning, artificial neural networks and our understanding of biological vision

Laurent Perrinet

[2023-05-10] NeuroSchool PhD Program in Neuroscience: Computation Neuroscience

logos qrcode

Contact me @ laurent.perrinet@univ-amu.fr