From the retina to action: Predictive processing in the visual system

Victor Boutin, Angelo Franciosini and Laurent Perrinet

HDR Robin Baurès, 25/3/2019

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement n°713750. Also, it has been carried out with the financial support of the Regional Council of Provence-Alpes-Côte d'Azur and with the financial support of the A*MIDEX (n°ANR-11-IDEX-0001-02). This work was granted access to the HPC resources of Aix-Marseille Université financed by the project Equip@Meso (ANR-10-EQPX-29-01) of the program "Investissements d’Avenir".

(AUTHOR) Hello, I am Laurent Perrinet from the Institute of Neurosciences of la Timone in Marseille, a joint unit from the CNRS and the AMU
(OBJECTIVE) Today I will present some recent results on Predictive processing in the visual system ... in visual systemS / From the retina to action
please interrupt
(ACKNO) this endeavour involves different techniques and tools ... From the head on, I wish to thanks people who collaborated and in particular .. mostly funded by the ANR horizontal V1 (fregnac chavane) + ANR TRAJECTORY (o marrre bruno cessac palacios )
LONDON (Jim Bednar, Friston)
(SHOW TITLE)

Visual areas are essential in transforming the raw luminous signal into a representation which efficiently conveys information about the environment. This process is constrained by various factors such as a wide variety of changes in the characteristics of the visual image but also by the necessity to be able to respond as quickly as possible to the incoming sensory stream, for instance to drive a movement of the eyes to the location of a potential danger.

As a consequence, and to rephrase [@Wigner90], "the Unreasonable Effectiveness of Vision in the Natural World" invites us to focus on this cognitive ability for a better understanding of the brain in general.

To achieve this, it is believed that the visual system takes advantage of the existence of a priori knowledge in the structure of visual information, such as the regularity in the shape and motion of visual objects. As such, the predictive coding coding framework offers a unified theory to explain many of the mechanisms at the different levels of the visual system and which were unveiled by decades of study in neurophysiology and psychophysics.

LP (2015) "Sparse models" in Biologically Inspired Computer Vision

Ravello, LP, Escobar, Palacios (2019) Scientific Reports

Kremkow, LP, Monier, Alonso, Aertsen, Fregnac, Masson (2016) Push-pull receptive field organization and synaptic depression: Mechanisms for reliably encoding naturalistic stimuli in V1

LP (2015) "Sparse models" in Biologically Inspired Computer Vision

Olshausen and Field (1997) Sparse coding with an overcomplete basis set: A strategy employed by V1?

a seminal idea is proposed by Olshausen: * this may be formalized as an inference problem:

edges are different sources, which are known to be sparse: by mixing these sources one forms the image (transparency hypothesis)

the sparseness is characterized by the pdf of the sources coefficients

this inference problem is an inverse problem:
while this problem may be hard to solve, this may be approached using a (conjugate) gradient descent which has a nice implementation in terms of neural networks

I wish to point here to an essential feature compared to classical feed-forward networks: you can not do the inference in one single shot in general; you need a recurrent / recursive network (see arrow) and this is precisely a possible function for one of the most numerous type of synapses: short-ranges lateral interactions

Olshausen and Field (1997) Sparse coding with an overcomplete basis set: A strategy employed by V1?

a seminal idea is proposed by Olshausen: * this may be formalized as an inference problem:

edges are different sources, which are known to be sparse: by mixing these sources one forms the image (transparency hypothesis)

the sparseness is characterized by the pdf of the sources coefficients

this inference problem is an inverse problem:
while this problem may be hard to solve, this may be approached using a (conjugate) gradient descent which has a nice implementation in terms of neural networks

Olshausen and Field (1997) Sparse coding with an overcomplete basis set: A strategy employed by V1?

a seminal idea is proposed by Olshausen: * this may be formalized as an inference problem:

edges are different sources, which are known to be sparse: by mixing these sources one forms the image (transparency hypothesis)

the sparseness is characterized by the pdf of the sources coefficients

this inference problem is an inverse problem:
while this problem may be hard to solve, this may be approached using a (conjugate) gradient descent which has a nice implementation in terms of neural networks

LP (2010) Neural Computation

Outline

Retina, Sparse coding and unsupervised learning

A model of Sparse Deep Predictive Coding

Emergence of mid-level features

let's move to the second part : learning

the main goal of Olshasuen was not only sparse coding but the fact that using the sparse code, a simple linear hebbian learning allows to separate independent sources

one can go one step before the cortex and ask the same question in the retina are the same process present ?

Theoretical advances in neural networks modelling have recently been pushed by the field of machine learning. These have proposed that biological neural networks could process information according to certain constraints of efficiency, such as minimizing the surprise of sensory states given noisy or ambiguous sensory inputs. We have developed such a Hierarchical (Deep) Predictive Coding model using unsupervised learning on (static) natural images, and that we wish to extend to (dynamic) sensory flows. Compared to classical Deep Learning models, one advantage of this approach is that the learned synaptic weights are meaningful, that is that they often emerge as to represent explicitly components of the input image in increasing levels of complexity: edges at the first layer, curvatures in the second, the eye or mouth in the third layer when presenting images of faces. Such networks can also be extended to supervised or reinforcement learning by adding efferent layers that would accept such labels.

Boutin, Franciosini, Ruffier, LP (2019) submitted

(from http://cs231n.github.io/convolutional-networks/)

Boutin, Franciosini, Ruffier, LP (2019) submitted

Outline

Retina, Sparse coding and unsupervised learning

A model of Sparse Deep Predictive Coding

Emergence of mid-level features

From the retina to action: Predictive processing in the visual system

Victor Boutin, Angelo Franciosini and Laurent Perrinet

HDR Robin Baurès, 25/3/2019

The free parameters of this network are the anatomical constraints that we impose to dimension layers. In the particular case of this networks, this is the number of layers, the number of channels (that is the number of overlays of the same map in a given layer), and the size of the kernels implementing the convolutions (in resemblance to receptive fields in biological NNs). Indeed, as in most classical Deep Learning neural networks, these parameters are hand tuned. However, we observed that these superstructural numbers have a great influence on the quality and type of filters that could emerge from the learning phase. Theoretically, this make sense as it imposes different levels of competition within each layer across channels but also by defining a characteristic size in local interactions.

We believe that this theoretical problem is similar to that faced by different the primary visual cortex in different species. Given a similar sensory input, the intra-cortical network imposes different levels of anatomical complexities. We predict that, given realistic orders of magnitudes for different species, we could differentially predict the emergence of different macrostructures in different species. In particular, the emergence of a topographically smooth representation of orientations is present in marmosets while mice have a salt-and-pepper representation. Thanks to this level of understanding, we will use our models to make predictions. In particular, we can generate images which are optimal to evoke activity targeted at a given layer, for instance for the second layer (curvatures) versus the first layer (edges). In practice, such images take the form of textures with different levels of structural complexity. For instance, stimuli targeted at the first layer would correspond to a sub-class of stimuli that we have widely used in previous experiments (Motion Clouds). We predict that more complex classes of stimuli should evoke differential activities in both species.

- Task 5: Bluesky: “Efficient hierarchical representations ” Overall, the role of visual information processing in cortical layers is to represent information robustly to noise and visual transformations (translations zooms rotations). There is thus a pressure on the structure of this processing at different temporal scales to maximize the efficiency of this code. This depends both on the statistics of the sensory input (both in space and time) but also on the repertoire of behavioral tasks that are embodied. We have recently developed a hierarchical unsupervised algorithm for neural networks which allows to generate such representations and that fits well to the processing performed by feed-forward, lateral and feedback connections. In such a model, the message passing between neurons is implemented by propagating waves which progressively refine the representation of sensory data. In particular, tuning the complexity of the representation at different layers allows to see the emergence of different structures. In particular, progressively increasing the complexity in lateral connections shows a phase transition from a salt-and-pepper organization to a more continuous representation of the space x orientation space. This could explain the different architectures observed in different animals. Such models could also make predictions on the adaptation of the network to novel environmental or behavioral contingencies. The goal of this Task is two-sided: first to understand the physiological and behavioral data in the light of predictive coding theories, rooting these results in a quantitative model. The second goal is to generate predictions. In particular, using our models, it will be possible to generate novel stimulus classes that specifically tackle the efficiency of the visual system at its different spatial and dynamic granularities. Lead Laurent/Lyle?

Thanks for your attention!

https://laurentperrinet.github.io/talk/2019-03-25-hdr-robin-baures

Ravello, Escobar, Palacios, LP (2019) in prep