We propose a neuromimetic architecture able to perform always-on pattern recognition. To achieve this, we extended an existing event-based algorithm [1], which introduced novel spatio-temporal features as a Hierarchy Of Time-Surfaces (HOTS). Built from asynchronous events acquired by a neuromorphic camera, these time surfaces allow to code the local dynamics of a visual scene and to create an efficient event-based pattern recognition architecture. Inspired by neuroscience, we extended this method to increase its performance. Our first contribution was to add a homeostatic gain control on the activity of neurons to improve the learning of spatio-temporal patterns [2]. A second contribution is to draw an analogy between the HOTS algorithm and Spiking Neural Networks (SNN). Following that analogy, our last contribution is to modify the classification layer and remodel the offline pattern categorization method previously used into an online and event-driven one. This classifier uses the spiking output of the network to define novel time surfaces and we then perform online classification with a neuromimetic implementation of a multinomial logistic regression. Not only do these improvements increase consistently the performances of the network, they also make this event-driven pattern recognition algorithm online and bio-realistic. Results were validated on different datasets: DVS barrel [3], Poker-DVS [4] and N-MNIST [5]. We foresee to develop the SNN version of the method and to extend this fully event-driven approach to more naturalistic tasks, notably for always-on, ultra-fast object categorization.

Motion Clouds are a generative model for naturalistic visual stimulation that offer full parametric control and more naturalism than the widely used alternatives of Random Dot Kinematograms (RDKs) or luminance gratings. We previously released an 3D FFT-based generation algorithm (Sanz-Leon et al., J Neurophysiol, 2012). Here, we present a novel implementation of motion clouds that uses an Auto-Regressive formulation so that any number of frames can be generated quickly with parameters changed in near real time, as needed in closed loop experiments. We demonstrate a version of the proposed toolbox that will be available online to illustrate the level of control available. With a graphic user interface, researchers can use interactive sliders to adjust motion cloud parameters like central frequency, orientations and bandwidths to get an intuitive feel for the parametric changes. We provide functions that can be easily integrated with psychophysics task tools like Psychtoolbox. Motion clouds can be used to generate trials of stand-alone moving luminance textures or added to other stimuli like images or videos as dynamic noise to disrupt visual processing. The toolbox can be run using GPUs to speed up generation to pseudo real-time for large stimulus arrays of about 1024 by 1024 pixels at 100Hz. We argue that this tool can enhance visual perception experiments in a range of contexts and would like it to be open to extensive testing, use and further development by the psychophysics, computational modelling, functional imaging and neurophysiology communities.

https://www.cerveauetpsycho.fr/sd/neurobiologie/le-mystere-de-la-joconde-elucide-par-les-neurosciences-26605.php https://www.facebook.com/photo/?fbid=10233017307913043&set=a.2288497170052 https://neuromatch.social/@laurentperrinet/113027202054980118 https://www.linkedin.com/posts/laurent-perrinet-1857b9_dans-le-dernier-num%C3%A9ro-de-cerveau-psycho-activity-7233740214886625280-Ivbf

Both biological and artificial neural networks inherently balance their performance with their operational cost, which balances their computational abilities. Typically, an efficient neuromorphic neural network is one that learns representations that reduce the redundancies and dimensionality of its input. This is for instance achieved in sparse coding, and sparse representations derived from natural images yield representations that are heterogeneous, both in their sampling of input features and in the variance of those features. Here, we investigated the connection between natural images’ structure, particularly oriented features, and their corresponding sparse codes. We showed that representations of input features scattered across multiple levels of variance substantially improve the sparseness and resilience of sparse codes, at the cost of reconstruction performance. This echoes the structure of the model’s input, allowing to account for the heterogeneously aleatoric structures of natural images. We demonstrate that learning kernel from natural images produces heterogeneity by balancing between approximate and dense representations, which improves all reconstruction metrics. Using a parametrized control of the kernels’ heterogeneity used by a convolutional sparse coding algorithm, we show that heterogeneity emphasizes sparseness, while homogeneity improves representation granularity. In a broader context, these encoding strategy can serve as inputs to deep convolutional neural networks. We prove that such variance-encoded sparse image datasets enhance computational efficiency, emphasizing the benefits of kernel heterogeneity to leverage naturalistic and variant input structures and possible applications to improve the throughput of neuromorphic hardware.

Recently, there has been an increase in interest in exploring the hypothesis that neural activity conveys information through precise spiking motifs. To investigate this phenomenon, several algorithms have been proposed to detect such motifs in Single Unit Activity recorded from populations of neurons. Based on the inversion of a generative model of raster plot synthesis, we present a novel detection model. This model derives an optimal detection procedure in the form of logistic regression combined with temporal convolution. Its differentiability allows for a supervised learning approach using gradient descent on the binary cross-entropy loss. To assess the model’s ability to detect spiking motifs in synthetic data, numerical evaluations are performed. This analysis emphasizes the benefits of utilizing spiking motifs instead of traditional firing rate-based population codes. Our learning method was able to successfully recover synthetically generated spiking motifs, indicating its potential for further applications. In the future, we aim to extend this method to real neurobiological data, where the ground truth is unknown, to explore and detect spiking motifs in a more natural and biologically relevant context.

for a follow-up, check out Jean-Nicolas Jérémie, Emmanuel Daucé, Laurent U Perrinet (2023). Retinotopic Mapping Enhances the Robustness of Convolutional Neural Networks. Submitted. PDF Cite DOI arXiv

Timing is essential for neural processing, but evidence for such temporal precision is still lacking. We have developed a theoretical model of representation based on spatio-temporal spiking motifs. Our goal is to develop a self-supervised learning method for optimal detection of such motifs in neurobiological data. To detect such motifs, we have extended the K-Means algorithm to process temporal data using a convolutional operator. A second pooling layer ensures that only one motif is used per time step. The results were improved by ensuring that the detected motifs are equiprobably activated using a homeostatic mechanism. We applied this algorithm to the Spiking Heidelberg database, which consists of the output of a realistic cochlear model to spoken digits. Qualitatively, the filters show a structure similar to the receptive fields found in the auditory cortex. Based on these promising results on this realistic yet synthetic dataset, future work will aim to apply his algorithm to neurological data to challenge the hypothesis of the role of precise spike timing in neural processes.

Grâce à nos yeux, l’organe sensible de la vision, nous pouvons aisément et instantanément explorer le monde visible qui nous entoure. C’est littéralement incroyable : la vision opère sans effort malgré la complexité des processus qui sont mis en œuvre. Mais en fait, comment fonctionnent nos yeux ? Quelles leçons pouvons-nous tirer de leur diversité dans le règne animal ? Est-il possible de remonter aux origines de leur évolution pour comprendre comment les yeux ont émergé au cours de l’évolution du vivant ?

Event cameras asynchronously report brightness changes with a temporal resolution in the order of microseconds, which makes them inherently suitable to address problems that involve rapid motion perception, such as ventral landing and fast obstacle avoidance. These problems are typically addressed by estimating a single global time-to-contact (TTC) measure, which explicitly assumes that the surface/obstacle is planar and fronto-parallel. We relax this assumption by proposing an incremental event-based method to estimate the TTC that jointly estimates the (up-to scale) inverse depth and global motion using a single event camera. The proposed method is reliable and fast while asynchronously maintaining a TTC map (TTCM), which provides per-pixel TTC estimates. As a side product, the proposed method can also estimate per-event optical flow. We achieve state-of-the-art performances on TTC estimation in terms of accuracy and runtime per event while achieving competitive performance on optical flow estimation.