[2026-09-18]

$logo$

Contact me @ laurent.perrinet@univ-amu.fr

Why Modeling Memory?

Modeling Memory : State-of-The-Art

Long short-term memory ([Wikipedia](https://en.wikipedia.org/wiki/Long_short-term_memory)) — Long short-term memory (Wikipedia)

Modeling Memory : State-of-The-Art

The Illustrated Transformer ([Jay Alammar](https://jalammar.github.io/illustrated-transformer/)) — The Illustrated Transformer (Jay Alammar)

Modeling Memory: Outline

Including memory in models

MNESIS: polychronous chains

Applications to neuroscience

Spiking Neural Networks

Spiking Neural Networks: Leaky Integrate-and-Fire

Review on Precise Spiking Motifs ([Grimaldi *et al*, 2023](https://laurentperrinet.github.io/publication/grimaldi-22-polychronies/)) — Review on Precise Spiking Motifs (Grimaldi *et al*, 2023)

Spiking Neural Networks: neurobiology

([Mainen & Sejnowski, 1995](https://github.com/SpikeAI/2022_polychronies-review/blob/main/src/Figure_2_MainenSejnowski1995.ipynb)) — (Mainen & Sejnowski, 1995)

Spiking Neural Networks: neurobiology

([Diesmann et al. 1999](https://github.com/SpikeAI/2022_polychronies-review/blob/main/src/Figure_3_Diesmann_et_al_1999.py)) — (Diesmann et al. 1999)

Spiking Neural Networks: neurobiology

Internal representation of hippocampal neuronal population spans a time-distance continuum. ([Haimerl et al, 2019](https://laurentperrinet.github.io/publication/grimaldi-22-polychronies/)) — Internal representation of hippocampal neuronal population spans a time-distance continuum. (Haimerl et al, 2019)

Spiking Neural Networks: Leaky Integrate-and-Fire

Spiking Neural Network: Polychronization

Spiking Neural Networks: Polychronization

Spiking Neural Network: Polychronization

Polychronization: Computation with Spikes ([Izhikevich, 2006](https://doi.org/10.1162/089976606775093882)) — Polychronization: Computation with Spikes (Izhikevich, 2006)

Spiking Neural Network: Polychronization

Methods

Methods: chaining Polychronization

Working Memory in a Recurrent Spiking Neural Networks ([LP, 2026](https://arxiv.org/abs/2604.14096)) — Working Memory in a Recurrent Spiking Neural Networks (LP, 2026)

Methods : BPTT (snnTorch)

Training SNNs Using Lessons From Deep Learning ([Eshraghian, 2023](https://ieeexplore.ieee.org/document/10242251)) — Training SNNs Using Lessons From Deep Learning (Eshraghian, 2023)

Methods : Weight initialization

$$ I_j(t) = \sum_{i=1}^{N} \bigl ( \sum_{d=1}^{D} \mathbf{W}_{j, i, d} \cdot s_i(t-d) \bigr ) $$ $$ u_j(t) = \beta \cdot u_j(t-1) \cdot (1 - s_j(t-1)) + I_j(t) $$ $$ s_j(t) = \mathbf{H}[u_j(t) \geq \vartheta] $$

$$ \mathbf{W} \mathbf{C} \approx \mathbf{S} $$

$$ w_{j, i, d} = \frac{1}{N \cdot D \cdot p_A \cdot M} \sum_{\mu=1}^{M} \sum_{t=D+1}^{T} s_{j}^{\mu}(t) \cdot s_i^{\mu}(t-d) $$

Results

Results : recall of target

Results : memory retrieval

Results : recall of target with noise

Results : recall of target with less trigger time

Results : recall of target with less trigger neurons

Results : robustness of target recall

Results : Analysing the network

Results : role of parameters

Modelling for neuroscience

Modelling for neuroscience: travelling waves

Modelling for neuroscience: Lorenz attractor

Modelling for neuroscience: encoding text

Trigger: severely. “What are you thinking of?”\n“I beg your pardon,” said Alice very humbly: “you had got to the fifth bend, I think?”\n“I had _not!_” cried the Mouse, sharply and very angrily.\n“A knot!” said Alice, always ready to make

Real: herself useful, and looking anxiously about her. “Oh, do let me help to undo it!”\n“I shall do nothing of the sort,” said the Mouse, getting up and walking away. “You insult me by talking such nonsense!”\n“I didn’t mean it!” pleaded poor Alice.

Prediction: herself useful, and looking anxiously about her. “Oh, do let me help to States it!”\n“I shall do nothing of the sort,” said the Mouse, getting up and walking away. “You insult me by talking such nonsense!”\n“I didn’t mean it!” pleaded poor Alice.

Modelling for neuroscience: encoding text

Trigger: —make—anything—prettier.”\n“Well, then,” the Gryphon went on, “if you don’t know what to uglify is, you _are_ a simpleton.”\nAlice did not feel encouraged to ask any more questions about it, so she turned to the Mock Turtle

Real: and said “What else had you to learn?”\n“Well, there was Mystery,” the Mock Turtle replied, counting off the subjects on his flappers,

Prediction: ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"

Modelling for neuroscience: encoding text

Trigger: one from this side of the ground.” So she tucked it away under her arm, that it might not escape again, and went back for a little more conversation with her friend.\nWhen she got back to the Cheshire Cat, she was surprised to find quite a large crowd collected round it: there was

Real: a dispute going on between the executioner, the King, and the Queen, who were all talking at once, while all the rest were quite silent, and looked very uncomfortable.\nThe moment Alice appeared, she was appea

Prediction: aI going on between, execution do, the King, and the Queen, who were all talking at once, while all the rest were quite silent, and looked very uncomfortable.\nThe moment Alice appeared, she was appea