In September 1953, a twenty-seven-year-old man named Henry Molaison underwent an experimental operation that removed his entire hippocampus, which, as doctors assumed, had been the source of his severe seizures. The surgery was deemed successful in one aspect; the intensity of his seizures significantly decreased, and he maintained his personality and intellect. However, people performing the intervention soon realised that the procedure had taken away something invaluable. Upon awakening, Molaison was utterly unable to form new memories. He could engage in a conversation for a minute or two, but shortly after, he would forget everything that had just transpired. Even forty years later, he could complete crossword puzzles with information from before 1953, but he could not recall facts that occurred afterwards. Molaison was effectively trapped in 1953.
Following his surgery, he became the most extensively studied patient in neuroscience history. What was it about the hippocampus that made it essential for forming new episodic memories while not affecting the retrieval of older ones?Â
In the brains of mammals, episodic memory arises from a collaboration between the ancient hippocampus and the more modern neocortex. The hippocampus is capable of rapidly learning patterns. Still, it cannot create a simulation of the world, whereas the neocortex can simulate intricate details of the world but struggles to learn new patterns quickly.
Do you recall the concept of catastrophic forgetting from the last blog? Neural networks tend to lose old patterns as they learn new ones. In mammals, the hippocampus helps the neocortex integrate new memories by retrieving and replaying recent memories alongside older ones, ensuring that the old ones remain intact. In the realm of AI, this is known as generative replay and experience replay, which aim to mitigate catastrophic forgetting, though neither method has completely resolved the issue.
Experience replay involves saving an agent’s previous interactions—tuples consisting of (state, action, reward, next state)—in a buffer. During training, samples from this buffer are randomly picked for learning, which disrupts correlations between consecutive experiences, enhances data efficiency, and stabilises the learning process.
Generative replay, on the other hand, employs a generative model (often a neural network) to produce synthetic samples that mimic data from past tasks, even if the original data is no longer accessible. By replaying these generated samples along with new data during continual or sequential learning, the AI can maintain old knowledge and lessen forgetting. Generative replay is especially advantageous when it is impractical to store real data.
Nonetheless, neither approach completely eliminates catastrophic forgetting, particularly as the number or complexity of tasks we want our AIs to perform grows. There are still challenges regarding generative quality, memory capacity, and scalability for both methods. If only machines could emulate the interaction between the hippocampus and other brain regions!