In 1989, Neal Cohen and Michael McCloskey were exploring how to teach artificial neural networks (ANNs) to perform addition. As neuroscientists at Johns Hopkins, they were particularly interested in the mechanisms of memory storage and maintenance in neural networks.
Cohen and McCloskey converted numerical values into neuron patterns and then trained a neural network to perform addition by transforming two input numbers (for example, 1 and 3) into the correct output number (which would be 4 in this instance). They first instructed the network to add ones (such as 1+2, 1+3, 1+4, etc.) until it mastered the task. Once it was proficient, they moved on to teaching the same network to add twos (like 2+1, 2+2, 2+3) until it excelled at that as well.
However, they soon encountered an issue.
After the network learned to add twos, it forgot how to add ones. When they backpropagated errors through the network and adjusted the weights to facilitate the addition of twos, the network effectively erased its memory of how to add ones. It successfully acquired the new skill but at the cost of its prior knowledge.
Cohen and McCloskey labelled this phenomenon in ANNs as the problem of catastrophic forgetting. This was not a minor discovery but rather a widespread and significant limitation of neural networks. When you train a neural network to recognise a new pattern or execute a new task, you run the risk of disrupting the network’s previously acquired knowledge.
How do contemporary AI systems tackle this issue? The truth is, they don’t yet.
Developers sidestep the problem by halting their AI systems once they have completed training. They do not allow AI systems to learn incrementally; instead, they learn everything at once and then cease to learn. ChatGPT, for instance, doesn’t learn from the multitude of users interacting with it; it stops learning the moment it is launched. A self-driving car doesn’t adapt while driving. Its software receives updates, which are essential for feeding the models new information and enhancing their performance. Current AI systems can only be updated after being retrained from the beginning, with human oversight closely monitoring their effectiveness across all relevant tasks. Catastrophic forgetting occurs because AI systems lack an understanding of causation. They recognise that certain events are linked to others but fail to determine which factors directly cause other outcomes. It’s akin to knowing that clouds increase the likelihood of rain without understanding that clouds are the cause of rain.
Biological brains do not encounter the issue of catastrophic forgetting because they do not initially learn patterns.
Several hypotheses exist regarding how vertebrate brains manage this.
One hypothesis suggests that the cortex’s capacity for pattern separation protects it from catastrophic forgetting. By distinguishing incoming patterns within the cortex, it is less likely that they will interfere with one another.
Another hypothesis proposes that learning in the cortex happens only during moments of surprise; synaptic weights are allowed to change only when the cortex detects a pattern that exceeds a certain novelty threshold. This selective learning helps maintain the stability of learned patterns over extended periods. Additionally, brains utilise various strategies to prevent catastrophic forgetting.
Following new experiences, the brain often “replays” both old and new memories during rest or sleep. This natural reactivation aids in the integration of new information while safeguarding older memories, effectively reorganising synaptic connections to avoid overlap and interference.
Additionally, the brain is equipped with multiple complementary learning systems. It utilises fast-learning areas for new information, such as the hippocampus, and slow-learning areas for stable, long-term storage, like the cortex. Over time, the hippocampus assists in “teaching” the cortex, a process referred to as consolidation. During sleep, the brain strengthens distinct, non-overlapping neuronal groups for each memory trace, ensuring that competing memories remain separate. Replay during sleep or offline periods promotes synaptic plasticity, allowing connections to adapt without erasing established knowledge.
The title of the first volume in the “Romy&Roby” AI Series for the entire family is “Romy, Roby and the Secrets of Sleep.” This choice is intentional. During the traumatic events in Como, Roby the robot sustains damage. Once rebooted and operational again, Mum and the family realise that he has experienced catastrophic forgetting and no longer remembers anything about his home. Roby is repaired with the assistance of sleep algorithms, which we will explore in future blog posts. These algorithms replicate the brain’s processes of replay and consolidation, aiming to minimise catastrophic forgetting in artificial neural networks.