Decoding the brain’s learning machine
In studies with monkeys, Johns Hopkins researchers report that they have uncovered significant new details about how the cerebellum — the “learning machine” of the mammalian brain — makes predictions and learns from its mistakes, helping us execute complex motor actions such as accurately shooting a basketball into a net or focusing your eyes on an object across the room.
In a summary of the study published on April 16 in the journal Nature Neuroscience, the investigators provide a better understanding of why degenerative diseases that affect the cerebellum cause people to lose control of their movements. Their results demonstrate that the cerebellum is organized in a very different way than current designs of artificial neural networks, which are currently used in machine learning and artificial intelligence.
Learning in the cerebellum — a part of the vertebrate brain located at the back of the skull that directs and regulates movements — is guided through a process of trial and error. For example, when learning to shoot a basketball, people usually miss many times before getting one shot through the hoop. As the arm moves, the cerebellum makes predictions about the consequences of the action. When the prediction does not match reality — that is, the ball misses the hoop — the cerebellum receives feedback from the eyes and the arm to learn from the error, fine-tuning factors such as aim, force and release to make a basket. This trial-to-trial learning from error produces gradual improvements in performance. However, research efforts have fallen short of decoding how the cells in the cerebellum make predictions and how they learn from their mistakes, according to Reza Shadmehr, Ph.D., professor of biomedical engineering and neuroscience at the Johns Hopkins University School of Medicine.
“Trying to understand the neural activity in the cerebellum is a bit like trying to decipher an alien language, a signal from a far-off galaxy,” he says.
To decode the signal, Shadmehr said, the team began with the principal neurons in the cerebellum, called Purkinje cells. Purkinje cells communicate through two types of electrical signals: simple spikes, which reflect information regarding the prediction that the cells are making, and complex spikes, which reflect information that is sent back to the Purkinje cells, informing them of the error in their prediction.
“You can think of the simple spikes as the ‘student’ that makes a prediction and the complex spikes as the ‘teacher’ that provides feedback,” says Shadmehr.
In a past study, Shadmehr and his research team found that the Purkinje cells organize into small groups of about 50 and together make predictions, sending their output of all the members simultaneously. The neurons that make up these groups share a critical feature: They all receive the same error signal. Therefore, the fundamental computational unit in the cerebellum was not a single Purkinje cell, but a small group that learns together from a common mistake. While earlier work had tried to decipher the simple spikes of individual Purkinje cells, these results suggested that the key to understanding the cerebellum was to organize the Purkinje cells into groups and then count the sum total of simple spikes being produced by the members of the group. By doing so, Shadmehr and his team discovered that the group generated simple spikes that precisely predicted the motion of a monkey’s eyes as it gazed around a screen.
Having found a way to decipher the language with which the cerebellum made predictions, the next step was to understand how the cerebellum learns from its prediction errors. To investigate this question, the researchers measured cerebellar activity in seven rhesus monkeys fitted with electrodes and trained them to follow a small target with their eyes. The researchers replicated a “mistake” by moving the target more quickly than the monkeys could accurately predict, causing them to frequently miss the target.
They found that simple spikes measured through the electrodes correlated with the intensity of movement — more simple spikes meant that the monkeys needed to move their eyes more quickly to catch the target. According to the researchers, the Purkinje cells produced an average of 50–70 spikes per second.
The researchers found that the complex spikes did not convey error in the same manner. Regardless of how far off the monkeys’ eyes were from the target, the number of complex spikes produced in the cerebellum stayed the same. Instead, they found that the direction of the error affected the probability of generating a complex spike, whereas the magnitude of the error affected the timing of the complex spike. Therefore, the language of prediction was in rate of simple spikes, whereas the language of prediction error was in timing of the complex spike.
For example, Shadmehr found that when the monkeys missed the target by 3 degrees of distance, a complex spike appeared after 100 milliseconds, and if the monkeys missed by 8 degrees, a complex spike appeared in 120 milliseconds.
“In short, the ‘language’ of the teacher is time, while the language of the student is rate,” says Shadmehr.
Shadmehr explains that current AI machines in the world rely on the same basic structure and are composed of many layers of neuronlike units that together make predictions. Layers of such units make calculations, passing results to the next layer until the process reaches the “output” layer that makes the final prediction. When an AI machine’s prediction is wrong, it experiences an error and, with current AI networks, sends that error back to all units.
“These machines are inspired by the way biological networks learn, but what is interesting is that the architecture of the cerebellum is somewhat different than what has been designed in artificial systems,” says Shadmehr.
Instead of all the cells receiving complex spikes after the error, in the cerebellum the Purkinje cells organize into small groups, each specializing in responding to specific errors. The Purkinje cells appear to be organized based on a preference for error in only a small part of the task space.
One of the advantages of the cerebellum’s architecture, says Shadmehr, is that it protects memories. When you experience an error to the right, it causes learning by engaging Purkinje cells that register that error. If on the next trial you experience an error to the left, the new error engages a new group of Purkinje cells. As a result, new errors do not erase the memory, but result in two independent memories.
Damage to the cerebellum can profoundly impair the ability to make movements, resulting in symptoms that together are called ataxia. A puzzle that has remained unsolved is how the cerebellum translates its predictions into motor commands that guide movements.
This final result may make it possible to refine diagnosis of cerebellar disease by measuring learning from error and allow scientists to connect motor learning impairment with the loss of specific groups of Purkinje cells.
Other researchers involved in this study include David Herzfeld of the Johns Hopkins University School of Medicine and Yoshiko Kojima and Robijanto Soetedjo of the Washington National Primate Center.
This research was supported by the National Institutes of Health (R01NS078311), the National Eye Institute (R01EY019258, R01EY023277), the Johns Hopkins University Science of Learning Institute and the Office of Naval Research (N00014-15-1-2312).