Motivation
Through our ability to create and store memories, we can flexibly query our knowledge of the world and apply it to novel situations with ease. To do so, we must differentiate between components of our experiences that are situational from those that are invariant. This mental process is known as abstraction, and it enables us to perform inference and generate predictions in novel situations. As neuroscientists, we seek to understand the link between the behavior of a model system and its internal representations, so we wondered: how is abstraction reflected in the cellular activity of the human brain?
Approach
We began our investigation by taking inspiration from work in non-human primates [1] which suggested that there was a connection between abstraction and the geometry of neural representations. Given its importance to this work, it’s worth spending a moment explaining what we mean when we say the “geometry of neural representations”. One oft-used measure of neuronal activity is the firing rate (i.e., number of spikes per unit time). For a single cell, the firing rate is a scalar value that describes the cell’s level of activity at any given time. For a population of N cells, this is a vector in N-dimensional space. Changes in the underlying neural activity can therefore be represented as trajectories in this N-dimensional space. For some time now, these representations and their geometrical properties have become the focus of extensive study in both the neuroscience and machine learning communities. Using this framework, we hypothesized that abstract reasoning was related to specific geometrical properties of neural representations.
We relied on three metrics to quantify the content and format of the underlying representational structure: 1. shattering dimensionality (SD), 2. cross-condition generalization performance (CCGP), and 3. parallelism score (PS). Please see [1] for a more complete definition of these metrics, and what aspect of the representational geometry they capture. Figure 1 shows what these metrics look like for three different configurations of 8 task conditions in a toy 3D neural representation. In this example, one of the 35 possible dichotomies is color-coded and labeled as “context”.
Figure 1: Three different geometries and their corresponding metrics (SD, CCGP, and PS). The color indicates context, and each of the dots represents a particular stimulus in that context. The axes are the firing rate of three hypothetical neurons.
The previous work on abstraction in non-human primates took place after extensively training the animals to perform the task. It therefore could not make statements about the representational changes that accompany the emergence of abstract knowledge. Through a collaboration between Cedars-Sinai Medical Center (CSMC), Toronto Western Hospital (UWH) and Columbia’s Center for Theoretical Neuroscience (CTN), we were able to observe this emergence for the first time, in humans, with single-cell resolution. To this end, we recorded the activity of more than 3000 neurons, across 6 different brain areas in 17 patients as they performed an inferential-reasoning task. The patients were undergoing monitoring for epileptic seizures using implanted depth electrodes and volunteered to participate in our study during their hospital stay. Among the brain areas we monitored was the medial frontal cortex, the ventral temporal cortex, the amygdala, and the hippocampus, which is thought to be critical for the implementation of abstraction and inference-related computations. In the task, patients had to learn that each image shown on the screen was associated with a correct response (left or right) and reward (5¢ or 25¢). Both the correct response and reward associated with a stimulus were dependent on a context that was not directly observable. Each individual learned these context-dependent associations at their own pace, and through their performance on critical trials, we could measure their level of understanding over time.
Figure 2: (a) Electrode placements across 6 brain regions. (b) The structure of the task that the participants performed. (c) Stimulus-response-outcome mappings across two contexts. (d) Example performance of a patient around an un-cued context switch. (e) Improved inference performance of a single patient over three sessions. (f) A 3D visualization (using multi-dimensional scaling) of the neural representations in the hippocampus for sessions where patients successfully performed inference.
Summary of results
In our analysis of the behavioral and neural data we observed that individuals who could correctly perform inference were different in two important ways: 1. there was a neural representation of the latent context, and 2. this representation was highly structured (compare Figure 2f with configuration 1 in Figure 1) with above chance CCGP for the context and task stimuli. This means that these variables were represented in an abstract format. One remarkable finding, which deviates from the previous work on abstraction in non-human primates, is the regional specificity of these effects in the human brain: the effects were observed in the hippocampus and not in the other brain areas we monitored. In fact, we observed dramatic changes in the firing rate properties and tuning of hippocampal cells as a function of learning, offering mechanistic insight into the learning process. And perhaps the most exciting result from this work, which could only be observed in humans, is that the representational properties which we linked to abstract reasoning could be achieved through either learning or verbal instruction. We were able to demonstrate this by comparing two types of sessions: 1. sessions where the patients received explicit task instructions detailing the stimulus-response-outcome mappings across the two contexts, and 2. sessions where the subjects learned these mappings entirely on their own. These results suggest that the hippocampus plays an important role in the rapid acquisition and deployment of abstract knowledge.
Epilogue
This work is the culmination of a multi-year effort, leveraging the varied expertise of a team that was split between two coasts. Our findings from this study highlight the importance of studying the same neural processes in different species and underscore the value of human electrophysiology. In our future work we would like to improve our understanding of abstraction by using richer and more naturalistic tasks, allowing us to probe the learning process with greater temporal resolution and to personalize the experience of the task to each individual participant.
References
[1] Bernardi, Silvia, et al. “The geometry of abstraction in the hippocampus and prefrontal cortex.” Cell 183.4 (2020): 954-967.