Study finds brain recalls sights with detail and sounds by meaning
Baycrest researchers used fMRI to compare visual and auditory memory, finding different weights on perceptual detail and gist.
By Priya Raghavan · Science Reporter
3 min read
Brain scans suggest people reconstruct visual and auditory memories in different ways, according to researchers at Baycrest's Rotman Research Institute. The finding matters because it may help scientists study memory changes in aging, learning and rehabilitation after impairment.
The study, published in The Journal of Neuroscience, reports that both sight and sound memories involve reinstatement, a process in which the brain reactivates patterns tied to an original experience. Baycrest said the difference lies in what the brain appears to emphasize during recall: visual memories kept more fine-grained sensory information, while sound memories leaned more on overall meaning.
Dr. Lei Zhang, a postdoctoral fellow at Baycrest's Rotman Research Institute and lead author of the study, said previous research had shown that people remember things they see and hear differently. Zhang said the new work helps identify brain mechanisms that may underlie that split.
The study was conducted with co-senior authors Dr. Claude Alain and Dr. Bradley Buchsbaum. Its full title is “Different Reliance on Sensory Reinstatement and Internally Transformed Representations during Vivid Retrieval of Visual and Auditory Episodes.”
How the study worked
The researchers used functional magnetic resonance imaging, or fMRI, to record brain activity in 25 young adults. Participants learned short naturalistic soundscapes and video clips that had been arranged in pairs.
During memory testing, one clip served as a cue for the other. Participants saw or heard the cue and were asked to mentally reconstruct the paired clip, then rated how vivid the memory felt after each trial.
The research team compared brain activity during learning and later recall. Baycrest said the analysis showed that both visual and auditory memories reactivated patterns in higher-order sensory regions, meaning the brain partially replayed parts of the original experience during retrieval.
Different paths to recall
The researchers found that stronger reactivation was linked with more vivid recall for both types of memory. The study also found that the replay differed by sensory system.
Visual memories were more closely associated with perceptual traces from the original experience, according to the study. Auditory memories relied more on internally reconstructed, meaning-based representations, often described as gist.
Baycrest said the findings point to a shared memory system for sights and sounds, with different weight placed on sensory detail or meaning depending on the kind of experience being remembered. The researchers described the work as among the first studies to directly compare visual and auditory memory reconstruction at the neural level.
Alain, a senior scientist at the Rotman Research Institute, said the findings could eventually support research into how memory shifts with age, how people learn effectively and how rehabilitation could be tailored for people with memory impairment. He also said future studies may examine whether auditory memory's focus on meaning offers benefits for long-term retention.
The paper lists Lei Zhang and colleagues as authors and carries the DOI 10.1523/jneurosci.1576-25.2026.
This story draws on original reporting from Medical Xpress.