Our brains are wired to fill in perceptual gaps in what we see, whether it’s a lion hiding in the trees or the shapes hidden in an optical illusion — but how does that wiring work? Neuroscientists are zeroing in on how special kinds of brain cells help us see things that aren’t actually there.
Researchers from Seattle’s Allen Institute for Brain Science and the University of California at Berkeley traced the role played by the cells, known as IC-encoder neurons, in a study published today by the journal Nature Neuroscience.
“The goal of this project was to understand the neural basis of pattern completion, or filling in when you are dealt ambiguous or missing data in your vision,” said senior study author Hillel Adesnik, a neuroscientist at Berkeley.
Such research could help scientists understand how our brains create a complete picture of the world around us from the data that our senses provide. It could also eventually reveal how hallucinations arise, or point the way to better computer vision systems.
The researchers behind the newly published study conducted their experiments using mice. They showed the mice a series of diagrams — some of which used Pacman-like circles to create an illusory black bar, and others in which the bar was explicitly outlined.
The Allen Institute’s OpenScope Brain Observatory team recorded how the brains of the mice responded to the different diagrams, millisecond by millisecond, and then delivered the data to the Berkeley team.
“The observatory provided access to what was going on during illusions across the brain,” Jerome Lecoq, an associate investigator at the Allen Institute, told GeekWire in an email. “This allowed the experiment at Berkeley in Hillel Adesnik’s lab to dive deeper into local mechanics that implemented those illusions.”
The Berkeley researchers identified a small set of neurons that responded to the diagrams with the illusion, but not to the diagrams with the outlined bars. They also used laser-based techniques known as two-photon calcium imaging and holographic optogenetics to trace the flow of activity between those IC-encoders — that is, illusion contour encoders — and other regions of the visual cortex.
The last set of experiments added one more twist: The researchers showed the mice a blank gray screen, but activated the IC-encoder neurons while they were doing it. Then they watched for activity in other neurons in the visual cortex.
“When we looked at those downstream neurons, their activity patterns looked more similar to when there was an actual illusory contour on the screen than when there was nothing. So we were sort of generating this controlled hallucination,” Adesnik said. “We can’t say that it actually made it to the highest levels of perception, assuming mice have perception. … But we could say that at the neural level, we could see neural pattern completion.”
Based on their analysis of the data, the researchers concluded that the IC-encoder neurons help manage the process of filling in the gaps in visual perception.
“The representation of the illusion arises in higher visual areas first, and then gets fed back to the primary visual cortex; and when that information is fed back, it’s received by these IC-encoders in the primary visual cortex,” lead study co-author Hyeyoung Shin, who went from Berkeley to Seoul National University in 2023, said in a news release.
Adesnik said future experiments could address at least some of the many questions raised by the newly published study. “If you really want to gauge what the mouse experiences perceptually, you’d have to train them to do so. And people have done this on all sorts of species — mice, monkeys, dogs, cats,” he said. “So we can do that.”
Insights from such research could eventually be applied to the human condition.
“In certain diseases you have patterns of activity that emerge in your brain that are abnormal, and in schizophrenia these are related to object representations that pop up randomly,” Lecoq said. “If you don’t understand how those objects are formed and a collective set of cells work together to make those representations emerge, you’re not going to be able to treat it; so understanding which cells and in which layer this activity occurs is helpful.”
Lecoq said the Allen Institute’s role in the research is consistent with the vision that gave rise to the OpenScope Brain Observatory back in 2018.
“The astronomical observatory analogy here is the observatory doing the survey of the sky, while the lab focuses on a local relevant constellation to untangle what is going on,” he said.
“This is how we view our Brain Observatory in the future: We will provide brain-wide recordings of specific and important behaviors, defined by one or more laboratories. Then those laboratories can analyze those datasets and sometimes perform complementary experiments to uncover specific computational mechanisms in the brain,” Lecoq said. “I am anticipating more research in neuroscience will be performed this way in the future.”
Previously: OpenScope team stirs up a brainstorm
In addition to Shin, Adesnik and Lecoq, the authors of the Nature Neuroscience study, titled “Recurrent Pattern Completion Drives the Neocortical Representation of Sensory Inference,” include Mora Ogando, Lamiae Abdeladim, Uday Jagadisan, Severine Durand, Ben Hardcastle, Hannah Belski, Hannah Cabasco, Henry Loefler, Ahad Bawany, Josh Wilkes, Katrina Nguyen, Lucas Suarez, Tye Johnson, Warren Han, Ben Ouellette, Conor Grasso, Jackie Swapp, Vivian Ha, Ahrial Young, Shiella Caldejon, Ali Williford, Peter Groblewski, Shawn Olsen and Carly Kiselycznyk.
Credit for Kanizsa Triangle graphic: Fibonacci, CC BY-SA 3.0, via Wikimedia Commons.