The digital twin was trained on large datasets of brain activity collected from the visual cortex of real mice as they watched movie clips. It could then predict the response of tens of thousands of neurons to new videos and images.

Digital twins could make studying the inner workings of the brain easier and more efficient.

“If you build a model of the brain and it’s very accurate, that means you can do a lot more experiments,” said Andreas Tolias, PhD, Stanford Medicine professor of ophthalmology and senior author of the study published April 10 in Nature. “The ones that are the most promising you can then test in the real brain.”

The lead author of the study is Eric Wang, PhD, a medical student at Baylor College of Medicine.

Beyond the training distribution

Unlike previous AI models of the visual cortex, which could simulate the brain’s response to only the type of stimuli they saw in the training data, the new model can predict the brain’s response to a wide range of new visual input. It can even surmise anatomical features of each neuron.

The new model is an example of a foundation model, a relatively new class of AI models capable of learning from large datasets, then applying that knowledge to new tasks and new types of data — or what researchers call “generalizing outside the training distribution.”

(ChatGPT is a familiar example of a foundation model that can learn from vast amounts of text to then understand and generate new text.)

“In many ways, the seed of intelligence is the ability to generalize robustly,” Tolias said. “The ultimate goal — the holy grail — is to generalize to scenarios outside your training distribution.”

Mouse movies

To train the new AI model, the researchers first recorded the brain activity of real mice as they watched movies — made-for-people movies. The films ideally would approximate what the mice might see in natural settings.

“It’s very hard to sample a realistic movie for mice, because nobody makes Hollywood movies for mice,” Tolias said. But action movies came close enough.

Mice have low-resolution vision — similar to our peripheral vision — meaning they mainly see movement rather than details or color. “Mice like movement, which strongly activates their visual system, so we showed them movies that have a lot of action,” Tolias said.

Over many short viewing sessions, the researchers recorded more than 900 minutes of brain activity from eight mice watching clips of action-packed movies, such as Mad Max. Cameras monitored their eye movements and behavior.

The researchers used the aggregated data to train a core model, which could then be customized into a digital twin of any individual mouse with a bit of additional training.

Accurate predictions

These digital twins were able to closely simulate the neural activity of their biological counterparts in response to a variety of new visual stimuli, including videos and static images. The large quantity of aggregated training data was key to the digital twins’ success, Tolias said. “They were impressively accurate because they were trained on such large datasets.”

Though trained only on neural activity, the new models could generalize to other types of data.

The digital twin of one particular mouse was able to predict the anatomical locations and cell type of thousands of neurons in the visual cortex as well as the connections between these neurons.

The researchers verified these predictions against high-resolution, electron microscope imaging of that mouse’s visual cortex, which was part of a larger project to map the structure and function of the mouse visual cortex in unprecedented detail. The results of that project, known as MICrONS, was published simultaneously in Nature.

Opening the black box

Because a digital twin can function long past the lifespan of a mouse, scientists could perform a virtually unlimited number of experiments on essentially the same animal. Experiments that would take years could be completed in hours, and millions of experiments could run simultaneously, speeding up research into how the brain processes information and the principles of intelligence.

“We’re trying to open the black box, so to speak, to understand the brain at the level of individual neurons or populations of neurons and how they work together to encode information,” Tolias said.

In fact, the new models are already yielding new insights. In another related study, also simultaneously published in Nature, researchers used a digital twin to discover how neurons in the visual cortex choose other neurons with which to form connections.

Scientists had known that similar neurons tend to form connections, like people forming friendships. The digital twin revealed which similarities mattered the most. Neurons prefer to connect with neurons that respond to the same stimulus — the color blue, for example — over neurons that respond to the same area of visual space.

“It’s like someone selecting friends based on what they like and not where they are,” Tolias said. “We learned this more precise rule of how the brain is organized.”

The researchers plan to extend their modeling into other brain areas and to animals, including primates, with more advanced cognitive capabilities.

“Eventually, I believe it will be possible to build digital twins of at least parts of the human brain,” Tolias said. “This is just the tip of the iceberg.”

Researchers from the University Göttingen and the Allen Institute for Brain Science contributed to the work.

The study received funding from the Intelligence Advanced Research Projects Activity, a National Science Foundation NeuroNex grant, the National Institute of Mental Health, the National Institute of Neurological Disorders and Stroke (grant U19MH114830), the National Eye Institute (grant R01 EY026927 and Core Grant for Vision Research T32-EY-002520-37), the European Research Council and the Deutsche Forschungsgemeinschaft.

Hyaline arteriolosclerosis is a condition that causes the small blood vessels to narrow, becoming thick and stiff. This makes it harder for blood to flow, which can damage the brain over time. It appears as lesions, areas of damaged tissue in the brain.

“Heavy alcohol consumption is a major global health concern linked to increased health problems and death,” said study author Alberto Fernando Oliveira Justo, PhD, of University of Sao Paulo Medical School in Brazil. “We looked at how alcohol affects the brain as people get older. Our research shows that heavy alcohol consumption is damaging to the brain, which can lead to memory and thinking problems.”

The study included 1,781 people who had an average age of 75 at death. All had brain autopsies.

Researchers examined brain tissue to look for signs of brain injury including tau tangles and hyaline arteriolosclerosis. They also measured brain weight and the height of each participant.

Family members answered questions about participants’ alcohol consumption.

Researchers then divided the participants into four groups: 965 people who never drank, 319 moderate drinkers who had seven or fewer drinks per week; 129 heavy drinkers who had eight or more drinks per week; and 368 former heavy drinkers. Researchers defined one drink as having 14 grams of alcohol, which is about 350 milliliters (ml) of beer, 150 ml of wine or 45 ml of distilled spirits.

Of those who never drank, 40% had vascular brain lesions. Of the moderate drinkers, 45% had vascular brain lesions. Of the heavy drinkers, 44% had vascular brain lesions. Of the former heavy drinkers, 50% had vascular brain lesions.

After adjusting for factors that could affect brain health such as age at death, smoking and physical activity, heavy drinkers had 133% higher odds of having vascular brain lesions compared to those who never drank, former heavy drinkers had 89% higher odds and moderate drinkers, 60%.

Researchers also found heavy and former heavy drinkers had higher odds of developing tau tangles, a biomarker associated with Alzheimer’s disease, with 41% and 31% higher odds, respectively.

Former heavy drinking was associated with a lower brain mass ratio, a smaller proportion of brain mass compared to body mass, and worse cognitive abilities. No link was found between moderate or heavy drinking and brain mass ratio or cognitive abilities.

Justo noted that, in addition to brain injuries, impaired cognitive abilities were observed only in former drinkers.

Researchers also found that heavy drinkers died an average of 13 years earlier than those who never drank.

“We found heavy drinking is directly linked to signs of injury in the brain, and this can cause long-term effects on brain health, which may impact memory and thinking abilities,” said Justo. “Understanding these effects is crucial for public health awareness and continuing to implement preventive measures to reduce heavy drinking.”

A limitation of the study was that it did not look at participants before death and did not have information on the duration of alcohol consumption and cognitive abilities.

The study was supported by The São Paulo Research Foundation.

University of Nebraska-Lincoln researchers made major contributions to the study, published online April 8 in the journal Nature Food.

“We are in a race to feed the world and to try to feed the population with the available agricultural land that we have,” said Patricio Grassini, Sunkist Distinguished Professor of Agronomy and one of the paper’s authors.

To do so requires estimates that predict both yield potential, as determined by weather and soil properties, and yield gaps, which is the difference between yield potential and current farm yields, which indicates the room that exists to increase food production on existing cropland. Those estimates are essential in making investments in agricultural research and development, both from public and private sources.

At issue is how best to compile those estimates.

In the Nature Food paper, a team that includes scientists from Nebraska and three other institutions calls into question the statistical methods now widely used. In addition to Grassini, Husker authors of the study included Fatima Tenorio, Fernando Aramburu Merlos and Juan Rattalino Edreira, research assistant professors of agronomy.

In the United States, for example, current statistical models tend to rely too heavily on best-case scenarios — the most productive counties with the most fertile soils in a year with the most favorable weather, Grassini said. The methods also extrapolate a single yield potential across large regions with a wide diversity of climates and soils that likely would produce a similarly wide range in yield potential.

“Therefore, if you use that year as a reference, you are going to be overestimating your production potential because the best county with the best soils in the best year doesn’t really represent your average climate or your most typical soil across the state,” Grassini said.

But in other parts of the world — Africa, for example — these models might underestimate crop yield. There, farmers may have limited access to inputs compared to producers in other areas, thus attaining yields far below what the climate can support.

This statistical approach also leads to conflicting results, with production potential estimates almost doubling from one method to another. Grassini said this approach — driven mostly by geographers and statisticians, not agronomists — has been largely accepted, and more rigorous analysis is needed.

The research team’s conclusions are explained in the paper, titled “Statistical approaches are inadequate for accurate estimation of yield potential and gaps at regional level.”

The study compared estimates of yield potential and yield gaps of major U.S. rainfed crops — corn, soybeans and wheat — derived from four statistical models against those derived from a “bottom-up” spatial scaling approach based on robust crop modeling and local weather and soil data, such as the Global Yield Gap and Water Productivity Atlas developed at Nebraska.

Process-based crop models used in this study have been rigorously validated for their capacity to estimate yield potential based on experimental data from well-managed crops grown across a wide range of environments. This bottom-up approach, which better incorporates long-term data and regional variations, is clearly superior, the team found.

“I expect some controversy,” Grassini said of the team’s conclusions challenging the conventional wisdom.

The approach recommended by the team should better capture yield gaps, which “can help identify regions with largest room to increase crop production, which, in turn provides a basis to orient agricultural research and development programs.”

“This is a call to set the record straight because if we are going to use this information to inform policy and our investments, we better make sure that the information is sound and has been validated,” Grassini said.

Additional team members included Romulo Lollato, associate professor of agronomy, Kansas State University; Sotirios Archontoulis, professor of agronomy, Iowa State University; and Antoine Couëdel, who completed his postdoctoral research at Nebraska and is a researcher at the French Agricultural Research Centre for International Development, France.