In the race towards developing Artificial General Intelligence (AGI), with influential entities like OpenAI and Google DeepMind leading the way, understanding and replicating human-like intelligence has become an important research interest. Central to these technological advancements is the Transformer model, whose fundamental principles are now being explored in new depth.

The key to powerful AI systems is grasping how they learn and remember information. The team applied principles of human brain learning, specifically concentrating on memory consolidation through the NMDA receptor in the hippocampus, to AI models.

The NMDA receptor is like a smart door in your brain that facilitates learning and memory formation. When a brain chemical called glutamate is present, the nerve cell undergoes excitation. On the other hand, a magnesium ion acts as a small gatekeeper blocking the door. Only when this ionic gatekeeper steps aside, substances are allowed to flow into the cell. This is the process that allows the brain to create and keep memories, and the gatekeeper’s (the magnesium ion) role in the whole process is quite specific.

The team made a fascinating discovery: the Transformer model seems to use a gatekeeping process similar to the brain’s NMDA receptor. This revelation led the researchers to investigate if the Transformer’s memory consolidation can be controlled by a mechanism similar to the NMDA receptor’s gating process.

In the animal brain, a low magnesium level is known to weaken memory function. The researchers found that long-term memory in Transformer can be improved by mimicking the NMDA receptor. Just like in the brain, where changing magnesium levels affect memory strength, tweaking the Transformer’s parameters to reflect the gating action of the NMDA receptor led to enhanced memory in the AI model. This breakthrough finding suggests that how AI models learn can be explained with established knowledge in neuroscience.

C. Justin LEE, who is a neuroscientist director at the institute, said, “This research makes a crucial step in advancing AI and neuroscience. It allows us to delve deeper into the brain’s operating principles and develop more advanced AI systems based on these insights.”

CHA Meeyoung, who is a data scientist in the team and at KAIST, notes, “The human brain is remarkable in how it operates with minimal energy, unlike the large AI models that need immense resources. Our work opens up new possibilities for low-cost, high-performance AI systems that learn and remember information like humans.”

What sets this study apart is its initiative to incorporate brain-inspired nonlinearity into an AI construct, signifying a significant advancement in simulating human-like memory consolidation. The convergence of human cognitive mechanisms and AI design not only holds promise for creating low-cost, high-performance AI systems but also provides valuable insights into the workings of the brain through AI models.

The idea of a runaway of the greenhouse effect is not new. In this scenario, a planet can evolve from a temperate state like on Earth to a true hell, with surface temperatures above 1000°C. The cause? Water vapor, a natural greenhouse gas. Water vapor prevents the solar irradiation absorbed by Earth to be reemitted towards the void of space, as thermal radiation. It traps heat a bit like a rescue blanket. A dash of greenhouse effect is useful — without it, Earth would have an average temperature below the freezing point of water, looking like a ball covered with ice and hostile to life.

On the opposite, too much greenhouse effect increases the evaporation of oceans, and thus the amount of water vapor in the atmosphere. “There is a critical threshold for this amount of water vapor, beyond which the planet cannot cool down anymore. From there, everything gets carried away until the oceans end up getting fully evaporated and the temperature reaches several hundred degrees,” explains Guillaume Chaverot, former postdoctoral scholar in the Department of Astronomy at the UNIGE Faculty of Science and main author of the study.

World premiere

“Until now, other key studies in climatology have focused solely on either the temperate state before the runaway, or either the inhabitable state post-runaway,” reveals Martin Turbet, researcher at CNRS laboratories of Paris and Bordeaux, and co-author of the study. “It is the first time a team has studied the transition itself with a 3D global climate model, and has checked how the climate and the atmosphere evolve during that process.”

One of the key points of the study describes the appearance of a very peculiar cloud pattern, increasing the runaway effect, and making the process irreversible. “From the start of the transition, we can observe some very dense clouds developing in the high atmosphere. Actually, the latter does not display anymore the temperature inversion characteristic of the Earth atmosphere and separating its two main layers: the troposphere and the stratosphere. The structure of the atmosphere is deeply altered,” points out Guillaume Chaverot.

Serious consequences for the search of life elsewhere

This discovery is a key feature for the study of climate on other planets, and in particular on exoplanets — planets orbiting other stars than the Sun. “By studying the climate on other planets, one of our strongest motivations is to determine their potential to host life,” indicates Émeline Bolmont, assistant professor and director of the UNIGE Life in the Universe Center (LUC), and co-author of the study.

The LUC leads state-of-the-art interdisciplinary research projects regarding the origins of life on Earth, and the quest for life elsewhere in our solar system and beyond, in exoplanetary systems. “After the previous studies, we suspected already the existence of a water vapor threshold, but the appearance of this cloud pattern is a real surprise!” discloses Émeline Bolmont. “We have also studied in parallel how this cloud pattern could create a specific signature, or ”fingerprint”, detectable when observing exoplanet atmospheres. The upcoming generation of instruments should be able to detect it,” unveils Martin Turbet. The team is also not aiming to stop there, Guillaume Chaverot having received a research grant to continue this study at the “Institut de Planétologie et d’Astrophysique de Grenoble” (IPAG). This new step of the research project will focus on the specific case of the Earth.

A planet Earth in a fragile equilibrium

With their new climate models, the scientists have calculated that a very small increase of the solar irradiation — leading to an increase of the global Earth temperature, of only a few tens of degrees — would be enough to trigger this irreversible runaway process on Earth and make our planet as inhospitable as Venus. One of the current climate goals is to limit global warming on Earth, induced by greenhouse gases, to only 1.5 degrees by 2050. One of the questions of Guillaume Chaverot’s research grant is to determine if greenhouse gases can trigger the runaway process as a slight increase of the Sun luminosity might do. If so, the next question will be to determine if the treshold temperatures are the same for both processes.

The Earth is thus not so far from this apocalyptical scenario. “Assuming this runaway process would be started on Earth, an evaporation of only 10 meters of the oceans’ surface would lead to a 1 bar increase of the atmospheric pressure at ground level. In just a few hundred years, we would reach a ground temperature of over 500°C. Later, we would even reach 273 bars of surface pressure and over 1 500°C, when all of the oceans would end up totally evaporated,” concludes Guillaume Chaverot.

The fifteen topics identified by the scan include:

• New sources of hydrogen for energy production, including seawater electrolysis
• Decarbonized ammonia production for fertilizer and fuel using water microdroplets, graphite mesh, and nitrogen
• Making food and animal feed with fish-like nutrients from autotrophic hydrogen-oxidizing bacteria
• Indoor agriculture in urban environments using light-free artificial photosynthesis
• Excessive usage of rock dusts to remineralize large amounts of carbon and put ecosystems at risk for heavy metal contamination
• Earthworm populations to continue declining, especially in farmland and broadleaved woodland area ecosystems, potentially due to pesticides
• Soil ecoacoustics, which project sounds like water moving through soil, poised to help monitor soil ecology and invertebrate populations
• Wildfires to change climate patterns like El Niño by releasing trapped aerosols
• Benchtop DNA printers, which can produce double-stranded DNA sequences on demand, to become increasingly advanced, widespread, and in need of regulation
• New methods to measure a chemical’s toxicity before adverse impacts surface
• Saudia Arabia’s planned “sustainable” ultra-tall skyscraper city to impact migratory birds
• Sea urchins to continue dying off rapidly worldwide, potentially due to a protist pathogen, which could threaten tropical ecosystems
• Testing methods to remove carbon dioxide from the ocean including using ocean fertilization, macroalgae, and rock injections
• Rising temperatures in the twilight zone to disturb the ocean’s biological carbon pump
• Melting Antarctic ice to alter deep sea ocean currents, reducing abyssal overturning by 40% by 2050

“The issues identified in this scan continue to reflect the juxtaposition between anthropogenic impacts on biodiversity and increasing technological capacity to mitigate those impacts,” write the researchers, led by conservationist William Sutherland of Cambridge University. “In some cases, new issues arise directly from efforts to mitigate other issues.”

The authors identify several common threads in the list of issues, including threats to marine ecosystems. They underscore the importance of recent marine policy initiatives to help address these threats while noting that these efforts will not help resolve the climate-related issues identified. They also discuss the continued importance of innovation in carbon capture and sustainable energy technologies.

“We anticipate continuing to highlight novel emerging impacts of climate change and technologies aiming to mitigate climate change and transition to more sustainable pathways in future horizon scans,” write the researchers.

This research was supported by funding from The Pew Charitable Trusts, the Natural Environment Research Council, and the Royal Society for the Protection of Birds.