For an AI agent, this is easier said than done. Current approaches often utilize multiple hand-crafted machine-learning models to tackle different parts of the task, which require a great deal of human effort and expertise to build. These methods, which use visual representations to directly make navigation decisions, demand massive amounts of visual data for training, which are often hard to come by.

To overcome these challenges, researchers from MIT and the MIT-IBM Watson AI Lab devised a navigation method that converts visual representations into pieces of language, which are then fed into one large language model that achieves all parts of the multistep navigation task.

Rather than encoding visual features from images of a robot’s surroundings as visual representations, which is computationally intensive, their method creates text captions that describe the robot’s point-of-view. A large language model uses the captions to predict the actions a robot should take to fulfill a user’s language-based instructions.

Because their method utilizes purely language-based representations, they can use a large language model to efficiently generate a huge amount of synthetic training data.

While this approach does not outperform techniques that use visual features, it performs well in situations that lack enough visual data for training. The researchers found that combining their language-based inputs with visual signals leads to better navigation performance.

“By purely using language as the perceptual representation, ours is a more straightforward approach. Since all the inputs can be encoded as language, we can generate a human-understandable trajectory,” says Bowen Pan, an electrical engineering and computer science (EECS) graduate student and lead author of a paper on this approach.

Pan’s co-authors include his advisor, Aude Oliva, director of strategic industry engagement at the MIT Schwarzman College of Computing, MIT director of the MIT-IBM Watson AI Lab, and a senior research scientist in the Computer Science and Artificial Intelligence Laboratory (CSAIL); Philip Isola, an associate professor of EECS and a member of CSAIL; senior author Yoon Kim, an assistant professor of EECS and a member of CSAIL; and others at the MIT-IBM Watson AI Lab and Dartmouth College. The research will be presented at the Conference of the North American Chapter of the Association for Computational Linguistics.

Solving a vision problem with language

Since large language models are the most powerful machine-learning models available, the researchers sought to incorporate them into the complex task known as vision-and-language navigation, Pan says.

But such models take text-based inputs and can’t process visual data from a robot’s camera. So, the team needed to find a way to use language instead.

Their technique utilizes a simple captioning model to obtain text descriptions of a robot’s visual observations. These captions are combined with language-based instructions and fed into a large language model, which decides what navigation step the robot should take next.

The large language model outputs a caption of the scene the robot should see after completing that step. This is used to update the trajectory history so the robot can keep track of where it has been.

The model repeats these processes to generate a trajectory that guides the robot to its goal, one step at a time.

To streamline the process, the researchers designed templates so observation information is presented to the model in a standard form — as a series of choices the robot can make based on its surroundings.

For instance, a caption might say “to your 30-degree left is a door with a potted plant beside it, to your back is a small office with a desk and a computer,” etc. The model chooses whether the robot should move toward the door or the office.

“One of the biggest challenges was figuring out how to encode this kind of information into language in a proper way to make the agent understand what the task is and how they should respond,” Pan says.

Advantages of language

When they tested this approach, while it could not outperform vision-based techniques, they found that it offered several advantages.

First, because text requires fewer computational resources to synthesize than complex image data, their method can be used to rapidly generate synthetic training data. In one test, they generated 10,000 synthetic trajectories based on 10 real-world, visual trajectories.

The technique can also bridge the gap that can prevent an agent trained with a simulated environment from performing well in the real world. This gap often occurs because computer-generated images can appear quite different from real-world scenes due to elements like lighting or color. But language that describes a synthetic versus a real image would be much harder to tell apart, Pan says.

Also, the representations their model uses are easier for a human to understand because they are written in natural language.

“If the agent fails to reach its goal, we can more easily determine where it failed and why it failed. Maybe the history information is not clear enough or the observation ignores some important details,” Pan says.

In addition, their method could be applied more easily to varied tasks and environments because it uses only one type of input. As long as data can be encoded as language, they can use the same model without making any modifications.

But one disadvantage is that their method naturally loses some information that would be captured by vision-based models, such as depth information.

However, the researchers were surprised to see that combining language-based representations with vision-based methods improves an agent’s ability to navigate.

“Maybe this means that language can capture some higher-level information than cannot be captured with pure vision features,” he says.

This is one area the researchers want to continue exploring. They also want to develop a navigation-oriented captioner that could boost the method’s performance. In addition, they want to probe the ability of large language models to exhibit spatial awareness and see how this could aid language-based navigation.

This research is funded, in part, by the MIT-IBM Watson AI Lab.

However, a new study has shown it is not only humans that are using forms of the chemical as a way to help them survive challenging conditions.

In tests conducted on beaches in Cornwall, and in the laboratory at the University of Plymouth, scientists confirmed the findings of previous studies which showed large sandhoppers (Talitrus saltator) increase the levels of magnesium ions in their bodies as temperatures fall. This slows them down so they are less active than they would be during the warmer months.

However, the new study has shown for the first time that when they want to enter a period of deep sleep the creatures have the means through which to increase their magnesium levels even further — in some instances more than doubling them.

Essentially acting as a natural narcotic, the magnesium puts the sandhopper into a torpid state. This enforced rest means that the creatures stay hidden in burrows up to 30cm beneath the beach surface, without the need to come up for food or water, and to some extent buffered from the wintery conditions at the surface of the sand.

The study — which focused on the sandhopper population at Portwrinkle, in South East Cornwall — was carried out by Professor of Marine Zoology John Spicer and BSc (Hons) Marine Biology graduate Jack Bush.

Writing in the Journal of Experimental Marine Biology and Ecology, they say it sheds further light on why large sandhoppers seem to disappear from sandy beaches during cold winter weather.

Professor Spicer has spent decades studying the impact of temperature on marine and coastal species, including a number of studies on sandhoppers in Scotland.

He said: “It has been known for over a century that large sandhoppers, relatives of shrimp and crabs, can overwinter buried deep in the sand at the top of beaches away from the reach of the tide. What our study shows is that they may help themselves ‘go to sleep’ by allowing a natural narcotising agent, magnesium ions, to build up in their body fluids.

“Fluids containing magnesium, like Epsom salts, are routinely used by humans to relax but also when aquatic animals are being examined as part of scientific investigations. Our study shows nature has also found a way to do that without outside involvement.

“That said, as it’s a temperature dependent process it does raise questions over what will happen as our world warms. Will sandhoppers no longer sleep and just eat decomposing wrack all day long? Or will they change their sleeping habits by adapting the way they manage the magnesium in their body fluids?”

A paper just published in Scientific Reports- Nature by Sergio Dempsey as first author with colleagues Dr Soroush Safaei, Dr Gonzalo Maso Talou at Auckland Bioengineering Institute, along with co-author Dr Holdsworth (Mātai and FMHS & CBR at the University of Auckland), describes the new metric based on 4D flow MRI technology.

This innovative metric is particularly crucial because increased vascular pulsatility is linked to several brain conditions, including Alzheimer’s disease and other forms of dementia. By accurately measuring how pulsatility is transmitted in the brain, researchers can better understand the underlying mechanism of these diseases and potentiall guide development f new treatments.

Current MRI methods face limitations due to anatomical variations and measurement constraints. The new technique removes this issue by integrating thousands of measurements across all brain vessels, rather than looking one spot at a time as the traditional methods. This provides a richer metric representative of the entire brain.

“The ability to measure how pulsatility is transmitted through the brain’s arteries could revolutionise our approach to neurological diseases, and support research in vascular damage hypotheses” explained Mr. Dempsey. “Our method allows for a detailed assessment of the brain’s vascular health, which is often compromised in neurodegenerative disorders.”

The study also highlighted the potential to enhance clinical assessments and research on brain health. By integrating this new metric into routine diagnostic procedures, healthcare providers can offer more precise and personalised care plans for individuals at risk of or suffering from cognitive impairments.

In addition to its implications for patient care, the researchers have made their tools publicly available, integrating them into pre-existing open-source software. This enables scientists and clinicians worldwide to adopt the advanced methodology, fostering further research and collaboration in the field of neurology.

The research team is planning further studies to explore the applications of this technique in larger and more diverse populations, beginning with the “Digital Twin Dementia Study” starting at Mātai later this month. Results from the initial study of the metric also identified important sex differences in vascular dynamics which has initiated a new study focussing on sex-related dynamics which is anticipated to begin at Mātai and the Centre for Advanced MRI (CAMRI) in November.

Microscopically, magnetism arises from a collection of tiny magnets associated with electrons, called spin. In ferromagnetic materials, all the electron spins point in the same direction, while in antiferromagnetic materials, the electron spins are aligned in opposite directions, half pointing one way and half the other, canceling out the net magnetism. Altermagnetic materials are proposed in theory to possess properties combining those of both antiferromagnetic and ferromagnetic materials. One potential application of altermagnetic materials is in spintronics technology, which aims to utilize the spin of electrons effectively in electronic devices such as next-generation magnetic memories. However, identifying altermagnets has been a challenge.

An international research group led by Associate Professor Atsushi Hariki from the Graduate School of Engineering at Osaka Metropolitan University pioneered a new method to identify altermagnets, using manganese telluride (α-MnTe) as a testbed.

With the aid of a supercomputer, the researchers theoretically predicted a fingerprint of altermagnetism in X-ray magnetic circular dichroism (XMCD), which measures the absorption difference between left- and right-circularly polarized light. Then, using the Diamond Light Source synchrotron in England, they experimentally demonstrated the XMCD spectrum for altermagnetic α-MnTe for the first time in the world.

“Our results show that XMCD is an effective method for the simple identification of altermagnetic materials,” Professor Hariki said. “Also, it can be expected to further accelerate the application of altermagnets in spintronics.”

The splitting of a single photon in two is one of the most useful tools in quantum photonics. It can create entangled photon pairs, single photons, squeezed light, and even more complicated states of light which are essential for optical quantum technologies. This process is known as spontaneous parametric down-conversion (SPDC).

SPDC is deeply linked to central symmetry. This is the symmetry with respect to a point — for instance, a square is centrally symmetric but a triangle is not. In its very essence — a splitting of one photon in two — SPDC breaks the central symmetry. Therefore, it is only possible in crystals whose elementary cell is centrally asymmetric. SPDC cannot happen in ordinary liquids or gases, because these materials are isotropic.

Recently, however, researchers have discovered liquid crystals that have a different structure, the so-called ferroelectric nematic liquid crystals. Despite being fluidic, these materials feature strong central symmetry breaking. Their molecules are elongated, asymmetric and, most importantly, they can be re-oriented by external electric field. Re-orientation of molecules changes the polarization of the generated photon pairs, as well as the generation rate. Given a proper packaging, a sample of such material can be a very useful device because it produces photon pairs efficiently, can be easily tuned with electric field, and can be integrated into more complex devices.

Using the samples prepared in Jozef Stefan Institute (Ljubljana, Slovenia) from a ferroelectric nematic liquid crystal synthesized by Merck Electronics KGaA, researchers at the Max-Planck Institute for the Science of Light have implemented SPDC, for the first time, in a liquid crystal. The efficiency of entangled photons generation is as high as in the best nonlinear crystals, such as lithium niobate, of similar thickness. By applying an electric field of just a few Volts, they were able to switch the generation of photon pairs on and off, as well as to change the polarization properties of these pairs. This discovery starts a new generation of quantum light sources: flexible, tunable, and efficient.

Antimicrobial resistance of pathogens is a worldwide problem, and recognized by the WHO as one of the top global public health and development threats. There are two ways to fight this: by developing new drugs, or by preventing the development of resistance. ‘We know that temperature affects the mutation rate in bacteria’, explains Timo van Eldijk, co-first author of the paper. ‘What we wanted to find out was how the increase in temperature associated with fever influences the mutation rate towards antibiotic resistance.’

Three antibiotics

‘Most studies on resistance mutations were done by lowering the ambient temperature, and none, as far as we know, used a moderate increase above normal body temperature,’ Van Eldijk reports. Together with Master’s student Eleanor Sheridan, Van Eldijk cultured E. coli bacteria at 37 or 40 degrees Celsius, and subsequently exposed them to three different antibiotics to assess the effect. ‘Again, some previous human trials have looked at temperature and antibiotics, but in these studies the type of drug was not controlled.’ In their laboratory study, the team used three different antibiotics with different modes of action: ciprofloxacin, rifampicin, and ampicillin.

The results showed that for two of the drugs, ciprofloxacin and rifampicin, increased temperature led to an increase in the mutation rate towards resistance. However, the third drug, ampicillin, caused a decrease in the mutation rate towards resistance at fever temperatures. ‘To be certain of this result, we actually replicated the study with ampicillin in two different labs, at the University of Groningen and the University of Montpellier, and got the same result,’ says Van Eldijk.

Fever-suppressing drugs

The researchers hypothesized that a temperature dependence of the efficacy of ampicillin could explain this result, and confirmed this in an experiment. This explains why ampicillin resistance is less likely to arise at 40 degrees Celsius. ‘Our study shows that a very mild change in temperature can drastically change the mutation rate towards resistance to antimicrobials,’ concludes Van Eldijk. ‘This is interesting, as other parameters such as the growth rate do not seem to change.’

If the results are replicated in humans, this could open the way to tackling antimicrobial resistance by lowering the temperature with fever-suppressing drugs, or by giving patients with a fever antimicrobial drugs with higher efficacy at higher temperatures. The team concludes in the paper: ‘An optimized combination of antibiotics and fever suppression strategies may be a new weapon in the battle against antibiotic resistance.’