A simpler method for learning to control a robot

Researchers from MIT and Stanford University have devised a new machine-learning approach that could be used to control a robot, such as a drone or autonomous vehicle, more effectively and efficiently in dynamic environments where conditions can change rapidly.

This technique could help an autonomous vehicle learn to compensate for slippery road conditions to avoid going into a skid, allow a robotic free-flyer to tow different objects in space, or enable a drone to closely follow a downhill skier despite being buffeted by strong winds.

The researchers’ approach incorporates certain structure from control theory into the process for learning a model in such a way that leads to an effective method of controlling complex dynamics, such as those caused by impacts of wind on the trajectory of a flying vehicle. One way to think about this structure is as a hint that can help guide how to control a system.

“The focus of our work is to learn intrinsic structure in the dynamics of the system that can be leveraged to design more effective, stabilizing controllers,” says Navid Azizan, the Esther and Harold E. Edgerton Assistant Professor in the MIT Department of Mechanical Engineering and the Institute for Data, Systems, and Society (IDSS), and a member of the Laboratory for Information and Decision Systems (LIDS). “By jointly learning the system’s dynamics and these unique control-oriented structures from data, we’re able to naturally create controllers that function much more effectively in the real world.”

Using this structure in a learned model, the researchers’ technique immediately extracts an effective controller from the model, as opposed to other machine-learning methods that require a controller to be derived or learned separately with additional steps. With this structure, their approach is also able to learn an effective controller using fewer data than other approaches. This could help their learning-based control system achieve better performance faster in rapidly changing environments.

“This work tries to strike a balance between identifying structure in your system and just learning a model from data,” says lead author Spencer M. Richards, a graduate student at Stanford University. “Our approach is inspired by how roboticists use physics to derive simpler models for robots. Physical analysis of these models often yields a useful structure for the purposes of control — one that you might miss if you just tried to naively fit a model to data. Instead, we try to identify similarly useful structure from data that indicates how to implement your control logic.”

Additional authors of the paper are Jean-Jacques Slotine, professor of mechanical engineering and of brain and cognitive sciences at MIT, and Marco Pavone, associate professor of aeronautics and astronautics at Stanford. The research will be presented at the International Conference on Machine Learning (ICML).

advertisement


Learning a controller

Determining the best way to control a robot to accomplish a given task can be a difficult problem, even when researchers know how to model everything about the system.

A controller is the logic that enables a drone to follow a desired trajectory, for example. This controller would tell the drone how to adjust its rotor forces to compensate for the effect of winds that can knock it off a stable path to reach its goal.

This drone is a dynamical system — a physical system that evolves over time. In this case, its position and velocity change as it flies through the environment. If such a system is simple enough, engineers can derive a controller by hand.

Modeling a system by hand intrinsically captures a certain structure based on the physics of the system. For instance, if a robot were modeled manually using differential equations, these would capture the relationship between velocity, acceleration, and force. Acceleration is the rate of change in velocity over time, which is determined by the mass of and forces applied to the robot.

But often the system is too complex to be exactly modeled by hand. Aerodynamic effects, like the way swirling wind pushes a flying vehicle, are notoriously difficult to derive manually, Richards explains. Researchers would instead take measurements of the drone’s position, velocity, and rotor speeds over time, and use machine learning to fit a model of this dynamical system to the data. But these approaches typically don’t learn a control-based structure. This structure is useful in determining how to best set the rotor speeds to direct the motion of the drone over time.

advertisement


Once they have modeled the dynamical system, many existing approaches also use data to learn a separate controller for the system.

“Other approaches that try to learn dynamics and a controller from data as separate entities are a bit detached philosophically from the way we normally do it for simpler systems. Our approach is more reminiscent of deriving models by hand from physics and linking that to control,” Richards says.

Identifying structure

The team from MIT and Stanford developed a technique that uses machine learning to learn the dynamics model, but in such a way that the model has some prescribed structure that is useful for controlling the system.

With this structure, they can extract a controller directly from the dynamics model, rather than using data to learn an entirely separate model for the controller.

“We found that beyond learning the dynamics, it’s also essential to learn the control-oriented structure that supports effective controller design. Our approach of learning state-dependent coefficient factorizations of the dynamics has outperformed the baselines in terms of data efficiency and tracking capability, proving to be successful in efficiently and effectively controlling the system’s trajectory,” Azizan says.

When they tested this approach, their controller closely followed desired trajectories, outpacing all the baseline methods. The controller extracted from their learned model nearly matched the performance of a ground-truth controller, which is built using the exact dynamics of the system.

“By making simpler assumptions, we got something that actually worked better than other complicated baseline approaches,” Richards adds.

The researchers also found that their method was data-efficient, which means it achieved high performance even with few data. For instance, it could effectively model a highly dynamic rotor-driven vehicle using only 100 data points. Methods that used multiple learned components saw their performance drop much faster with smaller datasets.

This efficiency could make their technique especially useful in situations where a drone or robot needs to learn quickly in rapidly changing conditions.

Plus, their approach is general and could be applied to many types of dynamical systems, from robotic arms to free-flying spacecraft operating in low-gravity environments.

In the future, the researchers are interested in developing models that are more physically interpretable, and that would be able to identify very specific information about a dynamical system, Richards says. This could lead to better-performing controllers.

This research is supported, in part, by the NASA University Leadership Initiative and the Natural Sciences and Engineering Research Council of Canada.

Share Button

Ancient DNA reveals diverse community in ‘Lost City of the Incas’

Who lived at Machu Picchu at its height? A new study, published today in Science Advances, used ancient DNA to find out for the first time where workers buried more than 500 years ago came from within the lost Inca Empire.

Researchers, including Jason Nesbitt, associate professor of archaeology at Tulane University School of Liberal Arts, performed genetic testing on individuals buried at Machu Picchu in order to learn more about the people who lived and worked there.

Machu Picchu is a UNESCO World Heritage Site located in the Cusco region of Peru. It is one of the most well-known archaeological sites in the world and attracts hundreds of thousands of visitors every year. It was once part of a royal estate of the Inca Empire.

Like other royal estates, Machu Picchu was home not only to royalty and other elite members of Inca society, but also to attendants and workers, many of whom lived in the estate year-round. These residents did not necessarily come from the local area, though it is only in this study that researchers have been able to confirm, with DNA evidence, the diversity of their backgrounds. “It’s telling us, not about elites and royalty, but lower status people,” Nesbitt said. “These were burials of the retainer population.”

This DNA analysis works in much the same way that modern genetic ancestry kits work. The researchers compared the DNA of 34 individuals buried at Machu Picchu to that of individuals from other places around the Inca Empire as well as some modern genomes from South America to see how closely related they might be.

The results of the DNA analysis showed that the individuals had come from throughout the Inca Empire, some as far away as Amazonia. Few of them had shared DNA with each other, showing that they had been brought to Machu Picchu as individuals rather than as part of a family or community group.

“Now, of course, genetics doesn’t translate into ethnicity or anything like that,” said Nesbitt of the results, “but that shows that they have distinct origins within different parts of the Inca Empire.”

“The study does really reinforce a lot of other types of research that have been done at Machu Picchu and other Inca sites,” Nesbitt said. The DNA analysis supports historical documentation and archaeological studies of the artifacts found associated with the burials.

This study is part of a larger movement in archaeology to combine traditional archaeological techniques with new technologies and scientific analyses. This combination of fields leads to a more complete understanding of the discoveries made.

Share Button

Blood-inquiry families heckle PM over compensation

Rishi Sunak said the government was working at pace but would wait for the final inquiry report.

Share Button

Unraveling a protein that may inspire a new biotechnology tool

Scientists have unraveled the step-by-step activation process of a protein with a deep evolutionary history in all domains of life, opening the door to harnessing its functions for use as a biotechnology tool.

The protein belongs to the “superfamily” of Argonaute proteins, which previous research has suggested to be involved in gene silencing, a fundamental process known as RNA interference.

These proteins are well-characterized in eukaryotes — the plants, fungi, animals, humans and other life forms with cells that have a defined nucleus. In prokaryotes that have no nucleus, there are two types of Argonaute proteins, long Argonautes and short Argonautes. The long Argonautes resemble their relatives in eukaryotes both structurally and functionally. In contrast, short Argonautes adopt different structures and perform different functions from other well-studied Argonautes.

This is the first study to detail structures and mechanisms of a short Argonaute, potentially sketching the beginnings of a blueprint for application to future therapeutic purposes.

“The short version of these prokaryotic proteins constitute 58% of all Argonautes, and are now emerging as a hot spot in the field,” said senior author Tianmin Fu, assistant professor of biological chemistry and pharmacology in The Ohio State University College of Medicine. “Among the capabilities we’ve identified is this protein’s precise role in the way bacteria trigger their own death to avoid losing power over their lifecycle through plasmid invasion. Understanding these types of mechanisms is the first step toward efforts to adapt highly effective natural functions for diagnostics and therapies.”

The study is published today (July 26, 2023) in Nature.

In this work, the research team focused on a protein called SPARTA, a short prokaryotic Argonaute (also referred to as Ago), specifically building upon other studies that showed this protein enables Maribacter polysiphoniae bacteria to program their death when they detect a plasmid invasion — when external DNA segments are trying to insert themselves to change bacterial properties.

advertisement


Ago proteins in eukaryotes are known to remain as simple molecules throughout activation, with the ability to bind only to other simple molecules. They also are established as participants in RNA interference, an evolutionary strategy to inhibit the expression of specific genes that may represent a threat to cell survival.

SPARTA, on the other hand, lacks certain structures that are needed to facilitate RNA interference. And though it starts out as a simple molecule like long prokaryotic and eukaryotic Agos, the activation similarities end there.

Using cryogenic electron microscopy, researchers identified SPARTA’s next steps: After it binds to RNA or DNA, it goes through numerous changes, eventually assembling into a larger multi-unit molecular complex.

Functional analysis of the complex revealed that the protein’s structural changes had to reach this point before it could produce the chemical reaction that allows threatened bacteria to program their own cell death — an enticing function scientists would like to manipulate to protect human health.

The researchers also introduced mutations to confirm that each step of the process was essential to maintaining SPARTA’s functionality.

All of this points to the fact that oligomerization — the methodical conversion of simple molecules into molecular complexes — is an essential part of activating short prokaryotic Argonaute proteins. While oligomerization of proteins is not rare, understanding its role in a protein’s activation is key to understanding how a protein interacts with other proteins and to determining its functional purpose.

advertisement


“When we talk about one protein that is expressed everywhere, in all organisms, we know this protein is inherently important, even if we don’t yet know all of its specific functions,” said first author Zhangfei Shen, a postdoctoral scholar in Fu’s lab. “Now that we know not just that it is oligomerized, but how it is oligomerized, and captured the intermediate states it is in during oligomerization, we’ve made good progress toward developing this protein as a tool.”

The possibilities envisioned by Fu’s lab include engineering short prokaryotic Agos that could help cells detect threats, or that could trigger molecules that threaten healthy cells to bring on their own death.

This work was supported by the National Institute of General Medical Sciences.

Additional co-authors include Xiao-Yuan Yang and Kotaro Nakanishi of Ohio State, Shiyu Xia of the California Institute of Technology, and Wei Huang and Derek Taylor of Case Western Reserve University.

Share Button

Earlier and earlier high-Arctic spring replaced by ‘extreme year-to-year variation’

About 15 years ago, researchers reported that the timing of spring in high-Arctic Greenland had advanced at some of the fastest rates of change ever seen anywhere in the world. But, according to new evidence reported in the journal Current Biology on July 26, that earlier pattern has since been completely erased. Instead of coming earlier and earlier, it seems the timing of Arctic spring is now driven by tremendous climate variability with drastic differences from one year to the next.

“As scientists we are obliged to revisit previous work to see whether the knowledge obtained at that time still holds,” says Niels Martin Schmidt of Aarhus University in Denmark. “We looked at previously reported extreme rates of phenological advancements in the Arctic and found that directional advancement is no longer the prevailing pattern. Actually, the previously observed trend has disappeared completely and has been replaced by extreme year-to-year variation in the onset of spring.”

Global changes in climate are expected to take place faster in the Arctic than in places at lower latitudes. To follow those trends, researchers at Zackenberg in Northeast Greenland launched an ecosystem-wide monitoring program in 1996. Among a suite of ecosystem variables, the program also tracks the timing of spring based on flowering plants, arthropod emergence, and bird nesting.

When the first 10 years of data were analyzed for 1996-2005, the findings showed a clear pattern of advancement across plants and animals included in the study. For instance, they saw some arthropods emerging up to 4 weeks earlier. In the new study, Schmidt and his colleagues wanted to see how these trends look now that they have 15 additional years of data available.

After analyzing the phenological data from 1996-2020, they report little evidence of directional change in the timing of events even as climate change continues. The researchers attribute this shift to a high degree of climate variability from year to year.

“That the extreme rates of phenological advancement we reported back in 2007 would not have continued unabated was not surprising to us,” Schmidt said. “However, that we see such a consistent shift from directional to extreme variability across so many different organisms and that the entire ecosystem now seems driven by variation in climatic conditions, was surprising.”

Schmidt says that the previous pattern showed steadily rising temperatures and declining snow cover. Now, what they see is a lot messier. Temperature increases have stalled while snow cover fluctuates dramatically from year to year.

advertisement


“Some years have almost no snow in spring, whereas others have snow on the ground way into the summer season,” he says. “This leaves us with a generally warmer but much more unpredictable spring climate — and this is where the second contributor to the observed phenological shift kicks in. Some species appear unable to take advantage of the warmer conditions in spring and appear to have reached the limits of their phenological plasticity.”

Plants and animals have some flexibility that allows them to track the climatic conditions in their environment, he explains. Arctic species in particular appear to have a high degree of phenological plasticity. Even so, the new evidence suggests that some species are already being pushed about as far as they can go. For instance, they don’t flower as early in warm summers as one might expect. As the Arctic continues to warm, the researchers predict that a growing number of species will become “increasingly out of sync with the climatic conditions.”

The new findings highlight the unfortunate reality that the lack in directional change does not mean that the climate is stable. In this case, quite the opposite is true. The climate pattern shows wide variation that may be pushing organisms and whole ecosystems to their limits. The researchers will continue to explore species-specific responses to the shifting climate pattern and its effects on essential interactions, such as pollination. They hope to learn how the responses of an individual species will cascade through the community. The findings are a reminder of the importance of long-term study.

“These insights can only be obtained because of sustained, ecosystem-wide, long-term monitoring with rigorous field sampling across more than 25 years at a very remote corner of the world,” Schmidt says. “Continued long-term monitoring is key to understand ecosystems and to detect changes in dynamics.”

This work was supported by the Danish Environmental Protection Agency, the Kvantum Institute at the University of Oulu, Academy of Finland, the Jane and Aatos Erkko Foundation, and the European Research Council.

Share Button

Family trees from the European Neolithic

The Neolithic burial site of Gurgy ‘les Noisats’ in France revealed two unprecedentedly large family trees which allowed a Franco-German team to explore the social organization of the 6,700-year-old community. Based on multiple lines of evidence, the team describes a close kin group which practiced monogamy and female exogamy, and experienced generally stable times.

The Neolithic lifestyle, based on farming instead of hunting and gathering, emerged in the Near East around 12,000 years ago and contributed profoundly to the modern way of life. The ability to produce and store extra food led Neolithic people to develop new social customs built on wealth, and therefore form social hierarchies. After an early phase of diffusion and having reached regions in western Europe, settled societies became more complex, which is sometimes reflected in the funerary world as well. The Paris Basin region in northern modern-day France is known for its monumental funerary sites, understood as being built for the society’s “elite.” In this context, the site of Gurgy ‘Les Noisats’, one of the biggest Neolithic funerary sites without monument in the region, begs the question who these people buried with different practices were.

Using new methods for obtaining and analysing ancient DNA data, and by sampling nearly every individual from the flat cemetery, researchers from the PACEA laboratory in Bordeaux, France, and from the Max Planck Institute for Evolutionary Anthropology in Leipzig, Germany, reveal two massive family trees which open a window into the lives of the people of this prehistoric community.

Massive family trees

In their study, the scientists analysed genome-wide ancient DNA data from 94 individuals buried at Gurgy, combined with strontium isotope ratio values, mitochondrial DNA (maternal lineages) and Y-chromosome (paternal lineages) data, age-at-death, and genetic sex. Two family trees could be reconstructed, the first connecting 64 individuals over seven generations is the largest pedigree reconstructed from ancient DNA to date, while the second connects twelve individuals over five generations.

“Since the beginning of the excavation, we found evidence of a complete control of the funerary space and only very few overlapping burials, which felt like the site was managed by a group of closely related individuals, or at least by people who knew who was buried where,” says Stéphane Rottier from the University of Bordeaux, the archaeo-anthropologist who excavated the site between 2004 and 2007. Indeed, a positive correlation between spatial and genetic distances showed that the deceased were likely to be buried close to a relative.

Insights into the social structure of Gurgy

Exploring the pedigrees revealed a strong patrilineal pattern, where each generation is almost exclusively linked to the previous generation through the biological father, which connects the entire group of Gurgy through the paternal line. At the same time, combined evidence from mitochondrial lineages and strontium stable isotope revealing a non-local origin of most women suggested the practice of patrilocality, meaning that the sons stayed where they were born, and had children with females from outside of Gurgy. Settling in with the male partner’s home community is known as virilocality. By contrast, most of the lineage adult daughters are missing, in line with female exogamy, potentially indicating a reciprocal exchange system. Interestingly, these “new incoming” female individuals were only very distantly related to each other, meaning that they must have come from a network of nearby communities, instead of just one nearby group. This lends support to the existence of a relatively wide and potentially fluid exchange network comprising many (including smaller) groups.

advertisement


Looking at the family trees, Maïté Rivollat, first author of the study, is amazed: “We observe a large number of full siblings who have reached reproductive age. Combined with the expected equal number of females and significant number of deceased infants, this indicates large family sizes, a high fertility rate and generally stable conditions of health and nutrition, which is quite striking for such ancient times.” Another notably unique feature at Gurgy is a lack of half-siblings, suggesting neither polygamous nor serial monogamous reproductive partnerships (or the exclusion of offspring from these unions from the main cemetery), when compared to the so far only other example of union practices from Neolithic megaliths.

A founding ancestor

In the frame of this patrilocal system, one male individual from which everyone in the largest family tree was descended could be identified as the “founding father” of the cemetery. His burial is unique at the site, as his skeletal remains were buried as a secondary deposit inside the grave pit of a woman, for whom, unfortunately, no genomic data could be obtained. Therefore, his bones must have been brought from wherever he had originally died to be reburied at Gurgy. “He must have represented a person of great significance for the founders of the Gurgy site to be brought there after a primary burial somewhere else,” explains Marie-France Deguilloux from the University of Bordeaux, co-senior author of the study.

Although the main pedigree spans seven generations, the demographic profile suggests that a large family group spanning several generations arrived at the site. With almost no subadults buried at the site during the first few generations, and by contrast no adult burials in the last generations, only a short use of the site is expected. The group must have left a previous site, leaving behind any previously deceased children but still brought the lineage father. Only a few generations later the same happened: the adult of the last generations left Gurgy for another place, leaving behind their own children. Hence, Gurgy was probably only used for three to four generations, or approximately one century.

These largest pedigrees reconstructed to date from ancient human DNA data, combined with multiple lines of evidence, represent an unprecedented step forward in our understanding of the social organization of past societies. “Only with the major advances in our field in very recent years and the full integration of context data it was possible to carry out such an extraordinary study. It is a dream come true for every anthropologist and archaeologist and opens up a new avenue for the study of the ancient human past,” concludes Wolfgang Haak of the Max Planck Institute for Evolutionary Anthropology, senior author of the study.

Share Button

Junior doctors to strike for four days in August

The walkout in England, beginning at 07:00 on 11 August, is the fifth round of strikes this year.

Share Button

Wall squats and planks best way to lower blood pressure

All exercise is good for blood pressure but research suggests strength training is most effective.

Share Button

Faulty concrete fears at 250 NHS Scotland sites

The buildings could contain a material which NHS Scotland warns is vulnerable to “catastrophic failure”.

Share Button

Gloomy climate calculation: Scientists predict a collapse of the Atlantic ocean current to happen mid-century

Important ocean currents that redistribute heat, cold and precipitation between the tropics and the northernmost parts of the Atlantic region will shut down around the year 2060 if current greenhouse gas emissions persist. This is the conclusion based on new calculations from the University of Copenhagen that contradict the latest report from the IPCC.

Contrary to what we may imagine about the impact of climate change in Europe, a colder future may be in store. In a new study, researchers from the University of Copenhagen’s Niels Bohr Institute and Department of Mathematical Sciences predict that the system of ocean currents which currently distributes cold and heat between the North Atlantic region and tropics will completely stop if we continue to emit the same levels of greenhouse gases as we do today.

Using advanced statistical tools and ocean temperature data from the last 150 years, the researchers calculated that the ocean current, known as the Thermohaline Circulation or the Atlantic Meridional Overturning Circulation (AMOC), will collapse — with 95 percent certainty — between 2025 and 2095. This will most likely occur in 34 years, in 2057, and could result in major challenges, particularly warming in the tropics and increased storminess in the North Atlantic region.

“Shutting down the AMOC can have very serious consequences for Earth’s climate, for example, by changing how heat and precipitation are distributed globally. While a cooling of Europe may seem less severe as the globe as a whole becomes warmer and heat waves occur more frequently, this shutdown will contribute to an increased warming of the tropics, where rising temperatures have already given rise to challenging living conditions,” says Professor Peter Ditlevsen from the Niels Bohr Institute.

“Our result underscores the importance of reducing global greenhouse gas emissions as soon as possible,” says the researcher.

The calculations, just published in the scientific journal, Nature Communications, contradict the message of the latest IPCC report, which, based on climate model simulations, considers an abrupt change in the thermohaline circulation very unlikely during this century.

Early warning signals present

The researchers’ prediction is based on observations of early warning signals that ocean currents exhibit as they become unstable. These Early Warning Signals for the Thermohaline Circulation have been reported previously, but only now has the development of advanced statistical methods made it possible to predict just when a collapse will occur.

advertisement


The researchers analysed sea surface temperatures in a specific area of the North Atlantic from 1870 to present days. These sea surface temperatures are “fingerprints” testifying the strength of the AMOC, which has only been measured directly for the past 15 years.

“Using new and improved statistical tools, we’ve made calculations that provide a more robust estimate of when a collapse of the Thermohaline Circulation is most likely to occur, something we had not been able to do before,” explains Professor Susanne Ditlevsen of UCPH’s Department of Mathematical Sciences.

The thermohaline circulation has operated in its present mode since the last ice age, where the circulation was indeed collapsed. Abrupt climate jumps between the present state of the AMOC and the collapsed state has been observed to happen 25 times in connection with iceage climate. These are the famed Dansgaard-Oeschger events first observed in ice cores from the Greenlandic ice sheet. At those events climate changes were extreme with 10-15 degrees changes over a decade, while present days climate change is 1.5 degrees warming over a century.

Facts:

  • The Atlantic Meridional Overturning Circulation (AMOC) is part of a global system of ocean currents. By far, it accounts for the most significant part of heat redistribution from the tropics to the northernmost regions of the Atlantic region — not least to Western Europe.
  • At the northernmost latitudes, circulation ensures that surface water is converted into deep, southbound ocean currents. The transformation creates space for additional surface water to be moved northward from equatorial regions. As such, thermohaline circulation is critical for maintaining the relatively mild climate of the North Atlantic region.
  • The work is supported by TiPES, a joint-European research collaboration focused on tipping points of the climate system. The TiPES project is an EU Horizon 2020 interdisciplinary climate research project focused on tipping points in the climate system.
Share Button