Humans are generally good at whole-body manipulation, but robots struggle with such tasks. To the robot, each spot where the box could touch any point on the carrier’s fingers, arms, and torso represents a contact event that it must reason about. With billions of potential contact events, planning for this task quickly becomes intractable.

Now MIT researchers found a way to simplify this process, known as contact-rich manipulation planning. They use an AI technique called smoothing, which summarizes many contact events into a smaller number of decisions, to enable even a simple algorithm to quickly identify an effective manipulation plan for the robot.

While still in its early days, this method could potentially enable factories to use smaller, mobile robots that can manipulate objects with their entire arms or bodies, rather than large robotic arms that can only grasp using fingertips. This may help reduce energy consumption and drive down costs. In addition, this technique could be useful in robots sent on exploration missions to Mars or other solar system bodies, since they could adapt to the environment quickly using only an onboard computer.

“Rather than thinking about this as a black-box system, if we can leverage the structure of these kinds of robotic systems using models, there is an opportunity to accelerate the whole procedure of trying to make these decisions and come up with contact-rich plans,” says H.J. Terry Suh, an electrical engineering and computer science (EECS) graduate student and co-lead author of a paper on this technique.

Joining Suh on the paper are co-lead author Tao Pang PhD ’23, a roboticist at Boston Dynamics AI Institute; Lujie Yang, an EECS graduate student; and senior author Russ Tedrake, the Toyota Professor of EECS, Aeronautics and Astronautics, and Mechanical Engineering, and a member of the Computer Science and Artificial Intelligence Laboratory (CSAIL). The research appears this week in IEEE Transactions on Robotics.

Learning about learning

Reinforcement learning is a machine-learning technique where an agent, like a robot, learns to complete a task through trial and error with a reward for getting closer to a goal. Researchers say this type of learning takes a black-box approach because the system must learn everything about the world through trial and error.

It has been used effectively for contact-rich manipulation planning, where the robot seeks to learn the best way to move an object in a specified manner.

But because there may be billions of potential contact points that a robot must reason about when determining how to use its fingers, hands, arms, and body to interact with an object, this trial-and-error approach requires a great deal of computation.

“Reinforcement learning may need to go through millions of years in simulation time to actually be able to learn a policy,” Suh adds.

On the other hand, if researchers specifically design a physics-based model using their knowledge of the system and the task they want the robot to accomplish, that model incorporates structure about this world that makes it more efficient.

Yet physics-based approaches aren’t as effective as reinforcement learning when it comes to contact-rich manipulation planning — Suh and Pang wondered why.

They conducted a detailed analysis and found that a technique known as smoothing enables reinforcement learning to perform so well.

Many of the decisions a robot could make when determining how to manipulate an object aren’t important in the grand scheme of things. For instance, each infinitesimal adjustment of one finger, whether or not it results in contact with the object, doesn’t matter very much. Smoothing averages away many of those unimportant, intermediate decisions, leaving a few important ones.

Reinforcement learning performs smoothing implicitly by trying many contact points and then computing a weighted average of the results. Drawing on this insight, the MIT researchers designed a simple model that performs a similar type of smoothing, enabling it to focus on core robot-object interactions and predict long-term behavior. They showed that this approach could be just as effective as reinforcement learning at generating complex plans.

“If you know a bit more about your problem, you can design more efficient algorithms,” Pang says.

A winning combination

Even though smoothing greatly simplifies the decisions, searching through the remaining decisions can still be a difficult problem. So, the researchers combined their model with an algorithm that can rapidly and efficiently search through all possible decisions the robot could make.

With this combination, the computation time was cut down to about a minute on a standard laptop.

They first tested their approach in simulations where robotic hands were given tasks like moving a pen to a desired configuration, opening a door, or picking up a plate. In each instance, their model-based approach achieved the same performance as reinforcement learning, but in a fraction of the time. They saw similar results when they tested their model in hardware on real robotic arms.

“The same ideas that enable whole-body manipulation also work for planning with dexterous, human-like hands. Previously, most researchers said that reinforcement learning was the only approach that scaled to dexterous hands, but Terry and Tao showed that by taking this key idea of (randomized) smoothing from reinforcement learning, they can make more traditional planning methods work extremely well, too,” Tedrake says.

However, the model they developed relies on a simpler approximation of the real world, so it cannot handle very dynamic motions, such as objects falling. While effective for slower manipulation tasks, their approach cannot create a plan that would enable a robot to toss a can into a trash bin, for instance. In the future, the researchers plan to enhance their technique so it could tackle these highly dynamic motions.

“If you study your models carefully and really understand the problem you are trying to solve, there are definitely some gains you can achieve. There are benefits to doing things that are beyond the black box,” Suh says.

This work is funded, in part, by Amazon, MIT Lincoln Laboratory, the National Science Foundation, and the Ocado Group.

Large spots are common features in the atmospheres of giant planets, the most famous being Jupiter’s Great Red Spot. On Neptune, a dark spot was first discovered by NASA’s Voyager 2 in 1989, before disappearing a few years later. “Since the first discovery of a dark spot, I’ve always wondered what these short-lived and elusive dark features are,” says Patrick Irwin, Professor at the University of Oxford in the UK and lead investigator of the study published today in Nature Astronomy.

Irwin and his team used data from ESO’s VLT to rule out the possibility that dark spots are caused by a ‘clearing’ in the clouds. The new observations indicate instead that dark spots are likely the result of air particles darkening in a layer below the main visible haze layer, as ices and hazes mix in Neptune’s atmosphere.

Coming to this conclusion was no easy feat because dark spots are not permanent features of Neptune’s atmosphere and astronomers had never before been able to study them in sufficient detail. The opportunity came after the NASA/ESA Hubble Space Telescope discovered several dark spots in Neptune’s atmosphere, including one in the planet’s northern hemisphere first noticed in 2018. Irwin and his team immediately got to work studying it from the ground — with an instrument that is ideally suited to these challenging observations.

Using the VLT’s Multi Unit Spectroscopic Explorer (MUSE), the researchers were able to split reflected sunlight from Neptune and its spot into its component colours, or wavelengths, and obtain a 3D spectrum [1]. This meant they could study the spot in more detail than was possible before. “I’m absolutely thrilled to have been able to not only make the first detection of a dark spot from the ground, but also record for the very first time a reflection spectrum of such a feature,” says Irwin.

Since different wavelengths probe different depths in Neptune’s atmosphere, having a spectrum enabled astronomers to better determine the height at which the dark spot sits in the planet’s atmosphere. The spectrum also provided information on the chemical composition of the different layers of the atmosphere, which gave the team clues as to why the spot appeared dark.

The observations also offered up a surprise result. “In the process we discovered a rare deep bright cloud type that had never been identified before, even from space,” says study co-author Michael Wong, a researcher at the University of California, Berkeley, USA. This rare cloud type appeared as a bright spot right beside the larger main dark spot, the VLT data showing that the new ‘deep bright cloud’ was at the same level in the atmosphere as the main dark spot. This means it is a completely new type of feature compared to the small ‘companion’ clouds of high-altitude methane ice that have been previously observed.

With the help of ESO’s VLT, it is now possible for astronomers to study features like these spots from Earth. “This is an astounding increase in humanity’s ability to observe the cosmos. At first, we could only detect these spots by sending a spacecraft there, like Voyager. Then we gained the ability to make them out remotely with Hubble. Finally, technology has advanced to enable this from the ground,” concludes Wong, before adding, jokingly: “This could put me out of work as a Hubble observer!“

Note

[1] MUSE is a 3D spectrograph that allows astronomers to observe the entirety of an astronomical object, like Neptune, in one go. At each pixel, the instrument measures the intensity of light as a function of its colour or wavelength. The resulting data form a 3D set in which each pixel of the image has a full spectrum of light. In total, MUSE measures over 3500 colours. The instrument is designed to take advantage of adaptive optics, which corrects for the turbulence in the Earth’s atmosphere, resulting in sharper images than otherwise possible. Without this combination of features, studying a Neptune dark spot from the ground would not have been possible.