The system, called MCUNet, designs compact neural networks that deliver unprecedented speed and accuracy for deep learning on IoT devices, despite limited memory and processing power. The technology could facilitate the expansion of the IoT universe while saving energy and improving data security.

The research will be presented at next month’s Conference on Neural Information Processing Systems. The lead author is Ji Lin, a PhD student in Song Han’s lab in MIT’s Department of Electrical Engineering and Computer Science. Co-authors include Han and Yujun Lin of MIT, Wei-Ming Chen of MIT and National University Taiwan, and John Cohn and Chuang Gan of the MIT-IBM Watson AI Lab.

The Internet of Things

The IoT was born in the early 1980s. Grad students at Carnegie Mellon University, including Mike Kazar ’78, connected a Cola-Cola machine to the internet. The group’s motivation was simple: laziness. They wanted to use their computers to confirm the machine was stocked before trekking from their office to make a purchase. It was the world’s first internet-connected appliance. “This was pretty much treated as the punchline of a joke,” says Kazar, now a Microsoft engineer. “No one expected billions of devices on the internet.”

Since that Coke machine, everyday objects have become increasingly networked into the growing IoT. That includes everything from wearable heart monitors to smart fridges that tell you when you’re low on milk. IoT devices often run on microcontrollers — simple computer chips with no operating system, minimal processing power, and less than one thousandth of the memory of a typical smartphone. So pattern-recognition tasks like deep learning are difficult to run locally on IoT devices. For complex analysis, IoT-collected data is often sent to the cloud, making it vulnerable to hacking.

“How do we deploy neural nets directly on these tiny devices? It’s a new research area that’s getting very hot,” says Han. “Companies like Google and ARM are all working in this direction.” Han is too.

With MCUNet, Han’s group codesigned two components needed for “tiny deep learning” — the operation of neural networks on microcontrollers. One component is TinyEngine, an inference engine that directs resource management, akin to an operating system. TinyEngine is optimized to run a particular neural network structure, which is selected by MCUNet’s other component: TinyNAS, a neural architecture search algorithm.

System-algorithm codesign

Designing a deep network for microcontrollers isn’t easy. Existing neural architecture search techniques start with a big pool of possible network structures based on a predefined template, then they gradually find the one with high accuracy and low cost. While the method works, it’s not the most efficient. “It can work pretty well for GPUs or smartphones,” says Lin. “But it’s been difficult to directly apply these techniques to tiny microcontrollers, because they are too small.”

So Lin developed TinyNAS, a neural architecture search method that creates custom-sized networks. “We have a lot of microcontrollers that come with different power capacities and different memory sizes,” says Lin. “So we developed the algorithm [TinyNAS] to optimize the search space for different microcontrollers.” The customized nature of TinyNAS means it can generate compact neural networks with the best possible performance for a given microcontroller — with no unnecessary parameters. “Then we deliver the final, efficient model to the microcontroller,” say Lin.

To run that tiny neural network, a microcontroller also needs a lean inference engine. A typical inference engine carries some dead weight — instructions for tasks it may rarely run. The extra code poses no problem for a laptop or smartphone, but it could easily overwhelm a microcontroller. “It doesn’t have off-chip memory, and it doesn’t have a disk,” says Han. “Everything put together is just one megabyte of flash, so we have to really carefully manage such a small resource.” Cue TinyEngine.

The researchers developed their inference engine in conjunction with TinyNAS. TinyEngine generates the essential code necessary to run TinyNAS’ customized neural network. Any deadweight code is discarded, which cuts down on compile-time. “We keep only what we need,” says Han. “And since we designed the neural network, we know exactly what we need. That’s the advantage of system-algorithm codesign.” In the group’s tests of TinyEngine, the size of the compiled binary code was between 1.9 and five times smaller than comparable microcontroller inference engines from Google and ARM. TinyEngine also contains innovations that reduce runtime, including in-place depth-wise convolution, which cuts peak memory usage nearly in half. After codesigning TinyNAS and TinyEngine, Han’s team put MCUNet to the test.

MCUNet’s first challenge was image classification. The researchers used the ImageNet database to train the system with labeled images, then to test its ability to classify novel ones. On a commercial microcontroller they tested, MCUNet successfully classified 70.7 percent of the novel images — the previous state-of-the-art neural network and inference engine combo was just 54 percent accurate. “Even a 1 percent improvement is considered significant,” says Lin. “So this is a giant leap for microcontroller settings.”

The team found similar results in ImageNet tests of three other microcontrollers. And on both speed and accuracy, MCUNet beat the competition for audio and visual “wake-word” tasks, where a user initiates an interaction with a computer using vocal cues (think: “Hey, Siri”) or simply by entering a room. The experiments highlight MCUNet’s adaptability to numerous applications.

“Huge potential”

The promising test results give Han hope that it will become the new industry standard for microcontrollers. “It has huge potential,” he says.

The advance “extends the frontier of deep neural network design even farther into the computational domain of small energy-efficient microcontrollers,” says Kurt Keutzer, a computer scientist at the University of California at Berkeley, who was not involved in the work. He adds that MCUNet could “bring intelligent computer-vision capabilities to even the simplest kitchen appliances, or enable more intelligent motion sensors.”

MCUNet could also make IoT devices more secure. “A key advantage is preserving privacy,” says Han. “You don’t need to transmit the data to the cloud.”

Analyzing data locally reduces the risk of personal information being stolen — including personal health data. Han envisions smart watches with MCUNet that don’t just sense users’ heartbeat, blood pressure, and oxygen levels, but also analyze and help them understand that information. MCUNet could also bring deep learning to IoT devices in vehicles and rural areas with limited internet access.

Plus, MCUNet’s slim computing footprint translates into a slim carbon footprint. “Our big dream is for green AI,” says Han, adding that training a large neural network can burn carbon equivalent to the lifetime emissions of five cars. MCUNet on a microcontroller would require a small fraction of that energy. “Our end goal is to enable efficient, tiny AI with less computational resources, less human resources, and less data,” says Han.

The study, funded by the Interdisciplinary Exchange for Utah Science (NEXUS) at the University of Utah, is published in the International Journal of Environmental Research and Public Health.

Life lived outdoors

People experiencing homelessness, particularly those who sleep outdoors at night, are the most vulnerable and exposed population to environmental hazards, says Daniel Mendoza, a research assistant professor in the Department of Atmospheric Sciences and visiting assistant professor in the Department of City & Metropolitan Planning. Mendoza also holds appointments as an adjunct assistant professor in the Pulmonary Division in the School of Medicine and as a senior scientist at NEXUS. “Many individuals sleep near a road or under a bridge,” he says, “which leads to exposure to high levels of traffic related emissions. Further compounding the issue is the fact that during sleep, many people breathe through their mouth and breathe more deeply.”

This life lived outdoors makes homelessness an environmental justice issue, says Jeff Rose, assistant professor in the Department of Parks, Recreation and Tourism.

“People experiencing unsheltered homelessness often live, eat, sleep, socialize, use the bathroom, and other basic human functions outdoors, with close and regular interaction with the environment,” he says. Environmental justice research looks at uneven exposures to pollution and other environmental risks. “Increasingly, scholars are considering people experiencing unsheltered homelessness as fitting in this framework.”

While other researchers have looked at how people experiencing homelessness experience environmental injustice in the form of access to safe drinking water or parks, the U team says it is among the first to look at how people experiencing homelessness also experience the intermittent poor air quality of Salt Lake County.

Gathering experiences

To collect people’s stories, Angelina DeMarco, a doctoral student in anthropology and Rebecca Hardenbrook, a doctoral student in mathematics, went to several Salt Lake City resource centers to meet with people experiencing sheltered homelessness.

“We sat in the dining hall of each center and invited all residents that came by to interview,” DeMarco says. In partnership with the Volunteers of America outreach team, they also interviewed people at the Salt Lake City library, on downtown streets, outside the St. Vincent de Paul dining hall and at local parks. Outdoor interviews took place often during harsh winter conditions, DeMarco says.

They interviewed everyone they encountered, 138 people total, and asked them open-ended questions about when and how they knew the air was polluted, and how air pollution make them feel. With the interviewees’ permission, the researchers also examined health records kept by the state Homeless Management Information System.

Sheltered and unsheltered

More than half of the participants reported having physical reactions to air pollution including headaches and difficulty breathing, and more than a third reported emotional stress associated with air pollution. 89% reported seeking medical treatment for their symptoms.

But the researchers also wanted to look at whether the duration of homelessness or residing within a shelter would affect individuals’ experiences with air pollution. Surprisingly, they found no significant differences in heart and lung health outcomes between sheltered and unsheltered individuals, as well as between people experiencing chronic (more than a year) or non-chronic homelessness.

“These results indicate that sheltered and unsheltered, short-term and long-term homeless populations experience negative health outcomes that are associated with air pollution,” DeMarco says. The mental health impacts of air pollution exposure, she says, merit additional study.

The message for governmental leaders, the researchers say, is that shelters and day centers that protect people from the elements may not be shielding them from air pollution and other environmental impacts, which can have a significant effect on their health. Affordable housing policies and efforts to place people experiencing homelessness in housing, they say, may do much more to protect a vulnerable population from an environmental hazard.

“The mantra of my lab since its inception has been that parasitic worms manipulate their hosts in very interesting ways to maintain their survival,” says Herbert, an associate professor of pathobiology in Penn’s School of Veterinary Medicine. “SARS-CoV-2 doesn’t care about staying in your body very long because it is transmitted so easily. Worms aren’t spread so easily, so they have to figure out how to persist.”

That focus has revealed a key insight about an immune signaling molecule, the cytokine IL-33, that is important not only in parasite infections, but in a range of other health conditions, such as asthma, obesity, and eczema. In a new study published in Science Immunology, Herbert and colleagues made insights that explain how IL-33 can both help defend the body against parasite infection, but also suppress chronic inflammation in diseases where the immune system is activated inappropriately and causes harmful pathology. A key discovery was that the activity of IL-33 depends upon which cell type is releasing it.

“Lots of people have been interested in IL-33 ever since two big genomic association studies implicated it and its receptor in the pathogenesis of asthma,” Herbert says. “Other researchers have looked at it in the context of infections and others in the context of the brain and development. And everyone knew this protein was in the nucleus, but no one understood how it got out of the cell to accomplish all of these things.

“I’m excited for this work because not only do we find this cytokine in a cell type that nobody was expecting, but we also present a mechanism that no one was expecting for how it could come out.”

IL-33 has been of major interest to immunologists focused on what are known as type 2 immune responses, typically associated with parasite infections or asthma and allergies. On the parasite front, researchers knew that IL-33 acted in part to “wake up” the immune system to the presence of a worm infection. In a mouse model, animals lacking IL-33 sustain worm infections much longer than those with IL-33 intact.

To find out whether it mattered which cell type was releasing the IL-33 signaling molecule, Herbert and colleagues used special mouse model in which only myeloid antigen-presenting cells (immune cells), or epithelial cells (those that line mucosal surfaces), failed to release IL-33.

“Sure enough, we found that when animals lacking the myeloid-derived IL-33 experienced a hookworm infection, they eliminated those hookworms quite fast,” Herbert says. Mice lacking IL-33 in the epithelial cells, however, were not able to readily clear the infection. The same results held up in another rodent model, this one of roundworm infection.

Dendritic cells, a type of myeloid antigen-presenting cell, produce IL-33, and further experiments showed that the cytokine produced by these cells supported a specific population of regulatory T cells (Tregs), which are cells “whose whole purpose is to suppress the immune response,” Herbert says.

Now understanding that dendritic cells were key to supporting Tregs, the researchers wanted to understand how the dendritic cells were delivering the IL-33. The team screened dendritic cells from mice with and without IL-33, identifying a protein called perforin-2 to be suppressed in expression from myeloid cells lacking IL-33.

Perforin-2, as its name suggests, forms a pore that spans the cell membrane, like a tunnel in a hillside, allowing the transport of proteins in and out. The find made complete sense to the researchers, providing an explanation for how dendritic cells could promote the release of IL-33 into the tissues to interact with Tregs. And when Herbert and colleagues experimentally eliminated perforin-2 from dendritic cells, they saw a subsequent lack of Treg growth.

To connect the findings in their animal model and lab dishes to humans, the team utilized patient samples from Penn otolaryngologist Noam Cohen. They found perforin-2 at the plasma membrane of cells from polyps removed from patients with chronic rhinosinusitis, suggesting that the significance of the findings extends to human health.

The study paves the way for even more translational work in immunology — and worms are to thank. “It’s kind of the missing link,” Herbert says. “It opens up a whole new direction for understanding how this cytokine could be involved in obesty, inflammatory bowel disease, Crohn’s, asthma, and development.”

Evangeline Ballerini, Ph.D. ’10, an assistant professor of biological sciences at California State University, Sacramento, said she and her collaborators — including Harvard’s Elena Kramer — settled on the name because the newly discovered gene calls the shots for floral nectar spurs the way Popovich does for his NBA team.

“I ended up choosing to name it after Gregg Popovich, in part, because the gene plays a regulatory role in spur development, kind of like a coach controls the development of their team,” said Ballerini, who is a long-time Golden State Warriors fan and a part-time Celtics fan because of her time in the Boston area, but respects the Spurs and admires Popovich’s leadership.

The work is described in a recently published study in PNAS.

Nectar spurs are the hollow tubes that bulge out from a number of flowers and are crucial to increasing biodiversity among flowering plants that have them. In many cases, species with nectar spurs are much more diverse than their close relative without this novel trait.

In the paper, the scientists identify the gene critical to controlling the development of these spurs in the common columbine, or Aquilegia. They found it acts as a master regulator that appears to control the creation of the spurs by regulating the activity of other genes, the way a coach decides who plays and when.

Aside from the quirky NBA reference, what really has evolutionary biologists excited about the discovery is that the findings have the potential to help them understand how organisms get their vast array of shapes and traits, and then how those traits evolve.

Nectar spurs are considered a key innovation in flowers, meaning they are considered a novel feature — one that helps organisms make the greatest use of their environment and leads to a diversity boom. Animals that evolved to have wings, for instance, have spun off into number of different species over millions of years. Other key innovations are eyes or the backbone in mammals.

Most key innovations happened deep in the past, making identifying their origin increasingly difficult. In the group of plants the researchers studied, however, floral nectar spurs have only been around for about 5 to 7 million years.

“Given that the Aquilegia nectar spur evolved relatively recently and is formed by modifications to a single floral organ, it provides a unique opportunity to begin to dissect the developmental and genetic basis of a key innovation, which, in turn, will provide insight into its origin,” the researchers wrote.

The researchers believe the gene is among the first key innovations for which scientists have identified the critical gene, opening the door to a number of areas in understanding how form and morphology are achieved in flowers and other living things.

“We’re particularly interested in novel features that seem to be very important for promoting speciation events,” said Kramer, Bussey Professor of Organismic and Evolutionary Biology and chair of the Department of Organismic and Evolutionary Biology. “In terms of a morphological trait, like the nectar spur, we’re asking: How did development [of the species] change? … It gives us, essentially, a handle, a starting place to try to start understanding this genetic network.”

Researchers made the discovery using a combination of techniques that included genetic sequencing and crossing species, and gene expression analyses. One of the keys was using a species of the Aquilegia native to China and known to be the only member of that genus, out of 60 to 70 species, to lack nectar spurs.

The team started by repeating a 1960 study by the Russian geneticist W. Pra?mo that crossed the spurless flower with a spurred species and suggested that a single, recessive gene was responsible for spur loss. Unlike Pra?mo, they had the genetic tools to finish the job, and sequenced the genome of about 300 offspring. That narrowed the search to just over 1,000 genes. Further genetic sleuthing led them to POPOVICH, which they call POP for short, and confirmed it using a genetically modified virus that knocks down, or suppresses, targeted genes.

“We took a species that has spurs and normally has POP expression, and we downregulated the expression of POP,” Kramer said. “We showed that it lost its spurs, and that result was the thing that ties it all together. Not only is this a gene that’s specifically expressed in spurs, but when you knock it down, it loses its spurs.”

While this is all strong evidence, more work is needed to confirm their findings.

“There are several directions that we’d like to go in, including trying to figure out how POP expression is controlled, which genes POP regulates the expression of, and what the POP gene is doing in the spurless relatives of Aquilegia,” Ballerini said.

According to Tao Li, professor of engineering and applied sciences at Nanjing University, “An ultra-compact metalens for imaging will miniaturize and even revolutionize conventional optical devices.” Despite all the ongoing work to improve metalenses, most research groups are using them as a substitute for conventional refractive lenses in conventional optical settings. For metalenses to move toward real-world application, it’s important to learn how to integrate metalenses into ultracompact optical devices.

In pursuit of a compact integrated microscope system, Li’s team mounted a metalens on a CMOS image sensor to create a prototype of a coin-sized imaging device. As reported in Advanced Photonics, their metalens-integrated imaging device (MIID) exhibits an ultracompact architecture with a working imaging distance in the hundreds of micrometers. Using a simple image-stitching process, they are able to obtain wide-field microscope imaging with large FOV and high resolution.

Pocket microscope system

The MIID prototype involves a millimeter-sized silicon metalens in a well-designed array of 6×6. Despite the integration of multiple lenses, imaging distance remains relatively small (~500 ?m) because each single lens is sized about 200 ?m. According to the authors, it can be extended to centimeter scale to cover the whole CMOS sensor.

The metalens array, which is a polarization multiplexer, has two different phase profiles corresponding to two circular light polarizations. According to Li, this arrangement ensures the elimination of blind areas.

The authors hope that the new MIID prototype heralds a new era of the pocket microscope system. They acknowledge that the imaging performance needs improvement and suggest a variety of approaches, such as adopting low-loss materials like GaN and SiN. They anticipate continuing advances in microscopy based on meta-technology in the future.

Story Source:

Materials provided by SPIE–International Society for Optics and Photonics. Note: Content may be edited for style and length.

When the goal is “not to sweat the small stuff,” mindfulness appears to offer little toward achieving that end.

The findings, published in the journal Personality and Social Psychology Bulletin, which measured the cardiovascular responses of 1,001 participants during stressful performance tasks, run contrary to previous research and pop culture assertions of how being mindful offers stress relief and coping benefits.

Where earlier work in this area suggests how mindfulness may help people manage active stressors, the current paper finds evidence for an opposite response. In the midst of stress, mindful participants demonstrated cardiovascular responses consistent with greater care and engagement. Put another way, they actually were “sweating the small stuff.”

Even more curiously, although the study’s participants demonstrated no physiological signs associated with positive stress responses, they did report having a positive experience afterward.

“What’s surprising, and particularly striking about our results, is that mindfulness didn’t seem to affect whether people had a more positive stress response in the moment,” said Thomas Saltsman, a researcher in UB’s psychology department and the paper’s lead author. “Did more mindful people actually feel confident, comfortable and capable while engaged in a stressful task? We didn’t see evidence of that, despite them reporting feeling better about the task afterward.”

Mindfulness does have benefits, but appears to be limited in what it can accomplish while people are actively engaged in stressful tasks, like taking a test, giving a speech or sitting for a job interview. Instead, being mindful may only benefit people’s perception of their stress experience after it has ended.

“Although our findings seem to go against a wholesome holy grail of stress and coping benefits associated with dispositional mindfulness, we believe that they instead point to its possible limitations,” says Saltsman. “Like an alleged holy grail of anything, its fruits are likely finite.”

Saltsman describes dispositional mindfulness as having a focused attention on the present. It’s a mindset that tries to avoid ruminating on past realities or considering future possibilities or consequences. It’s about being non-judgmental and relaxing critical interpretations. Mindfulness can be approached with formal training, but people can also be dispositionally higher or lower in mindfulness, which was the focus of their study.

Those high in dispositional mindfulness report greater well-being. They tend not to dwell on past events, and claim to manage stress well.

“Although those benefits seem unambiguous, the specific ways in which mindfulness should impact people’s psychological experiences during stress remain unclear,” says Saltsman. “So we used cardiovascular responses to capture what people were experiencing in a moment of stress, when they’re more or less dispositionally mindful.”

By measuring cardiovascular responses, Saltsman and the other researchers, including Mark Seery, an associate professor of psychology at UB, can tap into participants’ experiences during moments of stress — in this case, giving a speech or taking a reasoning-ability test.

Those responses include heart rate and how hard the heart is pumping. When people care more about the task they are completing, Seery says, their heart rate increases and beats harder. Other measures, like how much blood the heart is pumping and the degree to which blood vessels dilate, indicate how confident or capable one feels during the task.

“One thing these results say to me, in terms of what the average person is expecting when they casually get into mindfulness, is that what it’s actually doing for them could very well be mismatched from their expectations going in,” says Seery. “And this is an impressively large sample of more than a thousand participants, which makes the results particularly convincing.”

Story Source:

Materials provided by University at Buffalo. Original written by Bert Gambini. Note: Content may be edited for style and length.