MIT researchers have created a new method for computing directly on these devices, which drastically reduces this latency. Their technique shifts the memory-intensive steps of running a machine-learning model to a central server where components of the model are encoded onto light waves.

The waves are transmitted to a connected device using fiber optics, which enables tons of data to be sent lightning-fast through a network. The receiver then employs a simple optical device that rapidly performs computations using the parts of a model carried by those light waves.

This technique leads to more than a hundredfold improvement in energy efficiency when compared to other methods. It could also improve security, since a user’s data do not need to be transferred to a central location for computation.

This method could enable a self-driving car to make decisions in real-time while using just a tiny percentage of the energy currently required by power-hungry computers. It could also allow a user to have a latency-free conversation with their smart home device, be used for live video processing over cellular networks, or even enable high-speed image classification on a spacecraft millions of miles from Earth.

“Every time you want to run a neural network, you have to run the program, and how fast you can run the program depends on how fast you can pipe the program in from memory. Our pipe is massive — it corresponds to sending a full feature-length movie over the internet every millisecond or so. That is how fast data comes into our system. And it can compute as fast as that,” says senior author Dirk Englund, an associate professor in the Department of Electrical Engineering and Computer Science (EECS) and member of the MIT Research Laboratory of Electronics.

Joining Englund on the paper is lead author and EECS grad student Alexander Sludds; EECS grad student Saumil Bandyopadhyay, Research Scientist Ryan Hamerly, as well as others from MIT, the MIT Lincoln Laboratory, and Nokia Corporation. The research will be published in Science.

Lightening the load

Neural networks are machine-learning models that use layers of connected nodes, or neurons, to recognize patterns in datasets and perform tasks, like classifying images or recognizing speech. But these models can contain billions of weight parameters, which are numeric values that transform input data as they are processed. These weights must be stored in memory. At the same time, the data transformation process involves billions of algebraic computations, which require a great deal of power to perform.

The process of fetching data (the weights of the neural network, in this case) from memory and moving them to the parts of a computer that do the actual computation is one of the biggest limiting factors to speed and energy efficiency, says Sludds.

“So our thought was, why don’t we take all that heavy lifting — the process of fetching billions of weights from memory — move it away from the edge device and put it someplace where we have abundant access to power and memory, which gives us the ability to fetch those weights quickly?” he says.

The neural network architecture they developed, Netcast, involves storing weights in a central server that is connected to a novel piece of hardware called a smart transceiver. This smart transceiver, a thumb-sized chip that can receive and transmit data, uses technology known as silicon photonics to fetch trillions of weights from memory each second.

It receives weights as electrical signals and imprints them onto light waves. Since the weight data are encoded as bits (1s and 0s) the transceiver converts them by switching lasers; a laser is turned on for a 1 and off for a 0. It combines these light waves and then periodically transfers them through a fiber optic network so a client device doesn’t need to query the server to receive them.

“Optics is great because there are many ways to carry data within optics. For instance, you can put data on different colors of light, and that enables a much higher data throughput and greater bandwidth than with electronics,” explains Bandyopadhyay.

Trillions per second

Once the light waves arrive at the client device, a simple optical component known as a broadband “Mach-Zehnder” modulator uses them to perform super-fast, analog computation. This involves encoding input data from the device, such as sensor information, onto the weights. Then it sends each individual wavelength to a receiver that detects the light and measures the result of the computation.

The researchers devised a way to use this modulator to do trillions of multiplications per second, which vastly increases the speed of computation on the device while using only a tiny amount of power.

“In order to make something faster, you need to make it more energy efficient. But there is a trade-off. We’ve built a system that can operate with about a milliwatt of power but still do trillions of multiplications per second. In terms of both speed and energy efficiency, that is a gain of orders of magnitude,” Sludds says.

They tested this architecture by sending weights over an 86-kilometer fiber that connects their lab to MIT Lincoln Laboratory. Netcast enabled machine-learning with high accuracy — 98.7 percent for image classification and 98.8 percent for digit recognition — at rapid speeds.

“We had to do some calibration, but I was surprised by how little work we had to do to achieve such high accuracy out of the box. We were able to get commercially relevant accuracy,” adds Hamerly.

Moving forward, the researchers want to iterate on the smart transceiver chip to achieve even better performance. They also want to miniaturize the receiver, which is currently the size of a shoe box, down to the size of a single chip so it could fit onto a smart device like a cell phone.

The research is funded, in part, by NTT Research, the National Science Foundation, the Air Force Office of Scientific Research, the Air Force Research Laboratory, and the Army Research Office.

Researchers at the University of British Columbia’s Institute for the Oceans and Fisheries (IOF), the Department of Fisheries and Oceans (DFO), University of Toronto, Dalhousie University, and the Hakai Institute examined juvenile sockeye salmon from three locations along their migration route in coastal British Columbia. In 2016 and 2016, the body condition and nutritional health of juvenile sockeye was sampled in the northern Strait of Georgia, the highly tidally mixed Johnstone Strait, and Queen Charlotte Strait.

“We measured the fatty acids in the sockeye salmon tissues, specifically looking at stored fatty acids that are important for mobilizing energy during periods of starvation, such as what these salmon face in Johnstone Strait,” said lead author Dr. Jessica Garzke, an IOF postdoctoral fellow. “We found that short term changes in health differed between years but were not related to zooplankton biomass, abundance, average size, or species composition. Rather it was the quality of the zooplankton that played an important role.”

Leaving the generally productive and good foraging area of the Strait of Georgia, to the typically food-poor conditions of the Johnstone Strait has a powerful impact on the young salmon. When juvenile sockeye salmon entered Johnstone Strait in 2015, they were generally in good body and nutritional condition, and able to use stored energy resources to manage traversing this section, where prey availability was low, to get to Queen Charlotte Strait. In 2016, however, juvenile sockeye salmon entered Johnstone Strait with poor body and nutritional condition, and they struggled to regain condition when they reached Queen Charlotte Strait.

“It was clear from our study that differences in juvenile salmon condition between the two years were due to changes in the food quality available to the fish, more than any other factor. Incorporation of food quality measurements in addition to analysis of quantity of prey is vital to understanding changes in fish condition and survival between years,” Dr. Garzke said.

Understanding how diet affects the salmon’s growth and health during the early part of their lives could help researchers understand the long-term population trends of the species including their decline, says Dr. Garzke. Sockeye salmon are an iconic species in British Columbia, important both commercially and culturally, and are in decline.

The research highlighted how variable the ocean conditions are that juvenile salmon face in British Columbia, said senior author and IOF associate professor Dr. Brian Hunt, with migrating salmon experiencing “boom and bust” feeding conditions. “Importantly, feeding conditions are not consistent from one year to the next. They are impacted by changing climate, and areas that had good feeding conditions in one year can be bad the next. The more areas that switch to bad feeding conditions the tougher it is for juveniles to survive the coastal migration and their first winter at sea.”

Story Source:

Materials provided by University of British Columbia. Note: Content may be edited for style and length.

Disruptions to the body’s internal clock — which coordinates the timing of biochemical, physiological and behavioral processes — are associated with a range of diseases, including cancer, cardiovascular conditions and susceptibility to infections, as well as a higher risk for accidents. Common disruptions of circadian rhythms are jet lag and shift work, which is performed by some 30 million people in the U.S.

“We are studying two mutations, both of which affect our ability to respond to shifts of the light cycle,” says neurobiologist Eric Bittman, Professor Emeritus of Biology. “Both of them speed up the clock. They reveal how vulnerable we are to disruptions in the light:dark schedule.”

In mammals, circadian rhythms are generated internally by a master pacemaker in the suprachiasmatic nucleus of the hypothalamus in the brain. In addition, every cell in the body has its own circadian clock, which the master pacemaker coordinates. In the normal, light:dark and fluctuating environment, circadian clocks create 24-hour cycles. However, in constant conditions, such as when hamsters are studied in darkness, the rhythms generate cycles whose period is longer or shorter than 24 hours.

“What this reveals to us is that there’s some internal mechanism that is generating rhythmicity, and that the animals are using cues from the environment, the most powerful of which is the light:dark cycle, to sync it up to exactly 24 hours,” Bittman says.

In previous research, Bittman and team identified a recessive mutation, which they call duper, as a defect in the circadian regulator gene Cryptochrome 1 (CRY1) of Syrian hamsters. By improving the draft of the hamster genome using fast homozygosity mapping, they created a modern genetic research model for investigating human diseases.

In a follow-up paper published recently in the Proceedings of the National Academy of Sciences (PNAS), the research team focuses on the effects of duper in genetically altered hamsters. The scientists identify previously unsuspected functions of CRY1 in circadian entrainment, which is the synching of the biological clock with external signals, and heart disease.

“Duper speeds up the clock in constant conditions and it makes it be able to shift by as much as 180 degrees in response to even a brief pulse of light,” Bittman explains. “We suspect this might be relevant in understanding the effects of jet lag and shift work.”

The body’s organs reset their clocks at different rates after circadian disruption. This temporal misalignment is believed to cause the adverse health effects associated with shift work. “Almost all of our physiological processes are rhythmic,” Bittman says.

The researchers found that circadian re-entrainment is accelerated in duper mutant hamsters, independent of the speeding up of the clock. In an effort to understand more about the health consequences of circadian misalignment, the researchers examined effects of duper and phase shifts on a hamster model of heart disease known to be aggravated by phase shifts.

Simulated jet lag, in the form of eight-hour phase shifts every second week, shortened the lifespan of cardiomyopathic hamsters. However, the shortened lifespan was reversed in duper hamsters as the mutation accelerated their adjustment to the shift of the light:dark cycles. Bittman says the findings have implications for pinpointing the pathways involved in human biological clocks.

“For people with jet lag or the millions of shift workers, it can take days and sometimes weeks for the body — the different organs — to come back into their normal temporal relationship,” he says. Many of us disrupt our circadian system when we are exposed to light late in the evening, such as by looking at cell phones and computer screens. “It can take a couple of weeks until your brain is having the right relationship to your liver and kidneys,” Bittman adds.

The research suggests that we all need to be mindful of how the environment affects our biological clocks. Specifically, hospitals need to be sensitive to the timing of lights and darkness in patients’ rooms. “We need to pay attention to the temporal relationship between organs and between the master clock and the brain, and how it regulates the clock in the brain, as well as in the peripheral organs, and be sensitive to the light:dark environment,” Bittman says.

Additionally, understanding the rhythms of the organs is a crucial aspect in the timing of medical treatment. “An awful lot of drugs that are administered are more effective at one time of the day than another because the circadian clock is regulating all these metabolic pathways that in fact are being targeted by these drugs,” Bittman says.

Ongoing research will focus on identifying the underlying mechanisms of the biological clocks and their role in disease.