A science team has harnessed the power of deep learning to build a new laser-based system that can image around corners in real-time

Image Credit: Heiko Küverling via iStock - HDR tune by Universal-Sci

Deep learning is part of a larger family of methods of machine learning based on artificial neural networks. It can be applied in areas such as image identification, speech recognition, natural language processing, audio recognition, social network filtering, machine translation, bioinformatics, the creation of medicines, medical image recognition, etc. Machine learning can achieve results similar to or sometimes even better than human specialists.

Recently a science team harnessed the power of deep learning to build a new laser-based system that can image around corners in real-time. With some more improvements, the system could be able to let self-driving cars 'view' around things like parked cars or busy intersections to see potential hazards or pedestrians. It could also be fitted on satellites and spacecraft for assignments such as generating images of the inside of a cave on an asteroid.

The team has published their findings in the science journal: Optica. Christopher A. Metzler, team leader, stated that compared to other methods, their non-line-of-sight imaging system produces uniquely high resolutions and imaging speeds. According to him, these characteristics

allow for uses that wouldn't usually be possible. Think of reading the license plate of a car that is concealed or spotting a badge worn by someone walking on the other side of a corner.

The new system can detect minute details of concealed objects 1 meter away. It is intended to show tiny things at really great resolutions. However, it can also be merged with other imaging systems that produce low-resolution room-sized restorations of hidden objects. According to Felix Heide, one of the authors, their work takes a step towards significant applications in navigation, medical imaging robotics, and defense.

Although progress is made, the self-driving capabilities of contemporary cars are still relatively mundane in comparison to what might be possible in the future - Image Credit: Universal-Sci - CC BY 4.0

So how does it work?

The system conveniently uses a camera sensor that is already commercially available. In addition, it uses an enhanced standard laser source, akin to those one might find in a laser pointer.

The idea is that a laser beam rebounds off a visible wall onto concealed objects and then back onto the wall creating an interference pattern that encodes the shape of the concealed target objects. This interference pattern is known as a speckle pattern. The speckle effect is a consequence of the interference of numerous waves that are of the same frequency, having different amplitudes and, which add together to give a resultant wave whose amplitude, and therefore intensity, varies randomly.

After this, the speckle pattern has to be rebuilt into the concealed object digitally which requires the resolve of a difficult computational problem

Short exposure times are required for real-time picturing but generate too much noise for current algorithms to operate. The scientists used deep learning to alleviate this problem.

Co-author Prasanna Rangarajan stated that in comparison to other methods, their deep learning system is much less sensitive to noise and can, therefore, be operated with much shorter exposure times. The scientists precisely distinguished the noise, which made them able to synthesize data to train their algorithm to resolve the reconstruction issue without the need for retrieving expensive training data, which would have otherwise been necessary for regular machine learning.

The scientists tested their new method by recreating images of small (1 centimeter tall) numbers and letters obfuscated behind a corner. With an exposure time of only a quarter of a second, their method produced reconstructions in a 300-micron resolution. Currently, work is being done to make the system more practical for a wider variety of applications.

If the researchers succeed in their mission, traffic in the future might be a lot safer, and astronomers might have a new tool to uncover the hidden mysteries of our solar system. If you are interested in this subject and want more details, be sure to check out the paper listed below.

Sources and further reading: Deep-inverse correlography: towards real-time high-resolution non-line-of-sight imaging - Artificial Intelligence, Deep Learning - Speckle pattern