Seeing what’s happening on the other side of a corner isn’t as impossible as it sounds. Scientists have been working on the problem for years, using lasers to bounce light off unseen objects and detect what’s going beyond their line of sight. Now, researchers from MIT’s CSAIL have gone one step further: they’re using footage from an ordinary smartphone to “see” around corners by spotting subtle changes in light and shadow.
The premise of the work is simple: all objects reflect light, and, by closely studying the floor near a corner, you can see if something is moving on the other side based on changing shadows. These fluctuations are invisible to the human eye, but researchers were able to spot them by tweaking the footage from ordinary commercial cameras, and even an iPhone 5s.
The method has a number of serious limitations, though. For starters, you can’t make out any detail about the unseen object. You can identify how fast it’s moving, and get some idea of its position, but you can’t make out edges or shape or texture. And, unlike the laser method of seeing around corners, the unseen person or object has to be moving in a brightly lit space in order to be detected. As a last limitation, the source footage also has to be stable, although the researchers are working on how to use moving footage.
These caveats aside, the system is pretty robust. It works fine outside in bright sunlight, for example, and can even be used in the rain. Its creators think it could be used in the future for cameras on self-driving cars, allowing them to look around corners to spot pedestrians, cyclists, and other vehicles. In that sort of situation you don’t need to see any detail, you just need to know if something is there.
Speaking to The Verge by email, lead author of the research, Katie Bouman, says the MIT CSAIL team even tested exactly this sort of scenario. “In order to see how feasible the method would be for self-driving cars in our paper, we took a video when we were pretty far away from the corner at a shallow angle,” said Bouman. “We were still able to get a very clear signal, meaning the car could be quite far away from the corner and still work.”
The system — dubbed CornerCameras — currently needs a laptop to do the necessary image processing, but Bouman says this could be overcome in the future. “From a computing point of view, I think our system could be fully put on a phone. We just haven’t done it [yet],” she told The Verge. Just wait until there’s an app for that.