Deep learning speeds up video compressive sensing from days to minutes
DOI: 10.1063/10.0000928
Deep learning speeds up video compressive sensing from days to minutes lead image
Video compressive sensing, or snapshot compressive imaging, is the process of generating, high-dimensional data and video from a single two-dimensional image. Previously this has been achieved using iterative reconstructive algorithms, but the process of recovering the data from the image took hours.
Qiao et al. use deep learning-based reconstruction algorithms to recover information from a single compressed measurement (the image). Their use of deep learning throughout the process of video compressive sensing allowed them to pull information from an image within sub-seconds.
“In realistic applications, we will need the information immediately. Deep learning solves that problem. We compared several different frameworks and found a way to receive results within sub-seconds,” said author Xin Yuan.
The authors first built a snapshot compressive imaging device and then compared two deep learning regimes, an end-to-end convolutional neural network (E2E-CNN) and a Plug-and-Play (PnP) framework.
Though the E2E-CNN framework performed the fastest (sub-seconds) when given a pre-trained network, the PnP algorithm had the fastest speed (within 1 minute) of reconstruction among the iteration-based algorithms and was considered low cost and simple to use.
According to Yuan, the PnP algorithm would be suitable for cases where adaptive sensing is required, such as sports photography where the camera needs to change focus and frame rate. However, for applications where the cameras are fixed, E2E framework is preferred due to the possibility of pre-training the network.
“Existing surveillance cameras consume too much memory and bandwidth.” said Yuan. “If the video SCI cameras are deployed for traffic surveillance, our method would allow for quick and easy recovery of the data captured by the SCI cameras.”
Source: “Deep learning for video compressive sensing,” by Mu Qiao, Ziyi Meng, Jiawei Ma, and Xin Yuan, APL Photonics (2020). The article can be accessed at https://doi.org/10.1063/1.5140721 .
