Event-based Dual Photography for Transparent Scene Reconstruction

Xiaomeng Liu, Joshua D. Rego, Suren Jayasuriya, Sanjeev J. Koppal

Light transport contains all light information between the light source and the image sensor. As an important application of light transport, dual photography has been a popular research topic, but it is challenged by long acquisition time, low signal-noise ratio, storage and computation of a large number of measurements. In this paper, we propose a novel hardware setup that combines a flying-spot MEMS modulated projector with an event camera to implement dual photography for 3D scanning in both line-of-sight (LoS) and non-line-of-sight (NLoS) scenes with a transparent object. In particular using event light transport, depth extraction from the LoS scenes and 3D reconstruction of the object in a NLoS scene have been achieved.

Event Data Download (47.0 GB)

./video.mp4