Flying-Dot Photography
The light transport captures a scene’s visual complexity. Acquiring light transport for dynamic scenes is difficult, since any change in viewpoint, materials, illumination or geometry also varies the transport. One strategy to capture dynamic light transport is to use a fast “flying-dot” projector; i.e., where an impulse light-probe is quickly scanned across the scene. We have built a novel fast flying-dot projector prototype using a high speed camera and a scanning MEMS (Micro-electro-mechanical system) mirror. Our contributions are calibration strategies that enable dynamic light transport acquisition at near video rates with such a system. We develop new methods for overcoming the effects of MEMS mirror resonance. We utilize new algorithms for denoising impulse scanning at high frame rates and compare the trade-offs in visual quality between frame rate and illumination power. Finally, we show the utility of our calibrated setup by demonstrating graphics applications such as video relighting, direct/global separation, and dual videography for dynamic scenes such as fog, water, and glass.
Data is available upon request from the authors. The only fee would be to cover the cost of a hard disk and shipping for the full light transport dataset. Single frames of light transport are available upon request for free via electronic download. Please contact Xiaomeng Liu by liuxiaomeng51@ufl.edu.
Light transport contains all light information between the light source and the image sensor. As an important application of light transport, dual photography has been a popular research topic, but it is challenged by long acquisition time, low signal-noise ratio, storage and computation of a large number of measurements. In this paper, we propose a novel hardware setup that combines a flying-spot MEMS modulated projector with an event camera to implement dual photography for 3D scanning in both line-of-sight (LoS) and non-line-of-sight (NLoS) scenes with a transparent object. In particular using event light transport, depth extraction from the LoS scenes and 3D reconstruction of the object in a NLoS scene have been achieved.
Event Data Download (47.0 GB)
Design and Calibration of a Fast Flying-Dot Projector for Dynamic Light Transport Acquisition
Kristofer Henderson, Xiaomeng Liu, Justin Folden, Brevin Tilmon, Suren Jayasuriya, Sanjeev Koppal
Transactions on Computational Imaging 2020