Event-enhanced Passive Non-line-of-sight imaging for moving objects with Physical embedding

Abstract

Non-line-of-sight (NLOS) imaging with intelligent sensors emerges as a novel technique in imaging and sensing occluded objects around corners. With the innovation of bio-inspired neuromorphic sensors, the applications of novel sensors in unconventional imaging tasks like NLOS imaging have shown promising prospects in intelligent perception, encompassing autonomous driving, medical endoscopy and other sensing scenarios. However, the most challenging point of sensors application in computational imaging is the inverse problem established between sensors acquisition and reconstructions. Traditional physical retrieval methods with certain sensors applications usually result in poor reconstruction due to the highly ill-posedness, particularly in moving object imaging. Thanks to the development of neural networks, data-driven methods have greatly improved its accuracy, however, heavy reliance on data volume has put great pressure on data collection and dataset fabrication. To the best of our knowledge, we firstly propose a sensor-dominated restoration prototype termed "event enhanced passive NLOS imaging prototype for moving objects with physical embedding" (EPNP), which illustrated the application of dynamic vision sensors in NLOS imaging. EPNP induces an event camera for feature extraction of dynamic diffusion spot and leverages simulation dataset to pre-train the physical embedded model before fine-tuning with limited real-shot data. The proposed EPNP prototype is verified by simulation and real-world experiments, while the comparisons of data paradigms also validate the superiority of event-based sensor applications in passive NLOS imaging for moving objects and perspectives in advanced imaging techniques.