In-Sensor Motion Recognition with Memristive System and Light Sensing Surfaces

Abstract

In this paper, we introduce a novel device architecture that merges memristive devices with light-sensing surfaces, for energy-efficient motion recognition at the edge. Our light-sensing surface captures motion data through in-sensor computation. This data is then processed using a memristive system equipped with a HfO2-based synaptic device, coupled with a winner-take-all (WTA) circuit, tailored for low-power motion classification tasks. We validate our end-to-end system using four distinct human hand gestures - left-to-right, right-to-left, bottom-to-top, and top-to-bottom movements - to assess energy efficiency and classification robustness. Our experiments show that the system requires an average of only 4.17 nJ for taking our processed analog signal and mapping weights onto our memristive system and 0.952 nJ for testing per movement class, achieving 97.22% accuracy even under 5% noise interference. A key advantage of our proposed architecture is its low energy requirement, enabling the integration of energy-harvesting solutions such as solar power for sustainable autonomous operation. Additionally, our approach enhances data privacy by processing data locally, reducing the need for external data transmission and storage.