Recognition of auditory objects from continuous and ultra-sparse spike streams in neuromorphic devices

Abstract

Energy consumption is a major concern for modern artificial intelligence (AI) applications. The brain is thought to achieve energy-efficient computation through sparse coding schemes. By emulating the spiking dynamics of the brain, neuromorphic chips hold the potential to realize energy-efficient AI. In this work, we use birdsong to demonstrate that auditory object recognition can be achieved using ultra-sparse coding. Feature-detecting neurons tuned to short temporal features ($\sim$37 ms), sampled from exemplar syllables, transform continuous auditory signals into streams of ultra-sparse spikes that mark the onsets and offsets of features within the signal. Spike patterns elicited by the exemplars are stored as templates. During recognition, template matching is performed using axonal delays between feature-detecting and template-detecting neurons. Recognition is further enhanced by an additional group of specialized binary neurons trained to discriminate between syllable categories with subtle differences. When tested on Bengalese finch songs, our approach achieved error rates below 5% using 10 to 40 exemplars per syllable type. We then implemented this recognition algorithm using spiking neural networks on Intel's neuromorphic chip, Loihi, and found that response latencies remain constant (approximately 17 ms per syllable), regardless of the number of templates used. Our results demonstrate the potential of neuromorphic chips for low-latency, low-energy recognition of complex sounds, including speech.