Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2509.12275

Omni-CLST: Error-aware Curriculum Learning with guided Selective chain-of-Thought for audio question answering

Jinghua Zhao·Hang Su·Lichun Fan·Zhenbo Luo·Hui Wang·Haoqin Sun·Yong Qin
View original source ↗Read source document ↗Export metadata ↓

Abstract

With the rapid progress of large audio-language models (LALMs), audio question answering (AQA) has emerged as a challenging task requiring both fine-grained audio understanding and complex reasoning. While current methods mainly rely on constructing new datasets via captioning or reasoning traces, existing high-quality AQA data remains underutilized. To address this, we propose Omni-CLST, an error-aware Curriculum Learning framework with guided Selective Chain-of-Thought. The framework efficiently leverages existing high-quality dataset through two key strategies: an error-aware curriculum that organizes samples by difficulty, and a guided thought dropout mechanism that focuses reasoning on challenging cases. Experiments show that Omni-CLST achieves 73.80% on MMAU-mini and a new state of the art of 64.30% on MMAR, demonstrating robust generalization in multimodal audio-language understanding.

Explore related subjects

cs.SDcs.AIeess.AS

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Jinghua Zhao, Hang Su, Lichun Fan, Zhenbo Luo, Hui Wang, Haoqin Sun, Yong Qin. 2025-09-18. Omni-CLST: Error-aware Curriculum Learning with guided Selective chain-of-Thought for audio question answering. https://arxiv.org/abs/2509.12275

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Retain-Free Machine Unlearning for Speech Emotion Recognition

Speech Emotion Recognition (SER) infers a speaker's emotional state from speech and is increasingly deployed in human-computer interaction, education, and healthcare. Because speech also carries sensitive personal information, speakers may ask that some of their recordings be deleted, which requires removing the influence of those samples from an already trained SER model. Most machine unlearning methods can meet this request only with access to the remaining training data alongside the samples to be forgotten; this is impractical when the remaining data cannot be redistributed or has itself been deleted, and it adds storage and computation as the data grows. To this end, we propose a retain-free unlearning method that updates a pre-trained SER model using only the forget set. Our key idea is to synthesise adversarial samples from the forget set as a surrogate for the unavailable remaining data, and to constrain each parameter update by its estimated importance so that forgetting does not erase general knowledge. The experiments over several emotional-speech corpora and self-supervised backbones show that our method drives forget-set performance down to near chance while retaining much of the model's utility on the remaining and unseen-speaker data, narrowing the gap to methods that rely on the remaining set.

cs.SD↗
arXiv2026

SHINE: Sequential Hierarchical Integration Network for EEG and MEG

How natural speech is represented in the brain constitutes a major challenge for cognitive neuroscience. Reconstructing the speech envelope and Mel spectrogram from EEG and MEG provides a time-resolved way to study its temporal and spectral structure. Speech-related neural activity spans sensors and temporal scales; extracting these representations while adapting the use of context to each acoustic target is a central problem in speech reconstruction. We propose SHINE, a Sequential Hierarchical Integration Network for EEG and MEG. A residual sensor adapter unifies input dimensions, intermediate dilated-block states retain temporal depth, and a target- and time-dependent gate fuses local hierarchical and attention-enhanced context predictions. Across two EEG and two MEG datasets, SHINE has the highest mean envelope and mean-Mel Pearson correlations among nine local baseline implementations on all eight dataset-metric combinations. SHINE also placed second in the speech-detection Extended Track of the NeurIPS 2025 PNPL Competition. Code will be released at https://github.com/xuxiran/SHINE.

cs.SD↗
arXiv2026

Self-Distilled Pronunciation and Accent Control for Neural Text-to-Speech

Text-to-speech that reads raw text has no lexicon: a rare word is read as guessed, and a native Japanese listener accepts a word only if its reading and pitch accent are both right. A known remedy installs a reading-and-accent channel into a released model, but it needs many recordings. This paper removes the recordings: the frozen backbone reads a sentence containing a common word it already says correctly, and that output serves as the teacher for the same sentence with the word replaced by an annotated reading with its pitch accent. Screened raters judged the tag right on 0.80 to 0.93 of unseen difficult words on four backbones spanning autoregressive, diffusion, and encoder-decoder synthesis; plain kana, which cannot express an accent, got 0.38 to 0.60. On words needing no edit, naturalness is non-inferior on one backbone; on the other three, listeners prefer the unedited rendition by 0.19 to 0.26.

cs.SD↗