arXiv subjects

Masato Mimura

Publications and source records attributed to Masato Mimura.

At least 19 recordsLinked to original sources

arXiv2026

Improving Large-Scale Weakly Supervised ASR by Filtering and Selection

Leveraging large-scale weakly supervised datasets is crucial to train robust end-to-end automatic speech recognition (ASR) models. However, such datasets often contain noisy labels and lack domain specificity, limiting their effectiveness. To address these issues and make better use of weakly supervised datasets, we propose a novel training approach incorporating data filtering and selection. Our approach consists of three steps: pretraining on the entire dataset, continued pretraining on a filtered subset based on character error rate (CER), and fine-tuning on a small number of acoustically similar samples to the target domain, selected from the filtered subset. In experiments with a 90,000-hour weakly supervised Japanese dataset, the proposed filtering and selection methods synergistically reduced CER by up to 6.4% and 4.0%, respectively, even though these steps reused training samples already used in the first pretraining step.

eess.AS↗

arXiv2026

Progressive Alignment Objectives for Aligner-Encoder based ASR

Aligner-Encoders are recently proposed seq2seq end-to-end ASR models that replace decoder attention by predicting the uth token directly from the u-th encoder position, so the encoder must learn the alignment internally without cross-attention or a transducer lattice. In practice, this alignment often forms abruptly in the upper layers, making training sensitive and brittle on long utterances. We propose InterAligner, which adds an intermediate Aligner objective so alignment can form progressively across depth, together with an intermediate CTC loss (InterCTC) to stabilize optimization. On LibriSpeech with a 17-layer Conformer, a final-only Aligner reaches 5.0/7.8 WER (test-clean/other). InterCTC improves to 3.4/6.0, and InterAligner further reduces WER to 3.1/5.6 with the largest gains on long utterances.

eess.AS↗

arXiv2026

Coarse geometry of stable mixed commutator length I: duality and functional analysis on chains

Let $G$ be a group and $N$ its normal subgroup. On the mixed commutator subgroup $[G,N]$, the mixed stable commutator length $\mathrm{scl}_{G,N}$ and the restriction of the ordinary stable commutator length $\mathrm{scl}_{G}$ are defined. We characterize when they are bi-Lipschitz equivalent by the vanishing of a certain $\mathbb{R}$-linear space $\mathrm{W}(G,N)$ related to invariant quasimorphisms. For the proof, we obtain a refined version of the generalized mixed Bavard duality theorem, and perform functional analysis on the completion of a certain space of $1$-chains.

math.GR↗

arXiv2026

Chunkwise Aligners for Streaming Speech Recognition

We propose the Chunkwise Aligner, a novel architecture for streaming automatic speech recognition (ASR). While the Transducer is the standard model for streaming ASR, its training is costly due to the need to compute all possible audio-label alignments. The recently introduced Aligner reduces this cost by discarding explicit alignments, but this modification makes it unsuitable for streaming. Our approach overcomes this limitation by dividing the audio into chunks and aligning each label to the leftmost frames of its chunk, whereas transitions between chunks are managed by a learned end-of-chunk probability. Experiments show that the Chunkwise Aligner not only matches the Transducer's accuracy in both offline and streaming scenarios, but also offers superior training and decoding efficiencies.

eess.AS↗

arXiv2026

Decoder-only Conformer with Modality-aware Sparse Mixtures of Experts for ASR

We present a decoder-only Conformer for automatic speech recognition (ASR) that processes speech and text in a single stack without external speech encoders or pretrained large language models (LLM). The model uses a modality-aware sparse mixture of experts (MoE): disjoint expert pools for speech and text with hard routing and top-1 selection, embedded in hybrid-causality Conformer blocks (bidirectional for speech, causal for text). Training combines CTC on speech positions with label-smoothed cross-entropy for text generation. Our 113M-parameter model consistently improves WER over a 139M AED baseline on Librispeech (2.8% vs. 3.2% test-clean; 5.6% vs. 6.0% test-other). On Common Voice 16.1 with a single multilingual model across five languages, our approach reduces average WER from 12.2% to 10.6%. To our knowledge, this is the first randomly initialized decoder-only ASR that surpasses strong AED baselines via modality-aware routing and sparse MoE, achieving better accuracy with fewer active parameters and without alignment/adaptation modules.

eess.AS↗

arXiv2025

All-in-One ASR: Unifying Encoder-Decoder Models of CTC, Attention, and Transducer in Dual-Mode ASR

This paper proposes a unified framework, All-in-One ASR, that allows a single model to support multiple automatic speech recognition (ASR) paradigms, including connectionist temporal classification (CTC), attention-based encoder-decoder (AED), and Transducer, in both offline and streaming modes. While each ASR architecture offers distinct advantages and trade-offs depending on the application, maintaining separate models for each scenario incurs substantial development and deployment costs. To address this issue, we introduce a multi-mode joiner that enables seamless integration of various ASR modes within a single unified model. Experiments show that All-in-One ASR significantly reduces the total model footprint while matching or even surpassing the recognition performance of individually optimized ASR models. Furthermore, joint decoding leverages the complementary strengths of different ASR modes, yielding additional improvements in recognition accuracy.

eess.AS↗

arXiv2025

Flux homomorphism and bilinear form constructed from Shelukhin's quasimorphism

Given a closed connected symplectic manifold $(M,\omega)$, we construct an alternating $\mathbb{R}$-bilinear form $\mathfrak{b}=\mathfrak{b}_{\mu_{\mathrm{Sh}}}$ on the real first cohomology of $M$ from Shelukhin's quasimorphism $\mu_{\mathrm{Sh}}$. Here $\mu_{\mathrm{Sh}}$ is defined on the universal cover of the group of Hamiltonian diffeomorphisms on $(M,\omega)$. This bilinear form is invariant under the symplectic mapping class group action, and $\mathfrak{b}$ yields a constraint on the fluxes of commuting two elements in the group of symplectomorphisms on $(M,\omega)$. These results might be seen as an analog of Rousseau's result for an open connected symplectic manifold, where he recovered the symplectic pairing from the Calabi homomorphism. Furthermore, $\mathfrak{b}$ controls the extendability of Shelukhin's quasimorphisms, as well as the triviality of a characteristic class of Reznikov. To construct $\mathfrak{b}$, we build general machinery for a group $G$ of producing a real-valued $\mathbb{Z}$-bilinear form $\mathfrak{b}_{\mu}$ from a $G$-invariant quasimorphism $\mu$ on the commutator subgroup of $G$.

math.SG↗

arXiv2025

Non-extendablity of Shelukhin's quasimorphism and non-triviality of Reznikov's class

Shelukhin constructed a quasimorphism on the universal covering of the group of Hamiltonian diffeomorphisms for a general closed symplectic manifold. In the present paper, we prove the non-extendability of that quasimorphism for certain symplectic manifolds, such as a blow-up of torus and the product of a surface of genus at least two and a closed symplectic manifold. As its application, we prove the non-vanishing of Reznikov's characteristic class for the above symplectic manifolds.

math.SG↗

arXiv2025

Microphone Array Geometry Independent Multi-Talker Distant ASR: NTT System for the DASR Task of the CHiME-8 Challenge

In this paper, we introduce a multi-talker distant automatic speech recognition (DASR) system we designed for the DASR task 1 of the CHiME-8 challenge. Our system performs speaker counting, diarization, and ASR. It handles various recording conditions, from diner parties to professional meetings and from two to eight speakers. We perform diarization first, followed by speech enhancement, and then ASR as the challenge baseline. However, we introduced several key refinements. First, we derived a powerful speaker diarization relying on end-to-end speaker diarization with vector clustering (EEND-VC), multi-channel speaker counting using enhanced embeddings from EEND-VC, and target-speaker voice activity detection (TS-VAD). For speech enhancement, we introduced a novel microphone selection rule to better select the most relevant microphones among the distributed microphones and investigated improvements to beamforming. Finally, for ASR, we developed several models exploiting Whisper and WavLM speech foundation models. We present the results we submitted to the challenge and updated results we obtained afterward. Our strongest system achieves a 63% relative macro tcpWER improvement over the baseline and outperforms the challenge best results on the NOTSOFAR-1 meeting evaluation data among geometry-independent systems.

eess.AS↗

arXiv2024

Alignment-Free Training for Transducer-based Multi-Talker ASR

Extending the RNN Transducer (RNNT) to recognize multi-talker speech is essential for wider automatic speech recognition (ASR) applications. Multi-talker RNNT (MT-RNNT) aims to achieve recognition without relying on costly front-end source separation. MT-RNNT is conventionally implemented using architectures with multiple encoders or decoders, or by serializing all speakers' transcriptions into a single output stream. The first approach is computationally expensive, particularly due to the need for multiple encoder processing. In contrast, the second approach involves a complex label generation process, requiring accurate timestamps of all words spoken by all speakers in the mixture, obtained from an external ASR system. In this paper, we propose a novel alignment-free training scheme for the MT-RNNT (MT-RNNT-AFT) that adopts the standard RNNT architecture. The target labels are created by appending a prompt token corresponding to each speaker at the beginning of the transcription, reflecting the order of each speaker's appearance in the mixtures. Thus, MT-RNNT-AFT can be trained without relying on accurate alignments, and it can recognize all speakers' speech with just one round of encoder processing. Experiments show that MT-RNNT-AFT achieves performance comparable to that of the state-of-the-art alternatives, while greatly simplifying the training process.

eess.AS↗

arXiv2024

Boosting Hybrid Autoregressive Transducer-based ASR with Internal Acoustic Model Training and Dual Blank Thresholding

A hybrid autoregressive transducer (HAT) is a variant of neural transducer that models blank and non-blank posterior distributions separately. In this paper, we propose a novel internal acoustic model (IAM) training strategy to enhance HAT-based speech recognition. IAM consists of encoder and joint networks, which are fully shared and jointly trained with HAT. This joint training not only enhances the HAT training efficiency but also encourages IAM and HAT to emit blanks synchronously which skips the more expensive non-blank computation, resulting in more effective blank thresholding for faster decoding. Experiments demonstrate that the relative error reductions of the HAT with IAM compared to the vanilla HAT are statistically significant. Moreover, we introduce dual blank thresholding, which combines both HAT- and IAM-blank thresholding and a compatible decoding algorithm. This results in a 42-75% decoding speed-up with no major performance degradation.

eess.AS↗

arXiv2024

NTT Multi-Speaker ASR System for the DASR Task of CHiME-8 Challenge

We present a distant automatic speech recognition (DASR) system developed for the CHiME-8 DASR track. It consists of a diarization first pipeline. For diarization, we use end-to-end diarization with vector clustering (EEND-VC) followed by target speaker voice activity detection (TS-VAD) refinement. To deal with various numbers of speakers, we developed a new multi-channel speaker counting approach. We then apply guided source separation (GSS) with several improvements to the baseline system. Finally, we perform ASR using a combination of systems built from strong pre-trained models. Our proposed system achieves a macro tcpWER of 21.3 % on the dev set, which is a 57 % relative improvement over the baseline.

eess.AS↗

arXiv2024

Sentence-wise Speech Summarization: Task, Datasets, and End-to-End Modeling with LM Knowledge Distillation

This paper introduces a novel approach called sentence-wise speech summarization (Sen-SSum), which generates text summaries from a spoken document in a sentence-by-sentence manner. Sen-SSum combines the real-time processing of automatic speech recognition (ASR) with the conciseness of speech summarization. To explore this approach, we present two datasets for Sen-SSum: Mega-SSum and CSJ-SSum. Using these datasets, our study evaluates two types of Transformer-based models: 1) cascade models that combine ASR and strong text summarization models, and 2) end-to-end (E2E) models that directly convert speech into a text summary. While E2E models are appealing to develop compute-efficient models, they perform worse than cascade models. Therefore, we propose knowledge distillation for E2E models using pseudo-summaries generated by the cascade models. Our experiments show that this proposed knowledge distillation effectively improves the performance of the E2E model on both datasets.

cs.CL↗

arXiv2024

SpeakerBeam-SS: Real-time Target Speaker Extraction with Lightweight Conv-TasNet and State Space Modeling

Real-time target speaker extraction (TSE) is intended to extract the desired speaker's voice from the observed mixture of multiple speakers in a streaming manner. Implementing real-time TSE is challenging as the computational complexity must be reduced to provide real-time operation. This work introduces to Conv-TasNet-based TSE a new architecture based on state space modeling (SSM) that has been shown to model long-term dependency effectively. Owing to SSM, fewer dilated convolutional layers are required to capture temporal dependency in Conv-TasNet, resulting in the reduction of model complexity. We also enlarge the window length and shift of the convolutional (TasNet) frontend encoder to reduce the computational cost further; the performance decline is compensated by over-parameterization of the frontend encoder. The proposed method reduces the real-time factor by 78% from the conventional causal Conv-TasNet-based TSE while matching its performance.

eess.AS↗

arXiv2024

Invariant quasimorphisms and generalized mixed Bavard duality

This article provides an expository account of the celebrated duality theorem of Bavard and three its strengthenings. The Bavard duality theorem connects scl (stable commutator length) and quasimorphisms on a group. Calegari extended the framework from a group element to a chain on the group, and established the generalized Bavard duality. Kawasaki, Kimura, Matsushita and Mimura studied the setting of a pair of a group and its normal subgroup, and obtained the mixed Bavard duality. The first half of the present article is devoted to an introduction to these three Bavard dualities. In the latter half, we present a new strengthening, the generalized mixed Bavard duality, and provide a self-contained proof of it. This third strengthening recovers all of the Bavard dualities treated in the first half; thus, we supply complete proofs of these four Bavard dualities in a unified manner. In addition, we state several results on the space $\mathrm{W}(G,N)$ of non-extendable quasimorphisms, which is related to the comparison problem between scl and mixed scl via the mixed Bavard duality.

math.GT↗

arXiv2023

Coarse group theoretic study on stable mixed commutator length

Let $G$ be a group and $N$ a normal subgroup of $G$. We study the large scale behavior, not the exact values themselves, of the stable mixed commutator length $scl_{G,N}$ on the mixed commutator subgroup $[G,N]$; when $N=G$, $scl_{G,N}$ equals the stable commutator length $scl_G$ on the commutator subgroup $[G,G]$. For this purpose, we regard $scl_{G,N}$ not only as a function from $[G,N]$ to $\mathbb{R}_{\geq 0}$, but as a bi-invariant metric function $d^+_{scl_{G,N}}$ from $[G,N]\times [G,N]$ to $\mathbb{R}_{\geq 0}$. Our main focus is coarse group theoretic structures of $([G,N],d^+_{scl_{G,N}})$. Our preliminary result (the absolute version) connects, via the Bavard duality, $([G,N],d^+_{scl_{G,N}})$ and the quotient vector space of the space of $G$-invariant quasimorphisms on $N$ over one of such homomorphisms. In particular, we prove that the dimension of this vector space equals the asymptotic dimension of $([G,N],d^+_{scl_{G,N}})$. Our main result is the comparative version: we connect the coarse kernel, formulated by Leitner and Vigolo, of the coarse homomorphism $\iota_{G,N}\colon ([G,N],d^+_{scl_{G,N}})\to ([G,N],d^+_{scl_{G}})$; $y\mapsto y$, and a certain quotient vector space $W(G,N)$ of the space of invariant quasimorphisms. Assume that $N=[G,G]$ and that $W(G,N)$ is finite dimensional with dimension $\ell$. Then we prove that the coarse kernel of $\iota_{G,N}$ is isomorphic to $\mathbb{Z}^{\ell}$ as a coarse group. In contrast to the absolute version, the space $W(G,N)$ is finite dimensional in many cases, including all $(G,N)$ with finitely generated $G$ and nilpotent $G/N$. As an application of our result, given a group homomorphism $\varphi\colon G\to H$ between finitely generated groups, we define an $\mathbb{R}$-linear map `inside' the groups, which is dual to the naturally defined $\mathbb{R}$-linear map from $W(H,[H,H])$ to $W(G,[G,G])$ induced by $\varphi$.

math.GR↗

arXiv2023

Time-domain Speech Enhancement Assisted by Multi-resolution Frequency Encoder and Decoder

Time-domain speech enhancement (SE) has recently been intensively investigated. Among recent works, DEMUCS introduces multi-resolution STFT loss to enhance performance. However, some resolutions used for STFT contain non-stationary signals, and it is challenging to learn multi-resolution frequency losses simultaneously with only one output. For better use of multi-resolution frequency information, we supplement multiple spectrograms in different frame lengths into the time-domain encoders. They extract stationary frequency information in both narrowband and wideband. We also adopt multiple decoder outputs, each of which computes its corresponding resolution frequency loss. Experimental results show that (1) it is more effective to fuse stationary frequency features than non-stationary features in the encoder, and (2) the multiple outputs consistent with the frequency loss improve performance. Experiments on the Voice-Bank dataset show that the proposed method obtained a 0.14 PESQ improvement.

cs.SD↗

arXiv2022

Survey on invariant quasimorphisms and stable mixed commutator length

A homogeneous quasimorphism $\phi$ on a normal subgroup $N$ of $G$ is said to be $G$-invariant if $\phi(gxg^{-1}) = \phi(x)$ for every $g \in G$ and for every $x \in N$. Invariant quasimorphisms have naturally appeared in symplectic geometry and the extension problem of quasimorphisms. Moreover, it is known that the existence of non-extendable invariant quasimorphisms is closely related to the behavior of the stable mixed commutator length $\mathrm{scl}_{G,N}$, which is a certain generalization of the stable commutator length $\mathrm{scl}_G$. In this survey, we review the history and recent developments of invariant quasimorphisms and stable mixed commutator length. The topics we treat include several examples of invariant quasimorphisms, Bavard's duality theorem for invariant quasimorphisms, Aut-invariant quasimorphisms, and the estimation of the dimension of spaces of non-extendable quasimorphisms. We also mention the extension problem of partial quasimorphisms.

math.GR↗