Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2607.17544

X-Translator: A Real-Time Multilingual Speaker-Aware Speech-to-Speech Translation System

Yuxiang Zhao·Yichi Zhang·Yanjie An·Yanqiao Zhu·Zhanxun Liu·Yushen Chen·Qixi Zheng·Haina Zhu·Yunchong Xiao·Keqi Deng·Shuai Fan·Kai Yu·Xie Chen
View original source ↗Read source document ↗Export metadata ↓

Abstract

Real-time speech-to-speech translation (S2ST) systems must balance translation quality, latency, speech naturalness, and speaker consistency. Publicly documented S2ST systems have advanced direct, multilingual, streaming, and expressive modeling, while proprietary products and APIs increasingly expose real-time translation capabilities to users. However, practical deployment remains challenging for open and reproducible systems, especially in long-form and multi-speaker conversations where partial ASR hypotheses are unstable, turn boundaries are ambiguous, and target speech must be generated with an appropriate speaker prompt. We present X-Translator, a low-cost modular cascaded S2ST system that combines streaming ASR, machine translation, and prompt-conditioned TTS through a session-level runtime controller. The system uses incremental segment commitment to convert unstable ASR streams into translation-ready units, and an online speaker prompt manager to bind source speech spans to speaker-specific voice prompts for synthesis. We evaluate translation, speech quality, and latency with OpenSTBench, compare against proprietary speech translation APIs as behavioral baselines, measure long-form voice stability, evaluate speaker preservation in multi-speaker conversations, and assess multilingual translation quality. X-Translator provides an open platform for understanding the practical trade-offs of deployment-oriented S2ST. Code and demo are available at https://github.com/zhaoyx239/X-Translator.

Explore related subjects

eess.AScs.AI

Keep this discovery

BibTeXRIS

Yuxiang Zhao, Yichi Zhang, Yanjie An, Yanqiao Zhu, Zhanxun Liu, Yushen Chen, Qixi Zheng, Haina Zhu, Yunchong Xiao, Keqi Deng, Shuai Fan, Kai Yu, Xie Chen. 2026-07-20. X-Translator: A Real-Time Multilingual Speaker-Aware Speech-to-Speech Translation System. https://arxiv.org/abs/2607.17544

Cite the original work for its findings. Save a collection to share your selection of sources.

Discover connections

cs.AI subjecteess.AS subject

Connections use source metadata and explicit phrase matches, not verified experimental comparisons.

KEEP EXPLORING

Related papers

arXiv2026

A Closed-Loop Thermal Dynamic Model for AI Data Center Cooling Load Simulation

Cooling demand constitutes a significant and flexible component of AI data center electricity consumption, but time-synchronized measurements are scarce and constant coefficient-of-performance models cannot represent thermal dynamics. This letter proposes a closed-loop simulation model which couples a linear thermal dynamic model with deadband-based control to capture the nonlinear cooling dynamics. The model is validated using operational telemetry from the Marconi100 supercomputer. Compared with the baseline, the proposed model reduces the mean absolute error from 95.80 to 20.88~kW and the root-mean-square error from 109.79 to 27.27~kW. Evaluation over approximately 520 daily profiles further shows improved reproduction of daily peak demand and intraday variability. The proposed model provides a computationally tractable means of generating physically interpretable cooling load profiles for power system studies.

eess.SY↗
arXiv2026

Decoupled Data Consistency with Diffusion Purification for Image Restoration

Diffusion models have recently gained traction as a powerful class of deep generative priors, excelling in a wide range of image restoration tasks due to their exceptional ability to model data distributions. To solve image restoration problems, many existing techniques achieve data consistency by incorporating additional likelihood gradient steps into the reverse sampling process of diffusion models. However, the additional gradient steps pose a challenge for real-world practical applications as they incur a large computational overhead, thereby increasing inference time. They also present additional difficulties when using accelerated diffusion model samplers, as the number of data consistency steps is limited by the number of reverse sampling steps. In this work, we propose a novel diffusion-based image restoration solver that addresses these issues by decoupling the reverse process from the data consistency steps. Our method involves alternating between a reconstruction phase to maintain data consistency and a refinement phase that enforces the prior via diffusion purification. Our approach demonstrates versatility, making it highly adaptable for efficient problem-solving in latent space. Additionally, it reduces the necessity for numerous sampling steps through the integration of consistency models. The efficacy of our approach is validated through comprehensive experiments across various image restoration tasks, including image denoising, deblurring, inpainting, and super-resolution.

eess.IV↗
arXiv2026

Secure AI-Driven Super-Resolution for Real-Time Mixed Reality Applications

Immersive formats such as 360° and 6DoF point cloud videos require high bandwidth and low latency, posing challenges for real-time AR/VR streaming. This work focuses on reducing bandwidth consumption and encryption/decryption delay, two key contributors to overall latency. We design a system that downsamples point cloud content at the origin server and applies partial encryption. At the client, the content is decrypted and upscaled using an ML-based super-resolution model. Our evaluation demonstrates a nearly linear reduction in bandwidth/latency, and encryption/decryption overhead with lower downsampling resolutions, while the super-resolution model effectively reconstructs the original full-resolution point clouds with minimal error and modest inference time.

cs.CR↗