Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2609.35347

Beyond Teacher Assignment: Domain-Normalized Multi-Teacher On-Policy Distillation

Xin Li·Hao Jiang·Xin Gao·Annan Wang·Yuchen Xie·Jinghao Guo·Xingwei Qu·Yichi Zhang·Chau Yuen
View original source ↗Read source document ↗Export metadata ↓

Abstract

Reinforcement learning can turn one language model into several specialists, each excellent at a single skill such as mathematics, coding or following instructions, but users need one model with all of these skills. Multi-teacher on-policy distillation (MOPD) merges them by letting the specialists teach one student: the student answers each prompt, and the specialist for that prompt's domain gives feedback on every token. This routing decides which specialist teaches, but not how strongly its feedback moves the shared student. In Qwen3.5 models at three sizes, we find that MOPD's student does not beat one taught by the best single specialist and gains little of the mathematics specialist's advantage. The feedback is unbalanced: instruction-following feedback is several times more spread out than mathematics feedback and dominates the student's updates. We propose Domain-Normalized MOPD (DN-MOPD), which keeps the routing and rescales each domain's feedback by its measured spread. On six public benchmarks, DN-MOPD improves the average score over MOPD at every size, across three random seeds and under two answer-length limits, and recovers most of the lost mathematics gain. Controls with fixed domain weights show that the gain comes mainly from turning down instruction-following feedback rather than turning up mathematics alone, and that fixed weights close to those DN-MOPD measures perform comparably. Combining specialists therefore requires deciding not only which one teaches, but also how strongly its feedback counts.

Explore related subjects

cs.LG

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Xin Li, Hao Jiang, Xin Gao, Annan Wang, Yuchen Xie, Jinghao Guo, Xingwei Qu, Yichi Zhang, Chau Yuen. 2026-09-28. Beyond Teacher Assignment: Domain-Normalized Multi-Teacher On-Policy Distillation. https://arxiv.org/abs/2609.35347

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Efficient Policy Evaluation with Offline Data Informed Behavior Policy Design

Online Monte Carlo evaluation is a fundamental tool for assessing policy performance in reinforcement learning and sequential decision-making problems arising in operations research. However, achieving accurate estimates often requires extensive online interaction with the environment, which can be costly or impractical in many real-world settings. In this paper, we develop a framework that improves the sample efficiency of online Monte Carlo estimators while preserving unbiasedness. We first derive a closed-form optimal behavior policy that minimizes estimator variance under unbiasedness constraints. We then propose practical algorithms for learning the proposed behavior policy from previously collected offline data, enabling improved online evaluation without requiring estimation of the environment transition model. We provide theoretical analysis that quantifies the resulting variance reduction and analyzes the impact of approximation errors. Empirical studies across diverse environments demonstrate substantial improvements in online sample efficiency compared with standard on-policy Monte Carlo evaluation and existing baseline methods. Our results provide a unified framework for optimal behavior policy design in off-policy evaluation, with applications to reinforcement learning and operations research.

cs.LG↗
arXiv2026

MONOVAB : An Annotated Corpus for Bangla Multi-label Emotion Detection

In recent years, Sentiment Analysis (SA) and Emotion Recognition (ER) have been increasingly popular in the Bangla language, which is the seventh most spoken language throughout the entire world. However, the language is structurally complicated, which makes this field arduous to extract emotions in an accurate manner. Several distinct approaches such as the extraction of positive and negative sentiments as well as multiclass emotions, have been implemented in this field of study. Nevertheless, the extraction of multiple sentiments is an almost untouched area in this language. Which involves identifying several feelings based on a single piece of text. Therefore, this study demonstrates a thorough method for constructing an annotated corpus based on scrapped data from Facebook to bridge the gaps in this subject area to overcome the challenges. To make this annotation more fruitful, the context-based approach has been used. Bidirectional Encoder Representations from Transformers (BERT), a well-known methodology of transformers, have been shown the best results of all methods implemented. Finally, a web application has been developed to demonstrate the performance of the pre-trained top-performer model (BERT) for multi-label ER in Bangla.

cs.LG↗
arXiv2026

Online Regularized Statistical Learning in Reproducing Kernel Hilbert Space With Non-Stationary Data

We study recursive regularized learning algorithms in the reproducing kernel Hilbert space (RKHS) with non-stationary online data streams. We introduce the concept of a random Tikhonov regularization path and decompose the tracking error of the algorithm's output for the regularization path into random difference equations in RKHS. We show that the tracking error vanishes in mean square and almost surely if the regularization path is slowly time-varying. Then, leveraging the monotonicity of inverse operators and the spectral decomposition of compact operators, and introducing the RKHS persistence of excitation condition, we develop a dominated convergence method to prove the mean square and almost sure consistency between the regularization path and the unknown function to be learned. Especially, for independent and non-identically distributed data streams, the mean square and almost sure consistency between the algorithm's output and the unknown function is achieved if the input data's marginal probability measures are slowly time-varying and the average measure over each fixed-length time period is uniformly above a strictly positive finite Borel measure.

cs.LG↗