Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.14394

Understanding and Mitigating Training Instability in Large-Scale Recommender Systems: A Cascade Perspective

Yuening Li·Zeyu Wang·Zhen Zhang·Jiayi Liu·Zhimeng Jiang·Yuji Roh·Weikai Xu·Mingyan Gao·Onkar Dalal·Chuan He·Liang Liu
View original source ↗Read source document ↗Export metadata ↓

Abstract

Training large-scale recommender systems on dynamic industrial platforms, such as Short-Form Video (SFV) services with billions of daily interactions, faces critical stability challenges. Rapid distribution shifts in user data and increasing model complexity frequently trigger a divergence cascade, typically initiating with gradient explosion, propagating to weight explosion, and culminating in loss explosion. This G-W-L cascade severely impedes model convergence and performance. This paper provides a systematic analysis of this G-W-L cascade within an industrial SFV recommender context. We then propose and evaluate a structured mitigation framework where interventions are explicitly mapped to counteract specific stages of the cascade: gradient (G) control, weight/activation (W) stabilization, and loss (L) shaping. Our contribution provides a structured framework for understanding and tackling training divergence, offering practical insights for deploying robust, continuously updated recommenders in large-scale, non-stationary settings. We empirically validate the effectiveness of these strategies on a major industrial SFV recommendation system.

Explore related subjects

cs.IRcs.LG

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Yuening Li, Zeyu Wang, Zhen Zhang, Jiayi Liu, Zhimeng Jiang, Yuji Roh, Weikai Xu, Mingyan Gao, Onkar Dalal, Chuan He, Liang Liu. 2026-10-09. Understanding and Mitigating Training Instability in Large-Scale Recommender Systems: A Cascade Perspective. https://arxiv.org/abs/2610.14394

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Bridging Search Intent and Tabular Data in Dataset Search

Dataset search remains a bottleneck in data analysis workflows, as existing systems largely rely on keyword matching or shallow retrieval that fail to capture user intent. To facilitate intent-driven search, we present two retrieval components to equip multi-faceted dataset search interfaces through natural language. First, we propose Hypothetical Schema Embeddings (HySE) that uses a language model to generate task-aligned hypothetical table schemas which are used for embedding-based retrieval, bridging the gap between intent and table content. We introduce KaggleDS, a corpus comprising 6,503 tables and diverse task-queries derived from Kaggle, and show that HySE consistently improves recall over strong keyword and embedding-based baselines, as well as over a dense table retriever fine-tuned on KaggleDS. We also show that HySE generalizes to end-to-end question answering on, for example, the NQ-Tables dataset, a corpus of 169,898 Wikipedia tables with human-written queries, where HySE improves Recall@10 by +6.0 pp over dense retrieval baselines. Second, we introduce FlexPrune, which translates dataset constraints expressed in natural-language into SQL predicates over a customizable metadata index, enabling filtering across arbitrary metadata attributes. Combined with HySE, FlexPrune increases Recall@10 by +6.0 pp over dense retrieval followed by tag-based filtering. Our results show that natural-language-based retrieval supporting task and metadata queries offers a viable alternative to schema-level keyword search in dataset search.

cs.IR↗
arXiv2026

Predicting What Answer Engines Cite: Open Decision Models, Rerankers and a Distilled 4B Student on Ten Engines under Controlled Retrieval

Generative engine optimization needs a scorer that ranks candidate pages by the share of an answer they will receive as citations. We measure pretrained rerankers, a hosted typed decision model (Jev), eight open decision models and distilled students on GEO-Bench queries answered by ten engines that receive the same candidate sources in their prompt. Where several engines answer the same query, they agree with one another at a within-group Spearman correlation of 0.704. Jev and two open 27B decision models, used without training, are the strongest scorers at 0.575 to 0.578. A Qwen3-Reranker-4B fine-tuned with low-rank adapters on teacher labels for other GEO-Bench queries, released as GEO Citation Ranker 4B, reaches 0.542 to 0.549 across three seeds, and its differences with the 27B models and with Jev are not distinguishable from zero on 229 queries. Most of the remaining distance between the student and Jev disappears when Jev scores one passage at a time, as the student does. Students trained on teacher labels and on observed engine citations reach the same level, and labels from Jev alone match the two-teacher target at 0.6B. We release the model, the benchmark, the predictions of every reported scorer and the code.

cs.IR↗
arXiv2026

Role-Augmented Intent-Driven Generative Search Engine Optimization

Generative search engines (GSEs), which combine large language models (LLMs) with retrieval-augmented generation (RAG), are reshaping how users access Web information and making source visibility and influence in generated responses new optimization objectives. Existing generative search engine optimization (G-SEO) methods seek to improve content visibility, but many rely on direct access to target queries or GSE feedback elicited by those queries. However, Web content is typically optimized and published before future user queries are known, making G-SEO without access to target queries or their associated feedback an important yet underexplored problem. To address this, we propose RAID, a Role-Augmented Intent-Driven G-SEO method that operates without access to target queries or query-dependent GSE feedback. RAID infers a primary retrieval intent from source content, refines it through 4W multi-role deep reflection on prospective searchers' information needs, and uses both the primary and refined intents to jointly plan and rewrite source content. We further extend GEO-BENCH with diverse topic-related query variants, using both the original queries and their variants exclusively for evaluation to assess generalization to queries unseen during optimization. Across 100 samples and 500 evaluation queries, RAID achieves Relative Improvement Scores (RIS) of 3.00\% for Objective Overall and 1.32\% for Subjective Average, exceeding the best corresponding results among the nine predefined GEO rewriting strategies by 1.31 and 0.74 percentage points, respectively. With an optimization considered effective when $\mathrm{RIS}>0$, its effective optimization rates for Objective Overall and Subjective Average average 58.0\%, exceeding the highest corresponding average among the predefined strategies by 4.5 percentage points.

cs.IR↗