arXiv subjects

Xiaofeng Wang

Publications and source records attributed to Xiaofeng Wang.

At least 145 records · Page 8Linked to original sources

arXiv2025

Text-to-SQL for Enterprise Data Analytics

The introduction of large language models has brought rapid progress on Text-to-SQL benchmarks, but it is not yet easy to build a working enterprise solution. In this paper, we present insights from building an internal chatbot that enables LinkedIn's product managers, engineers, and operations teams to self-serve data insights from a large, dynamic data lake. Our approach features three components. First, we construct a knowledge graph that captures up-to-date semantics by indexing database metadata, historical query logs, wikis, and code. We apply clustering to identify relevant tables for each team or product area. Second, we build a Text-to-SQL agent that retrieves and ranks context from the knowledge graph, writes a query, and automatically corrects hallucinations and syntax errors. Third, we build an interactive chatbot that supports various user intents, from data discovery to query writing to debugging, and displays responses in rich UI elements to encourage follow-up chats. Our chatbot has over 300 weekly users. Expert review shows that 53% of its responses are correct or close to correct on an internal benchmark set. Through ablation studies, we identify the most important knowledge graph and modeling components, offering a practical path for developing enterprise Text-to-SQL solutions.

cs.CL↗

arXiv2025

ReconDreamer++: Harmonizing Generative and Reconstructive Models for Driving Scene Representation

Combining reconstruction models with generative models has emerged as a promising paradigm for closed-loop simulation in autonomous driving. For example, ReconDreamer has demonstrated remarkable success in rendering large-scale maneuvers. However, a significant gap remains between the generated data and real-world sensor observations, particularly in terms of fidelity for structured elements, such as the ground surface. To address these challenges, we propose ReconDreamer++, an enhanced framework that significantly improves the overall rendering quality by mitigating the domain gap and refining the representation of the ground surface. Specifically, ReconDreamer++ introduces the Novel Trajectory Deformable Network (NTDNet), which leverages learnable spatial deformation mechanisms to bridge the domain gap between synthesized novel views and original sensor observations. Moreover, for structured elements such as the ground surface, we preserve geometric prior knowledge in 3D Gaussians, and the optimization process focuses on refining appearance attributes while preserving the underlying geometric structure. Experimental evaluations conducted on multiple datasets (Waymo, nuScenes, PandaSet, and EUVS) confirm the superior performance of ReconDreamer++. Specifically, on Waymo, ReconDreamer++ achieves performance comparable to Street Gaussians for the original trajectory while significantly outperforming ReconDreamer on novel trajectories. In particular, it achieves substantial improvements, including a 6.1% increase in NTA-IoU, a 23. 0% improvement in FID, and a remarkable 4.5% gain in the ground surface metric NTL-IoU, highlighting its effectiveness in accurately reconstructing structured elements such as the road surface.

cs.CV↗

arXiv2025

EmbodieDreamer: Advancing Real2Sim2Real Transfer for Policy Training via Embodied World Modeling

The rapid advancement of Embodied AI has led to an increasing demand for large-scale, high-quality real-world data. However, collecting such embodied data remains costly and inefficient. As a result, simulation environments have become a crucial surrogate for training robot policies. Yet, the significant Real2Sim2Real gap remains a critical bottleneck, particularly in terms of physical dynamics and visual appearance. To address this challenge, we propose EmbodieDreamer, a novel framework that reduces the Real2Sim2Real gap from both the physics and appearance perspectives. Specifically, we propose PhysAligner, a differentiable physics module designed to reduce the Real2Sim physical gap. It jointly optimizes robot-specific parameters such as control gains and friction coefficients to better align simulated dynamics with real-world observations. In addition, we introduce VisAligner, which incorporates a conditional video diffusion model to bridge the Sim2Real appearance gap by translating low-fidelity simulated renderings into photorealistic videos conditioned on simulation states, enabling high-fidelity visual transfer. Extensive experiments validate the effectiveness of EmbodieDreamer. The proposed PhysAligner reduces physical parameter estimation error by 3.74% compared to simulated annealing methods while improving optimization speed by 89.91\%. Moreover, training robot policies in the generated photorealistic environment leads to a 29.17% improvement in the average task success rate across real-world tasks after reinforcement learning. Code, model and data will be publicly available.

cs.RO↗

arXiv2025

WonderFree: Enhancing Novel View Quality and Cross-View Consistency for 3D Scene Exploration

Interactive 3D scene generation from a single image has gained significant attention due to its potential to create immersive virtual worlds. However, a key challenge in current 3D generation methods is the limited explorability, which cannot render high-quality images during larger maneuvers beyond the original viewpoint, particularly when attempting to move forward into unseen areas. To address this challenge, we propose WonderFree, the first model that enables users to interactively generate 3D worlds with the freedom to explore from arbitrary angles and directions. Specifically, we decouple this challenge into two key subproblems: novel view quality, which addresses visual artifacts and floating issues in novel views, and cross-view consistency, which ensures spatial consistency across different viewpoints. To enhance rendering quality in novel views, we introduce WorldRestorer, a data-driven video restoration model designed to eliminate floaters and artifacts. In addition, a data collection pipeline is presented to automatically gather training data for WorldRestorer, ensuring it can handle scenes with varying styles needed for 3D scene generation. Furthermore, to improve cross-view consistency, we propose ConsistView, a multi-view joint restoration mechanism that simultaneously restores multiple perspectives while maintaining spatiotemporal coherence. Experimental results demonstrate that WonderFree not only enhances rendering quality across diverse viewpoints but also significantly improves global coherence and consistency. These improvements are confirmed by CLIP-based metrics and a user study showing a 77.20% preference for WonderFree over WonderWorld enabling a seamless and immersive 3D exploration experience. The code, model, and data will be publicly available.

cs.CV↗

arXiv2025

GigaVideo-1: Advancing Video Generation via Automatic Feedback with 4 GPU-Hours Fine-Tuning

Recent progress in diffusion models has greatly enhanced video generation quality, yet these models still require fine-tuning to improve specific dimensions like instance preservation, motion rationality, composition, and physical plausibility. Existing fine-tuning approaches often rely on human annotations and large-scale computational resources, limiting their practicality. In this work, we propose GigaVideo-1, an efficient fine-tuning framework that advances video generation without additional human supervision. Rather than injecting large volumes of high-quality data from external sources, GigaVideo-1 unlocks the latent potential of pre-trained video diffusion models through automatic feedback. Specifically, we focus on two key aspects of the fine-tuning process: data and optimization. To improve fine-tuning data, we design a prompt-driven data engine that constructs diverse, weakness-oriented training samples. On the optimization side, we introduce a reward-guided training strategy, which adaptively weights samples using feedback from pre-trained vision-language models with a realism constraint. We evaluate GigaVideo-1 on the VBench-2.0 benchmark using Wan2.1 as the baseline across 17 evaluation dimensions. Experiments show that GigaVideo-1 consistently improves performance on almost all the dimensions with an average gain of about 4% using only 4 GPU-hours. Requiring no manual annotations and minimal real data, GigaVideo-1 demonstrates both effectiveness and efficiency. Code, model, and data will be publicly available.

cs.CV↗

arXiv2025

A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment

The remarkable success of Large Language Models (LLMs) has illuminated a promising pathway toward achieving Artificial General Intelligence for both academic and industrial communities, owing to their unprecedented performance across various applications. As LLMs continue to gain prominence in both research and commercial domains, their security and safety implications have become a growing concern, not only for researchers and corporations but also for every nation. Currently, existing surveys on LLM safety primarily focus on specific stages of the LLM lifecycle, e.g., deployment phase or fine-tuning phase, lacking a comprehensive understanding of the entire "lifechain" of LLMs. To address this gap, this paper introduces, for the first time, the concept of "full-stack" safety to systematically consider safety issues throughout the entire process of LLM training, deployment, and eventual commercialization. Compared to the off-the-shelf LLM safety surveys, our work demonstrates several distinctive advantages: (I) Comprehensive Perspective. We define the complete LLM lifecycle as encompassing data preparation, pre-training, post-training, deployment and final commercialization. To our knowledge, this represents the first safety survey to encompass the entire lifecycle of LLMs. (II) Extensive Literature Support. Our research is grounded in an exhaustive review of over 800+ papers, ensuring comprehensive coverage and systematic organization of security issues within a more holistic understanding. (III) Unique Insights. Through systematic literature analysis, we have developed reliable roadmaps and perspectives for each chapter. Our work identifies promising research directions, including safety in data generation, alignment techniques, model editing, and LLM-based agent systems. These insights provide valuable guidance for researchers pursuing future work in this field.

cs.CR↗

arXiv2025

A post-common-envelope binary with double-peaked Balmer emission lines from TMTS

The dynamical method provides an efficient way to discover post-common-envelope binaries (PCEB) with faint white dwarfs (WDs), thanks to the development of time-domain survey projects. We perform a comprehensive analysis of the PCEB system TMTS J15530469+4457458 (J1553), discovered by the Tsinghua University-Ma Huateng Telescopes for Survey, to explore its physical origin and evolutionary fate. This system is characterized by double-peaked Balmer emission lines, and a cross-correlation function is applied to derive its radial velocity (RV) from a series of phase-resolved Keck spectra. Analyses with the cross-correlation function suggest that this system is a single-lined spectroscopic binary and only one star is optically visible. Further analysis through Doppler tomography indicates that J1553 is a detached binary without an accretion disk. Under such a configuration, the simultaneous light-curve and RV fitting reveal that this system contains an unseen WD with mass $M_{\rm A}=0.56\pm 0.09\, M_{\odot}$, and an M4 dwarf with mass $M_{\rm B}=0.37\pm 0.02\,M_{\odot}$ and radius $R_{\rm B}=0.403^{+0.014}_{-0.015}\,R_{\odot}$. The extra prominent Balmer emission lines seen in the spectra can trace the motion of the WD, which are likely formed near the WD surface as a result of wind accretion. According to the MESA simulation, J1553 could have evolved from a binary consisting of a 2.0-4.0 ${M}_{\odot}$ zero-age-main-sequence star and an M dwarf with an initial orbital period $P_i\approx 201-476$ d, and the system has undergone a common-envelope (CE) phase. After about $3.3\times10^6$ yr, J1553 should evolve into a cataclysmic variable, with a transient state as a supersoft X-ray source at the beginning. J1553 is an excellent system for studying wind accretion, CE ejection physics, and binary evolution theory.

astro-ph.SR↗

arXiv2025

SN 2021hpr: A Normal Type Ia Supernova Showing Excess Emission in the Early Rising Phase

We present extensive optical observations of a nearby Type Ia supernova (SN Ia), SN 2021hpr, located in the spiral galaxy NGC 3147 at a distance of $\sim$ 45 Mpc. Our observations cover a phase within $\sim 1-2$ days to $\sim 290$ days after the explosion. SN 2021hpr is found to be a spectroscopically normal SN Ia, with an absolute B-band peak magnitude of $M_{max}(B) \approx -19.16 \pm 0.14$ mag and a post-peak decline rate of $Δm_{15}(B)= 1.00 \pm 0.01 $ mag. Early-time light curves showed a $\sim 7.0 \%$ excess emission compared to a homogeneously expanding fireball model, likely due to SN ejecta interacting with a companion or immediate circumstellar matter. The optical spectra of SN 2021hpr are overall similar to those of normal SNe Ia, but characterized by prominent detached high-velocity features (HVFs) of Si {\sc ii} and Ca {\sc ii} in the early phase. After examining a small sample of well-observed normal SNe Ia, we find that the HVFs are likely common for the subgroup with early-excess emission. The association of early bump feature with the HVFs could be attributed to density or abundance enhancement at the outer layer of the exploding star, likely as a result of interactions with companion$/$CSM or experiencing more complete burning. Nevertheless, the redshifted Fe {\sc ii} and Ni {\sc ii} lines in the nebular-phase spectra of SN 2021hpr, contrary to the blueshift trend seen in other SNe Ia showing early bump features, indicate its peculiarity in the explosion that remains to be understood.

astro-ph.HE↗

arXiv2025

Investigating the Period-Luminosity Relations of delta Scuti Stars: A Pathway to Distance and 3-D Dust Map Inference

While delta Scuti stars are the most numerous class of kappa-mechanism pulsators in the instability strip, the short periods and small peak-to-peak amplitudes have left them understudied and underutilized. Recently, large-scale time-domain surveys have significantly increased the number of identified delta Scuti stars. Notably, the Tsinghua University-Ma Huateng Telescopes for Survey (TMTS), with its high-cadence observations at 1-minute intervals, has identified thousands of delta Scuti stars, greatly expanding the sample of these short-period pulsating variables. Using the delta Scuti stars from the TMTS catalogs of Periodic Variable Stars, we cross-matched the dataset with Pan-STARRS1, 2MASS, and WISE to obtain photometric measurements across optical and infrared bands. Parallax data, used as Bayesian priors, were retrieved from Gaia DR3, and line-of-sight dust extinction priors were estimated from a three-dimensional dust map. Using PyMC, we performed a simultaneous determination of the 11-band P-L relations of delta Scuti stars, which not only yields precise measurements of these relations, but also greatly improves constraints on the distance moduli and color excesses, as evidenced by the reduced uncertainties in the posterior distributions. Furthermore, our methodology enables an independent estimation of the color excess through the P-L relations, offering a potential complement to existing 3-D dust maps. Moreover, by cross-matching with LAMOST DR7, we investigated the influence of metallicity on the P-L relations. Our analysis reveals that incorporating metallicity might reduce the intrinsic scatter at longer wavelengths. However, this result does not achieve 3 sigma significance, leaving open the possibility that the observed reduction is attributable to statistical fluctuations.

astro-ph.SR↗

arXiv2025

Task-Parameter Nexus: Task-Specific Parameter Learning for Model-Based Control

This paper presents the Task-Parameter Nexus (TPN), a learning-based approach for online determination of the (near-)optimal control parameters of model-based controllers (MBCs) for tracking tasks. In TPN, a deep neural network is introduced to predict the control parameters for any given tracking task at runtime, especially when optimal parameters for new tasks are not immediately available. To train this network, we constructed a trajectory bank with various speeds and curvatures that represent different motion characteristics. Then, for each trajectory in the bank, we auto-tune the optimal control parameters offline and use them as the corresponding ground truth. With this dataset, the TPN is trained by supervised learning. We evaluated the TPN on the quadrotor platform. In simulation experiments, it is shown that the TPN can predict near-optimal control parameters for a spectrum of tracking tasks, demonstrating its robust generalization capabilities to unseen tasks.

cs.RO↗

arXiv2025

Supernovae at Distances < 40 Mpc: II. Supernova Rate in the Local Universe

Context.This is the second paper of a series aiming to determine the birth rates of supernovae in the local Universe. Aims. In this paper, we aim to estimate the SN rates in the local universe and fit the delay-time distribution of SNe Ia to put constraints on their progenitor scenarios. Methods.We performed a Monte-Carlo simulation to estimate the volumetric rates with the nearby SN sample introduced in Paper I of the series. The rate evolution of core-collapse SNe well traces the evolution of cosmic star formation history; while that of SNe Ia involves the convolution of cosmic star-formation history and a two-component delay-time distribution including a power law and a Gaussian component. Results.The volumetric rates of type Ia, Ibc and II SNe are derived as $0.325\pm0.040^{+0.016}_{-0.010}$, $0.160\pm0.028^{+0.044}_{-0.014}$, and $0.528\pm0.051^{+0.162}_{-0.013}$ (in unit of $10^{-4} yr^{-1} Mpc^{-3} h^3_{70}$), respectively. The rate of CCSNe is consistent with previous estimates. The newly derived local SN Ia rate is larger than existing results given at redshifts 0.01 < z < 0.1, favoring an increased rate from the universe at z ~ 0.1 to the local universe. A two-component model can well fit the rate variation, with the power law component accounting for the rate evolution at larger redshifts and the Gaussian component with a delay time of 12.63$\pm$0.38 Gyr accounting for the local rate evolution. This delayed component with such a longer delay time suggests that the progenitors of these SNe Ia were formed at around 1 Gyr after the birth of the universe, which could only be explained by a double-degenerate progenitor scenario. This is evidenced by the comparison with the PTF sample of SNe Ia at z = 0.073, which reveals that the increase in SN Ia rate at z < 0.01 is primarily due to the SNe Ia of massive E and S0 galaxies with old stellar populations.

astro-ph.HE↗

arXiv2025

Supernovae at Distances < 40 Mpc: I.Catalogues and fractions of Supernovae in a Complete Sample

Context.This is the first paper of a series aiming to determine the fractions and birth rates of various types of supernovae (SNe) in the local Universe. Aims. In this paper, we aim to construct a complete sample of SNe in the nearby universe and provide more precise measurement of subtype fractions. Methods.We carefully selected our SN sample at a distance of < 40 Mpc mainly from wide-field surveys conducted over the years from 2016 to 2023. Results.The sample contains a total of 211 SNe, including 109 SNe II, 69 SNe Ia, and 33 SNe Ibc. With the aid of sufficient spectra, we can obtain relatively accurate subtype classifications for all SNe in this sample. After corrections for the Malmquist bias, this volume-limited sample gives fractions of SNe Ia, SNe Ibc, and SNe II as $30.4^{+3.7}_{-11.5}\%$, $16.3^{+3.7}_{-7.4}\%$, and $53.3^{+9.5}_{-18.7}\%$, respectively.In the SN Ia sample, the fraction of the 91T-like subtype becomes relatively low (~5.4\%), while that of the 02cx-like subtype shows a moderate increase (~6.8\%). In the SN Ibc sample, we find significant fractions of broadlined SNe Ic (~18.0\%) and SNe Ibn (~8.8\%). The fraction of 87A-like subtype is determined as ~2.3\% for the first time, indicating rare explosions from blue supergiant stars. We find that SNe Ia show a double peak number distribution in S0- and Sc-type host galaxies, which may serve as a straightforward evidence for the presence of "prompt" and "delayed" progenitor components giving rise to SN Ia explosions. Several subtypes of SNe such as 02cx-like SNe Ia, broadlined SNe Ic, SNe IIn (and perhaps SNe Ibn) are found to occur preferentially in less massive spiral galaxies, favoring their associations with young stellar progenitors. Moreover, the 02cx-like subtype shows a trend of exploding in the outer skirt of their hosts, suggestive of metal-poor progenitors.

astro-ph.HE↗

arXiv2025

Optical and Near-infrared Observations of SN 2023ixf for over 600 days after the Explosion

Context.We present a comprehensive photometric and spectroscopic study of the nearby Type II supernova (SN) 2023ixf, with our extensive observations spanning the phases from ~3 to over 600 days after the first light.\\ Aims.The aim of this study is to obtain key information on the explosion properties of SN\,2023ixf and the nature of its progenitor.\\ Methods.The observational properties of SN\,2023ixf are compared with those of a sample of Type IIP/L SNe to investigate commonalities and diversities. We conduct a detailed analysis of temporal evolution of major spectral features observed throughout different phases of the SN\,2023ixf explosion. Several interpretations are addressed through a comparison between the data and the model spectra for progenitor stars within a range of zero-age main sequence (ZAMS) masses.\\ Results.Our observations indicate that SN\,2023ixf is a transitional SN that bridges the gap between Type IIP and IIL subclasses of H-rich SNe, characterized by a relatively short plateau ($\lesssim 70$\,d) in the light curve. It shows a rather prompt spectroscopic evolution toward the nebular phase; emission lines of Na, O, H, and Ca in nebular spectra all exhibit multipeak profiles, which might be attributed to bipolar distribution of the ejecta. In particular, the H$α$ profile can be separated into two central peaked components (with a velocity of about 1500\,km\,s$^{-1}$) that is likely due to nickel-powered ejecta and two outer peak/box components (with a velocity extending up to ~8000 km\,s$^{-1}$) that can arise from interaction of the outermost ejecta with a circumstellar shell at a distance of $\sim6.2\times10^{15}$cm. The nebular-phase spectra of SN\,2023ixf show good agreement with those predicted by model spectra for progenitor stars with a ZAMS mass ranging from 15 to 19\,M${_\odot}$. A distance $D = 6.35^{+0.31}_{-0.39}$\,Mpc is estimated for M101.

astro-ph.HE↗

arXiv2025

WonderTurbo: Generating Interactive 3D World in 0.72 Seconds

Interactive 3D generation is gaining momentum and capturing extensive attention for its potential to create immersive virtual experiences. However, a critical challenge in current 3D generation technologies lies in achieving real-time interactivity. To address this issue, we introduce WonderTurbo, the first real-time interactive 3D scene generation framework capable of generating novel perspectives of 3D scenes within 0.72 seconds. Specifically, WonderTurbo accelerates both geometric and appearance modeling in 3D scene generation. In terms of geometry, we propose StepSplat, an innovative method that constructs efficient 3D geometric representations through dynamic updates, each taking only 0.26 seconds. Additionally, we design QuickDepth, a lightweight depth completion module that provides consistent depth input for StepSplat, further enhancing geometric accuracy. For appearance modeling, we develop FastPaint, a 2-steps diffusion model tailored for instant inpainting, which focuses on maintaining spatial appearance consistency. Experimental results demonstrate that WonderTurbo achieves a remarkable 15X speedup compared to baseline methods, while preserving excellent spatial consistency and delivering high-quality output.

cs.CV↗

arXiv2025

A shock crashing into confined dense circumstellar matter brightens the nascent SN 2023ixf

Red supergiants may experience a short-lived period of episodic mass loss rather than steady winds before their core collapses, leading to dense circumstellar matter (CSM) close to core-collapse supernovae (SNe). Interaction of SN ejecta with such nearby CSM can generate additional radiation, appending to the cooling radiation from the shock breakout of the progenitor envelope, to brighten the nascent SN explosion. This phenomenon is conspicuous for SN 2023ixf as its V-band brightness showed a rapid increase of about three magnitudes from the first to the third day after the explosion, which is distinctive among type II SNe with flash ionized signatures. In this paper, we employ a Monte Carlo method to simulate the radiative diffusion process in the unshocked CSM. Considering a wide range of mass-loss rates from 10^-5 to 10^-2 Msun/yr, we confirmed that the fast-rising light curve of SN 2023ixf can be fitted by the interaction of the SN ejecta with a CSM having a mass-loss rate of about 10^-2 Msun/yr located within 10^15 cm to the progenitor.

astro-ph.HE↗

arXiv2025

HumanDreamer: Generating Controllable Human-Motion Videos via Decoupled Generation

Human-motion video generation has been a challenging task, primarily due to the difficulty inherent in learning human body movements. While some approaches have attempted to drive human-centric video generation explicitly through pose control, these methods typically rely on poses derived from existing videos, thereby lacking flexibility. To address this, we propose HumanDreamer, a decoupled human video generation framework that first generates diverse poses from text prompts and then leverages these poses to generate human-motion videos. Specifically, we propose MotionVid, the largest dataset for human-motion pose generation. Based on the dataset, we present MotionDiT, which is trained to generate structured human-motion poses from text prompts. Besides, a novel LAMA loss is introduced, which together contribute to a significant improvement in FID by 62.4%, along with respective enhancements in R-precision for top1, top2, and top3 by 41.8%, 26.3%, and 18.3%, thereby advancing both the Text-to-Pose control accuracy and FID metrics. Our experiments across various Pose-to-Video baselines demonstrate that the poses generated by our method can produce diverse and high-quality human-motion videos. Furthermore, our model can facilitate other downstream tasks, such as pose sequence prediction and 2D-3D motion lifting.

cs.CV↗

arXiv2025

Detection of [C I] Emission in Nebular Spectra of a Peculiar Type Ia Supernova 2022pul

SN 2022pul gains special attention due to its possible origin of a super-Chandarsekhar-mass white dwarf explosion (or called a 03fg-like type Ia supernova), which shows prominent [O I], [Ne II], and [Ca II] lines in its late-time spectra taken at $\sim+$300 days after the peak brightness. In this paper, we present new optical observations for this peculiar object, extending up to over 500 days after the peak brightness. In particular, in the $t\approx+515$ days spectrum, we identified for the first time the presence of narrow emission from [C I] $λ\lambda9824, 9850$, which appears asymmetric and quite similar to the accompanied [O I] $\lambda6300$ line in strength and profile. Based on the violent merger model that accounts well for previous observations but leaves little carbon in the center of the ejecta, this carbon line can be reproduced by increasing the degree of clumping in the ejecta and setting the carbon mass the same as that of oxygen ($\sim$0.06 $M_{\odot}$) in the innermost region ($\lesssim 2000$ km s$^{-1}$). In principle, the central carbon could come from the secondary white dwarf (WD) if it is ignited when hit by the shockwave of the explosion of the primary WD and explodes as a Ca-rich supernova, whereas pure deflagration of a super-Chandarsekhar-mass WD can account for such unburnt carbon more naturally.

astro-ph.HE↗

arXiv2025

Timestep Embedding Tells: It's Time to Cache for Video Diffusion Model

As a fundamental backbone for video generation, diffusion models are challenged by low inference speed due to the sequential nature of denoising. Previous methods speed up the models by caching and reusing model outputs at uniformly selected timesteps. However, such a strategy neglects the fact that differences among model outputs are not uniform across timesteps, which hinders selecting the appropriate model outputs to cache, leading to a poor balance between inference efficiency and visual quality. In this study, we introduce Timestep Embedding Aware Cache (TeaCache), a training-free caching approach that estimates and leverages the fluctuating differences among model outputs across timesteps. Rather than directly using the time-consuming model outputs, TeaCache focuses on model inputs, which have a strong correlation with the modeloutputs while incurring negligible computational cost. TeaCache first modulates the noisy inputs using the timestep embeddings to ensure their differences better approximating those of model outputs. TeaCache then introduces a rescaling strategy to refine the estimated differences and utilizes them to indicate output caching. Experiments show that TeaCache achieves up to 4.41x acceleration over Open-Sora-Plan with negligible (-0.07% Vbench score) degradation of visual quality.

cs.CV↗