Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.34704

Stable Continuous-Time Consistency Distillation: An Empirical Study with a Multistep Extension

Vinay Saji Mathew·Soundar R. Kumara·Gretta D Kellogg·William KM Lai
View original source ↗Read source document ↗Export metadata ↓

Abstract

Continuous-time consistency models are an attractive route to few-step generation. They remove the discretization error inherent to the timestep grid that discrete-time consistency models depend on. Until recently, however, training them was unstable. Lu & Song (2025) address this with the TrigFlow formulation and a coordinated set of architectural and training-objective changes. We conduct an independent empirical study of this framework on CIFAR-10 and ImageNet-64. No open-weight TrigFlow teachers exist at these scales, so we train them from scratch and then verify the consistency distillation (sCD) and consistency training (sCT) behavior of students initialized from these teachers. On CIFAR-10, our students reproduce the reported FID at one and two steps. On ImageNet-64, they fall short, a gap that appears related to our teacher and to architectural choices the paper leaves unspecified. We also report a parameter-count gap of about 5% between the model size stated in the paper and a faithful re-implementation, which we attribute to the scale-and-shift conditioning projections in Adaptive Double Normalization. We then propose Multistep sCD (MS-sCD), a continuous-time, segment-conditioned multistep generalization of sCD. The segment-conditioning idea is borrowed from the discrete-time multistep consistency distillation of Heek et al. (2024); we build on their preliminary continuous-time experiments within the TrigFlow/sCM framework. The rest of the algorithm follows sCD. MS-sCD replaces sCM's two-step intermediate time with a fixed segment-boundary schedule and recovers sCD at M = 1. We report FID at M = 2, 4, 8 and compare against the discrete-time MSCD baseline. Code, model weights, and training scripts are available on GitHub (Mathew et al., 2026).

Explore related subjects

cs.LG

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Vinay Saji Mathew, Soundar R. Kumara, Gretta D Kellogg, William KM Lai. 2026-10-26. Stable Continuous-Time Consistency Distillation: An Empirical Study with a Multistep Extension. https://arxiv.org/abs/2610.34704

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Generative Semi-supervised Graph Anomaly Detection

This work considers a practical semi-supervised graph anomaly detection (GAD) scenario, where part of the nodes in a graph are known to be normal, contrasting to the extensively explored unsupervised setting with a fully unlabeled graph. We reveal that having access to the normal nodes, even just a small percentage of normal nodes, helps enhance the detection performance of existing unsupervised GAD methods when they are adapted to the semi-supervised setting. However, their utilization of these normal nodes is limited. In this paper, we propose a novel Generative GAD approach (namely GGAD) for the semi-supervised scenario to better exploit the normal nodes. The key idea is to generate pseudo anomaly nodes, referred to as 'outlier nodes', for providing effective negative node samples in training a discriminative one-class classifier. The main challenge here lies in the lack of ground truth information about real anomaly nodes. To address this challenge, GGAD is designed to leverage two important priors about the anomaly nodes -- asymmetric local affinity and egocentric closeness -- to generate reliable outlier nodes that assimilate anomaly nodes in both graph structure and feature representations. Comprehensive experiments on six real-world GAD datasets are performed to establish a benchmark for semi-supervised GAD and show that GGAD substantially outperforms state-of-the-art unsupervised and semi-supervised GAD methods with varying numbers of training normal nodes. Code will be made available at https://github.com/mala-lab/GGAD.

cs.LG↗
arXiv2026

Adaptive Momentum Benefits Score-based Generative Models

Score-based Generative Models (SGMs) have achieved remarkable success in generation tasks by establishing a diffusion process that gradually perturbs real data into Gaussian noise and then learning a reverse process to reconstruct the data from noise. However, conventional denoising methods, such as Langevin dynamics and numerical stochastic differential equation solvers, rely on randomness but suffer from slow generation speeds due to the high number of score function evaluations. To address this, we propose a plug-and-play Markov adaptive momentum sampling method, inspired by the strong connection between the SGMs' sampling process and stochastic gradient descent optimization. Our approach accelerates the transformation process without introducing additional hyperparameters. Theoretically, we establish a rigorous foundation by analyzing the relationship between the distribution of samples generated by our method and the original data distribution. This provides a novel perspective on the analysis of sampling distributions. Empirically, our method demonstrates superior efficiency, generating high-quality images/graphs in significantly fewer sampling steps, achieving a speedup of 2x to 5x while maintaining or surpassing baseline performance on various image and graph generation benchmarks. These results underscore the effectiveness of adaptive momentum in enhancing the efficiency of SGMs.

cs.LG↗
arXiv2026

A Comparative Study of Convolutional and Recurrent Neural Networks for Storm Surge Prediction in Tampa Bay

In this paper, we compare the performance of three popular deep learning architectures: Long Short-Term Memory (LSTM), Convolutional Neural Networks (CNN), and CNN-LSTM in water level prediction over different time window sizes. We used wind, pressure, and tide data from National Oceanic and Atmospheric Administration (NOAA) stations. Our results show that the LSTM model generally outperformed the other models in forecasting water level trends. By applying LSTM, we achieved a mean squared error (MSE) of 0.006, making it the most reliable option for real-world use. We predicted water levels using wind, pressure, and tides. Wind, pressure, and tide data are time-based features that lack spatial patterns. Consequently, LSTM performed better than spatial methods, such as CNN. We also investigated Hurricane Ian, which occurred in Tampa Bay in September 2022 and caused a significant storm surge. Our findings show that CNN and CNN-LSTM performed slightly better than LSTMs during powerful storms, such as hurricanes. Hurricanes are seldom observed in training data, and CNN-based models are more reliable when predicting data that were not included in the training. To explain how environmental factors affect the predicted water level, we used the perturbation method. The perturbation shows that the tide is more effective at predicting water levels than wind and pressure. This study demonstrates the potential of deep learning to improve storm surge prediction accuracy.

cs.LG↗