Search arXiv⌕ Search

arXiv subjects

Zhiwei Ren

Publications and source records attributed to Zhiwei Ren.

4 recordsLinked to original sources

ReelCache: An Efficient KV Cache Compression and Management System for Multi-Round Video Generation

Autoregressive video diffusion models produce large KV caches during multi-round video generation. Keeping all these KV chunks would consume substantial memory, while rebuilding discarded caches would significantly delay video chunk generation. This paper presents ReelCache, a KV cache compression system for efficient multi-round, long video generation. ReelCache combines an intra-chunk compressor with an inter-chunk cache manager. The compressor applies low-rank decomposition across channels and video coding to the resulting coefficients, predicting the decomposition rank per transformer layer online to meet retained-energy targets. The cache manager shares basis directions across consecutive chunks and organizes chunk states in a tree, allowing generation branches to share historical state while supporting restoration for resume, rewind, and fork. Experiments across five video generation models show that ReelCache reduces KV storage by 14.46-21.13x relative to baselines and accelerates next-chunk generation by 1.25-3.36x, with less than 1% drop in six representative VBench metrics. GitHub: https://github.com/Tianyu9748/ReelCache

cs.CV↗

Predict, Reuse, and Repair: Accelerating Dynamic Sparse Attention for Long-Context LLM Decoding

Dynamic sparse attention (DSA) accelerates long-context LLM decoding by attending to only the top-K KV blocks relevant to each query, but it introduces a serialized selection-to-attention dependency that emerges as a new latency bottleneck. We present PRR, a speculate-reuse-repair runtime that exploits temporal locality in DSA selections to predict likely blocks, speculate the attention over them while selection is in flight, and incrementally repair missed blocks once the true selected set is known. PRR uses a lightweight EMA-based predictor, a profiling-guided speculation budget that keeps speculative work off the critical path, and a FlashAttention-based repair kernel that folds missed blocks into the partial attention state using online-softmax statistics. Across long-context benchmarks and representative DSA methods, PRR reduces per-token decoding latency by up to 40% while preserving downstream task accuracy. Github: https://github.com/Tianyu9748/Incremental_FlashAttention

cs.LG↗

Beyond Perfect APIs: A Comprehensive Evaluation of LLM Agents Under Real-World API Complexity

We introduce WildAGTEval, a benchmark designed to evaluate large language model (LLM) agents' function-calling capabilities under realistic API complexity. Unlike prior work that assumes an idealized API system and disregards real-world factors such as noisy API outputs, WildAGTEval accounts for two dimensions of real-world complexity: 1. API specification, which includes detailed documentation and usage constraints, and 2. API execution, which captures runtime challenges. Consequently, WildAGTEval offers (i) an API system encompassing 60 distinct complexity scenarios that can be composed into approximately 32K test configurations, and (ii) user-agent interactions for evaluating LLM agents on these scenarios. Using WildAGTEval, we systematically assess several advanced LLMs and observe that most scenarios are challenging, with irrelevant information complexity posing the greatest difficulty and reducing the performance of strong LLMs by 27.3%. Furthermore, our qualitative analysis reveals that LLMs occasionally distort user intent merely to claim task completion, critically affecting user satisfaction.

cs.CL↗

MuFA (Multi-type Fourier Analyzer): A tool for batch generation of MuMax3 input scripts and multi-type Fourier analysis from micromagnetic simulation output data

We present a tool for batch generation of input scripts and multi-type Fourier analysis from simulation results for the micromagnetic software MuMax3. The introduction of graphical user interface and parameter-sweeping functionality strongly speed up the input scripts creation and accelerate model optimization processes consequently. Three types of important Fourier analysis methods are provided for the acquisition of the quantitative frequency compositions, the spin-wave dispersion curve and the spatial distribution of spin-wave powers at different frequencies, respectively. Since the Fourier analysis is accelerated by parallel computations, the time cost is reduced to an acceptable level even in the presentation of tens of gigabytes data. With the MuMax3 and our proposal, a complete micromagnetic simulating tool chain from scripts generation to post analysis has been developed.

eess.SP↗