Search arXiv⌕ Search

arXiv subjects

Kunyang Li

Publications and source records attributed to Kunyang Li.

At least 19 recordsLinked to original sources

Video-Coder: Reinforcing Agentic Coding for Video Generation

Recent years have witnessed remarkable progress in video generation in terms of visual quality and temporal consistency. However, generating videos with precise spatial structure, coherent motion dynamics, and controllable scene composition remains challenging. In this paper, we introduce Video-Coder, the first attempt to train a code-driven video generation agent, which autonomously writes and executes code to construct a dynamic 3D scene (Draft Video) that provides structure and motion signals to downstream video generators. To support this paradigm, we design a dedicated data pipeline and curate two high-quality datasets, Video-Coder-SFT-6k and Video-Coder-RL-2k, which contain diverse prompts and ground-truth reference videos. We further develop the Video- Coder Harness, a unified interactive environment integrating code execution, 3D scene rendering, and execution feedback for agent training and inference. We then train Video-Coder with SFT fol- lowed by harness-native agentic reinforcement learning with dual reward feedback, which combines execution-based and video-based rewards. Additionally, we design an annealing training strategy that gradually reduces the execution reward over time, allowing the model to rely more on the video reward in the later training stages. Experiments show that Video-Coder achieves significant gains, improving Minimax-H3 by approximately 7 points on PhyGenBench and 24 points on VBench-2.0. We hope this work can serve as an open foundation for coding agents in video generation, and we fully open-source our data, models, and code

cs.CV↗

Set them free: extending RAMCOAL to model massive black hole triplets in hydrodynamical simulations of galaxies

Massive black hole binaries (MBHBs) and the higher-order multiples produced by repeated galaxy mergers evolve through regimes that cosmological simulations cannot resolve. These regimes determine merger delays, spins, recoils, and host galaxies at coalescence. We extend the RAMCOAL subgrid model to follow massive black hole triplets within hydrodynamical galaxy simulations. In the original three-stage binary model, black holes evolve as sink particles, undergo dynamical friction, and form bound binaries that harden through stellar scattering, gas torques, circumbinary-disc coupling, and gravitational-wave emission. When a hierarchical triplet becomes chaotic, RAMCOAL samples a library of outcomes from direct N body experiments and follows the resulting mergers, exchanges, and ejections alongside the accretion and spin evolution of each black hole. Isolated-galaxy tests with different triplet geometries show that orbital configuration alone can change which pair merges and its coalescence time. For the first time, we follow an MBH triplet to coalescence within a hydrodynamical simulation of gas-rich galaxies, coupling triplet-driven mergers to host-galaxy evolution. Subgrid integration costs less than 1 ms of single-core time per MBH system and time step, independently of particle resolution. The resulting coalescence properties reflect the environment encountered during subgrid evolution. RAMCOAL thus provides a step towards merger catalogues linking gravitational-wave signatures to the galaxies in which the black holes evolve.

astro-ph.GA↗

CineMR: Tool-Integrated Vision-Language Reasoning for Quantitative Cardiac MRI Assessment

Cardiovascular magnetic resonance (CMR), including cine imaging, is a reference standard for the noninvasive assessment of cardiac morphology and ventricular function. Cine CMR interpretation integrates qualitative visual assessment with quantitative measurements of ventricular volumes, ejection fraction, myocardial mass, wall thickness, and regional wall motion. Current medical vision-language models (VLMs) cannot reliably derive quantitative measurements from multidimensional cine images without analysis tools. We present CineMR, a tool-augmented VLM that invokes cardiac image-analysis tools and integrates their outputs into interleaved reasoning for quantitative CMR assessment. We also construct a multi-cohort visual question answering benchmark covering quantitative metric extraction, multiclass diagnosis, and differential diagnosis, together with tools for segmentation, phase selection, volumetry, morphometry, and regional wall motion analysis. CineMR is trained with supervised fine-tuning (SFT) on tool-interaction traces followed by Group Relative Policy Optimization (GRPO) with conditional tool-use rewards. On the multi-cohort cine CMR benchmark, CineMR achieves 35.9% pass@1 and 58.9% pass@4, compared with 1.5% pass@1 for the Qwen3-VL-8B backbone and 0.0% and 7.0% pass@1 for LLaVA-Med v1.5 and MedGemma-4B, respectively. Correct tool invocation reaches 99.8% after GRPO, up from 78.9% after SFT. Live tool outputs improve ventricular measurement accuracy by 20.4--23.7% over direct model predictions, and removing all tools reduces pass@1 from 35.9% to 27.9%. These results highlight the importance of reliable tool use for quantitative cine CMR reasoning and support CineMR as a promising approach for assistive cardiac image assessment. Code, benchmark resources, and model weights are available at https://github.com/AI-MIND-Lab/CineMR.

cs.CV↗

Keepsake: Selective Spatial Memory for Long-Horizon Video Generation

Long-horizon camera-controlled video generation relies on persistent memory to maintain scene consistency. Existing systems follow two strategies to achieve this consistency. Full-history approaches retain all generated observations, causing unbounded storage and retrieval costs. Selective-construction approaches reduce redundancy, but make one-time retention decisions that are never revisited, even as an observation's value changes with the evolving memory bank. Both strategies leave a shared question unresolved: as the generated history evolves, which stored observations should still remain in memory? Our key insight is that the value of a stored observation is not fixed, but relational: it depends on the alternatives currently available in the memory bank. A view supported by many geometrically and visually similar substitutes can be relinquished with little loss of coverage, whereas an observation with few viable alternatives should remain regardless of age. We introduce Keepsake, an online, training-free controller for fixed-capacity spatial memory. At each update, Keepsake constructs a pose-appearance graph over retained and newly generated observations, combining camera-pose proximity with visual similarity. A retention priority jointly captures the number of strong substitutes and the similarity of the closest alternative, allowing Keepsake to continually reassess memory value, preserve observations with little alternative support, and evict highly replaceable ones under a fixed budget. The controller modifies only the persistent-memory update; the host generator, denoising schedule, and retrieval rule remain unchanged. Across MemCam and WorldMem, Keepsake improves FVD and LPIPS under a fixed memory budget. On 180-second MemCam trajectories, it retains only 32 of 5,397 frames while reducing FVD by 35.1%.

cs.CV↗

MESA: Prioritizing Vulnerable Communication Channels for Securing Multi-Agent Systems

Multi-agent systems (MAS) are increasingly used to automate complex, distributed workflows. However, their inter-agent communication channels introduce new attack surfaces that remain poorly understood and are difficult to defend against. In this paper, we address how defenders should prioritize limited security effort to protect vulnerable communication channels before attacks are observed. This is motivated by our observation that the channel-level attack impact is highly non-uniform: a single compromised edge can account for up to 75% of total attack success. We introduce Mesa, a label-free framework for proactively ranking which MAS edges are most security-critical -- that is, most likely to affect the system's decision if compromised. Mesa combines six graph-theoretic metrics and two dynamic probes (ablation and masking) without requiring attack traces. We evaluate Mesa against a dynamic misinformation attack pipeline across three diverse MAS scenarios, eight network topologies, and five open-source LLMs from Qwen, Llama, and Gemma families. Mesa rankings correlate strongly with empirical per-edge attack success rate, achieving mean Spearman $ρ=+0.60$ (peaking at $+0.73$). In resource-constrained defense deployment, monitoring the top 10% of Mesa-ranked edges intercepts about 3x the successful attacks as random allocation. We further test Mesa under varying attacker and defender models and LangGraph workflows and characterize its limits under adaptive attacks and high-redundancy graphs. Overall, our results show that edge-level risk in MAS is often concentrated and predictable, allowing proactive hardening of multi-agent infrastructures.

cs.CR↗

Attend Locally, Remember Linearly: Linear Attention as Cross-Frame Memory for Autoregressive Video Diffusion

Autoregressive (AR) video diffusion is a powerful paradigm for streaming and interactive video generation. However, its reliance on softmax self-attention leads to quadratic compute complexity in sequence length and memory usage due to key-value caching, which limits its scalability to long video horizons. Existing remedies (e.g., sparse attention and KV-cache compression) reduce per-step cost but still rely on a linearly growing cache or irreversibly discard past context, and thus fail to address linear memory growth and streaming context management. To address this scalability bottleneck, we propose ARL2 (Attend Locally, Remember Linearly), a hybrid attention module that replaces quadratic cross-frame attention with a fixed-size recurrent state. We decompose self-attention into two branches: an intra-frame softmax branch for spatial detail and local dependencies, and an inter-frame gated recurrent linear branch that maintains a fixed-size state for streaming context. Our key insight is that softmax attention captures fine-grained local interactions, while a recurrent state provides controllable long-range memory. This design achieves linear-time scaling with constant memory while improving temporal consistency over the full-softmax model. To prevent noisy intermediate states from corrupting memory, we update the recurrent state only after the denoised pass. To avoid within-frame information asymmetry, all tokens share the same pre-update state rather than sequential updates. To the best of our knowledge, this is the first work to convert a pretrained AR video diffusion model into a hybrid linear attention architecture, through an efficient two-stage training scheme for AR video. With 75% of layers replaced by hybrid linear attention, the model achieves up to 2.26 wall-clock speedup and 54% memory reduction, while maintaining comparable quality with improving temporal consistency.

cs.CV↗

Aero-World: Action-Conditioned Aerial Video Generation from Inertial Controls

Foundation video models produce visually impressive results, but their use in embodied AI remains limited because they are primarily trained on natural language rather than low-level control signals. This limitation is especially pronounced for aerial flight, where motion occurs in unconstrained 6-DoF space and small errors in ego-motion can produce large trajectory drift. Generating aerial videos that follow fine-grained inertial actions can support scalable training and evaluation of aerial agents by providing a controllable proxy for real-world or expensive simulation data. To address this problem, we propose \textbf{Aero-World}, a method for converting a pretrained image-to-video diffusion model into a controllable aerial video generator. Aero-World injects sequences of translational acceleration and angular velocity into a pretrained latent diffusion transformer through an action-token stream. A frozen latent-space Physics Probe, trained independently on real video--IMU pairs, provides differentiable inertial-consistency supervision during LoRA finetuning while avoiding computationally expensive video decoding. We further propose \textbf{AeroBench}, a benchmark for evaluating whether generated drone videos adhere to low-level action signals. AeroBench uses Action Alignment Score (AAS) to measure agreement with commanded inertial actions and Physical Consistency Rate (PCR) to measure temporal motion stability. On AeroBench, Aero-World improves mean AAS from 57.7 to 63.6 over action-only finetuning and gives a stronger quality-control trade-off than AirScape, with lower FVD (596.5 vs. 1058.6), higher SSIM (0.595 vs. 0.505), and higher Flow-IMU correlation (0.44 vs. 0.20). These results suggest that frozen Physics Probe supervision is a practical mechanism for adapting pretrained video generators toward more action-aligned aerial motion.

cs.CV↗

The LISA Astrophysics MBHcatalogues Project: A comparison of predictions of simulated massive black hole binaries

In the hierarchical paradigm of galaxy formation, central massive black holes (MBHs) are expected to coalesce after the merger of their host galaxies. One of the main goals of the Laser Interferometer Space Antenna (LISA) is to constrain the origin and growth of MBHs through their merger rates and mass distribution. Predicting MBH merger rates requires not only tracing their statistical population from large to small physical scales (kpc to sub-pc) but also modelling their formation, accretion, dynamics, mergers, and their galactic physical processes across cosmic time. This project is the result of a large collaborative effort undertaken by the LISA Astrophysics Working Group, bringing together its collective expertise on MBH formation, evolution, and modelling, to build a comprehensive understanding of MBH merger rates across cosmic time. The project compares various theoretical predictions of MBH merger rates, quantifies the spread, and evaluates the global astrophysical uncertainties of the LISA event rates. To build a unique and complete view, our work is based on about 20 semi-analytical models and cosmological simulations from the literature, all employing distinct approaches to modelling MBH and galaxy physics. To compute the merger rates, we also incorporate delays arising from the dynamical phase of MBH hardening to coalescence. We present the expected LISA merger rates given current galaxy formation models and discuss how the merger rate depends on model assumptions, such as the seeding model and the resolution of cosmological simulations.

astro-ph.GA↗

A Non-compact Positivity-Preserving Numerical Scheme for Elliptic Differential Equations Based on Mathematical Expectation

We propose a novel non-compact, positivity-preserving scheme for linear non-divergence form elliptic equations. Based on the Feynman--Kac formula, the solution is represented as a conditional expectation associated with a diffusion process.Instead of using compact Markov chain approximations, we construct a wide-stencil scheme by approximating the expectation with carefully designed transition probabilities, ensuring both consistency and positivity preservation. The method is effective for anisotropic diffusion problems with mixed derivatives, where classical schemes typically fail unless the covariance matrix is diagonally dominant. A key feature of the proposed framework is its robust treatment of boundary conditions. For Dirichlet boundaries, we introduce a quadtree-based non-uniform stopping-time strategy, achieving $O(h)$ accuracy. For Neumann boundaries, a discrete specular reflection mechanism is employed, yielding $O(h^{1/2})$ convergence. Periodic boundaries are handled through modular wrapping, also achieving $O(h)$ accuracy. The resulting schemes are unconditionally stable and positivity-preserving due to their probabilistic structure. Numerical experiments confirm the theoretical convergence rates under all boundary conditions considered.

math.NA↗

The multimessenger view of Pulsar Timing Array black holes with the Horizon-AGN simulation

We use the Horizon-AGN cosmological simulation to study the properties of supermassive black hole binaries (MBHBs) contributing most to the gravitational wave background (GWB) signal expected in the pulsar timing array (PTA) band. We develop a pipeline to generate realistic populations of MBHBs, allowing us to estimate both the characteristic strain and GWB time series observable by PTA experiments. We identify potential continuous wave (CW) candidates standing above the background noise, using toy PTA sensitivities representing the current EPTA and future SKA. We estimate the probability of detecting at least one CW with signal-to-noise ratio $>3$ to be $4\%$ ($20\%$) for EPTA (SKA)-like sensitivities, assuming a 10-year baseline. We find the GWB to be dominated by hundreds to thousands of binaries at redshifts in the range $0.05-1$, with chirp masses of $10^{8.5}-10^{9.5}\, M_\odot$, hosted mainly in quiescent massive galaxies residing in halos of mass $\sim 10^{13}\, M_\odot$. CW candidates have larger masses, lower redshifts and are found in even more massive halos, typical of galaxy groups and clusters. The majority of these systems would appear as AGN rather than quasars, because of their low Eddington ratios. Nevertheless, CW candidates with $f_{\rm Edd}>10^{-3}$ can still outshine their hosts, particularly in radio and X-ray bands, suggesting them as the most promising route for identification. Our findings imply that optical and near-infrared searches based on light curve variability are challenging and biased toward more luminous systems. Finally, we highlight important caveats in the common method used to compare PTA observations with theoretical models. We find that GWB spectral inferences used by PTAs could be biased toward shallower slopes and higher amplitudes at $f=1/\rm yr$, thereby reducing the apparent tension between astrophysical expectations and PTA observations.

astro-ph.GA↗

PackCache: A Training-Free Acceleration Method for Unified Autoregressive Video Generation via Compact KV-Cache

A unified autoregressive model is a Transformer-based framework that addresses diverse multimodal tasks (e.g., text, image, video) as a single sequence modeling problem under a shared token space. Such models rely on the KV-cache mechanism to reduce attention computation from O(T^2) to O(T); however, KV-cache size grows linearly with the number of generated tokens, and it rapidly becomes the dominant bottleneck limiting inference efficiency and generative length. Unified autoregressive video generation inherits this limitation. Our analysis reveals that KV-cache tokens exhibit distinct spatiotemporal properties: (i) text and conditioning-image tokens act as persistent semantic anchors that consistently receive high attention, and (ii) attention to previous frames naturally decays with temporal distance. Leveraging these observations, we introduce PackCache, a training-free KV-cache management method that dynamically compacts the KV cache through three coordinated mechanisms: condition anchoring that preserves semantic references, cross-frame decay modeling that allocates cache budget according to temporal distance, and spatially preserving position embedding that maintains coherent 3D structure under cache removal. In terms of efficiency, PackCache accelerates end-to-end generation by 1.7-2.2x on 48-frame long sequences, showcasing its strong potential for enabling longer-sequence video generation. Notably, the final four frames - the portion most impacted by the progressively expanding KV-cache and thus the most expensive segment of the clip - PackCache delivers a 2.6x and 3.7x acceleration on A40 and H200, respectively, for 48-frame videos.

cs.CV↗

GVD: Guiding Video Diffusion Model for Scalable Video Distillation

To address the larger computation and storage requirements associated with large video datasets, video dataset distillation aims to capture spatial and temporal information in a significantly smaller dataset, such that training on the distilled data has comparable performance to training on all of the data. We propose GVD: Guiding Video Diffusion, the first diffusion-based video distillation method. GVD jointly distills spatial and temporal features, ensuring high-fidelity video generation across diverse actions while capturing essential motion information. Our method's diverse yet representative distillations significantly outperform previous state-of-the-art approaches on the MiniUCF and HMDB51 datasets across 5, 10, and 20 Instances Per Class (IPC). Specifically, our method achieves 78.29 percent of the original dataset's performance using only 1.98 percent of the total number of frames in MiniUCF. Additionally, it reaches 73.83 percent of the performance with just 3.30 percent of the frames in HMDB51. Experimental results across benchmark video datasets demonstrate that GVD not only achieves state-of-the-art performance but can also generate higher resolution videos and higher IPC without significantly increasing computational cost.

cs.CV↗

Tracking on-the-fly massive black hole binary evolution and coalescence in galaxy simulations: RAMCOAL

The detection of gravitational waves (GWs) from massive black hole binary (MBHB) coalescence motivates the development of a sub-grid model. We present RAMCOAL, integrated into the RAMSES code, which simulates the orbital evolution of MBHBs, accounting for stellar and gaseous dynamical friction (DF), stellar scattering, circumbinary disk interactions, and GW emission at scales below the simulation resolution. Unlike post-processing approaches, RAMCOAL tracks the real-time evolution of MBHBs within hydrodynamical simulations of galaxies using local quantities to model dynamics and accretion. This enables more accurate predictions of both GW signals and the properties of merging black holes. We validate RAMCOAL across isolated and merging galaxy setups at resolutions of 10, 50, and 100 pc, with and without black hole accretion and feedback. In addition, we test the model in seven galaxy merger scenarios at 100 pc resolution. These tests demonstrate that RAMCOAL is largely resolution-independent and successfully captures the effects of DF from stars, dark matter, and gas, loss-cone scattering, viscous drag from circumbinary disks, and GW emission -- all within a realistic galactic environment, even at low resolutions. With RAMCOAL, we can better estimate MBHB coalescence rates and the GW background, while providing insights into the electromagnetic counterparts of GW sources. This approach bridges the gap between electromagnetic observations and GW detection, offering a more comprehensive understanding of MBHB evolution in cosmological simulations.

astro-ph.GA↗

MP-GCAN: a highly accurate classifier for $α$-helical membrane proteins and $β$-barrel proteins

Membrane protein classification is a fundamental task in structural bioinformatics, critical to understanding protein functions and accelerating drug discovery. In this study, we propose MP-GCAN, a novel graph-based classification model that leverages both spatial and sequential features of proteins. MP-GCAN combines GCN, GAT, and GIN layers to capture hierarchical structural representations from 3D protein graphs, constructed from high-resolution PDB files with $α$-carbon coordinates and residue types. To evaluate performance, we curated a high-quality dataset of 500 membrane and 500 non-membrane proteins, and compared MP-GCAN with two baselines: a structure-confidence-based SGD classifier utilizing AlphaFold's pLDDT scores, and DeepTMHMM, a sequence-based deep learning model. Our experiments demonstrate that MP-GCAN significantly outperforms baselines, achieving an accuracy of 96% and strong F1-scores on both classes. The results highlight the importance of integrating pretrained GNN architectures with domain-specific structural data to enhance membrane protein classification.

q-bio.QM↗

Alignment and Adversarial Robustness: Are More Human-Like Models More Secure?

A small but growing body of work has shown that machine learning models which better align with human vision have also exhibited higher robustness to adversarial examples, raising the question: can human-like perception make models more secure? If true generally, such mechanisms would offer new avenues toward robustness. In this work, we conduct a large-scale empirical analysis to systematically investigate the relationship between representational alignment and adversarial robustness. We evaluate 114 models spanning diverse architectures and training paradigms, measuring their neural and behavioral alignment and engineering task performance across 105 benchmarks as well as their adversarial robustness via AutoAttack. Our findings reveal that while average alignment and robustness exhibit a weak overall correlation, specific alignment benchmarks serve as strong predictors of adversarial robustness, particularly those that measure selectivity toward texture or shape. These results suggest that different forms of alignment play distinct roles in model robustness, motivating further investigation into how alignment-driven approaches can be leveraged to build more secure and perceptually-grounded vision models.

cs.CV↗

On the Robustness Tradeoff in Fine-Tuning

Fine-tuning has become the standard practice for adapting pre-trained models to downstream tasks. However, the impact on model robustness is not well understood. In this work, we characterize the robustness-accuracy trade-off in fine-tuning. We evaluate the robustness and accuracy of fine-tuned models over 6 benchmark datasets and 7 different fine-tuning strategies. We observe a consistent trade-off between adversarial robustness and accuracy. Peripheral updates such as BitFit are more effective for simple tasks -- over 75% above the average measured by the area under the Pareto frontiers on CIFAR-10 and CIFAR-100. In contrast, fine-tuning information-heavy layers, such as attention layers via Compacter, achieves a better Pareto frontier on more complex tasks -- 57.5% and 34.6% above the average on Caltech-256 and CUB-200, respectively. Lastly, we observe that the robustness of fine-tuning against out-of-distribution data closely tracks accuracy. These insights emphasize the need for robustness-aware fine-tuning to ensure reliable real-world deployments.

cs.LG↗

All Languages Matter: Evaluating LMMs on Culturally Diverse 100 Languages

Existing Large Multimodal Models (LMMs) generally focus on only a few regions and languages. As LMMs continue to improve, it is increasingly important to ensure they understand cultural contexts, respect local sensitivities, and support low-resource languages, all while effectively integrating corresponding visual cues. In pursuit of culturally diverse global multimodal models, our proposed All Languages Matter Benchmark (ALM-bench) represents the largest and most comprehensive effort to date for evaluating LMMs across 100 languages. ALM-bench challenges existing models by testing their ability to understand and reason about culturally diverse images paired with text in various languages, including many low-resource languages traditionally underrepresented in LMM research. The benchmark offers a robust and nuanced evaluation framework featuring various question formats, including true/false, multiple choice, and open-ended questions, which are further divided into short and long-answer categories. ALM-bench design ensures a comprehensive assessment of a model's ability to handle varied levels of difficulty in visual and linguistic reasoning. To capture the rich tapestry of global cultures, ALM-bench carefully curates content from 13 distinct cultural aspects, ranging from traditions and rituals to famous personalities and celebrations. Through this, ALM-bench not only provides a rigorous testing ground for state-of-the-art open and closed-source LMMs but also highlights the importance of cultural and linguistic inclusivity, encouraging the development of models that can serve diverse global populations effectively. Our benchmark is publicly available.

cs.CV↗

Robust Lane Detection with Wavelet-Enhanced Context Modeling and Adaptive Sampling

Lane detection is critical for autonomous driving and ad-vanced driver assistance systems (ADAS). While recent methods like CLRNet achieve strong performance, they struggle under adverse con-ditions such as extreme weather, illumination changes, occlusions, and complex curves. We propose a Wavelet-Enhanced Feature Pyramid Net-work (WE-FPN) to address these challenges. A wavelet-based non-local block is integrated before the feature pyramid to improve global context modeling, especially for occluded and curved lanes. Additionally, we de-sign an adaptive preprocessing module to enhance lane visibility under poor lighting. An attention-guided sampling strategy further reffnes spa-tial features, boosting accuracy on distant and curved lanes. Experiments on CULane and TuSimple demonstrate that our approach signiffcantly outperforms baselines in challenging scenarios, achieving better robust-ness and accuracy in real-world driving conditions.

cs.CV↗