arXiv subjects

Jaeyeon Kim

Publications and source records attributed to Jaeyeon Kim.

At least 37 records · Page 2Linked to original sources

arXiv2026

Discrete Tilt Matching

Masked diffusion large language models (dLLMs) are a promising alternative to autoregressive generation. While reinforcement learning (RL) methods have recently been adapted to dLLM fine-tuning, their objectives typically depend on sequence-level marginal likelihoods, which are intractable for masked diffusion models. To address this, we derive Discrete Tilt Matching (DTM), a likelihood-free method that recasts dLLM fine-tuning as state-level matching of local unmasking posteriors under reward tilting. DTM takes the form of a weighted cross-entropy objective with explicit minimizer, and admits control variates that improve training stability. On a synthetic maze-planning task, we analyze how DTM's annealing schedule and control variates affect training stability and prevent mode collapse. At scale, fine-tuning LLaDA-8B-Instruct with DTM yields strong gains on Sudoku and Countdown while remaining competitive on MATH500 and GSM8K.

cs.LG↗

arXiv2026

Delta Rectified Flow Sampling for Text-to-Image Editing

We propose Delta Rectified Flow Sampling (DRFS), a novel inversion-free, path-aware editing framework within rectified flow models for text-to-image editing. DRFS is a distillation-based method that explicitly models the discrepancy between the source and target velocity fields in order to mitigate over-smoothing artifacts rampant in prior distillation sampling approaches. We further introduce a time-dependent shift term to push noisy latents closer to the target trajectory, enhancing the alignment with the target distribution. We theoretically demonstrate that disabling this shift recovers Delta Denoising Score (DDS), bridging score-based diffusion optimization and velocity-based rectified-flow optimization. Moreover, under rectified-flow dynamics, a linear shift schedule recovers the inversion-free method FlowEdit as a strict special case, yielding a unifying view of optimization and ODE editing. We conduct an analysis to guide the design of our shift term, and experimental results on the widely used PIE Benchmark indicate that DRFS achieves superior editing quality, fidelity, and controllability while requiring no architectural modifications. Code is available at https://github.com/Harvard-AI-and-Robotics-Lab/DeltaRectifiedFlowSampling.

cs.CV↗

arXiv2026

ERGO: Efficient High-Resolution Visual Understanding for Vision-Language Models

Efficient processing of high-resolution images is crucial for real-world vision-language applications. However, existing Large Vision-Language Models (LVLMs) incur substantial computational overhead due to the large number of vision tokens. With the advent of "thinking with images" models, reasoning now extends beyond text to the visual domain. This capability motivates our two-stage "coarse-to-fine" reasoning pipeline: first, a downsampled image is analyzed to identify task-relevant regions; then, only these regions are cropped at full resolution and processed in a subsequent reasoning stage. This approach reduces computational cost while preserving fine-grained visual details where necessary. A major challenge lies in inferring which regions are truly relevant to a given query. Recent related methods often fail in the first stage after input-image downsampling, due to perception-driven reasoning, where clear visual information is required for effective reasoning. To address this issue, we propose ERGO (Efficient Reasoning & Guided Observation) that performs reasoning-driven perception-leveraging multimodal context to determine where to focus. Our model can account for perceptual uncertainty, expanding the cropped region to cover visually ambiguous areas for answering questions. To this end, we develop simple yet effective reward components in a reinforcement learning framework for coarse-to-fine perception. Across multiple datasets, our approach delivers higher accuracy than the original model and competitive methods, with greater efficiency. For instance, ERGO surpasses Qwen2.5-VL-7B on the V* benchmark by 4.7 points while using only 23% of the vision tokens, achieving a 3x inference speedup. The code and models can be found at: https://github.com/nota-github/ERGO.

cs.CV↗

arXiv2026

Multi-Domain Audio Question Answering Benchmark Toward Acoustic Content Reasoning

We present Task 5 of the DCASE 2025 Challenge: an Audio Question Answering (AQA) benchmark spanning multiple domains of sound understanding. This task defines three QA subsets (Bioacoustics, Temporal Soundscapes, and Complex QA) to test audio-language models on interactive question-answering over diverse acoustic scenes. We describe the dataset composition (from marine mammal calls to soundscapes and complex real-world clips), the evaluation protocol (top-1 accuracy with answer-shuffling robustness), and baseline systems (Qwen2-Audio-7B, AudioFlamingo 2, Gemini-2-Flash). Preliminary results on the development set are compared, showing strong variation across models and subsets. This challenge aims to advance the audio understanding and reasoning capabilities of audio-language models toward human-level acuity, which are crucial for enabling AI agents to perceive and interact about the world effectively.

cs.SD↗

arXiv2026

The Interspeech 2026 Audio Reasoning Challenge: Evaluating Reasoning Process Quality for Audio Reasoning Models and Agents

Recent Large Audio Language Models (LALMs) excel in understanding but often lack transparent reasoning. To address this "black-box" limitation, we organized the Audio Reasoning Challenge at Interspeech 2026, the first shared task dedicated to evaluating Chain-of-Thought (CoT) quality in the audio domain. The challenge introduced MMAR-Rubrics, a novel instance-level protocol assessing the factuality and logic of reasoning chains. Featured Single Model and Agent tracks, the competition attracting 156 teams from 18 countries and regions. Results show agent systems currently lead in reasoning quality, utilizing iterative tool orchestration and cross-modal analysis. Besides, single models are rapidly advancing via reinforcement learning and sophisticated data pipeline. We details the challenge design, methodology, and a comprehensive analysis of state-of-the-art systems, providing new insights for explainable audio intelligence.

cs.SD↗

arXiv2026

Localized Deviations from the CO-PAH Relation in PHANGS-JWST Galaxies: Faint PAH Emission or Elevated CO Emissivity?

Polycyclic aromatic hydrocarbon (PAH) emission is widely used to trace the distribution of molecular gas in the interstellar medium, exhibiting a tight correlation with CO(2-1) emission across nearby galaxies. Using PHANGS-JWST and PHANGS-ALMA data, we identify localized regions where this correlation fails, with CO flux exceeding that predicted from 7.7$μ$m PAH emission by more than an order of magnitude. These outlier regions are found in 20 out of 70 galaxies and are located in galaxy centers and bars, without signs of massive star formation. We explore two scenarios to explain the elevated CO-to-PAH ratios, which can either be due to suppressed PAH emission or enhanced CO emissivity. We examine PAH emission in other bands (3.3$μ$m and 11.3$μ$m) and the dust continuum dominated bands (10$μ$m and 21$μ$m), finding consistently high CO-to-PAH (or CO-to-dust continuum) emission ratios, suggesting that 7.7$μ$m PAH emission is not particularly suppressed. In some outlier regions, PAH sizes and spectral energy distribution of the radiation differ slightly from nearby control regions with normal CO-to-PAH ratios, though without a consistent trend. We find that the outlier regions show higher CO velocity dispersions ($Δv_{\mathrm{CO}}$). This increase in $Δv_{\mathrm{CO}}$ lowers CO optical depth and raises its emissivity for a given gas mass. Our results favor a scenario where shear along the bar lanes and shocks at the bar ends elevate CO emissivity, leading to the breakdown of the CO-PAH correlation. Future JWST spectroscopy and deep ALMA observations of CO isotopologues will provide critical tests of this scenario.

astro-ph.GA↗

arXiv2025

Duration and properties of the embedded phase of star formation in 37 nearby galaxies from PHANGS-JWST

Light reprocessed by dust grains emitting in the infrared allows the study of the physics at play in dusty, embedded regions, where ultraviolet and optical wavelengths are attenuated. Infrared telescopes such as JWST have made it possible to study the earliest feedback phases, when stars are shielded by cocoons of gas and dust. This phase is crucial for unravelling the effects of feedback from young stars, leading to their emergence and the dispersal of their host molecular clouds. Here we show that the transition from the embedded to the exposed phase of star formation is short (< 4 Myr) and sometimes almost absent (< 1 Myr), across a sample of 37 nearby star-forming galaxies, covering a wide range of morphologies from massive barred spirals to irregular dwarfs. The short duration of the dust-clearing timescales suggests a predominant role of pre-supernova feedback mechanisms in revealing newborn stars, confirming previous results on smaller samples and allowing, for the first time, a statistical analysis of their dependencies. We find that the timescales associated with mid-infrared emission at 21 μm, tracing a dust-embedded feedback phase, are controlled by a complex interplay between giant molecular cloud properties (masses and velocity dispersions) and galaxy morphology. We report relatively longer durations of the embedded phase of star formation in barred spiral galaxies, while this phase is significantly reduced in low-mass irregular dwarf galaxies. We discuss tentative trends with gas-phase metallicity, which may favor faster cloud dispersal at low metallicities.

astro-ph.GA↗

arXiv2025

Gaze Beyond the Frame: Forecasting Egocentric 3D Visual Span

People continuously perceive and interact with their surroundings based on underlying intentions that drive their exploration and behaviors. While research in egocentric user and scene understanding has focused primarily on motion and contact-based interaction, forecasting human visual perception itself remains less explored despite its fundamental role in guiding human actions and its implications for AR/VR and assistive technologies. We address the challenge of egocentric 3D visual span forecasting, predicting where a person's visual perception will focus next within their three-dimensional environment. To this end, we propose EgoSpanLift, a novel method that transforms egocentric visual span forecasting from 2D image planes to 3D scenes. EgoSpanLift converts SLAM-derived keypoints into gaze-compatible geometry and extracts volumetric visual span regions. We further combine EgoSpanLift with 3D U-Net and unidirectional transformers, enabling spatio-temporal fusion to efficiently predict future visual span in the 3D grid. In addition, we curate a comprehensive benchmark from raw egocentric multisensory data, creating a testbed with 364.6K samples for 3D visual span forecasting. Our approach outperforms competitive baselines for egocentric 2D gaze anticipation and 3D localization while achieving comparable results even when projected back onto 2D image planes without additional 2D-specific training.

cs.CV↗

arXiv2025

PAH Marks the Spot: Digging for Buried Clusters in Nearby Star-forming Galaxies

The joint capabilities of the Hubble Space Telescope (HST) and JWST allow for an unparalleled look at the early lives of star clusters at near- and mid-infrared wavelengths. We present here a multiband analysis of embedded young stellar clusters in 11 nearby, star-forming galaxies, using the PHANGS-JWST and PHANGS-HST datasets. We use the Zooniverse citizen science platform to conduct an initial by-eye search for embedded clusters in near-UV/optical/near-infrared images that trace stellar continuum emission, the Paschen$α$ and H$α$ recombination lines, and the 3.3 $μ$m polycyclic aromatic hydrocarbon feature and its underlying continuum. With this approach, we identify 292 embedded cluster candidates for which we characterize their ages, masses, and levels of line-of-sight extinction by comparing the photometric data to predictions from stellar population models. The embedded cluster candidates have a median age of 4.5 Myr and an average line-of-sight extinction $\left< A_V \right> = 6.0$ mag. We determine lower limits on source stellar masses, resulting in a median stellar mass of $10^3$ $M_{\odot}$. We use this sample of embedded cluster candidates to train multiple convolutional neural network models to carry out deep transfer learning-based searches for embedded clusters. With the aim of optimizing models for future catalog production, we compare results for four variations of training data using two neural networks. Confusion matrices for all eight model configurations, as well as inter-model identification trends, are presented. With refinement of the training sample, we determine that optimized models could serve as a pathway for future embedded cluster identification beyond our 11 galaxy sample.

astro-ph.GA↗

arXiv2025

Selective Underfitting in Diffusion Models

Diffusion models have emerged as the principal paradigm for generative modeling across various domains. During training, they learn the score function, which in turn is used to generate samples at inference. They raise a basic yet unsolved question: which score do they actually learn? In principle, a diffusion model that matches the empirical score in the entire data space would simply reproduce the training data, failing to generate novel samples. Recent work addresses this question by arguing that diffusion models underfit the empirical score due to training-time inductive biases. In this work, we refine this perspective, introducing the notion of selective underfitting: instead of underfitting the score everywhere, better diffusion models more accurately approximate the score in certain regions of input space, while underfitting it in others. We characterize these regions and design empirical interventions to validate our perspective. Our results establish that selective underfitting is essential for understanding diffusion models, yielding new, testable insights into their generalization and generative performance.

cs.LG↗

arXiv2025

Any-Order Flexible Length Masked Diffusion

Masked diffusion models (MDMs) have recently emerged as a promising alternative to autoregressive models over discrete domains. MDMs generate sequences in an any-order, parallel fashion, enabling fast inference and strong performance on non-causal tasks. However, a crucial limitation is that they do not support token insertions and are thus limited to fixed-length generations. To this end, we introduce Flexible Masked Diffusion Models (FlexMDMs), a discrete diffusion paradigm that simultaneously can model sequences of flexible length while provably retaining MDMs' flexibility of any-order inference. Grounded in an extension of the stochastic interpolant framework, FlexMDMs generate sequences by inserting mask tokens and unmasking them. Empirically, we show that FlexMDMs match MDMs in perplexity while modeling length statistics with much higher fidelity. On a synthetic maze planning task, they achieve $\approx 60 \%$ higher success rate than MDM baselines. Finally, we show pretrained MDMs can easily be retrofitted into FlexMDMs: on 16 H100s, it takes only three days to fine-tune LLaDA-8B into a FlexMDM, achieving superior performance on math (GSM8K, $58\% \to 67\%$) and code infilling performance ($52\% \to 65\%$).

cs.LG↗

arXiv2025

Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions

In recent years, masked diffusion models (MDMs) have emerged as a promising alternative approach for generative modeling over discrete domains. Compared to autoregressive models (ARMs), MDMs trade off complexity at training time with flexibility at inference time. At training time, they must learn to solve an exponentially large number of infilling problems, but at inference time, they can decode tokens in essentially arbitrary order. In this work, we closely examine these two competing effects. On the training front, we theoretically and empirically demonstrate that MDMs indeed train on computationally intractable subproblems compared to their autoregressive counterparts. On the inference front, we show that a suitable strategy for adaptively choosing the token decoding order significantly enhances the capabilities of MDMs, allowing them to sidestep hard subproblems. On logic puzzles like Sudoku, we show that adaptive inference can boost solving accuracy in pretrained MDMs from $<7$% to $\approx 90$%, even outperforming ARMs with $7\times$ as many parameters and that were explicitly trained via teacher forcing to learn the right order of decoding.

cs.LG↗

arXiv2025

Task Diversity Shortens the ICL Plateau

In-context learning (ICL) describes a language model's ability to generate outputs based on a set of input demonstrations and a subsequent query. To understand this remarkable capability, researchers have studied simplified, stylized models. These studies have consistently observed long loss plateaus, during which models exhibit minimal improvement, followed by a sudden, rapid surge of learning. In this work, we reveal that training on multiple diverse ICL tasks simultaneously shortens the loss plateaus, making each task easier to learn. This finding is surprising as it contradicts the natural intuition that the combined complexity of multiple ICL tasks would lengthen the learning process, not shorten it. Our result suggests that the recent success in large-scale training of language models may be attributed not only to the richness of the data at scale but also to the easier optimization (training) induced by the diversity of natural language training data.

cs.LG↗

arXiv2025

ViSAGe: Video-to-Spatial Audio Generation

Spatial audio is essential for enhancing the immersiveness of audio-visual experiences, yet its production typically demands complex recording systems and specialized expertise. In this work, we address a novel problem of generating first-order ambisonics, a widely used spatial audio format, directly from silent videos. To support this task, we introduce YT-Ambigen, a dataset comprising 102K 5-second YouTube video clips paired with corresponding first-order ambisonics. We also propose new evaluation metrics to assess the spatial aspect of generated audio based on audio energy maps and saliency metrics. Furthermore, we present Video-to-Spatial Audio Generation (ViSAGe), an end-to-end framework that generates first-order ambisonics from silent video frames by leveraging CLIP visual features, autoregressive neural audio codec modeling with both directional and visual guidance. Experimental results demonstrate that ViSAGe produces plausible and coherent first-order ambisonics, outperforming two-stage approaches consisting of video-to-audio generation and audio spatialization. Qualitative examples further illustrate that ViSAGe generates temporally aligned high-quality spatial audio that adapts to viewpoint changes.

cs.SD↗

arXiv2025

Time-scales of polycyclic aromatic hydrocarbon and dust continuum emission from gas clouds compared to molecular gas cloud lifetimes in PHANGS-JWST galaxies

Recent JWST mid-infrared (mid-IR) images, tracing polycyclic aromatic hydrocarbons (PAHs) and dust continuum emission, provide detailed views of the interstellar medium (ISM) in nearby galaxies. Leveraging PHANGS-JWST Cycle 1 and PHANGS-MUSE data, we measure the PAH and dust continuum emission lifetimes of gas clouds across 17 nearby star-forming galaxies by analyzing the relative spatial distributions of mid-IR (7.7-11.3$μ$m) and H$α$ emission at various scales. We find that the mid-IR emitting time-scale of gas clouds in galaxy disks (excluding centers) ranges from 10 to 30Myr. After star formation is detected in H$α$, mid-IR emission persists for 3-7Myr during the stellar feedback phase, covering 70-80% of the H$α$ emission. This significant overlap is due to intense radiation from star-forming regions, illuminating the surrounding PAHs and dust grains. In most galaxies, the mid-IR time-scale closely matches the molecular cloud lifetime measured with CO. Although mid-IR emission is complex as influenced by ISM distribution, radiation, and abundances of dust and PAHs, the similarity between the two time-scales suggests that once gas clouds form with compact mid-IR emission, they quickly provide sufficient shielding for stable CO formation. This is likely due to our focus on molecular gas-rich regions of galaxies with near-solar metallicity. Finally, we find that the mid-IR emitting time-scale is longer in galaxies with well-defined HII regions and less structured backgrounds, allowing photons to more efficiently heat the ambient ISM surrounding the HII regions, rather than contributing to diffuse emission. This suggests that the shape of the ISM also influences mid-IR emission.

astro-ph.GA↗

arXiv2025

LoRA Training Provably Converges to a Low-Rank Global Minimum or It Fails Loudly (But it Probably Won't Fail)

Low-rank adaptation (LoRA) has become a standard approach for fine-tuning large foundation models. However, our theoretical understanding of LoRA remains limited as prior analyses of LoRA's training dynamics either rely on linearization arguments or consider highly simplified setups. In this work, we analyze the LoRA loss landscape without such restrictive assumptions. We define two regimes: a "special regime", which includes idealized setups where linearization arguments hold, and a "generic regime" representing more realistic setups where linearization arguments do not hold. In the generic regime, we show that LoRA training converges to a global minimizer with low rank and small magnitude, or a qualitatively distinct solution with high rank and large magnitude. Finally, we argue that the zero-initialization and weight decay in LoRA training induce an implicit bias toward the low-rank, small-magnitude region of the parameter space -- where global minima lie -- thus shedding light on why LoRA training usually succeeds in finding global minima.

cs.LG↗

arXiv2025

The impact of spiral arms on the star formation life cycle

The matter cycle between gas clouds and stars in galaxies plays a crucial role in regulating galaxy evolution through feedback mechanisms. In turn, the local and global galactic environments shape the interstellar medium and provide the initial conditions for star formation, potentially affecting the properties of this small-scale matter cycle. In particular, spiral arms have been proposed to play a pivotal role in the star formation life cycle, by enhancing the gas density and triggering star formation. However, their exact role is still debated. In this paper, we investigate the role of spiral arms in the giant molecular cloud evolutionary life cycle and on the star formation process in a sample of 22 nearby spiral galaxies from the PHANGS survey. We measure the cloud lifetime, the feedback timescale, the typical distance between independent regions and the star formation efficiency in spiral arms and inter-arm regions separately. We find that the distributions of the cloud lifetime as well as the feedback timescale are similar in both environments. This result suggests that spiral arms are unlikely to play a dominant role in triggering star formation. By contrast, the star formation efficiency appears to be slightly higher in inter-arm regions compared to spiral arms.

astro-ph.GA↗

arXiv2025

Relationships between PAHs, Small Dust Grains, H$_2$, and HI in Local Group Dwarf Galaxies NGC 6822 and WLM Using JWST, ALMA, and the VLA

We present 0.7-3.3 pc resolution mid-infrared (MIR) JWST images at 7.7 $μ$m (F770W) and 21 $μ$m (F2100W) covering the main star-forming regions of two of the closest star-forming low-metallicity dwarf galaxies, NGC6822 and Wolf-Lundmark-Melotte (WLM). The images of NGC6822 reveal filaments, edge-brightened bubbles, diffuse emission, and a plethora of point sources. By contrast, most of the MIR emission in WLM is point-like, with a small amount of extended emission. Compared to solar metallicity galaxies, the ratio of 7.7 $μ$m intensity ($I_ν^{F770W}$), tracing polycyclic aromatic hydrocarbons (PAHs), to 21 $μ$m intensity ($I_ν^{F2100W}$), tracing small, warm dust grain emission, is suppressed in these low-metallicity dwarfs. Using ALMA CO(2-1) observations, we find that detected CO intensity versus $I_ν^{F770W}$ at ~2 pc resolution in dwarfs follows a similar relationship to that at solar metallicity and lower resolution, while the CO versus $I_ν^{F2100W}$ relationship in dwarfs lies significantly below that derived from solar metallicity galaxies at lower resolution, suggesting more pronounced destruction of CO molecules at low metallicity. Finally, adding in Local Group L-Band Survey VLA 21 cm HI observations, we find that $I_ν^{F2100W}$ and $I_ν^{F770W}$ vs. total gas ratios are suppressed in NGC6822 and WLM compared to solar metallicity galaxies. In agreement with dust models, the level of suppression appears to be at least partly accounted for by the reduced galaxy-averaged dust-to-gas and PAH-to-dust mass ratios in the dwarfs. Remaining differences are likely due to spatial variations in dust model parameters, which should be an exciting direction for future work in local dwarf galaxies.

astro-ph.GA↗