arXiv subjects

Jaemin Han

Publications and source records attributed to Jaemin Han.

4 recordsLinked to original sources

arXiv2026

Clustering-Based Balanced Sampling and Allocation with Data Parallelism for High-Performance Fine-Tuning

Instruction-tuning datasets for large language models (LLMs) are often large, redundant, and imbalanced, limiting efficient adaptation. Naive large-batch fine-tuning repeatedly includes overrepresented sample groups while weakly covering underrepresented but informative ones, especially under data parallelism (DP) across multiple GPUs. We propose CluSTER, a Cluster-aware balanced Sampling framework for Training Efficient data Reduction in DP instruction tuning. CluSTER curates a representative reduced dataset through gradient-space clustering and DP-aware balanced allocation, ensuring dual-level coverage across clusters and workers, while preserving the original data distribution by weighted update. As a result, CluSTER reduces redundant computation and improves training stability without compromising model quality. Across multiple instruction-tuning datasets, CluSTER reduces training time by up to 69.6% with almost no accuracy loss compared to prior sampling and data reduction methods. Code is available at https://github.com/kaist-dmlab/CluSTER.

cs.LG↗

arXiv2026

TimelyFreeze: Adaptive Parameter Freezing Mechanism for Pipeline Parallelism

Pipeline parallelism enables training models that exceed single-device memory, but practical throughput remains limited by pipeline bubbles. Although parameter freezing can improve training throughput by adaptively skipping backward computation, existing methods often over-freeze parameters, resulting in unnecessary accuracy degradation. To address this issue, we propose TimelyFreeze, which models the pipeline schedule as a directed acyclic graph and solves a linear program to compute optimal freeze ratios that minimize batch execution time under accuracy constraints. Experiments show that TimelyFreeze achieves up to 40% training throughput improvement on LLaMA-8B with comparable accuracy. Overall, it enables faster large-scale model training without compromising convergence and generalizes across diverse pipeline-parallel settings.

cs.DC↗

arXiv2016

Linear Network Coding Capacity Region of The Smart Repeater with Broadcast Erasure Channels

This work considers the smart repeater network where a single source $s$ wants to send two independent packet streams to destinations $\{d_1,d_2\}$ with the help of relay $r$. The transmission from $s$ or $r$ is modeled by packet erasure channels: For each time slot, a packet transmitted by $s$ may be received, with some probabilities, by a random subset of $\{d_1,d_2,r\}$; and those transmitted by $r$ will be received by a random subset of $\{d_1,d_2\}$. Interference is avoided by allowing at most one of $\{s,r\}$ to transmit in each time slot. One example of this model is any cellular network that supports two cell-edge users when a relay in the middle uses the same downlink resources for throughput/safety enhancement. In this setting, we study the capacity region of $(R_1,R_2)$ when allowing linear network coding (LNC). The proposed LNC inner bound introduces more advanced packing-mixing operations other than the previously well-known butterfly-style XOR operation on overheard packets of two co-existing flows. A new LNC outer bound is derived by exploring the inherent algebraic structure of the LNC problem. Numerical results show that, with more than 85% of the experiments, the relative sum-rate gap between the proposed outer and inner bounds is smaller than 0.08% under the strong-relaying setting and 0.04% under arbitrary distributions, thus effectively bracketing the LNC capacity of the smart repeater problem.

cs.IT↗

arXiv2013

Graph-Theoretic Characterization of The Feasibility of The Precoding-Based 3-Unicast Interference Alignment Scheme

A new precoding-based intersession network coding (NC) scheme has recently been proposed, which applies the interference alignment technique, originally devised for wireless interference channels, to the 3-unicast problem of directed acyclic networks. The main result of this work is a graph-theoretic characterization of the feasibility of the 3-unicast interference alignment scheme. To that end, we first investigate several key relationships between the point-to-point network channel gains and the underlying graph structure. Such relationships turn out to be critical when characterizing graph-theoretically the feasibility of precoding-based solutions.

cs.IT↗