arXiv subjects

Jinyoung Kim

Publications and source records attributed to Jinyoung Kim.

35 records · Page 2Linked to original sources

arXiv2025

When Should Dense Retrievers Be Updated in Evolving Corpora? Detecting Out-of-Distribution Corpora Using GradNormIR

Dense retrievers encode texts into embeddings to efficiently retrieve relevant documents from large databases in response to user queries. However, real-world corpora continually evolve, leading to a shift from the original training distribution of the retriever. Without timely updates or retraining, indexing newly emerging documents can degrade retrieval performance for future queries. Thus, identifying when a dense retriever requires an update is critical for maintaining robust retrieval systems. In this paper, we propose a novel task of predicting whether a corpus is out-of-distribution (OOD) relative to a dense retriever before indexing. Addressing this task allows us to proactively manage retriever updates, preventing potential retrieval failures. We introduce GradNormIR, an unsupervised approach that leverages gradient norms to detect OOD corpora effectively. Experiments on the BEIR benchmark demonstrate that GradNormIR enables timely updates of dense retrievers in evolving document collections, significantly enhancing retrieval robustness and efficiency.

cs.IR↗

arXiv2025

TEOChat: A Large Vision-Language Assistant for Temporal Earth Observation Data

Large vision and language assistants have enabled new capabilities for interpreting natural images. These approaches have recently been adapted to earth observation data, but they are only able to handle single image inputs, limiting their use for many real-world tasks. In this work, we develop a new vision and language assistant called TEOChat that can engage in conversations about temporal sequences of earth observation data. To train TEOChat, we curate an instruction-following dataset composed of many single image and temporal tasks including building change and damage assessment, semantic change detection, and temporal scene classification. We show that TEOChat can perform a wide variety of spatial and temporal reasoning tasks, substantially outperforming previous vision and language assistants, and even achieving comparable or better performance than several specialist models trained to perform specific tasks. Furthermore, TEOChat achieves impressive zero-shot performance on a change detection and change question answering dataset, outperforms GPT-4o and Gemini 1.5 Pro on multiple temporal tasks, and exhibits stronger single image capabilities than a comparable single image instruction-following model on scene classification, visual question answering, and captioning. We publicly release our data, model, and code at https://github.com/ermongroup/TEOChat .

cs.CV↗

arXiv2024

CNG-SFDA:Clean-and-Noisy Region Guided Online-Offline Source-Free Domain Adaptation

Domain shift occurs when training (source) and test (target) data diverge in their distribution. Source-Free Domain Adaptation (SFDA) addresses this domain shift problem, aiming to adopt a trained model on the source domain to the target domain in a scenario where only a well-trained source model and unlabeled target data are available. In this scenario, handling false labels in the target domain is crucial because they negatively impact the model performance. To deal with this problem, we propose to update cluster prototypes (i.e., centroid of each sample cluster) and their structure in the target domain formulated by the source model in online manners. In the feature space, samples in different regions have different pseudo-label distribution characteristics affected by the cluster prototypes, and we adopt distinct training strategies for these samples by defining clean and noisy regions: we selectively train the target with clean pseudo-labels in the clean region, whereas we introduce mix-up inputs representing intermediate features between clean and noisy regions to increase the compactness of the cluster. We conducted extensive experiments on multiple datasets in online/offline SFDA settings, whose results demonstrate that our method, CNG-SFDA, achieves state-of-the-art for most cases. Code is available at https://github.com/hyeonwoocho7/CNG-SFDA.

cs.CV↗

arXiv2024

DynamicER: Resolving Emerging Mentions to Dynamic Entities for RAG

In the rapidly evolving landscape of language, resolving new linguistic expressions in continuously updating knowledge bases remains a formidable challenge. This challenge becomes critical in retrieval-augmented generation (RAG) with knowledge bases, as emerging expressions hinder the retrieval of relevant documents, leading to generator hallucinations. To address this issue, we introduce a novel task aimed at resolving emerging mentions to dynamic entities and present DynamicER benchmark. Our benchmark includes dynamic entity mention resolution and entity-centric knowledge-intensive QA task, evaluating entity linking and RAG model's adaptability to new expressions, respectively. We discovered that current entity linking models struggle to link these new expressions to entities. Therefore, we propose a temporal segmented clustering method with continual adaptation, effectively managing the temporal dynamics of evolving entities and emerging mentions. Extensive experiments demonstrate that our method outperforms existing baselines, enhancing RAG model performance on QA task with resolved mentions.

cs.CL↗

arXiv2024

A reaction network model of microscale liquid-liquid phase separation reveals effects of spatial dimension

Proteins can form droplets via liquid-liquid phase separation (LLPS) in cells. Recent experiments demonstrate that LLPS is qualitatively different on two-dimensional (2d) surfaces compared to three-dimensional (3d) solutions. In this paper, we use mathematical modeling to investigate the causes of the discrepancies between LLPS in 2d versus 3d. We model the number of proteins and droplets inducing LLPS by continuous-time Markov chains and use chemical reaction network theory to analyze the model. To reflect the influence of space dimension, droplet formation and dissociation rates are determined using the first hitting times of diffusing proteins. We first show that our stochastic model reproduces the appropriate phase diagram and is consistent with the relevant thermodynamic constraints. After further analyzing the model, we find that it predicts that the space dimension induces qualitatively different features of LLPS which are consistent with recent experiments. While it has been claimed that the differences between 2d and 3d LLPS stems mainly from different diffusion coefficients, our analysis is independent of the diffusion coefficients of the proteins since we use the stationary model behavior. Therefore, our results give new hypotheses about how space dimension affects LLPS.

q-bio.QM↗

arXiv2024

Joint-Embedding Predictive Architecture for Self-Supervised Learning of Mask Classification Architecture

In this work, we introduce Mask-JEPA, a self-supervised learning framework tailored for mask classification architectures (MCA), to overcome the traditional constraints associated with training segmentation models. Mask-JEPA combines a Joint Embedding Predictive Architecture with MCA to adeptly capture intricate semantics and precise object boundaries. Our approach addresses two critical challenges in self-supervised learning: 1) extracting comprehensive representations for universal image segmentation from a pixel decoder, and 2) effectively training the transformer decoder. The use of the transformer decoder as a predictor within the JEPA framework allows proficient training in universal image segmentation tasks. Through rigorous evaluations on datasets such as ADE20K, Cityscapes and COCO, Mask-JEPA demonstrates not only competitive results but also exceptional adaptability and robustness across various training scenarios. The architecture-agnostic nature of Mask-JEPA further underscores its versatility, allowing seamless adaptation to various mask classification family.

cs.CV↗

arXiv2024

GrowOVER: How Can LLMs Adapt to Growing Real-World Knowledge?

In the real world, knowledge is constantly evolving, which can render existing knowledge-based datasets outdated. This unreliability highlights the critical need for continuous updates to ensure both accuracy and relevance in knowledge-intensive tasks. To address this, we propose GrowOVER-QA and GrowOVER-Dialogue, dynamic open-domain QA and dialogue benchmarks that undergo a continuous cycle of updates, keeping pace with the rapid evolution of knowledge. Our research indicates that retrieval-augmented language models (RaLMs) struggle with knowledge that has not been trained on or recently updated. Consequently, we introduce a novel retrieval-interactive language model framework, where the language model evaluates and reflects on its answers for further re-retrieval. Our exhaustive experiments demonstrate that our training-free framework significantly improves upon existing methods, performing comparably to or even surpassing continuously trained language models.

cs.CL↗

arXiv2024

Groupwise Query Specialization and Quality-Aware Multi-Assignment for Transformer-based Visual Relationship Detection

Visual Relationship Detection (VRD) has seen significant advancements with Transformer-based architectures recently. However, we identify two key limitations in a conventional label assignment for training Transformer-based VRD models, which is a process of mapping a ground-truth (GT) to a prediction. Under the conventional assignment, an unspecialized query is trained since a query is expected to detect every relation, which makes it difficult for a query to specialize in specific relations. Furthermore, a query is also insufficiently trained since a GT is assigned only to a single prediction, therefore near-correct or even correct predictions are suppressed by being assigned no relation as a GT. To address these issues, we propose Groupwise Query Specialization and Quality-Aware Multi-Assignment (SpeaQ). Groupwise Query Specialization trains a specialized query by dividing queries and relations into disjoint groups and directing a query in a specific query group solely toward relations in the corresponding relation group. Quality-Aware Multi-Assignment further facilitates the training by assigning a GT to multiple predictions that are significantly close to a GT in terms of a subject, an object, and the relation in between. Experimental results and analyses show that SpeaQ effectively trains specialized queries, which better utilize the capacity of a model, resulting in consistent performance gains with zero additional inference cost across multiple VRD models and benchmarks. Code is available at https://github.com/mlvlab/SpeaQ.

cs.CV↗

arXiv2024

Pulse shape discrimination in an organic scintillation phoswich detector using machine learning techniques

We developed machine learning algorithms for distinguishing scintillation signals from a plastic-liquid coupled detector known as a phoswich. The challenge lies in discriminating signals from organic scintillators with similar shapes and short decay times. Using a single-readout phoswich detector, we successfully identified $γ$ radiation signals from two scintillating components. Our Boosted Decision Tree algorithm demonstrated a maximum discrimination power of 3.02 $\pm$ 0.85 standard deviation in the 950 keV region, providing an efficient solution for self-shielding and enhancing radiation detection capabilities.

physics.ins-det↗

arXiv2022

SAICL: Student Modelling with Interaction-level Auxiliary Contrastive Tasks for Knowledge Tracing and Dropout Prediction

Knowledge tracing and dropout prediction are crucial for online education to estimate students' knowledge states or to prevent dropout rates. While traditional systems interacting with students suffered from data sparsity and overfitting, recent sample-level contrastive learning helps to alleviate this issue. One major limitation of sample-level approaches is that they regard students' behavior interaction sequences as a bundle, so they often fail to encode temporal contexts and track their dynamic changes, making it hard to find optimal representations for knowledge tracing and dropout prediction. To apply temporal context within the sequence, this study introduces a novel student modeling framework, SAICL: \textbf{s}tudent modeling with \textbf{a}uxiliary \textbf{i}nteraction-level \textbf{c}ontrastive \textbf{l}earning. In detail, SAICL can utilize both proposed self-supervised/supervised interaction-level contrastive objectives: MilCPC (\textbf{M}ulti-\textbf{I}nteraction-\textbf{L}evel \textbf{C}ontrastive \textbf{P}redictive \textbf{C}oding) and SupCPC (\textbf{Sup}ervised \textbf{C}ontrastive \textbf{P}redictive \textbf{C}oding). While previous sample-level contrastive methods for student modeling are highly dependent on data augmentation methods, the SAICL is free of data augmentation while showing better performance in both self-supervised and supervised settings. By combining cross-entropy with contrastive objectives, the proposed SAICL achieved comparable knowledge tracing and dropout prediction performance with other state-of-art models without compromising inference costs.

cs.CY↗

arXiv2022

Investigation of the broadband emission of the gamma-ray binary HESS J0632+057 using an intrabinary shock model

We investigated a wealth of X-ray and gamma-ray spectral energy distribution (SED) and multi-band light curve (LC) data of the gamma-ray binary HESS J0632+057 using a phenomenological intrabinary shock (IBS) model. Our baseline model assumes that the IBS is formed by colliding winds from a putative pulsar and its Be companion, and particles accelerated in the IBS emit broadband radiation via synchrotron (SY) and inverse-Compton upscattering (ICS) processes. Adopting the latest orbital solution and system geometry (Tokayer et al. 2021), we reproduced the global X-ray and TeV LC features, two broad bumps at $ϕ\sim 0.3$ and $\sim0.7$, with the SY and ICS model components. We found these TeV LC peaks originate from ICS emission caused by the enhanced seed photon density near periastron and superior conjunction or Doppler-beamed emission of bulk-accelerated particles in the IBS at inferior conjunction. While our IBS model successfully explained most of the observed SED and LC data, we found that phase-resolved SED data in the TeV band require an additional component associated with ICS emission from pre-shock particles (produced by the pulsar wind). This finding indicates a possibility of delineating the IBS emission components and determining the bulk Lorentz factors of the pulsar wind at certain orbital phases.

astro-ph.HE↗

arXiv2022

NaI(Tl) crystal scintillator encapsulated in two organic-scintillator layers with pulse shape data analysis

Thallium-doped sodium iodide (NaI(Tl)) crystals are widely used in radiation detection applications, from gamma-ray spectroscopy to particle dark matter searches. However, if the crystal is exposed to relative humidity of even a few percent, its light emission degrades, making the crystal impractical as a detector. Surrounding the crystal with organic scintillators not only protects the surface of the crystal from humid air but also offers a new capability to tag backgrounds such as external gamma rays and surface contaminations. We developed a detector that is constructed by fully encasing a NaI(Tl) crystal in a plastic scintillator and then immersing the plastic-crystal assembly in liquid scintillator. Using data collected from this triple phoswich detector, a pulse shape analysis is able to identify the various radiation signals from the three scintillators. Additionally, we find that the crystal's emission quality is maintained for a month.

physics.ins-det↗

arXiv2020

Deep Cerebellar Nuclei Segmentation via Semi-Supervised Deep Context-Aware Learning from 7T Diffusion MRI

Deep cerebellar nuclei are a key structure of the cerebellum that are involved in processing motor and sensory information. It is thus a crucial step to accurately segment deep cerebellar nuclei for the understanding of the cerebellum system and its utility in deep brain stimulation treatment. However, it is challenging to clearly visualize such small nuclei under standard clinical magnetic resonance imaging (MRI) protocols and therefore precise segmentation is not feasible. Recent advances in 7 Tesla (T) MRI technology and great potential of deep neural networks facilitate automatic patient-specific segmentation. In this paper, we propose a novel deep learning framework (referred to as DCN-Net) for fast, accurate, and robust patient-specific segmentation of deep cerebellar dentate and interposed nuclei on 7T diffusion MRI. DCN-Net effectively encodes contextual information on the patch images without consecutive pooling operations and adding complexity via proposed dilated dense blocks. During the end-to-end training, label probabilities of dentate and interposed nuclei are independently learned with a hybrid loss, handling highly imbalanced data. Finally, we utilize self-training strategies to cope with the problem of limited labeled data. To this end, auxiliary dentate and interposed nuclei labels are created on unlabeled data by using DCN-Net trained on manual labels. We validate the proposed framework using 7T B0 MRIs from 60 subjects. Experimental results demonstrate that DCN-Net provides better segmentation than atlas-based deep cerebellar nuclei segmentation tools and other state-of-the-art deep neural networks in terms of accuracy and consistency. We further prove the effectiveness of the proposed components within DCN-Net in dentate and interposed nuclei segmentation.

eess.IV↗

arXiv2019

Attention-Aware Linear Depthwise Convolution for Single Image Super-Resolution

Although deep convolutional neural networks (CNNs) have obtained outstanding performance in image superresolution (SR), their computational cost increases geometrically as CNN models get deeper and wider. Meanwhile, the features of intermediate layers are treated equally across the channel, thus hindering the representational capability of CNNs. In this paper, we propose an attention-aware linear depthwise network to address the problems for single image SR, named ALDNet. Specifically, linear depthwise convolution allows CNN-based SR models to preserve useful information for reconstructing a super-resolved image while reducing computational burden. Furthermore, we design an attention-aware branch that enhances the representation ability of depthwise convolution layers by making full use of depthwise filter interdependency. Experiments on publicly available benchmark datasets show that ALDNet achieves superior performance to traditional depthwise separable convolutions in terms of quantitative measurements and visual quality.

eess.IV↗

arXiv2019

Continuous Dice Coefficient: a Method for Evaluating Probabilistic Segmentations

Objective: Overlapping measures are often utilized to quantify the similarity between two binary regions. However, modern segmentation algorithms output a probability or confidence map with continuous values in the zero-to-one interval. Moreover, these binary overlapping measures are biased to structure size. Addressing these challenges is the objective of this work. Methods: We extend the definition of the classical Dice coefficient (DC) overlap to facilitate the direct comparison of a ground truth binary image with a probabilistic map. We call the extended method continuous Dice coefficient (cDC) and show that 1) cDC is less or equal to 1 and cDC = 1 if-and-only-if the structures overlap is complete, and, 2) cDC is monotonically decreasing with the amount of overlap. We compare the classical DC and the cDC in a simulation of partial volume effects that incorporates segmentations of common targets for deep-brainstimulation. Lastly, we investigate the cDC for an automatic segmentation of the subthalamic-nucleus. Results: Partial volume effect simulation on thalamus (large structure) resulted with DC and cDC averages (SD) of 0.98 (0.006) and 0.99 (0.001), respectively. For subthalamic-nucleus (small structure) DC and cDC were 0.86 (0.025) and 0.97 (0.006), respectively. The DC and cDC for automatic STN segmentation were 0.66 and 0.80, respectively. Conclusion: The cDC is well defined for probabilistic segmentation, less biased to structure size and more robust to partial volume effects in comparison to DC. Significance: The proposed method facilitates a better evaluation of segmentation algorithms. As a better measurement tool, it opens the door for the development of better segmentation methods.

cs.CV↗

arXiv2018

Inception-Residual Block based Neural Network for Thermal Image Denoising

Thermal cameras show noisy images due to their limited thermal resolution, especially for the scenes of a low temperature difference. In order to deal with a noise problem, this paper proposes a novel neural network architecture with repeatable denoising inception-residual blocks(DnIRB) for noise learning. Each DnIRB has two sub-blocks with difference receptive fields and one shortcut connection to prevent a vanishing gradient problem. The proposed approach is tested for thermal images. The experimental results indicate that the proposed approach shows the best SQNR performance and reasonable processing time compared with state-of-the-art denoising methods.

cs.CV↗

arXiv2011

Auto-Guiding System for CQUEAN (Camera for QUasars in EArly uNiverse)

To perform imaging observation of optically red objects such as high redshift quasars and brown dwarfs, the Center for the Exploration of the Origin of the Universe (CEOU) recently developed an optical CCD camera, Camera for QUasars in EArly uNiverse(CQUEAN), which is sensitive at 0.7-1.1 um. To enable observations with long exposures, we developed an auto-guiding system for CQUEAN. This system consists of an off-axis mirror, a baffle, a CCD camera, a motor and a differential decelerator. To increase the number of available guiding stars, we designed a rotating mechanism for the off-axis guiding camera. The guiding field can be scanned along the 10 acrmin ring offset from the optical axis of the telescope. Combined with the auto-guiding software of the McDonald Observatory, we confirmed that a stable image can be obtained with an exposure time as long as 1200 seconds.

astro-ph.IM↗