Search arXiv⌕ Search

arXiv subjects

Yijun Hu

Publications and source records attributed to Yijun Hu.

6 recordsLinked to original sources

Exploiting Target Knowledge from MLLMs for Robust Few-Shot Segmentation

Few-shot segmentation (FSS) aims to segment unseen object categories with a few (e.g., one or five) labeled examples, enabling efficient adaptation to novel classes. Conventional models typically rely on appearance-based visual matching between support and query images for segmentation. While straightforward, these methods often struggle to handle significant appearance discrepancies and occlusions in the query image due to insufficient target knowledge. To mitigate this, we introduce a novel framework that mines target knowledge using the strong reasoning capacity of Multimodal Large Language Models (MLLMs) and employs it to enhance FSS. Specifically, building on SAM 2, our method, named MK-FSS, exploits two forms of complementary knowledge derived from a query image by an MLLM for FSS, including spatial knowledge, which provides a spatial prior indicating the potential target location, and semantic knowledge, which describes the target using text. The spatial knowledge is first encoded into a memory representation, and then resulting memory is integrated with the support-guided memory feature from query image through a carefully designed dual-memory debate-fusion (DMDF) module, yielding a more robust target memory feature. In parallel, the semantic knowledge is encoded into the textual feature, which is fused with multi-scale query features via a progressive cross-modal prompt generator (PCPG), producing a target-aware multimodal prompt for segmentation. Working together, the dual-memory feature and the multimodal prompt provide a comprehensive representation of the target, enabling more robust segmentation. In our extensive experiments, MK-FSS shows promising results and largely surpasses existing methods. Code will be released.

cs.CV↗

On data-driven robust distortion risk measures for non-negative risks with partial information

In this paper, by proposing two new kinds of distributional uncertainty sets, we explore robustness of distortion risk measures against distributional uncertainty. To be precise, we first consider a distributional uncertainty set which is characterized solely by a ball determined by general Wasserstein distance centered at certain empirical distribution function, and then further consider additional constraints of known first moment and any other higher moment of the underlying loss distribution function. Under the assumption that the distortion function is strictly concave and twice differentiable, and that the underlying loss random variable is non-negative and bounded, we derive closed-form expressions for the distribution functions which maximize a given distortion risk measure over the distributional uncertainty sets respectively. Moreover, we continue to study the general case of a concave distortion function and unbounded loss random variables. Comparisons with existing studies are also made. Finally, we provide a numerical study to illustrate the proposed models and results. Our work provides a novel generalization of several known achievements in the literature.

q-fin.RM↗

Robust Ego-Exo Correspondence with Long-Term Memory

Establishing object-level correspondence between egocentric and exocentric views is essential for intelligent assistants to deliver precise and intuitive visual guidance. However, this task faces numerous challenges, including extreme viewpoint variations, occlusions, and the presence of small objects. Existing approaches usually borrow solutions from video object segmentation models, but still suffer from the aforementioned challenges. Recently, the Segment Anything Model 2 (SAM 2) has shown strong generalization capabilities and excellent performance in video object segmentation. Yet, when simply applied to the ego-exo correspondence (EEC) task, SAM 2 encounters severe difficulties due to ineffective ego-exo feature fusion and limited long-term memory capacity, especially for long videos. Addressing these problems, we propose a novel EEC framework based on SAM 2 with long-term memories by presenting a dual-memory architecture and an adaptive feature routing module inspired by Mixture-of-Experts (MoE). Compared to SAM 2, our approach features (i) a Memory-View MoE module which consists of a dual-branch routing mechanism to adaptively assign contribution weights to each expert feature along both channel and spatial dimensions, and (ii) a dual-memory bank system with a simple yet effective compression strategy to retain critical long-term information while eliminating redundancy. In the extensive experiments on the challenging EgoExo4D benchmark, our method, dubbed LM-EEC, achieves new state-of-the-art results and significantly outperforms existing methods and the SAM 2 baseline, showcasing its strong generalization across diverse scenarios. Our code and model are available at https://github.com/juneyeeHu/LM-EEC.

cs.CV↗

On conditional distortion risk measures under uncertainty

Model uncertainty has been one prominent issue both in the theory of risk measures and in practice such as financial risk management and regulation. Motivated by this observation, in this paper, we take a new perspective to describe the model uncertainty, and thus propose a new class of risk measures under model uncertainty. More precisely, we use an auxiliary random variable to describe the model uncertainty. We first establish a conditional distortion risk measure under an auxiliary random variable. Then we axiomatically characterize it by proposing a set of new axioms. Moreover, its coherence and dual representation are investigated. Finally, we make comparisons with some known risk measures such as weighted value at risk (WVaR), range value at risk (RVaR) and $\sQ-$ mixture of ES. One advantage of our modeling is in its flexibility, as the auxiliary random variable can describe various contexts including model uncertainty. To illustrate the proposed framework, we also deduce new risk measures in the presence of background risk.This paper provides some theoretical results about risk measures under model uncertainty, being expected to make meaningful complement to the study of risk measures under model uncertainty.

econ.TH↗

On evaluation of joint risk for non-negative multivariate risks under dependence uncertainty

In this paper, we propose a novel axiomatic approach to evaluating the joint risk of multiple insurance risks under dependence uncertainty. Motivated by both the theory of expected utility and the Cobb-Dauglas utility function, we establish a joint risk measure for non-negative multivariate risks, which we refer to as a scalar distortion joint risk measure. After having studied its fundamental properties, we provide an axiomatic characterization of it by proposing a set of new axioms. The most novel axiom is the component-wise positive homogeneity. Then, based on the resulting distortion joint risk measures, we also propose a new class of vector-valued distortion joint risk measures for non-negative multivariate risks. Finally, we make comparisons with some vector-valued multivariate risk measures known in the literature, such as multivariate lower-orthant value at risk, multivariate upper-orthant conditional-tail-expectation, multivariate tail conditional expectation and multivariate tail distortion risk measures. It turns out that those vector-valued multivariate risk measures have forms of vector-valued distortion joint risk measures, respectively. This paper mainly gives some theoretical results about the evaluation of joint risk under dependence uncertainty, and it is expected to be helpful for measuring joint risk.

q-fin.RM↗

Quasiconvex risk measures with markets volatility

Since the quasiconvex risk measures is a bigger class than the well known convex risk measures, the study of quasiconvex risk measures makes sense especially in the financial markets with volatility. In this paper, we will study the quasiconvex risk measures defined on a special space $L^{p(\cdot)}$ where the variable exponent $p(\cdot)$ is no longer a given real number like the space $L^{p}$, but a random variable, which reflects the possible volatility of the financial markets. The dual representation for this quasiconvex risk measures will also provided.

q-fin.RM↗