arXiv subjects

Qing Cui

Publications and source records attributed to Qing Cui.

51 records · Page 3Linked to original sources

arXiv2024

Enhancing Recommender Systems with Large Language Model Reasoning Graphs

Recommendation systems aim to provide users with relevant suggestions, but often lack interpretability and fail to capture higher-level semantic relationships between user behaviors and profiles. In this paper, we propose a novel approach that leverages large language models (LLMs) to construct personalized reasoning graphs. These graphs link a user's profile and behavioral sequences through causal and logical inferences, representing the user's interests in an interpretable way. Our approach, LLM reasoning graphs (LLMRG), has four components: chained graph reasoning, divergent extension, self-verification and scoring, and knowledge base self-improvement. The resulting reasoning graph is encoded using graph neural networks, which serves as additional input to improve conventional recommender systems, without requiring extra user or item information. Our approach demonstrates how LLMs can enable more logical and interpretable recommender systems through personalized reasoning graphs. LLMRG allows recommendations to benefit from both engineered recommendation systems and LLM-derived reasoning graphs. We demonstrate the effectiveness of LLMRG on benchmarks and real-world scenarios in enhancing base recommendation models.

cs.IR↗

arXiv2024

Task-Driven Causal Feature Distillation: Towards Trustworthy Risk Prediction

Since artificial intelligence has seen tremendous recent successes in many areas, it has sparked great interest in its potential for trustworthy and interpretable risk prediction. However, most models lack causal reasoning and struggle with class imbalance, leading to poor precision and recall. To address this, we propose a Task-Driven Causal Feature Distillation model (TDCFD) to transform original feature values into causal feature attributions for the specific risk prediction task. The causal feature attribution helps describe how much contribution the value of this feature can make to the risk prediction result. After the causal feature distillation, a deep neural network is applied to produce trustworthy prediction results with causal interpretability and high precision/recall. We evaluate the performance of our TDCFD method on several synthetic and real datasets, and the results demonstrate its superiority over the state-of-the-art methods regarding precision, recall, interpretability, and causality.

cs.LG↗

arXiv2023

Unleashing the Power of Graph Data Augmentation on Covariate Distribution Shift

The issue of distribution shifts is emerging as a critical concern in graph representation learning. From the perspective of invariant learning and stable learning, a recently well-established paradigm for out-of-distribution generalization, stable features of the graph are assumed to causally determine labels, while environmental features tend to be unstable and can lead to the two primary types of distribution shifts. The correlation shift is often caused by the spurious correlation between environmental features and labels that differs between the training and test data; the covariate shift often stems from the presence of new environmental features in test data. However, most strategies, such as invariant learning or graph augmentation, typically struggle with limited training environments or perturbed stable features, thus exposing limitations in handling the problem of covariate shift. To address this challenge, we propose a simple-yet-effective data augmentation strategy, Adversarial Invariant Augmentation (AIA), to handle the covariate shift on graphs. Specifically, given the training data, AIA aims to extrapolate and generate new environments, while concurrently preserving the original stable features during the augmentation process. Such a design equips the graph classification model with an enhanced capability to identify stable features in new environments, thereby effectively tackling the covariate shift in data. Extensive experiments with in-depth empirical analysis demonstrate the superiority of our approach. The implementation codes are publicly available at https://github.com/yongduosui/AIA.

cs.LG↗

arXiv2023

Data-Centric Financial Large Language Models

Large language models (LLMs) show promise for natural language tasks but struggle when applied directly to complex domains like finance. LLMs have difficulty reasoning about and integrating all relevant information. We propose a data-centric approach to enable LLMs to better handle financial tasks. Our key insight is that rather than overloading the LLM with everything at once, it is more effective to preprocess and pre-understand the data. We create a financial LLM (FLLM) using multitask prompt-based finetuning to achieve data pre-processing and pre-understanding. However, labeled data is scarce for each task. To overcome manual annotation costs, we employ abductive augmentation reasoning (AAR) to automatically generate training data by modifying the pseudo labels from FLLM's own outputs. Experiments show our data-centric FLLM with AAR substantially outperforms baseline financial LLMs designed for raw text, achieving state-of-the-art on financial analysis and interpretation tasks. We also open source a new benchmark for financial analysis and interpretation. Our methodology provides a promising path to unlock LLMs' potential for complex real-world domains.

cs.CL↗

arXiv2023

Enhancing Asynchronous Time Series Forecasting with Contrastive Relational Inference

Asynchronous time series, also known as temporal event sequences, are the basis of many applications throughout different industries. Temporal point processes(TPPs) are the standard method for modeling such data. Existing TPP models have focused on parameterizing the conditional distribution of future events instead of explicitly modeling event interactions, imposing challenges for event predictions. In this paper, we propose a novel approach that leverages Neural Relational Inference (NRI) to learn a relation graph that infers interactions while simultaneously learning the dynamics patterns from observational data. Our approach, the Contrastive Relational Inference-based Hawkes Process (CRIHP), reasons about event interactions under a variational inference framework. It utilizes intensity-based learning to search for prototype paths to contrast relationship constraints. Extensive experiments on three real-world datasets demonstrate the effectiveness of our model in capturing event interactions for event sequence modeling tasks. Code will be integrated into the EasyTPP framework.

cs.LG↗

arXiv2023

Leveraging Large Language Models for Pre-trained Recommender Systems

Recent advancements in recommendation systems have shifted towards more comprehensive and personalized recommendations by utilizing large language models (LLM). However, effectively integrating LLM's commonsense knowledge and reasoning abilities into recommendation systems remains a challenging problem. In this paper, we propose RecSysLLM, a novel pre-trained recommendation model based on LLMs. RecSysLLM retains LLM reasoning and knowledge while integrating recommendation domain knowledge through unique designs of data, training, and inference. This allows RecSysLLM to leverage LLMs' capabilities for recommendation tasks in an efficient, unified framework. We demonstrate the effectiveness of RecSysLLM on benchmarks and real-world scenarios. RecSysLLM provides a promising approach to developing unified recommendation systems by fully exploiting the power of pre-trained language models.

cs.IR↗

arXiv2022

Robust Direct Learning for Causal Data Fusion

In the era of big data, the explosive growth of multi-source heterogeneous data offers many exciting challenges and opportunities for improving the inference of conditional average treatment effects. In this paper, we investigate homogeneous and heterogeneous causal data fusion problems under a general setting that allows for the presence of source-specific covariates. We provide a direct learning framework for integrating multi-source data that separates the treatment effect from other nuisance functions, and achieves double robustness against certain misspecification. To improve estimation precision and stability, we propose a causal information-aware weighting function motivated by theoretical insights from the semiparametric efficiency theory; it assigns larger weights to samples containing more causal information with high interpretability. We introduce a two-step algorithm, the weighted multi-source direct learner, based on constructing a pseudo-outcome and regressing it on covariates under a weighted least square criterion; it offers us a powerful tool for causal data fusion, enjoying the advantages of easy implementation, double robustness and model flexibility. In simulation studies, we demonstrate the effectiveness of our proposed methods in both homogeneous and heterogeneous causal data fusion scenarios.

stat.ML↗

arXiv2021

Multi-Interactive Attention Network for Fine-grained Feature Learning in CTR Prediction

In the Click-Through Rate (CTR) prediction scenario, user's sequential behaviors are well utilized to capture the user interest in the recent literature. However, despite being extensively studied, these sequential methods still suffer from three limitations. First, existing methods mostly utilize attention on the behavior of users, which is not always suitable for CTR prediction, because users often click on new products that are irrelevant to any historical behaviors. Second, in the real scenario, there exist numerous users that have operations a long time ago, but turn relatively inactive in recent times. Thus, it is hard to precisely capture user's current preferences through early behaviors. Third, multiple representations of user's historical behaviors in different feature subspaces are largely ignored. To remedy these issues, we propose a Multi-Interactive Attention Network (MIAN) to comprehensively extract the latent relationship among all kinds of fine-grained features (e.g., gender, age and occupation in user-profile). Specifically, MIAN contains a Multi-Interactive Layer (MIL) that integrates three local interaction modules to capture multiple representations of user preference through sequential behaviors and simultaneously utilize the fine-grained user-specific as well as context information. In addition, we design a Global Interaction Module (GIM) to learn the high-order interactions and balance the different impacts of multiple features. Finally, Offline experiment results from three datasets, together with an Online A/B test in a large-scale recommendation system, demonstrate the effectiveness of our proposed approach.

cs.IR↗

arXiv2020

Tight gaps in the cycle spectrum of 3-connected planar graphs

For any positive integer $k$, define $f(k)$ (respectively, $f_3(k)$) to be the minimal integer $\ge k$ such that every 3-connected planar graph $G$ (respectively, 3-connected cubic planar graph $G$) of circumference $\ge k$ has a cycle whose length is in the interval $[k, f(k)]$ (respectively, $[k, f_3(k)]$). Merker showed that $f_3(k) \le 2k + 9$ for any $k \ge 2$, and $f_3(k) \ge 2k + 2$ for any even $k \ge 4$. He conjectured that $f_3(k) \le 2k + 2$ for any $k \ge 2$. This conjecture was disproved by Zamfirescu, who gave an infinite family of counterexamples for every even $k \ge 6$ whose graphs have no cycle length in $[k, 2k + 2]$, i.e. $f_3(k) \ge 2k + 3$ for any even $k \ge 6$. However, the exact value of $f_3(k)$ was only known for $k \le 4$, and it was left open to determine $f_3(k)$ for $k \ge 5$. In this paper we improve Merker's upper bound, and give the exact value of $f_3(k)$ for every $k \ge 5$. We show that $f_3(5) = 10$, $f_3(7) = 15$, $f_3(9) = 20$, and $f_3(k) = 2k + 3$ for any $k = 6, 8$ or $\ge 10$. For general 3-connected planar graphs, Merker conjectured that there exists some positive integer $c$ such that $f(k) \le 2k + c$ for any positive integer $k$. We give a complete positive answer to this conjecture. We prove that $f(k) = 5$ for any $k \le 3$, $f(4) = 10$, and $f(k) = 2k + 3$ for any $k \ge 5$.

math.CO↗

arXiv2019

Value-aware Recommendation based on Reinforced Profit Maximization in E-commerce Systems

Existing recommendation algorithms mostly focus on optimizing traditional recommendation measures, such as the accuracy of rating prediction in terms of RMSE or the quality of top-$k$ recommendation lists in terms of precision, recall, MAP, etc. However, an important expectation for commercial recommendation systems is to improve the final revenue/profit of the system. Traditional recommendation targets such as rating prediction and top-$k$ recommendation are not directly related to this goal. In this work, we blend the fundamental concepts in online advertising and micro-economics into personalized recommendation for profit maximization. Specifically, we propose value-aware recommendation based on reinforcement learning, which directly optimizes the economic value of candidate items to generate the recommendation list. In particular, we generalize the basic concept of click conversion rate (CVR) in computational advertising into the conversation rate of an arbitrary user action (XVR) in E-commerce, where the user actions can be clicking, adding to cart, adding to wishlist, etc. In this way, each type of user action is mapped to its monetized economic value. Economic values of different user actions are further integrated as the reward of a ranking list, and reinforcement learning is used to optimize the recommendation list for the maximum total value. Experimental results in both offline benchmarks and online commercial systems verified the improved performance of our framework, in terms of both traditional top-$k$ ranking tasks and the economic profits of the system.

cs.IR↗

arXiv2018

Some sharp differential sphere theorems for nonnegative scalar curvature manifolds

In this paper, we obtain several new intrinsic and extrinsic differential sphere theorems via Ricci flow. For intrinsic case, we show that a closed simply connected $n(\ge 4)$-dimensional Riemannian manifold $M$ is diffeomorphic to $S^n$ if one of the following conditions holds pointwisely: $$ (i)\ R_0>\left(1-\frac{24(\sqrt{10}-3)}{n(n-1)}\right)K_{max};\quad \ (ii)\ \frac{Ric^{[4]}}{4(n-1)}>\left(1-\frac{6(\sqrt{10}-3)}{n-1}\right)K_{max}.$$ Here $K_{max}$, $Ric^{[k]}$ and $R_0$ stand for the maximal sectional curvature, the $k$-th weak Ricci curvature and the normalized scalar curvature. For extrinsic case, i.e., when $M$ is a closed simply connected $n(\ge 4)$-dimensional submanifold immersed in $\bar{M}$. We prove that $M$ is diffeomorphic to $S^n$ if it satisfies some pinching curvature conditions. The only involved extrinsic quantities in our pinching conditions are the maximal sectional curvature $\bar K_{max}$ and the squared norm of mean curvature vector $\vert H\vert^2$. More precisely, we show that $M$ is diffeomorphic to $S^n$ if one of the following conditions holds: \begin{itemize} \item[(1)] $R_0\ge \left(1-\frac{2}{n(n-1)}\right)\bar{K}_{max} +\frac{n(n-2)}{(n-1)^2}\vert H\vert^2$, and strict inequality is achieved at some point; \item[(2)] $\dfrac{Ric^{[2]}}{2}\ge (n-2)\bar K_{max}+\frac{n^2}{8}\vert H\vert^2,$ and strict inequality is achieved at some point; \item[(3)] $\dfrac{Ric^{[2]}}{2} \ge\frac{n(n-3)}{n-2}\left(\bar K_{max}+\vert H\vert^2\right),$ and strict inequality is achieved at some point. \end{itemize} It is worth pointing out that, in the proof of extrinsic case, we apply suitable complex orthonormal frame and simplify the calculations considerably. We also emphasize that both of the pinching constants in (2) and (3) are optimal for $n=4$.

math.DG↗

arXiv2017

Optimal lower eigenvalue estimates for Hodge-Laplacian and applications

In this paper, we consider the eigenvalue problem for Hodge-Laplacian on a Riemannian manifold $M$ isometrically immersed into another Riemannian manifold $\bar M$ for arbitrary codimension. We first assume the pull back Weitzenböck operator (defined in Section 2) of $\bar M$ bounded from below, and obtain an extrinsic lower bound for the first eigenvalue of Hodge-Laplacian. As applications, we obtain some rigidity results and a homology sphere theorem. Second, when the pull back Weitzenböck operator of $\bar M$ bounded from both sides, we give a lower bound of the first eigenvalue by the Ricci curvature of $M$ and some extrinsic geometry. As a consequence, we prove a weak Ejiri type theorem, that is, if the Ricci curvature bounded from below pointwisely by a function of the norm square of the mean curvature vector, then $M$ is a homology sphere. In the end, we give an example to show that all the eigenvalue estimates and homology sphere theorems are optimal when $\bar M$ has constant curvature.

math.DG↗

arXiv2017

On the volume of locally conformally flat 4 dimensional hypersphere

Let $M$ be a 5 dimensional Riemannian manifold with $Sec_M\in[0,1]$, $Σ$ be a locally conformally flat hypersphere in $M$ with mean curvature $H$. We prove that, there exists $\varepsilon_0>0$, such that $\int_Σ(1+H^2)^2 \ge 8π^2/3$, provided $H \le \varepsilon_0$. In particular, if $Σ$ is a locally conformally flat minimal hypersphere in $M$, then $Vol(Σ) \ge 8π^2/3$, which partially answer a question proposed by Mazet and Rosenberg \cite{Ma&Rosen}. For an $(n+1)-$ dimensional rotationally symmetric Riemannian manifold $M$, we show that an immersed hypersurface $Σ$ is locally conformally flat if and only if ($n-1$) of the principal curvatures of $Σ$ are the same, which is a generalization of Cartan's result \cite{Cartan}. As an application, we prove that if $M$ is (some special but large class) rotationally symmetric 5-manifold with $Sec_M\in [0,1]$, and $Σ$ is a locally conformally flat hypersphere with mean curvature $H$, the inequality $\int_Σ(1+H^2)^2 \ge 8π^2/3$ holds for all $H$.

math.DG↗

arXiv2016

Second Order Necessary Conditions for Optimal Control Problems on Riemannian Manifolds

This work is concerned with an optimal control problem on a Riemannian manifold, for which two typical cases are considered. The first case is when the endpoint is free. For this case, the control set is assumed to be a separable metric space. By introducing suitable dual equations, which depend on the curvature tensor of the manifold, we establish the second order necessary and sufficient optimality conditions of integral form. In particular, when the control set is a Polish space, the second order necessary condition is reduced to a pointwise form. As a key preliminary result and also an interesting byproduct, we derive a geometric lemma, which may have some independent interest. The second case is when the endpoint is fixed. For this more difficult case, the control set is assumed to be open in an Euclidian space. We obtain the second order necessary and sufficient optimality conditions, in which the curvature tensor also appears explicitly. Our optimality conditions can be used to recover the following famous geometry result: Any geodesic connecting two fixed points on a Riemannian manifold satisfies the second variation of energy; while the existing optimality conditions in control literatures fail to give the same result.

math.OC↗

arXiv2014

KNET: A General Framework for Learning Word Embedding using Morphological Knowledge

Neural network techniques are widely applied to obtain high-quality distributed representations of words, i.e., word embeddings, to address text mining, information retrieval, and natural language processing tasks. Recently, efficient methods have been proposed to learn word embeddings from context that captures both semantic and syntactic relationships between words. However, it is challenging to handle unseen words or rare words with insufficient context. In this paper, inspired by the study on word recognition process in cognitive psychology, we propose to take advantage of seemingly less obvious but essentially important morphological knowledge to address these challenges. In particular, we introduce a novel neural network architecture called KNET that leverages both contextual information and morphological word similarity built based on morphological knowledge to learn word embeddings. Meanwhile, the learning architecture is also able to refine the pre-defined morphological knowledge and obtain more accurate word similarity. Experiments on an analogical reasoning task and a word similarity task both demonstrate that the proposed KNET framework can greatly enhance the effectiveness of word embeddings.

cs.CL↗