arXiv subjects

Lei Shi

Publications and source records attributed to Lei Shi.

At least 199 records · Page 11Linked to original sources

arXiv2024

Forward selection and post-selection inference in factorial designs

Ever since the seminal work of R. A. Fisher and F. Yates, factorial designs have been an important experimental tool to simultaneously estimate the effects of multiple treatment factors. In factorial designs, the number of treatment combinations grows exponentially with the number of treatment factors, which motivates the forward selection strategy based on the sparsity, hierarchy, and heredity principles for factorial effects. Although this strategy is intuitive and has been widely used in practice, its rigorous statistical theory has not been formally established. To fill this gap, we establish design-based theory for forward factor selection in factorial designs based on the potential outcome framework. We not only prove a consistency property for the factor selection procedure but also discuss statistical inference after factor selection. In particular, with selection consistency, we quantify the advantages of forward selection based on asymptotic efficiency gain in estimating factorial effects. With inconsistent selection in higher-order interactions, we propose two strategies and investigate their impact on subsequent inference. Our formulation differs from the existing literature on variable selection and post-selection inference because our theory is based solely on the physical randomization of the factorial design and does not rely on a correctly specified outcome model.

stat.ME↗

arXiv2024

DONAPI: Malicious NPM Packages Detector using Behavior Sequence Knowledge Mapping

With the growing popularity of modularity in software development comes the rise of package managers and language ecosystems. Among them, npm stands out as the most extensive package manager, hosting more than 2 million third-party open-source packages that greatly simplify the process of building code. However, this openness also brings security risks, as evidenced by numerous package poisoning incidents. In this paper, we synchronize a local package cache containing more than 3.4 million packages in near real-time to give us access to more package code details. Further, we perform manual inspection and API call sequence analysis on packages collected from public datasets and security reports to build a hierarchical classification framework and behavioral knowledge base covering different sensitive behaviors. In addition, we propose the DONAPI, an automatic malicious npm packages detector that combines static and dynamic analysis. It makes preliminary judgments on the degree of maliciousness of packages by code reconstruction techniques and static analysis, extracts dynamic API call sequences to confirm and identify obfuscated content that static analysis can not handle alone, and finally tags malicious software packages based on the constructed behavior knowledge base. To date, we have identified and manually confirmed 325 malicious samples and discovered 2 unusual API calls and 246 API call sequences that have not appeared in known samples.

cs.CR↗

arXiv2024

Spectral Algorithms on Manifolds through Diffusion

The existing research on spectral algorithms, applied within a Reproducing Kernel Hilbert Space (RKHS), has primarily focused on general kernel functions, often neglecting the inherent structure of the input feature space. Our paper introduces a new perspective, asserting that input data are situated within a low-dimensional manifold embedded in a higher-dimensional Euclidean space. We study the convergence performance of spectral algorithms in the RKHSs, specifically those generated by the heat kernels, known as diffusion spaces. Incorporating the manifold structure of the input, we employ integral operator techniques to derive tight convergence upper bounds concerning generalized norms, which indicates that the estimators converge to the target function in strong sense, entailing the simultaneous convergence of the function itself and its derivatives. These bounds offer two significant advantages: firstly, they are exclusively contingent on the intrinsic dimension of the input manifolds, thereby providing a more focused analysis. Secondly, they enable the efficient derivation of convergence rates for derivatives of any k-th order, all of which can be accomplished within the ambit of the same spectral algorithms. Furthermore, we establish minimax lower bounds to demonstrate the asymptotic optimality of these conclusions in specific contexts. Our study confirms that the spectral algorithms are practically significant in the broader context of high-dimensional approximation.

stat.ML↗

arXiv2024

The de Haas-van Alphen quantum oscillations in the kagome metal RbTi3Bi5

Kagome system usually attracts great interest in condensed matter physics due to its unique structure hosting various exotic states such as superconductivity (SC), charge density wave (CDW), and nontrivial topological states. Topological semimetal RbTi3Bi5 consisting of the kagome layer of Ti shares a similar crystal structure to topological correlated materials AV3Sb5 (A = K, Rb, Cs) but with the absence of CDW and SC. Systematic de Haas-van Alphen (dHvA) oscillation measurements are performed on the single crystals of RbTi3Bi5 to pursue nontrivial topological physics and exotic states. Combining with theoretical calculations, detailed Fermi surface topology and band structure are investigated. A two-dimensional (2D) Fermi pocket \b{eta} is revealed with a light-effective mass in consistent with the semimetal predictions. Landau Fan of RbTi3Bi5 reveals a zero Berry phase for the \b{eta} oscillation in contrast to that of CsTi3Bi5. These results suggest the kagome RbTi3Bi5 is a good candidate to explore nontrivial topological exotic states and topological correlated physics.

cond-mat.str-el↗

arXiv2024

A Safe Screening Rule with Bi-level Optimization of $ν$ Support Vector Machine

Support vector machine (SVM) has achieved many successes in machine learning, especially for a small sample problem. As a famous extension of the traditional SVM, the $ν$ support vector machine ($ν$-SVM) has shown outstanding performance due to its great model interpretability. However, it still faces challenges in training overhead for large-scale problems. To address this issue, we propose a safe screening rule with bi-level optimization for $ν$-SVM (SRBO-$ν$-SVM) which can screen out inactive samples before training and reduce the computational cost without sacrificing the prediction accuracy. Our SRBO-$ν$-SVM is strictly deduced by integrating the Karush-Kuhn-Tucker (KKT) conditions, the variational inequalities of convex problems and the $ν$-property. Furthermore, we develop an efficient dual coordinate descent method (DCDM) to further improve computational speed. Finally, a unified framework for SRBO is proposed to accelerate many SVM-type models, and it is successfully applied to one-class SVM. Experimental results on 6 artificial data sets and 30 benchmark data sets have verified the effectiveness and safety of our proposed methods in supervised and unsupervised tasks.

cs.LG↗

arXiv2024

Self-Supervised Representation Learning with Meta Comprehensive Regularization

Self-Supervised Learning (SSL) methods harness the concept of semantic invariance by utilizing data augmentation strategies to produce similar representations for different deformations of the same input. Essentially, the model captures the shared information among multiple augmented views of samples, while disregarding the non-shared information that may be beneficial for downstream tasks. To address this issue, we introduce a module called CompMod with Meta Comprehensive Regularization (MCR), embedded into existing self-supervised frameworks, to make the learned representations more comprehensive. Specifically, we update our proposed model through a bi-level optimization mechanism, enabling it to capture comprehensive features. Additionally, guided by the constrained extraction of features using maximum entropy coding, the self-supervised learning model learns more comprehensive features on top of learning consistent features. In addition, we provide theoretical support for our proposed method from information theory and causal counterfactual perspective. Experimental results show that our method achieves significant improvement in classification, object detection and instance segmentation tasks on multiple benchmark datasets.

cs.CV↗

arXiv2024

Stability of pairwise social dilemma games: destructive agents, constructive agents, and their joint effects

Destructive agents, who opt out of the game and indiscriminately harm others, paradoxically foster cooperation, representing an intriguing variant of the voluntary participation strategy. Yet, their impact on cooperation remains inadequately understood, particularly in the context of pairwise social dilemma games and in comparison to their counterparts, constructive agents, who opt out of the game but indiscriminately benefit others. Furthermore, little is known about the combined effects of both agent types on cooperation dynamics. Using replicator dynamics in infinite and well-mixed populations, we find that, contrary to their role in facilitating cooperation in multiplayer games, destructive agents fail to encourage cooperation in pairwise social dilemmas. Instead, they destabilize and may even replace defection in the prisoners' dilemma and stag-hunt games. Similarly, in the chicken game, they can destabilize or replace the mixed equilibrium of cooperation and defection, and they undermine cooperation in the harmony game. Conversely, constructive agents, when their payoffs exceed their contributions to opponents, can exhibit effects similar to destructive agents. However, if their payoffs are lower, while they destabilize defection in prisoners' dilemma and stag-hunt games, they do not disrupt the cooperation equilibrium in harmony games and have a negligible impact on the coexistence of cooperation in chicken games. The combination of destructive and constructive agents does not facilitate cooperation but instead generates complex evolutionary dynamics, including bi-stable, tri-stable, and quad-stable states, with outcomes contingent on their relative payoffs and game types. These results, taken together, enhance our understanding of the impact of the voluntary participation mechanism on cooperation, contributing to a more comprehensive understanding of its influence.

physics.soc-ph↗

arXiv2024

Statistical Optimality of Divide and Conquer Kernel-based Functional Linear Regression

Previous analysis of regularized functional linear regression in a reproducing kernel Hilbert space (RKHS) typically requires the target function to be contained in this kernel space. This paper studies the convergence performance of divide-and-conquer estimators in the scenario that the target function does not necessarily reside in the underlying RKHS. As a decomposition-based scalable approach, the divide-and-conquer estimators of functional linear regression can substantially reduce the algorithmic complexities in time and memory. We develop an integral operator approach to establish sharp finite sample upper bounds for prediction with divide-and-conquer estimators under various regularity conditions of explanatory variables and target function. We also prove the asymptotic optimality of the derived rates by building the mini-max lower bounds. Finally, we consider the convergence of noiseless estimators and show that the rates can be arbitrarily fast under mild conditions.

stat.ML↗

arXiv2024

RJUA-QA: A Comprehensive QA Dataset for Urology

We introduce RJUA-QA, a novel medical dataset for question answering (QA) and reasoning with clinical evidence, contributing to bridge the gap between general large language models (LLMs) and medical-specific LLM applications. RJUA-QA is derived from realistic clinical scenarios and aims to facilitate LLMs in generating reliable diagnostic and advice. The dataset contains 2,132 curated Question-Context-Answer pairs, corresponding about 25,000 diagnostic records and clinical cases. The dataset covers 67 common urological disease categories, where the disease coverage exceeds 97.6\% of the population seeking medical services in urology. Each data instance in RJUA-QA comprises: (1) a question mirroring real patient to inquiry about clinical symptoms and medical conditions, (2) a context including comprehensive expert knowledge, serving as a reference for medical examination and diagnosis, (3) a doctor response offering the diagnostic conclusion and suggested examination guidance, (4) a diagnosed clinical disease as the recommended diagnostic outcome, and (5) clinical advice providing recommendations for medical examination. RJUA-QA is the first medical QA dataset for clinical reasoning over the patient inquiries, where expert-level knowledge and experience are required for yielding diagnostic conclusions and medical examination advice. A comprehensive evaluation is conducted to evaluate the performance of both medical-specific and general LLMs on the RJUA-QA dataset. Our data is are publicly available at \url{https://github.com/alipay/RJU_Ant_QA}.

cs.CL↗

arXiv2024

On the maximal and minimal degree components of the cocenter of the cyclotomic KLR algebras

Let $\mathscr{R}_α^Λ$ be the cyclotomic KLR algebra associated to a symmetrizable Kac-Moody Lie algebra $\mathfrak{g}$ and polynomials $\{Q_{ij}(u,v)\}_{i,j\in I}$. Shan, Varagnolo and Vasserot show that, when the ground field $K$ has characteristic $0$, the degree $d$ component of the cocenter $Tr(\mathscr{R}_α^Λ)$ is nonzero only if $0\leq d\leq d_{Λ,α}$. In this paper we show that this holds true for arbitrary ground field $K$, arbitrary $\mathfrak{g}$ and arbitrary polynomials $\{Q_{ij}(u,v)\}_{i,j\in I}$. We generalize our earlier results on the $K$-linear generators of $Tr(\mathscr{R}_α^Λ), Tr(\mathscr{R}_α^Λ)_0, Tr(\mathscr{R}_α^Λ)_{d_{Λ,α}}$ to arbitrary ground field $K$. Moreover, we show that the dimension of the degree $0$ component $Tr(\mathscr{R}_α^Λ)_0$ is always equal to $\dim V(Λ)_{Λ-α}$, where $V(Λ)$ is the integrable highest weight $U(\mathfrak{g})$-module with highest weight $Λ$, and we obtain a basis for $Tr(\mathscr{R}_α^Λ)_0$.

math.RT↗

arXiv2023

Mindful Explanations: Prevalence and Impact of Mind Attribution in XAI Research

When users perceive AI systems as mindful, independent agents, they hold them responsible instead of the AI experts who created and designed these systems. So far, it has not been studied whether explanations support this shift in responsibility through the use of mind-attributing verbs like "to think". To better understand the prevalence of mind-attributing explanations we analyse AI explanations in 3,533 explainable AI (XAI) research articles from the Semantic Scholar Open Research Corpus (S2ORC). Using methods from semantic shift detection, we identify three dominant types of mind attribution: (1) metaphorical (e.g. "to learn" or "to predict"), (2) awareness (e.g. "to consider"), and (3) agency (e.g. "to make decisions"). We then analyse the impact of mind-attributing explanations on awareness and responsibility in a vignette-based experiment with 199 participants. We find that participants who were given a mind-attributing explanation were more likely to rate the AI system as aware of the harm it caused. Moreover, the mind-attributing explanation had a responsibility-concealing effect: Considering the AI experts' involvement lead to reduced ratings of AI responsibility for participants who were given a non-mind-attributing or no explanation. In contrast, participants who read the mind-attributing explanation still held the AI system responsible despite considering the AI experts' involvement. Taken together, our work underlines the need to carefully phrase explanations about AI systems in scientific writing to reduce mind attribution and clearly communicate human responsibility.

cs.AI↗

arXiv2023

Neural Reasoning About Agents' Goals, Preferences, and Actions

We propose the Intuitive Reasoning Network (IRENE) - a novel neural model for intuitive psychological reasoning about agents' goals, preferences, and actions that can generalise previous experiences to new situations. IRENE combines a graph neural network for learning agent and world state representations with a transformer to encode the task context. When evaluated on the challenging Baby Intuitions Benchmark, IRENE achieves new state-of-the-art performance on three out of its five tasks - with up to 48.9% improvement. In contrast to existing methods, IRENE is able to bind preferences to specific agents, to better distinguish between rational and irrational agents, and to better understand the role of blocking obstacles. We also investigate, for the first time, the influence of the training tasks on test performance. Our analyses demonstrate the effectiveness of IRENE in combining prior knowledge gained during training for unseen evaluation tasks.

cs.AI↗

arXiv2023

Provably Accelerated Decentralized Gradient Method Over Unbalanced Directed Graphs

We consider the decentralized optimization problem, where a network of $n$ agents aims to collaboratively minimize the average of their individual smooth and convex objective functions through peer-to-peer communication in a directed graph. To tackle this problem, we propose two accelerated gradient tracking methods, namely APD and APD-SC, for non-strongly convex and strongly convex objective functions, respectively. We show that APD and APD-SC converge at the rates $O\left(\frac{1}{k^2}\right)$ and $O\left(\left(1 - C\sqrt{\fracμ{L}}\right)^k\right)$, respectively, up to constant factors depending only on the mixing matrix. APD and APD-SC are the first decentralized methods over unbalanced directed graphs that achieve the same provable acceleration as centralized methods. Numerical experiments demonstrate the effectiveness of both methods.

math.OC↗

arXiv2023

Improving Self-supervised Molecular Representation Learning using Persistent Homology

Self-supervised learning (SSL) has great potential for molecular representation learning given the complexity of molecular graphs, the large amounts of unlabelled data available, the considerable cost of obtaining labels experimentally, and the hence often only small training datasets. The importance of the topic is reflected in the variety of paradigms and architectures that have been investigated recently. Yet the differences in performance seem often minor and are barely understood to date. In this paper, we study SSL based on persistent homology (PH), a mathematical tool for modeling topological features of data that persist across multiple scales. It has several unique features which particularly suit SSL, naturally offering: different views of the data, stability in terms of distance preservation, and the opportunity to flexibly incorporate domain knowledge. We (1) investigate an autoencoder, which shows the general representational power of PH, and (2) propose a contrastive loss that complements existing approaches. We rigorously evaluate our approach for molecular property prediction and demonstrate its particular features in improving the embedding space: after SSL, the representations are better and offer considerably more predictive power than the baselines over different probing tasks; our loss increases baseline performance, sometimes largely; and we often obtain substantial improvements over very small datasets, a common scenario in practice.

cs.LG↗

arXiv2023

A Comparative Analysis of the COVID-19 Infodemic in English and Chinese: Insights from Social Media Textual Data

The COVID-19 infodemic, characterized by the rapid spread of misinformation and unverified claims related to the pandemic, presents a significant challenge. This paper presents a comparative analysis of the COVID-19 infodemic in the English and Chinese languages, utilizing textual data extracted from social media platforms. To ensure a balanced representation, two infodemic datasets were created by augmenting previously collected social media textual data. Through word frequency analysis, the thirty-five most frequently occurring infodemic words are identified, shedding light on prevalent discussions surrounding the infodemic. Moreover, topic clustering analysis uncovers thematic structures and provides a deeper understanding of primary topics within each language context. Additionally, sentiment analysis enables comprehension of the emotional tone associated with COVID-19 information on social media platforms in English and Chinese. This research contributes to a better understanding of the COVID-19 infodemic phenomenon and can guide the development of strategies to combat misinformation during public health crises across different languages.

cs.SI↗

arXiv2023

Encoding Enhanced Complex CNN for Accurate and Highly Accelerated MRI

Magnetic resonance imaging (MRI) using hyperpolarized noble gases provides a way to visualize the structure and function of human lung, but the long imaging time limits its broad research and clinical applications. Deep learning has demonstrated great potential for accelerating MRI by reconstructing images from undersampled data. However, most existing deep conventional neural networks (CNN) directly apply square convolution to k-space data without considering the inherent properties of k-space sampling, limiting k-space learning efficiency and image reconstruction quality. In this work, we propose an encoding enhanced (EN2) complex CNN for highly undersampled pulmonary MRI reconstruction. EN2 employs convolution along either the frequency or phase-encoding direction, resembling the mechanisms of k-space sampling, to maximize the utilization of the encoding correlation and integrity within a row or column of k-space. We also employ complex convolution to learn rich representations from the complex k-space data. In addition, we develop a feature-strengthened modularized unit to further boost the reconstruction performance. Experiments demonstrate that our approach can accurately reconstruct hyperpolarized 129Xe and 1H lung MRI from 6-fold undersampled k-space data and provide lung function measurements with minimal biases compared with fully-sampled image. These results demonstrate the effectiveness of the proposed algorithmic components and indicate that the proposed approach could be used for accelerated pulmonary MRI in research and clinical lung disease patient care.

eess.IV↗

arXiv2023

Graded dimensions and monomial bases for the cyclotomic quiver Hecke algebras

In this paper we give a closed formula for the graded dimension of the cyclotomic quiver Hecke algebra $R^Λ(β)$ associated to an {\it arbitrary} symmetrizable Cartan matrix $A=(a_{ij})_{i,j}\in I$, where $Λ\in P^+$ and $β\in Q_n^+$. As applications, we obtain some {\it necessary and sufficient conditions} for the KLR idempotent $e(ν)$ (for any $ν\in I^β$) to be nonzero in the cyclotomic quiver Hecke algebra $R^Λ(β)$. We prove several level reduction results which decomposes $\dim R^Λ(β)$ into a sum of some products of $\dim R^{Λ^i}(β_i)$ with $Λ=\sum_iΛ^i$ and $β=\sum_{i}β_i$, where $Λ^i\in P^+, β^i\in Q^+$ for each $i$. We construct some explicit monomial bases for the subspaces $e(\widetildeν)R^Λ(β)e(μ)$ and $e(\widetildeν)R^Λ(β)e(μ)$ of $R^Λ(β)$, where $μ\in I^β$ is {\it arbitrary} and $\widetildeν\in I^β$ is a certain specific $n$-tuple (see Section 4).Finally, we use our graded dimension formulae to provide some examples which show that $R^Λ(n)$ is in general not graded free over its natural embedded subalgebra $R^Λ(m)$ with $m<n$.

math.RT↗

arXiv2023

Transformers over Directed Acyclic Graphs

Transformer models have recently gained popularity in graph representation learning as they have the potential to learn complex relationships beyond the ones captured by regular graph neural networks. The main research question is how to inject the structural bias of graphs into the transformer architecture, and several proposals have been made for undirected molecular graphs and, recently, also for larger network graphs. In this paper, we study transformers over directed acyclic graphs (DAGs) and propose architecture adaptations tailored to DAGs: (1) An attention mechanism that is considerably more efficient than the regular quadratic complexity of transformers and at the same time faithfully captures the DAG structure, and (2) a positional encoding of the DAG's partial order, complementing the former. We rigorously evaluate our approach over various types of tasks, ranging from classifying source code graphs to nodes in citation networks, and show that it is effective in two important aspects: in making graph transformers generally outperform graph neural networks tailored to DAGs and in improving SOTA graph transformer performance in terms of both quality and efficiency.

cs.LG↗