arXiv subjects

Lin Liu

Publications and source records attributed to Lin Liu.

At least 235 records · Page 13Linked to original sources

arXiv2023

Attention Weighted Mixture of Experts with Contrastive Learning for Personalized Ranking in E-commerce

Ranking model plays an essential role in e-commerce search and recommendation. An effective ranking model should give a personalized ranking list for each user according to the user preference. Existing algorithms usually extract a user representation vector from the user behavior sequence, then feed the vector into a feed-forward network (FFN) together with other features for feature interactions, and finally produce a personalized ranking score. Despite tremendous progress in the past, there is still room for improvement. Firstly, the personalized patterns of feature interactions for different users are not explicitly modeled. Secondly, most of existing algorithms have poor personalized ranking results for long-tail users with few historical behaviors due to the data sparsity. To overcome the two challenges, we propose Attention Weighted Mixture of Experts (AW-MoE) with contrastive learning for personalized ranking. Firstly, AW-MoE leverages the MoE framework to capture personalized feature interactions for different users. To model the user preference, the user behavior sequence is simultaneously fed into expert networks and the gate network. Within the gate network, one gate unit and one activation unit are designed to adaptively learn the fine-grained activation vector for experts using an attention mechanism. Secondly, a random masking strategy is applied to the user behavior sequence to simulate long-tail users, and an auxiliary contrastive loss is imposed to the output of the gate network to improve the model generalization for these users. This is validated by a higher performance gain on the long-tail user test set. Experiment results on a JD real production dataset and a public dataset demonstrate the effectiveness of AW-MoE, which significantly outperforms state-of-art methods. Notably, AW-MoE has been successfully deployed in the JD e-commerce search engine, ...

cs.IR↗

arXiv2023

Causal Effect Estimation with Variational AutoEncoder and the Front Door Criterion

An essential problem in causal inference is estimating causal effects from observational data. The problem becomes more challenging with the presence of unobserved confounders. When there are unobserved confounders, the commonly used back-door adjustment is not applicable. Although the instrumental variable (IV) methods can deal with unobserved confounders, they all assume that the treatment directly affects the outcome, and there is no mediator between the treatment and the outcome. This paper aims to use the front-door criterion to address the challenging problem with the presence of unobserved confounders and mediators. In practice, it is often difficult to identify the set of variables used for front-door adjustment from data. By leveraging the ability of deep generative models in representation learning, we propose FDVAE to learn the representation of a Front-Door adjustment set with a Variational AutoEncoder, instead of trying to search for a set of variables for front-door adjustment. Extensive experiments on synthetic datasets validate the effectiveness of FDVAE and its superiority over existing methods. The experiments also show that the performance of FDVAE is not sensitive to the causal strength of unobserved confounders and is feasible in the case of dimensionality mismatch between learned representations and the ground truth. We further apply the method to three real-world datasets to demonstrate its potential applications.

cs.LG↗

arXiv2023

A Hybrid Deep Feature-Based Deformable Image Registration Method for Pathology Images

Pathologists need to combine information from differently stained pathology slices for accurate diagnosis. Deformable image registration is a necessary technique for fusing multi-modal pathology slices. This paper proposes a hybrid deep feature-based deformable image registration framework for stained pathology samples. We first extract dense feature points via the detector-based and detector-free deep learning feature networks and perform points matching. Then, to further reduce false matches, an outlier detection method combining the isolation forest statistical model and the local affine correction model is proposed. Finally, the interpolation method generates the deformable vector field for pathology image registration based on the above matching points. We evaluate our method on the dataset of the Non-rigid Histology Image Registration (ANHIR) challenge, which is co-organized with the IEEE ISBI 2019 conference. Our technique outperforms the traditional approaches by 17% with the Average-Average registration target error (rTRE) reaching 0.0034. The proposed method achieved state-of-the-art performance and ranked 1st in evaluating the test dataset. The proposed hybrid deep feature-based registration method can potentially become a reliable method for pathology image registration.

eess.IV↗

arXiv2023

Linking a predictive model to causal effect estimation

A predictive model makes outcome predictions based on some given features, i.e., it estimates the conditional probability of the outcome given a feature vector. In general, a predictive model cannot estimate the causal effect of a feature on the outcome, i.e., how the outcome will change if the feature is changed while keeping the values of other features unchanged. This is because causal effect estimation requires interventional probabilities. However, many real world problems such as personalised decision making, recommendation, and fairness computing, need to know the causal effect of any feature on the outcome for a given instance. This is different from the traditional causal effect estimation problem with a fixed treatment variable. This paper first tackles the challenge of estimating the causal effect of any feature (as the treatment) on the outcome w.r.t. a given instance. The theoretical results naturally link a predictive model to causal effect estimations and imply that a predictive model is causally interpretable when the conditions identified in the paper are satisfied. The paper also reveals the robust property of a causally interpretable model. We use experiments to demonstrate that various types of predictive models, when satisfying the conditions identified in this paper, can estimate the causal effects of features as accurately as state-of-the-art causal effect estimation methods. We also show the potential of such causally interpretable predictive models for robust predictions and personalised decision making.

cs.LG↗

arXiv2023

Effects of Epileptiform Activity on Discharge Outcome in Critically Ill Patients

Epileptiform activity (EA) is associated with worse outcomes including increased risk of disability and death. However, the effect of EA on the neurologic outcome is confounded by the feedback between treatment with anti-seizure medications (ASM) and EA burden. A randomized clinical trial is challenging due to the sequential nature of EA-ASM feedback, as well as ethical reasons. However, some mechanistic knowledge is available, e.g., how drugs are absorbed. This knowledge together with observational data could provide a more accurate effect estimate using causal inference. We performed a retrospective cross-sectional study with 995 patients with the modified Rankin Scale (mRS) at discharge as the outcome and the EA burden defined as the mean or maximum proportion of time spent with EA in six-hour windows in the first 24 hours of electroencephalography as the exposure. We estimated the change in discharge mRS if everyone in the dataset had experienced a certain EA burden and were untreated. We combined pharmacological modeling with an interpretable matching method to account for confounding and EA-ASM feedback. Our matched groups' quality was validated by the neurologists. Having a maximum EA burden greater than 75% when untreated had a 22% increased chance of a poor outcome (severe disability or death), and mild but long-lasting EA increased the risk of a poor outcome by 14%. The effect sizes were heterogeneous depending on pre-admission profile, e.g., patients with hypoxic-ischemic encephalopathy (HIE) or acquired brain injury (ABI) were more affected. Interventions should put a higher priority on patients with an average EA burden higher than 10%, while treatment should be more conservative when the maximum EA burden is low.

stat.ME↗

arXiv2023

AST-SED: An Effective Sound Event Detection Method Based on Audio Spectrogram Transformer

In this paper, we propose an effective sound event detection (SED) method based on the audio spectrogram transformer (AST) model, pretrained on the large-scale AudioSet for audio tagging (AT) task, termed AST-SED. Pretrained AST models have recently shown promise on DCASE2022 challenge task4 where they help mitigate a lack of sufficient real annotated data. However, mainly due to differences between the AT and SED tasks, it is suboptimal to directly utilize outputs from a pretrained AST model. Hence the proposed AST-SED adopts an encoder-decoder architecture to enable effective and efficient fine-tuning without needing to redesign or retrain the AST model. Specifically, the Frequency-wise Transformer Encoder (FTE) consists of transformers with self attention along the frequency axis to address multiple overlapped audio events issue in a single clip. The Local Gated Recurrent Units Decoder (LGD) consists of nearest-neighbor interpolation (NNI) and Bidirectional Gated Recurrent Units (Bi-GRU) to compensate for temporal resolution loss in the pretrained AST model output. Experimental results on DCASE2022 task4 development set have demonstrated the superiority of the proposed AST-SED with FTE-LGD architecture. Specifically, the Event-Based F1-score (EB-F1) of 59.60% and Polyphonic Sound detection Score scenario1 (PSDS1) score of 0.5140 significantly outperform CRNN and other pretrained AST-based systems.

eess.AS↗

arXiv2023

New $\sqrt{n}$-consistent, numerically stable higher-order influence function estimators

Higher-Order Influence Functions (HOIFs) provide a unified theory for constructing rate-optimal estimators for a large class of low-dimensional (smooth) statistical functionals/parameters (and sometimes even infinite-dimensional functions) that arise in substantive fields including epidemiology, economics, and the social sciences. Since the introduction of HOIFs by Robins et al. (2008), they have been viewed mostly as a theoretical benchmark rather than a useful tool for statistical practice. Works aimed to flip the script are scant, but a few recent papers Liu et al. (2017, 2021b) make some partial progress. In this paper, we take a fresh attempt at achieving this goal by constructing new, numerically stable HOIF estimators (or sHOIF estimators for short with ``s'' standing for ``stable'') with provable statistical, numerical, and computational guarantees. This new class of sHOIF estimators (up to the 2nd order) was foreshadowed in synthetic experiments conducted by Liu et al. (2020a).

math.ST↗

arXiv2023

Magnetic detection under high pressures using designed silicon vacancy centers in silicon carbide

Pressure-induced magnetic phase transition is attracting interest due to its ability to detect superconducting behaviour at high pressures in diamond anvil cells. However, detection of the local sample magnetic properties is a great challenge due to the small sample chamber volume. Recently, optically detected magnetic resonance (ODMR) of nitrogen vacancy (NV) centers in diamond have been used for in-situ pressure-induced phase transition detection. However, owing to their four orientation axes and temperature-dependent zero-field-splitting, interpreting the observed ODMR spectra of NV centers remain challenging. Here, we study the optical and spin properties of implanted silicon vacancy defects in 4H-SiC, which is single-axis and temperature-independent zero-field-splitting. Using this technique, we observe the magnetic phase transition of Nd2Fe14B at about 7 GPa and map the critical temperature-pressure phase diagram of the superconductor YBa2Cu3O6.6. These results highlight the potential of silicon vacancy-based quantum sensors for in-situ magnetic detection at high pressures.

physics.app-ph↗

arXiv2022

DeepMed: Semiparametric Causal Mediation Analysis with Debiased Deep Learning

Causal mediation analysis can unpack the black box of causality and is therefore a powerful tool for disentangling causal pathways in biomedical and social sciences, and also for evaluating machine learning fairness. To reduce bias for estimating Natural Direct and Indirect Effects in mediation analysis, we propose a new method called DeepMed that uses deep neural networks (DNNs) to cross-fit the infinite-dimensional nuisance functions in the efficient influence functions. We obtain novel theoretical results that our DeepMed method (1) can achieve semiparametric efficiency bound without imposing sparsity constraints on the DNN architecture and (2) can adapt to certain low dimensional structures of the nuisance functions, significantly advancing the existing literature on DNN-based semiparametric causal inference. Extensive synthetic experiments are conducted to support our findings and also expose the gap between theory and practice. As a proof of concept, we apply DeepMed to analyze two real datasets on machine learning fairness and reach conclusions consistent with previous findings.

stat.ML↗

arXiv2022

Data-Driven Prognosis of Failure Detection and Prediction of Lithium-ion Batteries

Battery prognostics and health management predictive models are essential components of safety and reliability protocols in battery management system frameworks. Overall, developing a robust and efficient battery model that aligns with the current literature is a useful step in ensuring the safety of battery function. For this purpose, a multi-physics, multi-scale deterministic data-driven prognosis (DDP) is proposed that only relies on in situ measurements of data and estimates the failure based on the curvature information extracted from the system. Unlike traditional applications that require explicit expression of conservation principle, the proposed method devices a local conservation functional in the neighborhood of each data point which is represented as the minimization of curvature in the system. By eliminating the need for offline training, the method can predict the onset of instability for a variety of systems over a prediction horizon. The prediction horizon to prognosticate the instability, alternatively, is considered as the remaining useful life (RUL) metric. The framework is then employed to analyze the health status of Li-ion batteries. Based on the results, it has demonstrated that the DDP technique can accurately predict the onset of failure of Li-ion batteries.

physics.data-an↗

arXiv2022

Causal Inference with Conditional Instruments using Deep Generative Models

The instrumental variable (IV) approach is a widely used way to estimate the causal effects of a treatment on an outcome of interest from observational data with latent confounders. A standard IV is expected to be related to the treatment variable and independent of all other variables in the system. However, it is challenging to search for a standard IV from data directly due to the strict conditions. The conditional IV (CIV) method has been proposed to allow a variable to be an instrument conditioning on a set of variables, allowing a wider choice of possible IVs and enabling broader practical applications of the IV approach. Nevertheless, there is not a data-driven method to discover a CIV and its conditioning set directly from data. To fill this gap, in this paper, we propose to learn the representations of the information of a CIV and its conditioning set from data with latent confounders for average causal effect estimation. By taking advantage of deep generative models, we develop a novel data-driven approach for simultaneously learning the representation of a CIV from measured variables and generating the representation of its conditioning set given measured variables. Extensive experiments on synthetic and real-world datasets show that our method outperforms the existing IV methods.

cs.LG↗

arXiv2022

Formulations of the Elastodynamic Equations in Anisotropic and Multiphasic Porous Media from the Principle of Energy Conservation

Elastodynamic equations have been formulated with either Newton's second law of motion, Lagrange's equation, or Hamilton's principle for over 150 years. In this work, contrary to classical continuum mechanics, a novel strategic methodology is proposed for formulating general mechanical equations using the principle of energy conservation. Firstly, based on Hamilton's principle, Hamilton's equations, Lagrange's equation, and the elastodynamic equation of motion are derived in arbitrarily anisotropic and multiphasic porous elastic media, for the first time. Secondly, these equations are all formulated using the principle of energy conservation for the related media. Both formulation results using the two kinds of principles are compared and validated by each other. The advantages of our methodology lie in that, the elastodynamic equation of motion, Lagrange's equation, and Hamilton's equations in continuum mechanics are directly formulated using a simple constraint of energy conservation without introducing variational concepts. It is easy to understand and has clear physical meanings. Our methodology unlocks the physics essences of Hamilton's principle in continuum mechanics, which is a consequence of the principle of energy conservation. Although the linear stress-strain constitutive relation is considered, our methodology can still be used in a nonlinear dynamical system. The methodology also paves an alternative way of treating other complex continuous dynamical systems in a broad sense. In addition, as an application, the continuity conditions at various medium interfaces are also revisited and extended using our proposed approach, which explains the law of reflections and refractions.

physics.class-ph↗

arXiv2022

Linear mixed model vs two-stage methods: Developing prognostic models of diabetic kidney disease progression

Identifying prognostic factors for disease progression is a cornerstone of medical research. Repeated assessments of a marker outcome are often used to evaluate disease progression, and the primary research question is to identify factors associated with the longitudinal trajectory of this marker. Our work is motivated by diabetic kidney disease (DKD), where serial measures of estimated glomerular filtration rate (eGFR) are the longitudinal measure of kidney function, and there is notable interest in identifying factors, such as metabolites, that are prognostic for DKD progression. Linear mixed models (LMM) with serial marker outcomes (e.g., eGFR) are a standard approach for prognostic model development, namely by evaluating the time and prognostic factor (e.g., metabolite) interaction. However, two-stage methods that first estimate individual-specific eGFR slopes, and then use these as outcomes in a regression framework with metabolites as predictors are easy to interpret and implement for applied researchers. Herein, we compared the LMM and two-stage methods, in terms of bias and mean squared error via analytic methods and simulations, allowing for irregularly spaced measures and missingness. Our findings provide novel insights into when two-stage methods are suitable longitudinal prognostic modeling alternatives to the LMM. Notably, our findings generalize to other disease studies.

stat.CO↗

arXiv2022

Honor of Kings Arena: an Environment for Generalization in Competitive Reinforcement Learning

This paper introduces Honor of Kings Arena, a reinforcement learning (RL) environment based on Honor of Kings, one of the world's most popular games at present. Compared to other environments studied in most previous work, ours presents new generalization challenges for competitive reinforcement learning. It is a multi-agent problem with one agent competing against its opponent; and it requires the generalization ability as it has diverse targets to control and diverse opponents to compete with. We describe the observation, action, and reward specifications for the Honor of Kings domain and provide an open-source Python-based interface for communicating with the game engine. We provide twenty target heroes with a variety of tasks in Honor of Kings Arena and present initial baseline results for RL-based methods with feasible computing resources. Finally, we showcase the generalization challenges imposed by Honor of Kings Arena and possible remedies to the challenges. All of the software, including the environment-class, are publicly available at https://github.com/tencent-ailab/hok_env . The documentation is available at https://aiarena.tencent.com/hok/doc/ .

cs.LG↗

arXiv2022

Requirements Engineering for Machine Learning: A Review and Reflection

Today, many industrial processes are undergoing digital transformation, which often requires the integration of well-understood domain models and state-of-the-art machine learning technology in business processes. However, requirements elicitation and design decision making about when, where and how to embed various domain models and end-to-end machine learning techniques properly into a given business workflow requires further exploration. This paper aims to provide an overview of the requirements engineering process for machine learning applications in terms of cross domain collaborations. We first review the literature on requirements engineering for machine learning, and then go through the collaborative requirements analysis process step-by-step. An example case of industrial data-driven intelligence applications is also discussed in relation to the aforementioned steps.

cs.SE↗

arXiv2022

On genuine entanglement for tripartite systems

We investigate the genuine entanglement in tripartite systems based on partial transposition and the norm of correlation tensors of the density matrices. We first derive an analytical sufficient criterion to detect genuine entanglement of tripartite qubit quantum states combining with the partial transposition of the density matrices. Then we use the norm of correlation tensors to study genuine entanglement for tripartite qudit quantum states and obtain a genuine entanglement criterion by constructing certain matrices. With detailed examples our results are seen to be able to detect more genuine tripartite entangled states than previous studies.

quant-ph↗

arXiv2022

Dynamic Event-Triggered Consensus of Multi-agent Systems on Matrix-weighted Networks

This paper examines the event-triggered consensus of the multi-agent system on matrix-weighted networks, where the interdependencies among higher-dimensional states of neighboring agents are characterized by matrix-weighted edges in the network. Specifically, a novel distributed dynamic event-triggered coordination strategy is proposed for this category of generalized networks, in which an auxiliary system is employed for each agent to dynamically adjust the triggering threshold, which plays an essential role in guaranteeing that the triggering time sequence does not exhibit Zeno behavior. Distributed event-triggered control protocols are proposed to guarantee leaderless and leader-follower consensus for multi-agent systems on matrix-weighted networks, respectively. Remarkably, the spectrum of matrix-valued weights is crucial in event-triggered mechanism design for matrix-weighted networks, generalizing those results only applicable for scalar-weighted networks. The proposed approach allows each agent to broadcast and receive information only at its triggering instants. Finally, simulation examples are provided to demonstrate the theoretical results.

eess.SY↗

arXiv2022

Fiber-integrated silicon carbide silicon vacancy-based magnetometer

Silicon vacancy in silicon carbide has drawn much attention for various quantum sensing. However, most of the previous experiments are achieved using confocal scanning systems, which limit its applications in practical applications. In this work, we demonstrate a compact fiber-integrated silicon carbide silicon vacancy-based vector magnetometer at room temperature. First, we effectively couple the silicon vacancy in a tiny silicon carbide slice to an optical fiber tip and realize the readout of the spin signal through the fiber at the same time. We then study the optically detected magnetic resonance spectra at different laser and microwave powers, obtaining an optimized magnetic field sensitivity of 12.3 μT/Hz1/2. Based on this, the magnetometer is performed to measure the strength and polar angle of an external magnetic field, respectively. Through these experiments, we have paved the way for fiber-integrated silicon vacancy-based magnetometer applications in practical environments such as geophysics and biomedical sensing.

quant-ph↗