arXiv subjects

Runze Li

Publications and source records attributed to Runze Li.

At least 109 records · Page 6Linked to original sources

arXiv2023

RECLIP: Resource-efficient CLIP by Training with Small Images

We present RECLIP (Resource-efficient CLIP), a simple method that minimizes computational resource footprint for CLIP (Contrastive Language Image Pretraining). Inspired by the notion of coarse-to-fine in computer vision, we leverage small images to learn from large-scale language supervision efficiently, and finetune the model with high-resolution data in the end. Since the complexity of the vision transformer heavily depends on input image size, our approach significantly reduces the training resource requirements both in theory and in practice. Using the same batch size and training epoch, RECLIP achieves highly competitive zero-shot classification and image-text retrieval accuracy with 6 to 8x less computational resources and 7 to 9x fewer FLOPs than the baseline. Compared to the state-of-the-art contrastive learning methods, RECLIP demonstrates 5 to 59x training resource savings while maintaining highly competitive zero-shot classification and retrieval performance. Finally, RECLIP matches the state of the art in transfer learning to open-vocabulary detection tasks, achieving 32 APr on LVIS. We hope this work will pave the path for the broader research community to explore language supervised pretraining in resource-friendly settings.

cs.CV↗

arXiv2023

Predictable gate-field control of spin in altermagnets with spin-layer coupling

Spintronics, a technology harnessing electron spin for information transmission, offers a promising avenue to surpass the limitations of conventional electronic devices. While the spin directly interacts with the magnetic field, its control through the electric field is generally more practical, and has become a focal point in the field of spintronics. Current methodologies for generating spin polarization via an electric field generally necessitate spin-orbit coupling. Here, we propose an innovative mechanism that accomplishes this task without dependence on spin-orbit coupling. Our method employs two-dimensional altermagnets with valley-mediated spin-layer coupling (SLC), in which electronic states display symmetry-protected and valley-contrasted spin and layer polarization. The SLC facilitates predictable, continuous, and reversible control of spin polarization using a gate electric field. Through symmetry analysis and ab initio calculations, we pinpoint high-quality material candidates that exhibit SLC. We ascertain that applying a gate field of $0.2$ eV/Å~ to monolayer Ca(CoN)$_2$ can induce significant spin splitting up to 123 meV. As a result, perfect and switchable spin/valley-currents, and substantial tunneling magnetoresistance can be achieved in these materials using only a gate field. These findings provide new opportunities for generating predictable spin polarization and designing novel spintronic devices based on coupled spin, valley and layer physics.

physics.app-ph↗

arXiv2023

Enhancing Robustness of Gradient-Boosted Decision Trees through One-Hot Encoding and Regularization

Gradient-boosted decision trees (GBDT) are widely used and highly effective machine learning approach for tabular data modeling. However, their complex structure may lead to low robustness against small covariate perturbation in unseen data. In this study, we apply one-hot encoding to convert a GBDT model into a linear framework, through encoding of each tree leaf to one dummy variable. This allows for the use of linear regression techniques, plus a novel risk decomposition for assessing the robustness of a GBDT model against covariate perturbations. We propose to enhance the robustness of GBDT models by refitting their linear regression forms with $L_1$ or $L_2$ regularization. Theoretical results are obtained about the effect of regularization on the model performance and robustness. It is demonstrated through numerical experiments that the proposed regularization approach can enhance the robustness of the one-hot-encoded GBDT models.

stat.ML↗

arXiv2023

Propensity score regression for causal inference with treatment heterogeneity

Understanding how treatment effects vary on individual characteristics is critical in the contexts of personalized medicine, personalized advertising and policy design. When the characteristics are of practical interest are only a subset of full covariate, non-parametric estimation is often desirable; but few methods are available due to the computational difficult. Existing non-parametric methods such as the inverse probability weighting methods have limitations that hinder their use in many practical settings where the values of propensity scores are close to 0 or 1. We propose the propensity score regression (PSR) that allows the non-parametric estimation of the heterogeneous treatment effects in a wide context. PSR includes two non-parametric regressions in turn, where it first regresses on the propensity scores together with the characteristics of interest, to obtain an intermediate estimate; and then, regress the intermediate estimates on the characteristics of interest only. By including propensity scores as regressors in the non-parametric manner, PSR is capable of substantially easing the computational difficulty while remain (locally) insensitive to any value of propensity scores. We present several appealing properties of PSR, including the consistency and asymptotical normality, and in particular the existence of an explicit variance estimator, from which the analytical behaviour of PSR and its precision can be assessed. Simulation studies indicate that PSR outperform existing methods in varying settings with extreme values of propensity scores. We apply our method to the national 2009 flu survey (NHFS) data to investigate the effects of seasonal influenza vaccination and having paid sick leave across different age groups.

stat.ME↗

arXiv2023

Reconstruction and fast prediction of a 3D flow field based on a variational autoencoder

Reconstruction and fast prediction of flow fields are important for the improvement of data center operations and energy savings. In this study, an artificial neural network (ANN) and variational autoencoder (VAE) composite model is proposed for the reconstruction and prediction of 3D flowfields with high accuracy and efficiency. The VAE model is trained to extract features of the problem and to realize 3D physical field reconstruction. The ANN is employed to achieve the constructability of the extracted features. A dataset of steady temperature/velocity fields is acquired by computational fluid dynamics and heat transfer (CFD/HT) and fed to train the deep learning model. The proposed ANN-VAE model is experimentally proven to achieve promising field prediction accuracy with a significantly reduced computational cost. Compared to the CFD/HT method, the ANN-VAE method speeds up the physical field prediction by approximately 380,000 times, with mean accuracies of 97.3% for temperature field prediction and 97.9% for velocity field prediction, making it feasible for real-time physical field acquisition.

physics.flu-dyn↗

arXiv2023

Room-temperature van der Waals 2D ferromagnet switching by spin-orbit torques

Emerging wide varieties of the two-dimensional (2D) van der Waals (vdW) magnets with atomically thin and smooth interfaces holds great promise for next-generation spintronic devices. However, due to the lower Curie temperature of the vdW 2D ferromagnets than room temperature, electrically manipulating its magnetization at room temperature has not been realized. In this work, we demonstrate the perpendicular magnetization of 2D vdW ferromagnet Fe3GaTe2 can be effectively switched at room temperature in Fe3GaTe2/Pt bilayer by spin-orbit torques (SOTs) with a relatively low current density of 1.3 10^7A/cm2. Moreover, the high SOT efficiency of ξ_{DL}~0.22 is quantitatively determined by harmonic measurements, which is higher than those in Pt-based heavy metal/conventional ferromagnet devices. Our findings of room-temperature vdW 2D ferromagnet switching by SOTs provide a significant basis for the development of vdW-ferromagnet-based spintronic applications.

physics.app-ph↗

arXiv2023

Detection and Estimation of Structural Breaks in High-Dimensional Functional Time Series

In this paper, we consider detecting and estimating breaks in heterogeneous mean functions of high-dimensional functional time series which are allowed to be cross-sectionally correlated and temporally dependent. A new test statistic combining the functional CUSUM statistic and power enhancement component is proposed with asymptotic null distribution theory comparable to the conventional CUSUM theory derived for a single functional time series. In particular, the extra power enhancement component enlarges the region where the proposed test has power, and results in stable power performance when breaks are sparse in the alternative hypothesis. Furthermore, we impose a latent group structure on the subjects with heterogeneous break points and introduce an easy-to-implement clustering algorithm with an information criterion to consistently estimate the unknown group number and membership. The estimated group structure can subsequently improve the convergence property of the post-clustering break point estimate. Monte-Carlo simulation studies and empirical applications show that the proposed estimation and testing techniques have satisfactory performance in finite samples.

stat.ME↗

arXiv2023

Functional-Coefficient Quantile Regression for Panel Data with Latent Group Structure

This paper considers estimating functional-coefficient models in panel quantile regression with individual effects, allowing the cross-sectional and temporal dependence for large panel observations. A latent group structure is imposed on the heterogenous quantile regression models so that the number of nonparametric functional coefficients to be estimated can be reduced considerably. With the preliminary local linear quantile estimates of the subject-specific functional coefficients, a classic agglomerative clustering algorithm is used to estimate the unknown group structure and an easy-to-implement ratio criterion is proposed to determine the group number. The estimated group number and structure are shown to be consistent. Furthermore, a post-grouping local linear smoothing method is introduced to estimate the group-specific functional coefficients, and the relevant asymptotic normal distribution theory is derived with a normalisation rate comparable to that in the literature. The developed methodologies and theory are verified through a simulation study and showcased with an application to house price data from UK local authority districts, which reveals different homogeneity structures at different quantile levels.

econ.EM↗

arXiv2022

Anomalous spatial shifts in interface electronic reflection beyond linear approximation

Recently, the electronic analogy of the anomalous spatial shift, including Goos-Hänchen and Imbert-Fedorov effects, has been attracting widespread interest. The current research on the anomalous spatial shift in interface electronic reflection is based on the paradigm of linear approximation, under which the center position of the incident and reflected beams are obtained by expanding the phases of relevant basis states and scattering amplitudes to the first order of incident momentum. However, in a class of normal cases, the linear approximation can lead to a divergent spatial shift in reflection for certain incident angles even though the corresponding reflection possibility is finite. In this work, we show that such non-physical results are caused by an abrupt change in the number of the propagating states at critical parameters, and can be resolved by calculating the center positions of the scattering beams beyond the linear approximation. Moreover, we find that the beam width has an important influence on the spatial shift near the critical angles. We demonstrate our idea via concrete calculations of Goos-Hänchen and Imbert-Fedorov shift on two representative models. These results are beneficial for clarifying the scope of application of the linear approximation in the study of anomalous spatial shifts.

cond-mat.mes-hall↗

arXiv2022

Generalized Varying Coefficient Mediation Models

Motivated by an analysis of causal mechanism from economic stress to entrepreneurial withdrawals through depressed affect, we develop a two-layer generalized varying coefficient mediation model. This model captures the bridging effects of mediators that may vary with another variable, by treating them as smooth functions of this variable. It also allows various response types by introducing the generalized varying coefficient model in the first layer. The varying direct and indirect effects are estimated through spline expansion. The theoretical properties of the estimated direct and indirect coefficient functions, including estimation biases, asymptotic distributions, and so forth, are explored. Simulation studies validate the finite-sample performance of the proposed estimation method. A real data analysis based on the proposed model discovers some interesting behavioral economic phenomenon, that self-efficacy influences the deleterious impact of economic stress, both directly and indirectly through depressed affect, on business owners' withdrawl intentions.

math.ST↗

arXiv2022

A causal approach to functional mediation analysis with application to a smoking cessation intervention

The increase in the use of mobile and wearable devices now allows dense assessment of mediating processes over time. For example, a pharmacological intervention may have an effect on smoking cessation via reductions in momentary withdrawal symptoms. We define and identify the causal direct and indirect effects in terms of potential outcomes on the mean difference and odds ratio scales, and present a method for estimating and testing the indirect effect of a randomized treatment on a distal binary variable as mediated by the nonparametric trajectory of an intensively measured longitudinal variable (e.g., from ecological momentary assessment). Coverage of a bootstrap test for the indirect effect is demonstrated via simulation. An empirical example is presented based on estimating later smoking abstinence from patterns of craving during smoking cessation treatment. We provide an R package, funmediation, available on CRAN, to conveniently apply this technique. We conclude by discussing possible extensions to multiple mediators and directions for future research.

stat.ME↗

arXiv2022

Tunable Rapid Electron Transport in Titanium Oxide Thin Films

Rapid electron transport in the quantum well triggers many novel physical phenomena and becomes a critical point for the high-speed electronics. Here, we found electrical properties of the titanium oxide changed from semiconducting to metallic as the degree of oxidation decreased and Schottky quantum well was formed at the interface. We take the asymmetry interface electron scattering effect into consideration when studying the electrical transport properties of the multilayer thin films. A novel physical conductivity model for the multilayer thin films was developed. We found electron would be transferred from the low-mobility semiconducting and metallic conductive channels to the high-mobility Schottky quantum well conductive channel with an in-plane applied electric field. Electron concentration and mobility of the forming 2DEG in the Schottky quantum well could be tuned thus the nano-devices exhibited non-linear voltage-current curves. The differential resistivity of the nano-devices could decrease by two orders with increasing electric field at room temperature. Weak electron localization of electrons has been experimentally observed in our nano-devices at low temperature, which further demonstrated the existence of 2DEG in the Schottky quantum well. Our work will provide us new physics about the rapid electron transport in the multilayer thin films, and bring novel functional devices for the modern microelectronic industry.

physics.app-ph↗

arXiv2022

Central limit theorem and Berry-Esseen bounds for a branching random walk with immigration in a random environment

We consider a branching random walk on $d$-dimensional real space with immigration in a time-dependent random environment. Let $Z_n(\mathbf t)$ be the so-called partition function of the process, namely, the moment generating function of the counting measure describing the dispersion of individuals at time $n$. For $\mathbf t$ fixed, the logarithm $\log Z_n(\mathbf t)$ satisfies a central limit theorem. By studying the logarithmic moments of the intrinsic submartingale of the system and its convergence rates, we establish the uniform and non-uniform Berry-Esseen bounds corresponding to the central limit theorem, and discover the exact convergence rate in the central limit theorem.

math.PR↗

arXiv2022

Linear maps preserving the Lorentz spectrum of $3 \times 3$ matrices

For a given $3 \times 3$ real matrix $A$, the eigenvalue complementarity problem relative to the Lorentz cone consists of finding a real number $λ$ and a nonzero vector $x \in \mathbb{R}^3$ such that $x^T(A-λI)x=0$ and both $x$ and $(A-λI)x$ lie in the Lorentz cone, which is comprised of all vectors in $\mathbb{R}^3$ forming a $45^\circ$ or smaller angle with the positive $z$-axis. We refer to the set of all solutions $λ$ to this eigenvalue complementarity problem as the Lorentz spectrum of $A$. Our work concerns the characterization of the linear preservers of the Lorentz spectrum on the space $M_3$ of $3 \times 3$ real matrices, that is, the linear maps $ϕ: M_3 \to M_3$ such that the Lorentz spectra of $A$ and $ϕ(A)$ are the same for all $A$. We have proven that all such linear preservers take the form $ϕ(A) = (Q \oplus [1])A(Q^T \oplus [1])$, where $Q$ is an orthogonal $2 \times 2$ matrix.

math.RA↗

arXiv2022

MonoIndoor++:Towards Better Practice of Self-Supervised Monocular Depth Estimation for Indoor Environments

Self-supervised monocular depth estimation has seen significant progress in recent years, especially in outdoor environments. However, depth prediction results are not satisfying in indoor scenes where most of the existing data are captured with hand-held devices. As compared to outdoor environments, estimating depth of monocular videos for indoor environments, using self-supervised methods, results in two additional challenges: (i) the depth range of indoor video sequences varies a lot across different frames, making it difficult for the depth network to induce consistent depth cues for training; (ii) the indoor sequences recorded with handheld devices often contain much more rotational motions, which cause difficulties for the pose network to predict accurate relative camera poses. In this work, we propose a novel framework-MonoIndoor++ by giving special considerations to those challenges and consolidating a set of good practices for improving the performance of self-supervised monocular depth estimation for indoor environments. First, a depth factorization module with transformer-based scale regression network is proposed to estimate a global depth scale factor explicitly, and the predicted scale factor can indicate the maximum depth values. Second, rather than using a single-stage pose estimation strategy as in previous methods, we propose to utilize a residual pose estimation module to estimate relative camera poses across consecutive frames iteratively. Third, to incorporate extensive coordinates guidance for our residual pose estimation module, we propose to perform coordinate convolutional encoding directly over the inputs to pose networks. The proposed method is validated on a variety of benchmark indoor datasets, i.e., EuRoC MAV, NYUv2, ScanNet and 7-Scenes, demonstrating the state-of-the-art performance.

cs.CV↗

arXiv2022

Study of transfer learning from 2D supercritical airfoils to 3D transonic swept wings

Machine learning has been widely utilized in fluid mechanics studies and aerodynamic optimizations. However, most applications, especially flow field modeling and inverse design, involve two-dimensional flows and geometries. The dimensionality of three-dimensional problems is so high that it is too difficult and expensive to prepare sufficient samples. Therefore, transfer learning has become a promising approach to reuse well-trained two-dimensional models and greatly reduce the need for samples for three-dimensional problems. This paper proposes to reuse the baseline models trained on supercritical airfoils to predict finite-span swept supercritical wings, where the simple swept theory is embedded to improve the prediction accuracy. Two baseline models for transfer learning are investigated: one is commonly referred to as the forward problem of predicting the pressure coefficient distribution based on the geometry, and the other is the inverse problem that predicts the geometry based on the pressure coefficient distribution. Two transfer learning strategies are compared for both baseline models. The transferred models are then tested on the prediction of complete wings. The results show that transfer learning requires only approximately 500 wing samples to achieve good prediction accuracy on different wing planforms and different free stream conditions. Compared to the two baseline models, the transferred models reduce the prediction error by 60% and 80%, respectively.

physics.flu-dyn↗

arXiv2022

Model-Free Statistical Inference on High-Dimensional Data

This paper aims to develop an effective model-free inference procedure for high-dimensional data. We first reformulate the hypothesis testing problem via sufficient dimension reduction framework. With the aid of new reformulation, we propose a new test statistic and show that its asymptotic distribution is $χ^2$ distribution whose degree of freedom does not depend on the unknown population distribution. We further conduct power analysis under local alternative hypotheses. In addition, we study how to control the false discovery rate of the proposed $χ^2$ tests, which are correlated, to identify important predictors under a model-free framework. To this end, we propose a multiple testing procedure and establish its theoretical guarantees. Monte Carlo simulation studies are conducted to assess the performance of the proposed tests and an empirical analysis of a real-world data set is used to illustrate the proposed methodology.

stat.ME↗

arXiv2022

Multiple-Splitting Projection Test for High-Dimensional Mean Vectors

We propose a multiple-splitting projection test (MPT) for one-sample mean vectors in high-dimensional settings. The idea of projection test is to project high-dimensional samples to a 1-dimensional space using an optimal projection direction such that traditional tests can be carried out with projected samples. However, estimation of the optimal projection direction has not been systematically studied in the literature. In this work, we bridge the gap by proposing a consistent estimation via regularized quadratic optimization. To retain type I error rate, we adopt a data-splitting strategy when constructing test statistics. To mitigate the power loss due to data-splitting, we further propose a test via multiple splits to enhance the testing power. We show that the $p$-values resulted from multiple splits are exchangeable. Unlike existing methods which tend to conservatively combine dependent $p$-values, we develop an exact level $α$ test that explicitly utilizes the exchangeability structure to achieve better power. Numerical studies show that the proposed test well retains the type I error rate and is more powerful than state-of-the-art tests.

stat.ME↗