Search arXiv⌕ Search

arXiv subjects

Siyu Chen

Publications and source records attributed to Siyu Chen.

At least 109 records · Page 6Linked to original sources
arXiv2024

Origin of competing charge density waves in kagome metal ScV$_6$Sn$_6$

Understanding competing charge density wave (CDW) orders in the bilayer kagome metal ScV$_6$Sn$_6$ remains challenging. Experimentally, upon cooling, short-range order with wave vector $\mathbf{q}_{2}=(\frac{1}{3},\frac{1}{3},\frac{1}{2})$ forms, which is subsequently suppressed by the condensation of long-range $\mathbf{q}_{3}=(\frac{1}{3},\frac{1}{3},\frac{1}{3})$ CDW order at lower temperature. Theoretically, however, the $\mathbf{q}_{2}$ CDW is predicted as the ground state, leaving the CDW mechanism elusive. Here, using anharmonic phonon-phonon calculations combined with density functional theory, we predict a temperature-driven structural phase transitions from the high-temperature pristine phase to the $\mathbf{q}_{2}$ CDW, followed by the low-temperature $\mathbf{q}_{3}$ CDW, explaining experimental observations. We demonstrate that semi-core electron states stabilize the $\mathbf{q}_{3}$ CDW over the $\mathbf{q}_{2}$ CDW. Furthermore, we find that the out-of-plane lattice parameter controls the competing CDWs, motivating us to propose compressive bi-axial strain as an experimental protocol to stabilize the $\mathbf{q}_{2}$ CDW. Finally, we suggest Ge or Pb doping at the Sn site as another potential avenue to control CDW instabilities. Our work provides a full theory of CDWs in ScV$_6$Sn$_6$, rationalizing experimental observations and resolving earlier discrepancies between theory and experiment.

cond-mat.str-el↗
arXiv2024

Giant Electron-Phonon Coupling Induced Band-Gap Renormalization in Anharmonic Silver Chalcohalide Antiperovskites

Silver chalcohalide antiperovskites (CAP), Ag$_{3}$XY (X = S, Se; Y = Br, I), are a family of highly anharmonic inorganic compounds with great potential for energy applications. However, a substantial and unresolved discrepancy exists between the optoelectronic properties predicted by theoretical first-principles methods and those measured experimentally at room temperature, hindering the fundamental understanding and rational engineering of CAP. In this work, we employ density functional theory, tight-binding calculations, and anharmonic Fröhlich theory to investigate the optoelectronic properties of CAP at finite temperatures. Near room temperature, we observe a giant band-gap ($E_{g}$) reduction of approximately $20$-$60$\% relative to the value calculated at $T = 0$ K, bringing the estimated $E_{g}$ into excellent agreement with experimental measurements. This relative $T$-induced band-gap renormalization is roughly twice the largest value previously reported in the literature for similar temperature ranges. Low-energy optical polar phonon modes, which break inversion symmetry and promote the overlap between silver and chalcogen $s$ electronic orbitals in the conduction band, are identified as the primary contributors to this giant $E_{g}$ reduction. Furthermore, when considering temperature effects, the optical absorption coefficient of CAP increases by nearly an order of magnitude for visible light frequencies. These insights not only bridge a crucial gap between theory and experiment but also open pathways for future technologies where temperature, electric fields, or light dynamically tailor optoelectronic behavior, positioning CAP as a versatile platform for next-generation energy applications.

cond-mat.mtrl-sci↗
arXiv2024

Constraining Axion-Like Particles from observations of AGN B2 2234+28A and 3C 454.3

Axion-photon oscillation effect provides a possible explanation for the presence of very-high-energy (VHE) $γ$-ray signals from distant sources. In this work, we propose a model-dependent method to select possible sources that may give sufficient constraints on the axion parameters. We investigate such effect in the spectra of active galactic nuclei (AGN) B2 2234+28A and 3C 454.3 based on data obtained from Fermi Large Area Telescope (Fermi-LAT) and MAGIC U.L. We utilize the Markov Chain Monte Carlo method to fit the axion parameters, yielding a result of $g_{aγ}=3.05^{+0.51}_{-0.31} \times 10^{-11}$ GeV$^{-1}$ for the axion-photon coupling strength and $m_{a}=5.25^{+2.35}_{-2.65} \times 10^{-8} $ eV for the axion mass. We also perform 95\% confidence level (CL) constraints to set an upper limit for $g_{aγ}$.

astro-ph.HE↗
arXiv2024

D$^3$epth: Self-Supervised Depth Estimation with Dynamic Mask in Dynamic Scenes

Depth estimation is a crucial technology in robotics. Recently, self-supervised depth estimation methods have demonstrated great potential as they can efficiently leverage large amounts of unlabelled real-world data. However, most existing methods are designed under the assumption of static scenes, which hinders their adaptability in dynamic environments. To address this issue, we present D$^3$epth, a novel method for self-supervised depth estimation in dynamic scenes. It tackles the challenge of dynamic objects from two key perspectives. First, within the self-supervised framework, we design a reprojection constraint to identify regions likely to contain dynamic objects, allowing the construction of a dynamic mask that mitigates their impact at the loss level. Second, for multi-frame depth estimation, we introduce a cost volume auto-masking strategy that leverages adjacent frames to identify regions associated with dynamic objects and generate corresponding masks. This provides guidance for subsequent processes. Furthermore, we propose a spectral entropy uncertainty module that incorporates spectral entropy to guide uncertainty estimation during depth fusion, effectively addressing issues arising from cost volume computation in dynamic environments. Extensive experiments on KITTI and Cityscapes datasets demonstrate that the proposed method consistently outperforms existing self-supervised monocular depth estimation baselines. Code is available at \url{https://github.com/Csyunling/D3epth}.

cs.CV↗
arXiv2024

Characterization of Spin-Orbit Effects in Superconductors In$_5$Bi$_3$ and In$_5$Sb$_3$

We report a first principles computational analysis of two phonon-mediated superconductors, In$_{5}$Bi$_{3}$ and In$_{5}$Sb$_{3}$. We show that spin-orbit coupling leads to splitting of electron bands around the Fermi energy, resulting in a suppression of the electronic density of states in both compounds. In In$_{5}$Bi$_{3}$, the spin-orbit coupling is essential for the dynamical stability of the experimentally observed phase, and the calculated superconducting critical temperature is in close agreement with measurements. In In$_{5}$Sb$_{3}$, the spin-orbit coupling significantly reduces the calculated superconducting critical temperature compared to calculations neglecting relativistic effects. Our work emphasises the subtle interplay between spin-orbit interactions and phonon-mediated superconductivity.

cond-mat.supr-con↗
arXiv2024

On metric choice in dimension reduction for Fréchet regression

Fréchet regression is becoming a mainstay in modern data analysis for analyzing non-traditional data types belonging to general metric spaces. This novel regression method is especially useful in the analysis of complex health data such as continuous monitoring and imaging data. Fréchet regression utilizes the pairwise distances between the random objects, which makes the choice of metric crucial in the estimation. In this paper, existing dimension reduction methods for Fréchet regression are reviewed, and the effect of metric choice on the estimation of the dimension reduction subspace is explored for the regression between random responses and Euclidean predictors. Extensive numerical studies illustrate how different metrics affect the central and central mean space estimators. Two real applications involving analysis of brain connectivity networks of subjects with and without Parkinson's disease and an analysis of the distributions of glycaemia based on continuous glucose monitoring data are provided, to demonstrate how metric choice can influence findings in real applications.

stat.ME↗
arXiv2024

Unveiling Induction Heads: Provable Training Dynamics and Feature Learning in Transformers

In-context learning (ICL) is a cornerstone of large language model (LLM) functionality, yet its theoretical foundations remain elusive due to the complexity of transformer architectures. In particular, most existing work only theoretically explains how the attention mechanism facilitates ICL under certain data models. It remains unclear how the other building blocks of the transformer contribute to ICL. To address this question, we study how a two-attention-layer transformer is trained to perform ICL on $n$-gram Markov chain data, where each token in the Markov chain statistically depends on the previous $n$ tokens. We analyze a sophisticated transformer model featuring relative positional embedding, multi-head softmax attention, and a feed-forward layer with normalization. We prove that the gradient flow with respect to a cross-entropy ICL loss converges to a limiting model that performs a generalized version of the induction head mechanism with a learned feature, resulting from the congruous contribution of all the building blocks. In the limiting model, the first attention layer acts as a $\mathit{copier}$, copying past tokens within a given window to each position, and the feed-forward network with normalization acts as a $\mathit{selector}$ that generates a feature vector by only looking at informationally relevant parents from the window. Finally, the second attention layer is a $\mathit{classifier}$ that compares these features with the feature at the output position, and uses the resulting similarity scores to generate the desired output. Our theory is further validated by experiments.

cs.LG↗
arXiv2024

Unveiling the Statistical Foundations of Chain-of-Thought Prompting Methods

Chain-of-Thought (CoT) prompting and its variants have gained popularity as effective methods for solving multi-step reasoning problems using pretrained large language models (LLMs). In this work, we analyze CoT prompting from a statistical estimation perspective, providing a comprehensive characterization of its sample complexity. To this end, we introduce a multi-step latent variable model that encapsulates the reasoning process, where the latent variable encodes the task information. Under this framework, we demonstrate that when the pretraining dataset is sufficiently large, the estimator formed by CoT prompting is equivalent to a Bayesian estimator. This estimator effectively solves the multi-step reasoning problem by aggregating a posterior distribution inferred from the demonstration examples in the prompt. Moreover, we prove that the statistical error of the CoT estimator can be decomposed into two main components: (i) a prompting error, which arises from inferring the true task using CoT prompts, and (ii) the statistical error of the pretrained LLM. We establish that, under appropriate assumptions, the prompting error decays exponentially to zero as the number of demonstrations increases. Additionally, we explicitly characterize the approximation and generalization errors of the pretrained LLM. Notably, we construct a transformer model that approximates the target distribution of the multi-step reasoning problem with an error that decreases exponentially in the number of transformer blocks. Our analysis extends to other variants of CoT, including Self-Consistent CoT, Tree-of-Thought, and Selection-Inference, offering a broad perspective on the efficacy of these methods. We also provide numerical experiments to validate the theoretical findings.

cs.AI↗
arXiv2024

Practical Marketplace Optimization at Uber Using Causally-Informed Machine Learning

Budget allocation of marketplace levers, such as incentives for drivers and promotions for riders, has long been a technical and business challenge at Uber; understanding lever budget changes' impact and estimating cost efficiency to achieve predefined budgets is crucial, with the goal of optimal allocations that maximize business value; we introduce an end-to-end machine learning and optimization procedure to automate budget decision-making for cities, relying on feature store, model training and serving, optimizers, and backtesting; proposing state-of-the-art deep learning (DL) estimator based on S-Learner and a novel tensor B-Spline regression model, we solve high-dimensional optimization with ADMM and primal-dual interior point convex optimization, substantially improving Uber's resource allocation efficiency.

cs.LG↗
arXiv2024

Fintech and MSEs Innovation: an Empirical Analysis

Employing a comprehensive survey of micro and small enterprises (MSEs) and the Digital Financial Inclusion Index in China, this study investigates the influence of fintech on MSE innovation empirically. Our findings indicate that fintech advancement substantially enhances the likelihood of MSEs engaging in innovative endeavors and boosts both the investment and outcomes of their innovation processes. The underlying mechanisms are attributed to fintech's role in fostering long-term strategic incentives and investment in human capital. This includes the use of promotions and stock options as rewards, rather than traditional perks like gifts or trips, the attraction of a greater number of university graduates, and the increase in both training expenses and the remuneration of technical staff. Our heterogeneity analysis reveals that fintech exerts a more pronounced effect on MSEs situated in economically developed areas, those that are five years old or younger, and businesses with limited assets and workforce. Additionally, we uncover that fintech stimulates the innovation of MSEs' independent research and development (R\&D) efforts. This paper contributes to the understanding of the nuanced ways in which fintech impacts MSE innovation and offers policy insights aimed at unleashing the full potential of MSEs' innovative capabilities.

econ.GN↗
arXiv2024

From Words to Actions: Unveiling the Theoretical Underpinnings of LLM-Driven Autonomous Systems

In this work, from a theoretical lens, we aim to understand why large language model (LLM) empowered agents are able to solve decision-making problems in the physical world. To this end, consider a hierarchical reinforcement learning (RL) model where the LLM Planner and the Actor perform high-level task planning and low-level execution, respectively. Under this model, the LLM Planner navigates a partially observable Markov decision process (POMDP) by iteratively generating language-based subgoals via prompting. Under proper assumptions on the pretraining data, we prove that the pretrained LLM Planner effectively performs Bayesian aggregated imitation learning (BAIL) through in-context learning. Additionally, we highlight the necessity for exploration beyond the subgoals derived from BAIL by proving that naively executing the subgoals returned by LLM leads to a linear regret. As a remedy, we introduce an $ε$-greedy exploration strategy to BAIL, which is proven to incur sublinear regret when the pretraining error is small. Finally, we extend our theoretical framework to include scenarios where the LLM Planner serves as a world model for inferring the transition model of the environment and to multi-agent settings, enabling coordination among multiple Actors.

cs.LG↗
arXiv2024

Contractual Reinforcement Learning: Pulling Arms with Invisible Hands

The agency problem emerges in today's large scale machine learning tasks, where the learners are unable to direct content creation or enforce data collection. In this work, we propose a theoretical framework for aligning economic interests of different stakeholders in the online learning problems through contract design. The problem, termed \emph{contractual reinforcement learning}, naturally arises from the classic model of Markov decision processes, where a learning principal seeks to optimally influence the agent's action policy for their common interests through a set of payment rules contingent on the realization of next state. For the planning problem, we design an efficient dynamic programming algorithm to determine the optimal contracts against the far-sighted agent. For the learning problem, we introduce a generic design of no-regret learning algorithms to untangle the challenges from robust design of contracts to the balance of exploration and exploitation, reducing the complexity analysis to the construction of efficient search algorithms. For several natural classes of problems, we design tailored search algorithms that provably achieve $\tilde{O}(\sqrt{T})$ regret. We also present an algorithm with $\tilde{O}(T^{2/3})$ for the general problem that improves the existing analysis in online contract design with mild technical assumptions.

cs.LG↗
arXiv2024

Why does in-context learning fail sometimes? Evaluating in-context learning on open and closed questions

We measure the performance of in-context learning as a function of task novelty and difficulty for open and closed questions. For that purpose, we created a novel benchmark consisting of hard scientific questions, each paired with a context of various relevancy. We show that counter-intuitively, a context that is more aligned with the topic does not always help more than a less relevant context. This effect is especially visible for open questions and questions of high difficulty or novelty. This result reveals a fundamental difference between the treatment of close-form and open-form questions by large-language models and shows a need for a more robust evaluation of in-context learning on the variety of different types of questions. It also poses a new question of how to optimally select a context for large language models, especially in the context of Retrieval Augmented Generation (RAG) systems. Our results suggest that the answer to this question can be highly application-dependent and might be contingent on factors including the format of the question, the perceived difficulty level of the questions, and the novelty or popularity of the information we seek.

cs.CL↗
arXiv2024

Training Dynamics of Multi-Head Softmax Attention for In-Context Learning: Emergence, Convergence, and Optimality

We study the dynamics of gradient flow for training a multi-head softmax attention model for in-context learning of multi-task linear regression. We establish the global convergence of gradient flow under suitable choices of initialization. In addition, we prove that an interesting "task allocation" phenomenon emerges during the gradient flow dynamics, where each attention head focuses on solving a single task of the multi-task model. Specifically, we prove that the gradient flow dynamics can be split into three phases -- a warm-up phase where the loss decreases rather slowly and the attention heads gradually build up their inclination towards individual tasks, an emergence phase where each head selects a single task and the loss rapidly decreases, and a convergence phase where the attention parameters converge to a limit. Furthermore, we prove the optimality of gradient flow in the sense that the limiting model learned by gradient flow is on par with the best possible multi-head softmax attention model up to a constant factor. Our analysis also delineates a strict separation in terms of the prediction accuracy of ICL between single-head and multi-head attention models. The key technique for our convergence analysis is to map the gradient flow dynamics in the parameter space to a set of ordinary differential equations in the spectral domain, where the relative magnitudes of the semi-singular values of the attention weights determines task allocation. To our best knowledge, our work provides the first convergence result for the multi-head softmax attention model.

cs.LG↗
arXiv2024

Digital finance, Bargaining Power and Gender Wage Gap

The proliferation of internet technology has catalyzed the rapid development of digital finance, significantly impacting the optimization of resource allocation in China and exerting a substantial and enduring influence on the structure of employment and income distribution. This research utilizes data sourced from the Chinese General Social Survey and the Digital Financial Inclusion Index to scrutinize the influence of digital finance on the gender wage disparity in China. The findings reveal that digital finance reduces the gender wage gap, and this conclusion remains robust after addressing endogeneity problem using instrumental variable methods. Further analysis of the underlying mechanisms indicates that digital finance facilitates female entrepreneurship by lowering financing barriers, thereby promoting employment opportunities for women and also empowering them to negotiate higher wages. Specially, digital finance enhances women's bargaining power within domestic settings, therefore exerts a positive influence on the wages of women. Sub-sample regressions demonstrate that women from economically disadvantaged backgrounds, with lower human capital, benefit more from digital finance, underscoring its inclusive nature. This study provides policy evidence for empowering vulnerable groups to increase their wages and addressing the persistent issue of gender income disparity in the labor market.

econ.GN↗
arXiv2024

A Platform for All-optical Thomson/ Compton Scattering with Versatile Parameters

A dual-beam platform for all-optical electron-photon scattering, or Thomson/Compton scattering, with adjustable collision-angle and parameter tuning ability has been developed, which, in principle, can be used for the verification of strong-field quantum electrodynamics effects. Combining this platform with a 200 TW Ti:Sapphire laser system, we demonstrated the generation of inverse Compton scattering X/gamma-rays with tunable energies from tens of keV to MeV. The polarization of X/gamma radiation was manipulated by controlling the polarization of scattering laser. In the near future, by combining this experimental platform with multi-PW laser facilities, it is proposed to experimentally generate X/gamma radiation with orbital angular momentum for the nuclear isomer excitation, and more importantly, to explore the regime transition from nonlinear Thomson scattering to nonlinear Compton scattering, eventually to demonstrate the verification of theories on extremely strong field quantum electrodynamics effects.

hep-ex↗
arXiv2024

A minimal model of boosting and waning iin a recurrent seasonal epidemic

We propose a model of the immunity to a cyclical epidemic disease taking account not only of seasonal boosts during the infectious season, but also of residual immunity remaining from one season to the next. The focus is on the exponential waning process over successive cycles, imposed on the temporal distribution of infections or exposures over a season. This distribution, interacting with the waning function, is all that is necessary to reproduce, in mathematically closed form, the mechanical cycle of boosting and waning immunity characteristic of recurrent seasonal infectious disease. Distinct from epidemiological models predicting numbers of individuals moving between infectivity compartments, our result enables us to directly estimate parameters of waning and the infectivity distribution. We can naturally iterate the cyclical process to simulate immunity trajectories over many years and thus to quantify the strong relationship between residual immunity and the time elapsed between annual infectivity peaks.

q-bio.PE↗