arXiv subjects

Siyu Chen

Publications and source records attributed to Siyu Chen.

At least 127 records · Page 7Linked to original sources

arXiv2024

Salient Sparse Visual Odometry With Pose-Only Supervision

Visual Odometry (VO) is vital for the navigation of autonomous systems, providing accurate position and orientation estimates at reasonable costs. While traditional VO methods excel in some conditions, they struggle with challenges like variable lighting and motion blur. Deep learning-based VO, though more adaptable, can face generalization problems in new environments. Addressing these drawbacks, this paper presents a novel hybrid visual odometry (VO) framework that leverages pose-only supervision, offering a balanced solution between robustness and the need for extensive labeling. We propose two cost-effective and innovative designs: a self-supervised homographic pre-training for enhancing optical flow learning from pose-only labels and a random patch-based salient point detection strategy for more accurate optical flow patch extraction. These designs eliminate the need for dense optical flow labels for training and significantly improve the generalization capability of the system in diverse and challenging environments. Our pose-only supervised method achieves competitive performance on standard datasets and greater robustness and generalization ability in extreme and unseen scenarios, even compared to dense optical flow-supervised state-of-the-art methods.

cs.CV↗

arXiv2024

Wasserstein Flow Meets Replicator Dynamics: A Mean-Field Analysis of Representation Learning in Actor-Critic

Actor-critic (AC) algorithms, empowered by neural networks, have had significant empirical success in recent years. However, most of the existing theoretical support for AC algorithms focuses on the case of linear function approximations, or linearized neural networks, where the feature representation is fixed throughout training. Such a limitation fails to capture the key aspect of representation learning in neural AC, which is pivotal in practical problems. In this work, we take a mean-field perspective on the evolution and convergence of feature-based neural AC. Specifically, we consider a version of AC where the actor and critic are represented by overparameterized two-layer neural networks and are updated with two-timescale learning rates. The critic is updated by temporal-difference (TD) learning with a larger stepsize while the actor is updated via proximal policy optimization (PPO) with a smaller stepsize. In the continuous-time and infinite-width limiting regime, when the timescales are properly separated, we prove that neural AC finds the globally optimal policy at a sublinear rate. Additionally, we prove that the feature representation induced by the critic network is allowed to evolve within a neighborhood of the initial one.

cs.LG↗

arXiv2024

Adaptive Parameter Estimation under Finite Excitation

Although persistent excitation is often acknowledged as a sufficient condition to exponentially converge in the field of adaptive parameter estimation, it must be noted that in practical applications this may be unguaranteed. Recently, more attention has turned to another relaxed condition, i.e., finite excitation. In this paper, for a class of nominal nonlinear systems with unknown constant parameters, a novel method that combines the Newton algorithm and the time-varying factor is proposed, which can achieve exponential convergence under finite excitation. First, by introducing pre-filtering, the nominal system is transformed to a linear parameterized form. Then the detailed mathematical derivation is outlined from an estimation error accumulated cost function. And it is given that the theoretical analysis of the proposed method in stability and robustness. Finally, comparative numerical simulations are given to illustrate the superiority of the proposed method.

eess.SY↗

arXiv2024

Implicit Regularization of Gradient Flow on One-Layer Softmax Attention

We study gradient flow on the exponential loss for a classification problem with a one-layer softmax attention model, where the key and query weight matrices are trained separately. Under a separability assumption on the data, we show that when gradient flow achieves the minimal loss value, it further implicitly minimizes the nuclear norm of the product of the key and query weight matrices. Such implicit regularization can be described by a Support Vector Machine (SVM) problem with respect to the attention weights. This finding contrasts with prior results showing that the gradient descent induces an implicit regularization on the Frobenius norm on the product weight matrix when the key and query matrices are combined into a single weight matrix for training. For diagonal key and query matrices, our analysis builds upon the reparameterization technique and exploits approximate KKT conditions of the SVM associated with the classification data. Moreover, the results are extended to general weights configurations given proper alignment of the weight matrices' singular spaces with the data features at initialization.

cs.LG↗

arXiv2024

Cooperative Learning with Gaussian Processes for Euler-Lagrange Systems Tracking Control under Switching Topologies

This work presents an innovative learning-based approach to tackle the tracking control problem of Euler-Lagrange multi-agent systems with partially unknown dynamics operating under switching communication topologies. The approach leverages a correlation-aware cooperative algorithm framework built upon Gaussian process regression, which adeptly captures inter-agent correlations for uncertainty predictions. A standout feature is its exceptional efficiency in deriving the aggregation weights achieved by circumventing the computationally intensive posterior variance calculations. Through Lyapunov stability analysis, the distributed control law ensures bounded tracking errors with high probability. Simulation experiments validate the protocol's efficacy in effectively managing complex scenarios, establishing it as a promising solution for robust tracking control in multi-agent systems characterized by uncertain dynamics and dynamic communication structures.

cs.MA↗

arXiv2024

On the dynamical stability of copper-doped lead apatite

The recent claim of room temperature superconductivity in a copper-doped lead apatite compound, called LK-99, has sparked remarkable interest and controversy. Subsequent experiments have largely failed to reproduce the claimed superconductivity, while theoretical works have identified multiple key features including strong electronic correlation, structural instabilities, and dopability constraints. A puzzling claim of several recent theoretical studies is that both parent and copper-doped lead apatite structures are dynamically unstable at the harmonic level, questioning decades of experimental reports of the parent compound structures and the recently proposed copper-doped structures. In this work, we demonstrate that both parent and copper-doped lead apatite structures are dynamically stable at room temperature. Anharmonic phonon-phonon interactions play a key role in stabilizing some copper-doped phases, while most phases are largely stable even at the harmonic level. We also show that dynamical stability depends on both volume and correlation strength, suggesting controllable ways of exploring the copper-doped lead apatite structural phase diagram. Our results fully reconcile the theoretical description of the structures of both parent and copper-doped lead apatite with experiment.

cond-mat.supr-con↗

arXiv2024

The planted XY model: thermodynamics and inference

In this paper we study a fully connected planted spin glass named the planted XY model. Motivation for studying this system comes both from the spin glass field and the one of statistical inference where it models the angular synchronization problem. We derive the replica symmetric (RS) phase diagram in the temperature, ferromagnetic bias plane using the approximate message passing (AMP) algorithm and its state evolution (SE). While the RS predictions are exact on the Nishimori line (i.e. when the temperature is matched to the ferromagnetic bias), they become inaccurate when the parameters are mismatched, giving rise to a spin glass phase where AMP is not able to converge. To overcome the defects of the RS approximation we carry out a one-step replica symmetry breaking (1RSB) analysis based on the approximate survey propagation (ASP) algorithm. Exploiting the state evolution of ASP, we count the number of metastable states in the measure, derive the 1RSB free entropy and find the behavior of the Parisi parameter throughout the spin glass phase.

cond-mat.dis-nn↗

arXiv2023

Context Matters: Data-Efficient Augmentation of Large Language Models for Scientific Applications

In this paper, we explore the challenges inherent to Large Language Models (LLMs) like GPT-4, particularly their propensity for hallucinations, logic mistakes, and incorrect conclusions when tasked with answering complex questions. The capacity of LLMs to present erroneous answers in a coherent and semantically rigorous manner further complicates the detection of factual inaccuracies. This issue is especially pronounced in fields that require specialized expertise. Our work delves into these challenges, aiming to enhance the understanding and mitigation of such errors, thereby contributing to the improvement of LLM accuracy and reliability in scientific and other specialized domains. Our findings reveal a non-linear relationship between the context's relevancy and the answers' measured quality. In addition, we demonstrate that with the correct calibration, it is possible to automate the grading procedure -- a finding suggesting that, at least to some degree, the LLMs can be used to self-examine the quality of their own performance. Finally, we describe an experimental platform that can be seen as a proof-of-concept of the techniques described in this work.

cs.CL↗

arXiv2023

Towards a Muon Collider

A muon collider would enable the big jump ahead in energy reach that is needed for a fruitful exploration of fundamental interactions. The challenges of producing muon collisions at high luminosity and 10 TeV centre of mass energy are being investigated by the recently-formed International Muon Collider Collaboration. This Review summarises the status and the recent advances on muon colliders design, physics and detector studies. The aim is to provide a global perspective of the field and to outline directions for future work.

physics.acc-ph↗

arXiv2023

Temperature effects in topological insulators of transition metal dichalcogenide monolayers

We investigate the role of temperature on the topological insulating state of metal dichalcogenide monolayers, 1T$^\prime$-MX$_2$ (M=W, Mo and X=S, Se). Using first principles calculations based on density functional theory, we consider three temperature-related contributions to the topological band gap: electrons coupling with short-wavelength phonons, with long-wavelength phonons \textit{via} Fröhlich coupling, and thermal expansion. We find that electron-phonon coupling promotes the topology of the electronic structures of all 1T$^\prime$-MX$_2$ monolayers, while thermal expansion acts as a counteracting effect. Additionally, we derive the band renormalization from Fröhlich coupling in the two-dimensional context and observe its relatively modest contribution to 1T$^\prime$-MX$_2$ monolayers. Finally, we present a simplified physical picture to understand the "inverse Varshni" effect driven by band inversion in topological insulators. Our work reveals that, among the four 1T$^\prime$-MX$_2$ studied monolayers, MoSe$_2$ is a promising candidate for room temperature applications as its band gap displays remarkable resilience against thermal expansion, while the topological order of WS$_2$ can be tuned under the combined influence of strain and temperature. Both materials represent novel examples of temperature promoted topological insulators.

cond-mat.mtrl-sci↗

arXiv2023

Few-shot Forgery Detection via Guided Adversarial Interpolation

The increase in face manipulation models has led to a critical issue in society - the synthesis of realistic visual media. With the emergence of new forgery approaches at an unprecedented rate, existing forgery detection methods suffer from significant performance drops when applied to unseen novel forgery approaches. In this work, we address the few-shot forgery detection problem by 1) designing a comprehensive benchmark based on coverage analysis among various forgery approaches, and 2) proposing Guided Adversarial Interpolation (GAI). Our key insight is that there exist transferable distribution characteristics between majority and minority forgery classes1. Specifically, we enhance the discriminative ability against novel forgery approaches via adversarially interpolating the forgery artifacts of the minority samples to the majority samples under the guidance of a teacher network. Unlike the standard re-balancing method which usually results in over-fitting to minority classes, our method simultaneously takes account of the diversity of majority information as well as the significance of minority information. Extensive experiments demonstrate that our GAI achieves state-of-the-art performances on the established few-shot forgery detection benchmark. Notably, our method is also validated to be robust to choices of majority and minority forgery approaches. The formal publication version is available in Pattern Recognition.

cs.CV↗

arXiv2023

Computational prediction of samarium hydride at megabar pressure

Samarium hydrides, belonging to the broad class of lanthanide polyhydrides, have yet to be experimentally tested at high pressure. In this study, we use random structure searches to explore multiple possible stoichiometries and propose SmH$_2$ with a layered hexagonal structure in the $P6/mmm$ space group and SmH$_6$ with hydrogen clathrate structures in the Im-3m space group as theoretically stable phases of samarium hydrides at a wide range of pressures centered around $200$ GPa. We further combine the first-principles methods of density functional theory and dynamical mean-field theory to explore many-body correlations in samarium hydrides, reporting electron and phonon dispersions and densities of states, and also evaluate the electron-phonon driven superconductivity to investigate low critical temperatures at $200$ GPa.

cond-mat.str-el↗

arXiv2023

Boosting likelihood learning with event reweighting

Extracting maximal information from experimental data requires access to the likelihood function, which however is never directly available for complex experiments like those performed at high energy colliders. Theoretical predictions are obtained in this context by Monte Carlo events, which do furnish an accurate but abstract and implicit representation of the likelihood. Strategies based on statistical learning are currently being developed to infer the likelihood function explicitly by training a continuous-output classifier on Monte Carlo events. In this paper, we investigate the usage of Monte Carlo events that incorporate the dependence on the parameters of interest by reweighting. This enables more accurate likelihood learning with less training data and a more robust learning scheme that is more suited for automation and extensive deployment. We illustrate these advantages in the context of LHC precision probes of new Effective Field Theory interactions.

hep-ph↗

arXiv2023

Learning to Incentivize Information Acquisition: Proper Scoring Rules Meet Principal-Agent Model

We study the incentivized information acquisition problem, where a principal hires an agent to gather information on her behalf. Such a problem is modeled as a Stackelberg game between the principal and the agent, where the principal announces a scoring rule that specifies the payment, and then the agent then chooses an effort level that maximizes her own profit and reports the information. We study the online setting of such a problem from the principal's perspective, i.e., designing the optimal scoring rule by repeatedly interacting with the strategic agent. We design a provably sample efficient algorithm that tailors the UCB algorithm (Auer et al., 2002) to our model, which achieves a sublinear $T^{2/3}$-regret after $T$ iterations. Our algorithm features a delicate estimation procedure for the optimal profit of the principal, and a conservative correction scheme that ensures the desired agent's actions are incentivized. Furthermore, a key feature of our regret bound is that it is independent of the number of states of the environment.

cs.LG↗

arXiv2023

Actions Speak What You Want: Provably Sample-Efficient Reinforcement Learning of the Quantal Stackelberg Equilibrium from Strategic Feedbacks

We study reinforcement learning (RL) for learning a Quantal Stackelberg Equilibrium (QSE) in an episodic Markov game with a leader-follower structure. In specific, at the outset of the game, the leader announces her policy to the follower and commits to it. The follower observes the leader's policy and, in turn, adopts a quantal response policy by solving an entropy-regularized policy optimization problem induced by leader's policy. The goal of the leader is to find her optimal policy, which yields the optimal expected total return, by interacting with the follower and learning from data. A key challenge of this problem is that the leader cannot observe the follower's reward, and needs to infer the follower's quantal response model from his actions against leader's policies. We propose sample-efficient algorithms for both the online and offline settings, in the context of function approximation. Our algorithms are based on (i) learning the quantal response model via maximum likelihood estimation and (ii) model-free or model-based RL for solving the leader's decision making problem, and we show that they achieve sublinear regret upper bounds. Moreover, we quantify the uncertainty of these estimators and leverage the uncertainty to implement optimistic and pessimistic algorithms for online and offline settings. Besides, when specialized to the linear and myopic setting, our algorithms are also computationally efficient. Our theoretical analysis features a novel performance-difference lemma which incorporates the error of quantal response model, which might be of independent interest.

cs.LG↗

arXiv2023

Image Quality-aware Diagnosis via Meta-knowledge Co-embedding

Medical images usually suffer from image degradation in clinical practice, leading to decreased performance of deep learning-based models. To resolve this problem, most previous works have focused on filtering out degradation-causing low-quality images while ignoring their potential value for models. Through effectively learning and leveraging the knowledge of degradations, models can better resist their adverse effects and avoid misdiagnosis. In this paper, we raise the problem of image quality-aware diagnosis, which aims to take advantage of low-quality images and image quality labels to achieve a more accurate and robust diagnosis. However, the diversity of degradations and superficially unrelated targets between image quality assessment and disease diagnosis makes it still quite challenging to effectively leverage quality labels to assist diagnosis. Thus, to tackle these issues, we propose a novel meta-knowledge co-embedding network, consisting of two subnets: Task Net and Meta Learner. Task Net constructs an explicit quality information utilization mechanism to enhance diagnosis via knowledge co-embedding features, while Meta Learner ensures the effectiveness and constrains the semantics of these features via meta-learning and joint-encoding masking. Superior performance on five datasets with four widely-used medical imaging modalities demonstrates the effectiveness and generalizability of our method.

eess.IV↗

arXiv2023

Pressure-induced transitions in FePS$_3$: Structural, magnetic and electronic properties

FePS$_3$ is a prototype van der Waals layered antiferromagnet and a Mott insulator under ambient conditions, which has been recently reported to go through a pressure-induced dimensionality crossover and an insulator-to-metal transition. These transitions also lead to the appearance of a novel magnetic metallic state. To further understand these emergent structural and physical properties, we have performed a first-principles study using van der Waals and Hubbard $U$ corrected density functional theory including a random structure search. Our computational study attempts to interpret the experimental coexistence of the low- and intermediate-pressure phases and we predict a novel high-pressure phase with distinctive dimensionality and different possible origins of metallicity.

cond-mat.str-el↗

arXiv2023

Unsupervised detection of small hyperreflective features in ultrahigh resolution optical coherence tomography

Recent advances in optical coherence tomography such as the development of high speed ultrahigh resolution scanners and corresponding signal processing techniques may reveal new potential biomarkers in retinal diseases. Newly visible features are, for example, small hyperreflective specks in age-related macular degeneration. Identifying these new markers is crucial to investigate potential association with disease progression and treatment outcomes. Therefore, it is necessary to reliably detect these features in 3D volumetric scans. Because manual labeling of entire volumes is infeasible a need for automatic detection arises. Labeled datasets are often not publicly available and there are usually large variations in scan protocols and scanner types. Thus, this work focuses on an unsupervised approach that is based on local peak-detection and random walker segmentation to detect small features on each B-scan of the volume.

eess.IV↗