arXiv subjects

Yi Liu

Publications and source records attributed to Yi Liu.

At least 505 records · Page 28Linked to original sources

arXiv2023

Random Entity Quantization for Parameter-Efficient Compositional Knowledge Graph Representation

Representation Learning on Knowledge Graphs (KGs) is essential for downstream tasks. The dominant approach, KG Embedding (KGE), represents entities with independent vectors and faces the scalability challenge. Recent studies propose an alternative way for parameter efficiency, which represents entities by composing entity-corresponding codewords matched from predefined small-scale codebooks. We refer to the process of obtaining corresponding codewords of each entity as entity quantization, for which previous works have designed complicated strategies. Surprisingly, this paper shows that simple random entity quantization can achieve similar results to current strategies. We analyze this phenomenon and reveal that entity codes, the quantization outcomes for expressing entities, have higher entropy at the code level and Jaccard distance at the codeword level under random entity quantization. Therefore, different entities become more easily distinguished, facilitating effective KG representation. The above results show that current quantization strategies are not critical for KG representation, and there is still room for improvement in entity distinguishability beyond current strategies. The code to reproduce our results is available at https://github.com/JiaangL/RandomQuantization.

cs.AI↗

arXiv2023

MUSCLE: Multi-task Self-supervised Continual Learning to Pre-train Deep Models for X-ray Images of Multiple Body Parts

While self-supervised learning (SSL) algorithms have been widely used to pre-train deep models, few efforts [11] have been done to improve representation learning of X-ray image analysis with SSL pre-trained models. In this work, we study a novel self-supervised pre-training pipeline, namely Multi-task Self-super-vised Continual Learning (MUSCLE), for multiple medical imaging tasks, such as classification and segmentation, using X-ray images collected from multiple body parts, including heads, lungs, and bones. Specifically, MUSCLE aggregates X-rays collected from multiple body parts for MoCo-based representation learning, and adopts a well-designed continual learning (CL) procedure to further pre-train the backbone subject various X-ray analysis tasks jointly. Certain strategies for image pre-processing, learning schedules, and regularization have been used to solve data heterogeneity, overfitting, and catastrophic forgetting problems for multi-task/dataset learning in MUSCLE.We evaluate MUSCLE using 9 real-world X-ray datasets with various tasks, including pneumonia classification, skeletal abnormality classification, lung segmentation, and tuberculosis (TB) detection. Comparisons against other pre-trained models [7] confirm the proof-of-concept that self-supervised multi-task/dataset continual pre-training could boost the performance of X-ray image analysis.

cs.CV↗

arXiv2023

Incorporating Pre-trained Model Prompting in Multimodal Stock Volume Movement Prediction

Multimodal stock trading volume movement prediction with stock-related news is one of the fundamental problems in the financial area. Existing multimodal works that train models from scratch face the problem of lacking universal knowledge when modeling financial news. In addition, the models ability may be limited by the lack of domain-related knowledge due to insufficient data in the datasets. To handle this issue, we propose the Prompt-based MUltimodal Stock volumE prediction model (ProMUSE) to process text and time series modalities. We use pre-trained language models for better comprehension of financial news and adopt prompt learning methods to leverage their capability in universal knowledge to model textual information. Besides, simply fusing two modalities can cause harm to the unimodal representations. Thus, we propose a novel cross-modality contrastive alignment while reserving the unimodal heads beside the fusion head to mitigate this problem. Extensive experiments demonstrate that our proposed ProMUSE outperforms existing baselines. Comprehensive analyses further validate the effectiveness of our architecture compared to potential variants and learning mechanisms.

cs.CL↗

arXiv2023

Simulation study of BESIII with stitched CMOS pixel detector using ACTS

Reconstruction of tracks of charged particles with high precision is very crucial for HEP experiments to achieve their physics goals. As the tracking detector of BESIII experiment, the BESIII drift chamber has suffered from aging effects resulting in degraded tracking performance after operation for about 15 years. To preserve and enhance the tracking performance of BESIII, one of the proposals is to add one layer of thin CMOS pixel sensor in cylindrical shape based on the state-of-the-art stitching technology, between the beam pipe and the drift chamber. The improvement of tracking performance of BESIII with such an additional pixel detector compared to that with only the existing drift chamber is studied using the modern common tracking software ACTS, which provides a set of detector-agnostic and highly performant tracking algorithms that have demonstrated promising performance for a few high energy physics and nuclear physics experiments.

physics.ins-det↗

arXiv2023

ASTER: Automatic Speech Recognition System Accessibility Testing for Stutterers

The popularity of automatic speech recognition (ASR) systems nowadays leads to an increasing need for improving their accessibility. Handling stuttering speech is an important feature for accessible ASR systems. To improve the accessibility of ASR systems for stutterers, we need to expose and analyze the failures of ASR systems on stuttering speech. The speech datasets recorded from stutterers are not diverse enough to expose most of the failures. Furthermore, these datasets lack ground truth information about the non-stuttered text, rendering them unsuitable as comprehensive test suites. Therefore, a methodology for generating stuttering speech as test inputs to test and analyze the performance of ASR systems is needed. However, generating valid test inputs in this scenario is challenging. The reason is that although the generated test inputs should mimic how stutterers speak, they should also be diverse enough to trigger more failures. To address the challenge, we propose ASTER, a technique for automatically testing the accessibility of ASR systems. ASTER can generate valid test cases by injecting five different types of stuttering. The generated test cases can both simulate realistic stuttering speech and expose failures in ASR systems. Moreover, ASTER can further enhance the quality of the test cases with a multi-objective optimization-based seed updating algorithm. We implemented ASTER as a framework and evaluated it on four open-source ASR models and three commercial ASR systems. We conduct a comprehensive evaluation of ASTER and find that it significantly increases the word error rate, match error rate, and word information loss in the evaluated ASR systems. Additionally, our user study demonstrates that the generated stuttering audio is indistinguishable from real-world stuttering audio clips.

cs.SD↗

arXiv2023

MDS Array Codes With Small Sub-packetization Levels and Small Repair Degrees

High-rate minimum storage regenerating (MSR) codes are known to require a large sub-packetization level, which can make meta-data management difficult and hinder implementation in practical systems. A few maximum distance separable (MDS) array code constructions have been proposed to attain a much smaller sub-packetization level by sacrificing a bit of repair bandwidth. However, to the best of our knowledge, only one construction by Guruswami et al. can support the repair of a failed node without contacting all the surviving nodes. This construction is certainly of theoretical interest but not yet practical due to its requirement for very large code parameters. In this paper, we propose a generic transformation that can convert any $(\overline{n}, \overline{k})$ MSR code with a repair degree of $\overline{d}<\overline{n}-1$ into another $(n=s\overline{n},k)$ MDS array code that supports $d<n-1$ with a small sub-packetization level and $(1+ε)$-optimal repair bandwidth (i.e., $1+ε$ times the optimal value) under a specific condition. We obtain three MDS array codes with small sub-packetization levels and $(1+ε)$-optimal repair bandwidth by applying this transformation to three known MSR codes. All the new MDS array codes have a small repair degree of $d<n-1$ and work for both small and large code parameters.

cs.IT↗

arXiv2023

Finite quotients, arithmetic invariants, and hyperbolic volume

For any pair of orientable closed hyperbolic $3$--manifolds, this paper shows that any isomorphism between the profinite completions of their fundamental groups witnesses a bijective correspondence between the Zariski dense $\mathrm{PSL}(2,\mathbb{Q}^{\mathtt{ac}})$--representations of their fundamental groups, up to conjugacy; moreover, corresponding pairs of representations have identical invariant trace fields and isomorphic invariant quaternion algebras. (Here, $\mathbb{Q}^{\mathtt{ac}}$ denotes an algebraic closure of $\mathbb{Q}$.) Next, assuming the $p$--adic Borel regulator injectivity conjecture for number fields, this paper shows that uniform lattices in $\mathrm{PSL}(2,\mathbb{C})$ with isomorphic profinite completions have identical invariant trace fields, isomorphic invariant quaternion algebras, identical covolume, and identical arithmeticity.

math.GT↗

arXiv2023

Self-Supervised Dynamic Hypergraph Recommendation based on Hyper-Relational Knowledge Graph

Knowledge graphs (KGs) are commonly used as side information to enhance collaborative signals and improve recommendation quality. In the context of knowledge-aware recommendation (KGR), graph neural networks (GNNs) have emerged as promising solutions for modeling factual and semantic information in KGs. However, the long-tail distribution of entities leads to sparsity in supervision signals, which weakens the quality of item representation when utilizing KG enhancement. Additionally, the binary relation representation of KGs simplifies hyper-relational facts, making it challenging to model complex real-world information. Furthermore, the over-smoothing phenomenon results in indistinguishable representations and information loss. To address these challenges, we propose the SDK (Self-Supervised Dynamic Hypergraph Recommendation based on Hyper-Relational Knowledge Graph) framework. This framework establishes a cross-view hypergraph self-supervised learning mechanism for KG enhancement. Specifically, we model hyper-relational facts in KGs to capture interdependencies between entities under complete semantic conditions. With the refined representation, a hypergraph is dynamically constructed to preserve features in the deep vector space, thereby alleviating the over-smoothing problem. Furthermore, we mine external supervision signals from both the global perspective of the hypergraph and the local perspective of collaborative filtering (CF) to guide the model prediction process. Extensive experiments conducted on different datasets demonstrate the superiority of the SDK framework over state-of-the-art models. The results showcase its ability to alleviate the effects of over-smoothing and supervision signal sparsity.

cs.IR↗

arXiv2023

Online Local Differential Private Quantile Inference via Self-normalization

Based on binary inquiries, we developed an algorithm to estimate population quantiles under Local Differential Privacy (LDP). By self-normalizing, our algorithm provides asymptotically normal estimation with valid inference, resulting in tight confidence intervals without the need for nuisance parameters to be estimated. Our proposed method can be conducted fully online, leading to high computational efficiency and minimal storage requirements with $\mathcal{O}(1)$ space. We also proved an optimality result by an elegant application of one central limit theorem of Gaussian Differential Privacy (GDP) when targeting the frequently encountered median estimation problem. With mathematical proof and extensive numerical testing, we demonstrate the validity of our algorithm both theoretically and experimentally.

stat.ME↗

arXiv2023

ExploitFlow, cyber security exploitation routes for Game Theory and AI research in robotics

This paper addresses the prevalent lack of tools to facilitate and empower Game Theory and Artificial Intelligence (AI) research in cybersecurity. The primary contribution is the introduction of ExploitFlow (EF), an AI and Game Theory-driven modular library designed for cyber security exploitation. EF aims to automate attacks, combining exploits from various sources, and capturing system states post-action to reason about them and understand potential attack trees. The motivation behind EF is to bolster Game Theory and AI research in cybersecurity, with robotics as the initial focus. Results indicate that EF is effective for exploring machine learning in robot cybersecurity. An artificial agent powered by EF, using Reinforcement Learning, outperformed both brute-force and human expert approaches, laying the path for using ExploitFlow for further research. Nonetheless, we identified several limitations in EF-driven agents, including a propensity to overfit, the scarcity and production cost of datasets for generalization, and challenges in interpreting networking states across varied security settings. To leverage the strengths of ExploitFlow while addressing identified shortcomings, we present Malism, our vision for a comprehensive automated penetration testing framework with ExploitFlow at its core.

cs.RO↗

arXiv2023

Amplification of light pulses with orbital angular momentum (OAM) in nitrogen ions lasing

Nitrogen ions pumped by intense femtosecond laser pulses give rise to optical amplification in the ultraviolet range. Here, we demonstrated that a seed light pulse carrying orbital angular momentum (OAM) can be significantly amplified in nitrogen plasma excited by a Gaussian femtosecond laser pulse. With the topological charge of +1 and -1, we observed an energy amplification of the seed light pulse by two orders of magnitude, while the amplified pulse carries the same OAM as the incident seed pulse. Moreover, we show that a spatial misalignment of the plasma amplifier with the OAM seed beam leads to an amplified emission of Gaussian mode without OAM, due to the special spatial profile of the OAM seed pulse that presents a donut-shaped intensity distribution. Utilizing this misalignment, we can implement an optical switch that toggles the output signal between Gaussian mode and OAM mode. This work not only certifies the phase transfer from the seed light to the amplified signal, but also highlights the important role of spatial overlap of the donut-shaped seed beam with the gain region of the nitrogen plasma for the achievement of OAM beam amplification.

physics.optics↗

arXiv2023

Exploring high scale seesaw models through a supersymmetric portal

The seesaw scale is a priori unknown. If the seesaw scale is low, one may directly look for the new particles predicted by seesaw models. If the seesaw scale is high, such an approach is unfeasible. We show that in some supersymmetric seesaw models the large Yukawa couplings of high scale seesaw models leave their fingerprints to Higgs-slepton couplings and that this can result in decays of the type $\tildeν_{2}\rightarrow \tildeν_{1}h$ in Type-I and Type-III seesaw models and $\tilde{\ell}^{\pm}_{2}\rightarrow \tilde{\ell}^{\pm}_{1}h$ in the latter. Unfortunately the current exclusion bounds make it impossible to see a significant signal even at the High-Luminosity phase of the LHC. In this paper, we highlight that the High-Energy phase of the LHC (with $\sqrt{s}=27$~TeV) could afford one with some sensitivity to those in the single lepton channel.

hep-ph↗

arXiv2023

A Joint Design for Full-duplex OFDM AF Relay System with Precoded Short Guard Interval

In-band full-duplex relay (FDR) has attracted much attention as an effective solution to improve the coverage and spectral efficiency in wireless communication networks. The basic problem for FDR transmission is how to eliminate the inherent self-interference and re-use the residual self-interference (RSI) at the relay to improve the end-to-end performance. Considering the RSI at the FDR, the overall equivalent channel can be modeled as an infinite impulse response (IIR) channel. For this IIR channel, a joint design for precoding, power gain control and equalization of cooperative OFDM relay systems is presented. Compared with the traditional OFDM systems, the length of the guard interval for the proposed design can be distinctly reduced, thereby improving the spectral efficiency. By analyzing the noise sources, this paper evaluates the signal to noise ratio (SNR) of the proposed scheme and presents a power gain control algorithm at the FDR. Compared with the existing schemes, the proposed scheme shows a superior bit error rate (BER) performance.

cs.IT↗

arXiv2023

Enhanced Population on Ionic Excited States by Synchronized Ionization and Multiphoton Resonance

We study population distributions and lasing actions of N_2^+ driven by femtosecond lasers with various wavelengths, and uncover an efficient ionic excitation mechanism induced by synchronized ionization and multiphoton resonance. Our results show that the strongest N_2^+ lasing appears around 1000 nm pump wavelength. At the optimal wavelength, the pump-energy threshold for air lasing generation is reduced by five folds compared with that required by the previous 800 nm pump laser. Simulations based on the ionization-coupling model indicate that although the Stark-assisted three-photon resonance can be satisfied within a broad pump wavelength range, the optimal pump wavelength arises when the dynamic three-photon resonance temporally synchronizes with the ionization injection. In this case, the ionic dipoles created at each half optical cycle have the same phase. The dipole phase locking promotes the continuous population transfer from ionic ground state to the excited state, giving rise to a dramatic increase of excited-state population. This work provides new insight on the photoexcitation mechanism of ions in strong laser fields, and opens up a route for optimizing ionic radiations.

physics.atom-ph↗

arXiv2023

Implementation of ACTS for STCF track reconstruction

With an electron-positron collider operating at center-of-mass-energy 2-7 GeV and a peak luminosity above $0.5\times10^{35} cm^{-2} s^{-1}$, the STCF physics program will provide an unique platform for in-depth studies of hadron structure and non-perturbative strong interaction as well as probing new physics beyond the Standard Model in the $τ$-Charm sector, succeeding the present Beijing Electron-Positron Collider. To fulfill the physics targets and further maximize the physics potential at STCF, the STCF tracking software should have capability to reconstruct charged particles with high efficiency and excellent momentum resolution, especially for the charged particles with low transverse momentum down to 50 MeV. A Common Tracking Software (ACTS) providing a set of detector-independent tracking algorithms is adopted for reconstructing charged tracks with the information of two sub-detectors, a $μ$RWELL-based inner tracker and a drift chamber, at STCF. This is the first demonstration of ACTS for a drift chamber. The implementation details and performance of track reconstruction are presented.

hep-ex↗

arXiv2023

SFP: Spurious Feature-targeted Pruning for Out-of-Distribution Generalization

Model substructure learning aims to find an invariant network substructure that can have better out-of-distribution (OOD) generalization than the original full structure. Existing works usually search the invariant substructure using modular risk minimization (MRM) with fully exposed out-domain data, which may bring about two drawbacks: 1) Unfairness, due to the dependence of the full exposure of out-domain data; and 2) Sub-optimal OOD generalization, due to the equally feature-untargeted pruning on the whole data distribution. Based on the idea that in-distribution (ID) data with spurious features may have a lower experience risk, in this paper, we propose a novel Spurious Feature-targeted model Pruning framework, dubbed SFP, to automatically explore invariant substructures without referring to the above drawbacks. Specifically, SFP identifies spurious features within ID instances during training using our theoretically verified task loss, upon which, SFP attenuates the corresponding feature projections in model space to achieve the so-called spurious feature-targeted pruning. This is typically done by removing network branches with strong dependencies on identified spurious features, thus SFP can push the model learning toward invariant features and pull that out of spurious features and devise optimal OOD generalization. Moreover, we also conduct detailed theoretical analysis to provide the rationality guarantee and a proof framework for OOD structures via model sparsity, and for the first time, reveal how a highly biased data distribution affects the model's OOD generalization. Experiments on various OOD datasets show that SFP can significantly outperform both structure-based and non-structure-based OOD generalization SOTAs, with accuracy improvement up to 4.72% and 23.35%, respectively

cs.LG↗

arXiv2023

ADENIUM -- A demonstrator for a next-generation beam telescope at DESY

High-resolution beam telescopes for charged particle tracking are one of the most important and equally demanding infrastructure items at test beam facilities. The main purpose of beam telescopes is to provide precise reference track information of beam particles to measure the performance of a device under test (DUT). In this report the development of the ADENIUM beam telescope (ALPIDE sensor based DESY Next test beam Instrument) as a demonstrator and prototype for a next-generation beam telescope is presented. The ADENIUM beam telescope features up to six pixelated reference planes framed by plastic scintillators for triggering. ADENIUM is capable of replacing the currently used EUDET-type beam telescopes without impacting existing DUT implementations due to the integration of the telescope DAQ into EUDAQ2. In this report the concept and design of the ADENIUM telescope as well as its performance are discussed. The telescope's pointing resolution is determined in different configurations. For an optimal setup at an momentum of 5.6 GeV with an ALPIDE as DUT, a resolution better than 3 um has been extracted. No rate limitations have been observed at the DESY II test beam.

physics.ins-det↗

arXiv2023

An Ultra-low Power TinyML System for Real-time Visual Processing at Edge

Tiny machine learning (TinyML), executing AI workloads on resource and power strictly restricted systems, is an important and challenging topic. This brief firstly presents an extremely tiny backbone to construct high efficiency CNN models for various visual tasks. Then, a specially designed neural co-processor (NCP) is interconnected with MCU to build an ultra-low power TinyML system, which stores all features and weights on chip and completely removes both of latency and power consumption in off-chip memory access. Furthermore, an application specific instruction-set is further presented for realizing agile development and rapid deployment. Extensive experiments demonstrate that the proposed TinyML system based on our model, NCP and instruction set yields considerable accuracy and achieves a record ultra-low power of 160mW while implementing object detection and recognition at 30FPS. The demo video is available on \url{https://www.youtube.com/watch?v=mIZPxtJ-9EY}.

eess.IV↗