arXiv subjects

Jie Ren

Publications and source records attributed to Jie Ren.

At least 127 records · Page 7Linked to original sources

arXiv2024

Machine Learning of Knot Topology in Non-Hermitian Band Braids

The deep connection among braids, knots and topological physics has provided valuable insights into studying topological states in various physical systems. However, identifying distinct braid groups and knot topology embedded in non-Hermitian systems is challenging and requires significant efforts. Here, we demonstrate that an unsupervised learning with the representation basis of $su(n)$ Lie algebra on $n$-fold extended non-Hermitian bands can fully classify braid group and knot topology therein, without requiring any prior mathematical knowledge or any pre-defined topological invariants. We demonstrate that the approach successfully identifies different topological elements, such as unlink, unknot, Hopf link, Solomon ring, trefoil, and so on, by employing generalized Gell-Mann matrices in non-Hermitian models with $n$=2 and $n$=3 energy bands. Moreover, since eigenstate information of non-Hermitian bands is incorporated in addition to eigenvalues, the approach distinguishes the different parity-time symmetry and breaking phases, recognizes the opposite chirality of braids and knots, and identifies out distinct topological phases that were overlooked before. Our study shows significant potential of machine learning in classification of knots, braid groups, and non-Hermitian topological phases.

cond-mat.mes-hall↗

arXiv2024

Spin-Transfer-Torque Induced Spatially Nonuniform Switching in Ferrimagnets

Ferrimagnet (FiM), (FeCo)1-xGdx, attracts research attention due to its ultrafast magnetic dynamics and finite net magnetization. Incorporating FiM into the magnetic tunnel junction will be beneficial to further improve the writing speed of magnetic random access memory (MRAM). It is commonly assumed that the FeCo and Gd atoms are switched together due to the strong exchange coupling, which remains valid even if one performs the two-sublattice macrospin simulation. Interestingly, using the atomistic model developed by our group, it is clearly seen that different atoms are not switched together. In addition, our study reveals that the nature of switching is spatially nonuniform even in the small sample with the dimension of 20 nm-20 nm. Furthermore, the characteristics of nonuniformity are completely different for samples with different Gd composition (x). When x is close to the magnetization compensation point, successful switching cannot be obtained, but is accompanied by the stable oscillation. The atom type that dominates the oscillation is different from that predicted by the two-sublattice macrospin model. In addition, the size of singular region is a non-monotonic function of current density. All these results can only be understood by considering the spatial nonuniform magnetization dynamics.

cond-mat.mes-hall↗

arXiv2023

Superdiffusive Transport in Quasi-Particle Dephasing Models

Investigating the behavior of noninteracting fermions subjected to local dephasing, we reveal that quasi-particle dephasing can induce superdiffusive transport. This superdiffusion arises from nodal points within the momentum distribution of local dephasing quasi-particles, leading to asymptotic long-lived modes. By studying the dynamics of the Wigner function, we rigorously elucidate how the dynamics of these enduring modes give rise to Lévy walk processes, a renowned mechanism underlying superdiffusion phenomena. Our research demonstrates the controllability of dynamical scaling exponents by selecting quasi-particles and extends its applicability to higher dimensions, underlining the pervasive nature of superdiffusion in dephasing models.

cond-mat.stat-mech↗

arXiv2023

Self-Evaluation Improves Selective Generation in Large Language Models

Safe deployment of large language models (LLMs) may benefit from a reliable method for assessing their generated content to determine when to abstain or to selectively generate. While likelihood-based metrics such as perplexity are widely employed, recent research has demonstrated the limitations of using sequence-level probability estimates given by LLMs as reliable indicators of generation quality. Conversely, LLMs have demonstrated strong calibration at the token level, particularly when it comes to choosing correct answers in multiple-choice questions or evaluating true/false statements. In this work, we reformulate open-ended generation tasks into token-level prediction tasks, and leverage LLMs' superior calibration at the token level. We instruct an LLM to self-evaluate its answers, employing either a multi-way comparison or a point-wise evaluation approach, with the option to include a ``None of the above'' option to express the model's uncertainty explicitly. We benchmark a range of scoring methods based on self-evaluation and evaluate their performance in selective generation using TruthfulQA and TL;DR. Through experiments with PaLM-2 and GPT-3, we demonstrate that self-evaluation based scores not only improve accuracy, but also correlate better with the overall quality of generated content.

cs.CL↗

arXiv2023

Improving the Robustness of Summarization Models by Detecting and Removing Input Noise

The evaluation of abstractive summarization models typically uses test data that is identically distributed as training data. In real-world practice, documents to be summarized may contain input noise caused by text extraction artifacts or data pipeline bugs. The robustness of model performance under distribution shift caused by such noise is relatively under-studied. We present a large empirical study quantifying the sometimes severe loss in performance (up to 12 ROUGE-1 points) from different types of input noise for a range of datasets and model sizes. We then propose a light-weight method for detecting and removing such noise in the input during model inference without requiring any extra training, auxiliary models, or even prior knowledge of the type of noise. Our proposed approach effectively mitigates the loss in performance, recovering a large fraction of the performance drop, sometimes as large as 11 ROUGE-1 points.

cs.CL↗

arXiv2023

Interpreting and Disentangling Feature Components of Various Complexity from DNNs

This paper aims to define, quantify, and analyze the feature complexity that is learned by a DNN. We propose a generic definition for the feature complexity. Given the feature of a certain layer in the DNN, our method disentangles feature components of different complexity orders from the feature. We further design a set of metrics to evaluate the reliability, the effectiveness, and the significance of over-fitting of these feature components. Furthermore, we successfully discover a close relationship between the feature complexity and the performance of DNNs. As a generic mathematical tool, the feature complexity and the proposed metrics can also be used to analyze the success of network compression and knowledge distillation.

cs.LG↗

arXiv2023

A Unified Approach to Interpreting and Boosting Adversarial Transferability

In this paper, we use the interaction inside adversarial perturbations to explain and boost the adversarial transferability. We discover and prove the negative correlation between the adversarial transferability and the interaction inside adversarial perturbations. The negative correlation is further verified through different DNNs with various inputs. Moreover, this negative correlation can be regarded as a unified perspective to understand current transferability-boosting methods. To this end, we prove that some classic methods of enhancing the transferability essentially decease interactions inside adversarial perturbations. Based on this, we propose to directly penalize interactions during the attacking process, which significantly improves the adversarial transferability.

cs.LG↗

arXiv2023

Universal Self-Consistency for Large Language Model Generation

Self-consistency with chain-of-thought prompting (CoT) has demonstrated remarkable performance gains on various challenging tasks, by utilizing multiple reasoning paths sampled from large language models (LLMs). However, self-consistency relies on the answer extraction process to aggregate multiple solutions, which is not applicable to free-form answers. In this work, we propose Universal Self-Consistency (USC), which leverages LLMs themselves to select the most consistent answer among multiple candidates. We evaluate USC on a variety of benchmarks, including mathematical reasoning, code generation, long-context summarization, and open-ended question answering. On open-ended generation tasks where the original self-consistency method is not applicable, USC effectively utilizes multiple samples and improves the performance. For mathematical reasoning, USC matches the standard self-consistency performance without requiring the answer formats to be similar. Finally, without access to execution results, USC also matches the execution-based voting performance on code generation.

cs.CL↗

arXiv2023

Quantum complexity and bulk timelike singularities

Quantum complexity has already shed light on CFT states dual to bulk geometries containing spacelike singularities \cite{Barbon:2015ria, Bolognesi:2018ion, Caputa:2021pad}. In this work, we turn our attention to quantum complexity of CFT/quantum gravity states dual to bulk geometries with a naked timelike singularity. The appearance of naked timelike singularities in semiclassical gravity is allowed in string theory, particularly in the context of holography, so long as they satisfy the \emph{Gubser criterion} \cite{Gubser:2000nd, Gursoy:2008za}. In this work, we use holographic complexity as a probe on geometries containing naked timelike singularities and explore potential relation to the Gubser criterion for detecting allowable naked timelike singularities. We study three specific cases, namely the negative mass Schwarzschild-AdS spacetime, the timelike Kasner-AdS \cite{Ren:2016xhb} and Einstein-dilaton system \cite{Ren:2019lgw}. The first two cases are outright ruled out by the Gubser criterion while the third case is more subtle - according to the Gubser criterion the singularity switches from forbidden to admissible as the parameter $α$ is dialed in the range $[0,1]$ across the transition point at $α= 1/\sqrt{3}$. We probe all three geometries using two holographic complexity prescriptions, namely CA and CV. For the case of the negative mass SAdS and timelike Kasner-AdS$_4$ the complexities display no sign of pathology (both receive finite contribution from the naked singularity). For the Einstein-Dilaton case, action-complexity does display a sharp transition from physical positive values to patholgical negative divergent values (arising from the singularity) as one transcends the Gubser bound. Our study suggests that neither action-complexity (CA) nor volume-complexity (CV) can serve as a sensitive tool to investigate (naked) timelike singularities.

hep-th↗

arXiv2023

Probabilistic Categorical Adversarial Attack & Adversarial Training

The existence of adversarial examples brings huge concern for people to apply Deep Neural Networks (DNNs) in safety-critical tasks. However, how to generate adversarial examples with categorical data is an important problem but lack of extensive exploration. Previously established methods leverage greedy search method, which can be very time-consuming to conduct successful attack. This also limits the development of adversarial training and potential defenses for categorical data. To tackle this problem, we propose Probabilistic Categorical Adversarial Attack (PCAA), which transfers the discrete optimization problem to a continuous problem that can be solved efficiently by Projected Gradient Descent. In our paper, we theoretically analyze its optimality and time complexity to demonstrate its significant advantage over current greedy based attacks. Moreover, based on our attack, we propose an efficient adversarial training framework. Through a comprehensive empirical study, we justify the effectiveness of our proposed attack and defense algorithms.

cs.LG↗

arXiv2023

QUIK: Towards End-to-End 4-Bit Inference on Generative Large Language Models

Large Language Models (LLMs) from the GPT family have become extremely popular, leading to a race towards reducing their inference costs to allow for efficient local computation. Yet, the vast majority of existing work focuses on weight-only quantization, which can reduce runtime costs in the memory-bound one-token-at-a-time generative setting, but does not address them in compute-bound scenarios, such as batched inference or prompt processing. In this paper, we address the general quantization problem, where both weights and activations should be quantized. We show, for the first time, that the majority of inference computations for large generative models such as LLaMA, OPT, and Falcon can be performed with both weights and activations being cast to 4 bits, in a way that leads to practical speedups, while at the same time maintaining good accuracy. We achieve this via a hybrid quantization strategy called QUIK, which compresses most of the weights and activations to 4-bit, while keeping some outlier weights and activations in higher-precision. The key feature of our scheme is that it is designed with computational efficiency in mind: we provide GPU kernels matching the QUIK format with highly-efficient layer-wise runtimes, which lead to practical end-to-end throughput improvements of up to 3.4x relative to FP16 execution. We provide detailed studies for models from the OPT, LLaMA-2 and Falcon families, as well as a first instance of accurate inference using quantization plus 2:4 sparsity. Code is available at: https://github.com/IST-DASLab/QUIK.

cs.LG↗

arXiv2023

Dataset for Investigating Anomalies in Compute Clusters

The dataset was collected for 332 compute nodes throughout May 19 - 23, 2023. May 19 - 22 characterizes normal compute cluster behavior, while May 23 includes an anomalous event. The dataset includes eight CPU, 11 disk, 47 memory, and 22 Slurm metrics. It represents five distinct hardware configurations and contains over one million records, totaling more than 180GB of raw data.

cs.DC↗

arXiv2023

On the Generalization of Training-based ChatGPT Detection Methods

ChatGPT is one of the most popular language models which achieve amazing performance on various natural language tasks. Consequently, there is also an urgent need to detect the texts generated ChatGPT from human written. One of the extensively studied methods trains classification models to distinguish both. However, existing studies also demonstrate that the trained models may suffer from distribution shifts (during test), i.e., they are ineffective to predict the generated texts from unseen language tasks or topics. In this work, we aim to have a comprehensive investigation on these methods' generalization behaviors under distribution shift caused by a wide range of factors, including prompts, text lengths, topics, and language tasks. To achieve this goal, we first collect a new dataset with human and ChatGPT texts, and then we conduct extensive studies on the collected dataset. Our studies unveil insightful findings which provide guidance for developing future methodologies or data collection strategies for ChatGPT detection.

cs.CL↗

arXiv2023

Retrieve-Rewrite-Answer: A KG-to-Text Enhanced LLMs Framework for Knowledge Graph Question Answering

Despite their competitive performance on knowledge-intensive tasks, large language models (LLMs) still have limitations in memorizing all world knowledge especially long tail knowledge. In this paper, we study the KG-augmented language model approach for solving the knowledge graph question answering (KGQA) task that requires rich world knowledge. Existing work has shown that retrieving KG knowledge to enhance LLMs prompting can significantly improve LLMs performance in KGQA. However, their approaches lack a well-formed verbalization of KG knowledge, i.e., they ignore the gap between KG representations and textual representations. To this end, we propose an answer-sensitive KG-to-Text approach that can transform KG knowledge into well-textualized statements most informative for KGQA. Based on this approach, we propose a KG-to-Text enhanced LLMs framework for solving the KGQA task. Experiments on several KGQA benchmarks show that the proposed KG-to-Text augmented LLMs approach outperforms previous KG-augmented LLMs approaches regarding answer accuracy and usefulness of knowledge statements.

cs.CL↗

arXiv2023

Development of 15kA/cm$^2$ Fabrication Process for Superconducting Integrated Digital Circuits

A new fabrication process for superconducting integrated digital circuits is reported. We have developed the "SIMIT Nb04" fabrication technique for superconducting integrated circuits with Nb-based Josephson junctions based on the validated "SIMIT Nb03" process and Chemical Mechanical Planarization (CMP) technology. Seven Nb superconducting layers and one Mo resistor layer are included in the "SIMIT Nb04" process with 19 mask levels. The device structure is composed of active layers including junctions at the bottom, two passive transmission line (PTL) layers in the middle and a DC power layer at the top. The circuit fabrication started with the fabrication of Mo resistors with a target sheet resistance Rsh of 3 $Ω$, followed by the deposition of Nb/Al-AlO$_x$/Nb trilayer Josephson-junction with a target critical current density Jc at 15 kA/cm$^2$. To increase the Al-AlO$_x$ barrier layer etching's repeatability, an additional barrier protection layer was applied. To accomplish high-quality planarization, we created a planarization procedure coupled with dummy filling. To assess the process dependability and controllability, a set of process control monitors (PCMs) for monitoring fabrication and design parameters was designed and monitored. The successful manufacturing and testing of a few small-scale circuits, like our standard library cells, further attests to the viability of our fabrication process for superconducting integrated circuits.

physics.app-ph↗

arXiv2023

Holographic complexity of hyperbolic black holes

We calculate the holographic complexity of a family of hyperbolic black holes in an Einstein-Maxwell-dilaton (EMD) system by applying the complexity=action (CA) conjecture. While people previously studied spherical black holes in the same system, we show that hyperbolic black holes have intriguing features. We confirm that the complexity expression mainly depends on the spacetime causal structure despite the rich thermodynamics. The nontrivial neutral limit that exists only for hyperbolic black holes enables us to analytically obtain the complexity growth rate during phase transitions. We find that the dilaton accelerates the growth rate of complexity, and the Lloyd bound can be violated. This is in contrast to the spherical case where the dilation slows the complexity growth rate down, and the Lloyd bound is always satisfied. As a special case, we study the holographic complexity growth rate of the neutral hairy black holes and find that the Lloyd bound is always violated.

hep-th↗

arXiv2023

Quantum many-body scars in spin-1 Kitaev chain with uniaxial single-ion anisotropy

To establish a solid-state-based framework for the coexistence of quantum many-body scars and quantum criticality, we investigate the spin-1 Kitaev chain with uniaxial single-ion anisotropy (SIA). In the subspace with uniform $\mathbb{Z}_2$ gauge fields, this model can be exactly mapped to the spin-1/2 effective detuned PXP Hamiltonian, where the SIA plays a role of the static detuning term. The quench dynamics starting from the product states is symmetric between positive and negative values of the SIA, while a quantum phase transition from the Kitaev spin liquid to the dimer phase only occurs at the critical point with a negative $D_c$, implying the spontaneous breaking of the translational symmetry. We find that the coherent oscillations of quantum fidelity and certain local observables are sustained against small SIA perturbations in a quantum quench from special initial states. While the oscillation amplitudes of these observables decay with time as the SIA strength is increased, the system completely thermalizes upon approaching the critical point. In contrast, the initial polarized state, which shows an absence of revivals of quantum fidelity, will exhibit long revivals for $D<D_c$. Finally, we investigate the evolution of phase boundaries of the Kitaev spin liquid and dimer phase by introducing Heisenberg interactions, which spoil the $\mathbb{Z}_2$ gauge fields. A complete phase diagram is given by the infinite time-evolving block decimation method and the ground state properties of each phase are accurately captured by various spin correlations. Our work opens the door to understanding exotic connections between many-body scars and quantum criticality in systems with higher spins.

cond-mat.str-el↗

arXiv2023

Holographic Renyi entropies from hyperbolic black holes with scalar hair

The Renyi entropies as a generalization of the entanglement entropy imply much more information. We analytically calculate the Renyi entropies (with a spherical entangling surface) by means of a class of neutral hyperbolic black holes with scalar hair as a one-parameter generalization of the MTZ black hole. The zeroth-order and third-order phase transitions of black holes lead to discontinuity of the Renyi entropies and their second derivatives, respectively. From the Renyi entropies that are analytic at $n=\infty$, we can express the entanglement spectrum as an infinite sum in terms of the Bell polynomials. We show that the analytic treatment is in agreement with numerical calculations for the low-lying entanglement spectrum in a wide range of parameters.

hep-th↗