arXiv subjects

Jie Liu

Publications and source records attributed to Jie Liu.

At least 307 records · Page 17Linked to original sources

arXiv2025

Entanglement transfer between giant atoms in waveguide-QED systems

We investigate the entanglement transfer between giant atoms in waveguide-QED systems. The system consists of two pairs of two-level giant atoms, $ab$ and $cd$, each independently coupled to its respective one-dimensional waveguide. Initially, entangled states are stored in atom pair $ac$. There we consider three giant atom coupling configurations: separated, braided, and nested. For comparison, the entanglement transfer for small atoms configuration is also studied here. We focus on the entanglement transfer from atom pair $ac$ to atoms pair $bd$ and atom pair $ab$ in these four coupling configurations. It is shown that the transfer of entanglement in each coupling configuration strongly depends on phase shift. In particular, the braided configuration demonstrates superior performance in entanglement transfer. For the entanglement transfer from atom pair $ac$ to atom pair $bd$, complete entanglement transfer is presented in braided configuration, a behavior not found in small atom or other giant atom configurations. For the entanglement transfer from atom pair $ac$ to atom pair $ab$, although the maximum entanglement transferred to atom pair $ab$ in the braided configuration is half of that one to atom pair $bd$, it is still higher than that in the small atom, separated, and nested configurations. This study lays the foundation for entanglement transfer between giant atoms in waveguide-QED platforms.

quant-ph↗

arXiv2024

DAMPER: A Dual-Stage Medical Report Generation Framework with Coarse-Grained MeSH Alignment and Fine-Grained Hypergraph Matching

Medical report generation is crucial for clinical diagnosis and patient management, summarizing diagnoses and recommendations based on medical imaging. However, existing work often overlook the clinical pipeline involved in report writing, where physicians typically conduct an initial quick review followed by a detailed examination. Moreover, current alignment methods may lead to misaligned relationships. To address these issues, we propose DAMPER, a dual-stage framework for medical report generation that mimics the clinical pipeline of report writing in two stages. In the first stage, a MeSH-Guided Coarse-Grained Alignment (MCG) stage that aligns chest X-ray (CXR) image features with medical subject headings (MeSH) features to generate a rough keyphrase representation of the overall impression. In the second stage, a Hypergraph-Enhanced Fine-Grained Alignment (HFG) stage that constructs hypergraphs for image patches and report annotations, modeling high-order relationships within each modality and performing hypergraph matching to capture semantic correlations between image regions and textual phrases. Finally,the coarse-grained visual features, generated MeSH representations, and visual hypergraph features are fed into a report decoder to produce the final medical report. Extensive experiments on public datasets demonstrate the effectiveness of DAMPER in generating comprehensive and accurate medical reports, outperforming state-of-the-art methods across various evaluation metrics.

cs.CV↗

arXiv2024

Iterated relation systems on Riemannian manifolds

For fractals on Riemannian manifolds, the theory of iterated function systems often does not apply well directly, as fractal sets are often defined by relations that are multivalued or non-contractive. To overcome this difficulty, we introduce the notion of iterated relation systems. We study the attractor of an iterated relation system and formulate a condition under which such an attractor can be identified with that of an associated graph-directed iterated function system. Using this method, we obtain dimension formulas for the attractor of an iterated relation system under the graph open set condition or the graph finite type condition. This method improves the one in [Ngai-Xu, J. Geom. Anal. {\bf 33} (2023), 262], which relies on knowing the specific structure of the attractor.

math.DS↗

arXiv2024

Advocating for the Silent: Enhancing Federated Generalization for Non-Participating Clients

Federated Learning (FL) has surged in prominence due to its capability of collaborative model training without direct data sharing. However, the vast disparity in local data distributions among clients, often termed the Non-Independent Identically Distributed (Non-IID) challenge, poses a significant hurdle to FL's generalization efficacy. The scenario becomes even more complex when not all clients participate in the training process, a common occurrence due to unstable network connections or limited computational capacities. This can greatly complicate the assessment of the trained models' generalization abilities. While a plethora of recent studies has centered on the generalization gap pertaining to unseen data from participating clients with diverse distributions, the distinction between the training distributions of participating clients and the testing distributions of non-participating ones has been largely overlooked. In response, our paper unveils an information-theoretic generalization framework for FL. Specifically, it quantifies generalization errors by evaluating the information entropy of local distributions and discerning discrepancies across these distributions. Inspired by our deduced generalization bounds, we introduce a weighted aggregation approach and a duo of client selection strategies. These innovations are designed to strengthen FL's ability to generalize and thus ensure that trained models perform better on non-participating clients by incorporating a more diverse range of client data distributions. Our extensive empirical evaluations reaffirm the potency of our proposed methods, aligning seamlessly with our theoretical construct.

cs.LG↗

arXiv2024

ContextModule: Improving Code Completion via Repository-level Contextual Information

Large Language Models (LLMs) have demonstrated impressive capabilities in code completion tasks, where they assist developers by predicting and generating new code in real-time. However, existing LLM-based code completion systems primarily rely on the immediate context of the file being edited, often missing valuable repository-level information, user behaviour and edit history that could improve suggestion accuracy. Additionally, challenges such as efficiently retrieving relevant code snippets from large repositories, incorporating user behavior, and balancing accuracy with low-latency requirements in production environments remain unresolved. In this paper, we propose ContextModule, a framework designed to enhance LLM-based code completion by retrieving and integrating three types of contextual information from the repository: user behavior-based code, similar code snippets, and critical symbol definitions. By capturing user interactions across files and leveraging repository-wide static analysis, ContextModule improves the relevance and precision of generated code. We implement performance optimizations, such as index caching, to ensure the system meets the latency constraints of real-world coding environments. Experimental results and industrial practise demonstrate that ContextModule significantly improves code completion accuracy and user acceptance rates.

cs.SE↗

arXiv2024

An Entailment Tree Generation Approach for Multimodal Multi-Hop Question Answering with Mixture-of-Experts and Iterative Feedback Mechanism

With the rise of large-scale language models (LLMs), it is currently popular and effective to convert multimodal information into text descriptions for multimodal multi-hop question answering. However, we argue that the current methods of multi-modal multi-hop question answering still mainly face two challenges: 1) The retrieved evidence containing a large amount of redundant information, inevitably leads to a significant drop in performance due to irrelevant information misleading the prediction. 2) The reasoning process without interpretable reasoning steps makes the model difficult to discover the logical errors for handling complex questions. To solve these problems, we propose a unified LLMs-based approach but without heavily relying on them due to the LLM's potential errors, and innovatively treat multimodal multi-hop question answering as a joint entailment tree generation and question answering problem. Specifically, we design a multi-task learning framework with a focus on facilitating common knowledge sharing across interpretability and prediction tasks while preventing task-specific errors from interfering with each other via mixture of experts. Afterward, we design an iterative feedback mechanism to further enhance both tasks by feeding back the results of the joint training to the LLM for regenerating entailment trees, aiming to iteratively refine the potential answer. Notably, our method has won the first place in the official leaderboard of WebQA (since April 10, 2024), and achieves competitive results on MultimodalQA.

cs.CL↗

arXiv2024

A Spectrum Evaluation Benchmark for Medical Multi-Modal Large Language Models

The significant breakthroughs of Medical Multi-Modal Large Language Models (Med-MLLMs) renovate modern healthcare with robust information synthesis and medical decision support. However, these models are often evaluated on benchmarks that are unsuitable for the Med-MLLMs due to the complexity of real-world diagnostics across diverse specialties. To address this gap, we introduce Asclepius, a novel Med-MLLM benchmark that comprehensively assesses Med-MLLMs in terms of: distinct medical specialties (cardiovascular, gastroenterology, etc.) and different diagnostic capacities (perception, disease analysis, etc.). Grounded in 3 proposed core principles, Asclepius ensures a comprehensive evaluation by encompassing 15 medical specialties, stratifying into 3 main categories and 8 sub-categories of clinical tasks, and exempting overlap with existing VQA dataset. We further provide an in-depth analysis of 6 Med-MLLMs and compare them with 3 human specialists, providing insights into their competencies and limitations in various medical contexts. Our work not only advances the understanding of Med-MLLMs' capabilities but also sets a precedent for future evaluations and the safe deployment of these models in clinical environments.

cs.CL↗

arXiv2024

STANet: A Novel Spatio-Temporal Aggregation Network for Depression Classification with Small and Unbalanced FMRI Data

Accurate diagnosis of depression is crucial for timely implementation of optimal treatments, preventing complications and reducing the risk of suicide. Traditional methods rely on self-report questionnaires and clinical assessment, lacking objective biomarkers. Combining fMRI with artificial intelligence can enhance depression diagnosis by integrating neuroimaging indicators. However, the specificity of fMRI acquisition for depression often results in unbalanced and small datasets, challenging the sensitivity and accuracy of classification models. In this study, we propose the Spatio-Temporal Aggregation Network (STANet) for diagnosing depression by integrating CNN and RNN to capture both temporal and spatial features of brain activity. STANet comprises the following steps:(1) Aggregate spatio-temporal information via ICA. (2) Utilize multi-scale deep convolution to capture detailed features. (3) Balance data using the SMOTE to generate new samples for minority classes. (4) Employ the AFGRU classifier, which combines Fourier transformation with GRU, to capture long-term dependencies, with an adaptive weight assignment mechanism to enhance model generalization. The experimental results demonstrate that STANet achieves superior depression diagnostic performance with 82.38% accuracy and a 90.72% AUC. The STFA module enhances classification by capturing deeper features at multiple scales. The AFGRU classifier, with adaptive weights and stacked GRU, attains higher accuracy and AUC. SMOTE outperforms other oversampling methods. Additionally, spatio-temporal aggregated features achieve better performance compared to using only temporal or spatial features. STANet outperforms traditional or deep learning classifiers, and functional connectivity-based classifiers, as demonstrated by ten-fold cross-validation.

eess.IV↗

arXiv2024

Kawamata--Miyaoka type inequality for $\mathbb{Q}$-Fano varieties with canonical singularities

Let $X$ be an $n$-dimensional normal $\mathbb{Q}$-factorial projective variety with canonical singularities and Picard number one such that $X$ is smooth in codimension two, $-K_X$ is ample and $n\geq 2$. We prove that $X$ satisfies the following Kawamata--Miyaoka type inequality: \[ c_1(X)^n< 4 c_2(X)\cdot c_1(X)^{n-2}. \] If additionally $X$ is a threefold with terminal singularities, then a stronger inequality is also obtained.

math.AG↗

arXiv2024

Large Language Models for Lossless Image Compression: Next-Pixel Prediction in Language Space is All You Need

We have recently witnessed that ``Intelligence" and `` Compression" are the two sides of the same coin, where the language large model (LLM) with unprecedented intelligence is a general-purpose lossless compressor for various data modalities. This attribute particularly appeals to the lossless image compression community, given the increasing need to compress high-resolution images in the current streaming media era. Consequently, a spontaneous envision emerges: Can the compression performance of the LLM elevate lossless image compression to new heights? However, our findings indicate that the naive application of LLM-based lossless image compressors suffers from a considerable performance gap compared with existing state-of-the-art (SOTA) codecs on common benchmark datasets. In light of this, we are dedicated to fulfilling the unprecedented intelligence (compression) capacity of the LLM for lossless image compression tasks, thereby bridging the gap between theoretical and practical compression performance. Specifically, we propose P$^{2}$-LLM, a next-pixel prediction-based LLM, which integrates various elaborated insights and methodologies, \textit{e.g.,} pixel-level priors, the in-context ability of LLM, and a pixel-level semantic preservation strategy, to enhance the understanding capacity of pixel sequences for better next-pixel predictions. Extensive experiments on benchmark datasets demonstrate that P$^{2}$-LLM can beat SOTA classical and learned codecs.

cs.CV↗

arXiv2024

Weak-to-Strong Search: Align Large Language Models via Searching over Small Language Models

Large language models are usually fine-tuned to align with human preferences. However, fine-tuning a large language model can be challenging. In this work, we introduce $\textit{weak-to-strong search}$, framing the alignment of a large language model as a test-time greedy search to maximize the log-probability difference between small tuned and untuned models while sampling from the frozen large model. This method serves both as (1) a compute-efficient model up-scaling strategy that avoids directly tuning the large model and as (2) an instance of weak-to-strong generalization that enhances a strong model with weak test-time guidance. Empirically, we demonstrate the flexibility of weak-to-strong search across different tasks. In controlled-sentiment generation and summarization, we use tuned and untuned $\texttt{gpt2}$s to improve the alignment of large models without additional training. Crucially, in a more difficult instruction-following benchmark, AlpacaEval 2.0, we show that reusing off-the-shelf small models (e.g., $\texttt{zephyr-7b-beta}$ and its untuned version) can improve the length-controlled win rates of both white-box and black-box large models against $\texttt{gpt-4-turbo}$ (e.g., $34.4\% \rightarrow 37.9\%$ for $\texttt{Llama-3-70B-Instruct}$ and $16.0\% \rightarrow 20.1\%$ for $\texttt{gpt-3.5-turbo-instruct}$), despite the small models' low win rates $\approx 10.0\%$.

cs.CL↗

arXiv2024

ezyMRI: How to build an MRI machine from scratch -- Experience from a four-day hackathon

Nuclear magnetic resonance instruments are becoming available to the do-it-yourself community. The challenges encountered in the endeavor to build a magnetic resonance imaging instrument from scratch were confronted in a four-day hackathon at Singapore University of Technology and Design in spring 2024. One day was devoted to educational lectures and three days to system construction and testing. Seventy young researchers from all parts of the world formed six teams focusing on magnet, gradient coil, RF coil, console, system integration, and design, which together produced a working MRI instrument in three days. The different steps, encountered challenges, and their solutions are reported.

physics.med-ph↗

arXiv2024

DeUEDroid: Detecting Underground Economy Apps Based on UTG Similarity

In recent years, the underground economy is proliferating in the mobile system. These underground economy apps (UEware) make profits from providing non-compliant services, especially in sensitive areas such as gambling, pornography, and loans. Unlike traditional malware, most of them (over 80%) do not have malicious payloads. Due to their unique characteristics, existing detection approaches cannot effectively and efficiently mitigate this emerging threat. To address this problem, we propose a novel approach to effectively and efficiently detect UEware by considering their UI transition graphs (UTGs). Based on the proposed approach, we design and implement a system named DeUEDroid to perform the detection. To evaluate DeUEDroid, we collect 25,717 apps and build the first large-scale ground-truth dataset (1,700 apps) of UEware. The evaluation result based on the ground-truth dataset shows that DeUEDroid can cover new UI features and statically construct precise UTG. It achieves 98.22% detection F1-score and 98.97% classification accuracy, significantly outperforming traditional approaches. The evaluation involving 24,017 apps demonstrates the effectiveness and efficiency of UEware detection in real-world scenarios. Furthermore, the result reveals that UEware are prevalent, with 54% of apps in the wild and 11% of apps in app stores being UEware. Our work sheds light on future work in analyzing and detecting UEware.

cs.CR↗

arXiv2024

Skin-topological effect in two-dimensional nonreciprocal topological superconductor

We investigate the interplay between non-Hermitian skin effect (NHSE) and topological properties in two-dimensional topological superconductor. Two kinds of non-Hermiticity are considered. The first is the spin-independent non-reciprocal hopping, which respects particle-hole symmetry (PHS) and dictates the bulk as well as topological edge modes localize at opposite corners, manifested as the $Z_2$ NHSE protected by PHS. The direction of the localization can be conveniently characterized by the sign of the winding numbers. The other is the spin-dependent non-reciprocal hopping, where the NHSE is protected by time-reversal symmetry (TRS). The Kramers doublets are localized at opposite corners, namely the TRS protected $Z_2$ NHSE. When apply an external magnetic field, its internal symmetry changes from the symplectic class into the orthogonal class, eliminating the NHSE for the states in the spectrum continuum. For the zero energy states which are isolated, the NHSE still has its effects despite the orthogonality. For the spin-independent case, edge states can be effectively tuned by the direction of Zeeman field, where at certain directions the zero-energy edge state can be free of NHSE and uniformly distributed. Our work paves the way for the study of the interplay between topology and non-Hermiticity in superconducting systems.

cond-mat.supr-con↗

arXiv2024

Edge supercurrent in Josephson junctions based on topological materials

The interplay between novel topological states and superconductivity has garnered substantial interest due to its potential for topological quantum computing. The Josephson effect serves as a useful probe for edge superconductivity in these hybrid topological materials. In Josephson junctions based on topological materials, supercurrents exhibit unique quantum interference patterns, including the conventional Fraunhofer oscillations, the $Φ_0$-periodic oscillation, and the $2Φ_0$-periodic oscillation in response to the external magnetic field ($Φ_0 = h/2e$ is the flux quantum, $h$ the Planck constant, and $e$ the electron charge). These interference patterns stem from varied Andreev reflection mechanisms and the associated current density profiles. This review seeks to comprehensively examine the theoretical and experimental advancements in understanding the quantum interference patterns of edge supercurrents in Josephson junctions based on quantum spin Hall, quantum Hall, and quantum anomalous Hall systems.

cond-mat.supr-con↗

arXiv2024

Anomalous Fraunhofer-like patterns in quantum anomalous Hall Josephson junction

The intriguing interplay between topology and superconductivity has attracted significant attention, given its potential for realizing topological superconductivity. In the quantum anomalous Hall insulators (QAHIs)-based junction, the supercurrents are carried by the chiral edge states, characterized by a $2Φ_0$ magnetic flux periodicity ($Φ_0 = h/2e$ is the flux quantum, $h$ the Planck constant, and $e$ the electron charge). However, experimental observations indicate the presence of bulk carriers in QAHI samples due to magnetic dopants. In this study, we reveal a systematic transition from edge-state to bulk-state dominant supercurrents as the chemical potential varies from the bulk gap to the conduction band. This results in an evolution from a $2Φ_0$-periodic oscillation pattern to an asymmetric Fraunhofer pattern. Furthermore, a novel Fraunhoher-like pattern emerges due to the coexistence of chiral edge states and bulk states caused by magnetic {\color{black}domains}, even when the chemical potential resides within the gap. These findings not only advance the theoretical understanding but also pave the way for the experimental discovery of the chiral Josephson effect based on QAHI doped with magnetic impurities.

cond-mat.mes-hall↗

arXiv2024

Best Practices for Distilling Large Language Models into BERT for Web Search Ranking

Recent studies have highlighted the significant potential of Large Language Models (LLMs) as zero-shot relevance rankers. These methods predominantly utilize prompt learning to assess the relevance between queries and documents by generating a ranked list of potential documents. Despite their promise, the substantial costs associated with LLMs pose a significant challenge for their direct implementation in commercial search systems. To overcome this barrier and fully exploit the capabilities of LLMs for text ranking, we explore techniques to transfer the ranking expertise of LLMs to a more compact model similar to BERT, using a ranking loss to enable the deployment of less resource-intensive models. Specifically, we enhance the training of LLMs through Continued Pre-Training, taking the query as input and the clicked title and summary as output. We then proceed with supervised fine-tuning of the LLM using a rank loss, assigning the final token as a representative of the entire sentence. Given the inherent characteristics of autoregressive language models, only the final token can encapsulate all preceding tokens. Additionally, we introduce a hybrid point-wise and margin MSE loss to transfer the ranking knowledge from LLMs to smaller models like BERT. This method creates a viable solution for environments with strict resource constraints. Both offline and online evaluations have confirmed the efficacy of our approach, and our model has been successfully integrated into a commercial web search engine as of February 2024.

cs.IR↗

arXiv2024

Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencent

In this paper, we introduce Hunyuan-Large, which is currently the largest open-source Transformer-based mixture of experts model, with a total of 389 billion parameters and 52 billion activation parameters, capable of handling up to 256K tokens. We conduct a thorough evaluation of Hunyuan-Large's superior performance across various benchmarks including language understanding and generation, logical reasoning, mathematical problem-solving, coding, long-context, and aggregated tasks, where it outperforms LLama3.1-70B and exhibits comparable performance when compared to the significantly larger LLama3.1-405B model. Key practice of Hunyuan-Large include large-scale synthetic data that is orders larger than in previous literature, a mixed expert routing strategy, a key-value cache compression technique, and an expert-specific learning rate strategy. Additionally, we also investigate the scaling laws and learning rate schedule of mixture of experts models, providing valuable insights and guidances for future model development and optimization. The code and checkpoints of Hunyuan-Large are released to facilitate future innovations and applications. Codes: https://github.com/Tencent/Hunyuan-Large Models: https://huggingface.co/tencent/Tencent-Hunyuan-Large

cs.CL↗