arXiv subjects

Peng Zhang

Publications and source records attributed to Peng Zhang.

At least 361 records · Page 20Linked to original sources

arXiv2024

A Comprehensive Summarization and Evaluation of Feature Refinement Modules for CTR Prediction

Click-through rate (CTR) prediction is widely used in academia and industry. Most CTR tasks fall into a feature embedding \& feature interaction paradigm, where the accuracy of CTR prediction is mainly improved by designing practical feature interaction structures. However, recent studies have argued that the fixed feature embedding learned only through the embedding layer limits the performance of existing CTR models. Some works apply extra modules on top of the embedding layer to dynamically refine feature representations in different instances, making it effective and easy to integrate with existing CTR methods. Despite the promising results, there is a lack of a systematic review and summarization of this new promising direction on the CTR task. To fill this gap, we comprehensively summarize and define a new module, namely \textbf{feature refinement} (FR) module, that can be applied between feature embedding and interaction layers. We extract 14 FR modules from previous works, including instances where the FR module was proposed but not clearly defined or explained. We fully assess the effectiveness and compatibility of existing FR modules through comprehensive and extensive experiments with over 200 augmented models and over 4,000 runs for more than 15,000 GPU hours. The results offer insightful guidelines for researchers, and all benchmarking code and experimental results are open-sourced. In addition, we present a new architecture of assigning independent FR modules to separate sub-networks for parallel CTR models, as opposed to the conventional method of inserting a shared FR module on top of the embedding layer. Our approach is also supported by comprehensive experiments demonstrating its effectiveness.

cs.IR↗

arXiv2024

AOTree: Aspect Order Tree-based Model for Explainable Recommendation

Recent recommender systems aim to provide not only accurate recommendations but also explanations that help users understand them better. However, most existing explainable recommendations only consider the importance of content in reviews, such as words or aspects, and ignore the ordering relationship among them. This oversight neglects crucial ordering dimensions in the human decision-making process, leading to suboptimal performance. Therefore, in this paper, we propose Aspect Order Tree-based (AOTree) explainable recommendation method, inspired by the Order Effects Theory from cognitive and decision psychology, in order to capture the dependency relationships among decisive factors. We first validate the theory in the recommendation scenario by analyzing the reviews of the users. Then, according to the theory, the proposed AOTree expands the construction of the decision tree to capture aspect orders in users' decision-making processes, and use attention mechanisms to make predictions based on the aspect orders. Extensive experiments demonstrate our method's effectiveness on rating predictions, and our approach aligns more consistently with the user' s decision-making process by displaying explanations in a particular order, thereby enhancing interpretability.

cs.IR↗

arXiv2024

Electronic Correlations and Hund's Rule Coupling in Trilayer Nickelate La4Ni3O10

Trilayer Ruddlesden-Popper phase La4Ni3O10 has been observed with Tc of about 30 K at high pressure in recent experiment, which further expanded the family of nickelate superconductors. In this study, we explored the effects of electronic correlations in La4Ni3O10 using density function theory plus dynamical mean-field theory at ambient and high pressures. Our derived spectral functions and Fermi surface of ambient pressure phase are nicely consistent with the experimental results by angle-resolved photoemission spectroscopy, which emphasized the importance of electronic correlations in La4Ni3O10. We also found the electronic correlations in pressured La4Ni3O10 are both orbital-dependent and layer-dependent due to the presence of Hund's rule coupling. There is a competition between the Hund's rule coupling and the crystal-field splitting, and therefore the Ni-O layers with weaker crystal-field splitting energy would have stronger electronic correlations.

cond-mat.str-el↗

arXiv2024

Focus, Distinguish, and Prompt: Unleashing CLIP for Efficient and Flexible Scene Text Retrieval

Scene text retrieval aims to find all images containing the query text from an image gallery. Current efforts tend to adopt an Optical Character Recognition (OCR) pipeline, which requires complicated text detection and/or recognition processes, resulting in inefficient and inflexible retrieval. Different from them, in this work we propose to explore the intrinsic potential of Contrastive Language-Image Pre-training (CLIP) for OCR-free scene text retrieval. Through empirical analysis, we observe that the main challenges of CLIP as a text retriever are: 1) limited text perceptual scale, and 2) entangled visual-semantic concepts. To this end, a novel model termed FDP (Focus, Distinguish, and Prompt) is developed. FDP first focuses on scene text via shifting the attention to the text area and probing the hidden text knowledge, and then divides the query text into content word and function word for processing, in which a semantic-aware prompting scheme and a distracted queries assistance module are utilized. Extensive experiments show that FDP significantly enhances the inference speed while achieving better or competitive retrieval accuracy compared to existing methods. Notably, on the IIIT-STR benchmark, FDP surpasses the state-of-the-art model by 4.37% with a 4 times faster speed. Furthermore, additional experiments under phrase-level and attribute-aware scene text retrieval settings validate FDP's particular advantages in handling diverse forms of query text. The source code will be publicly available at https://github.com/Gyann-z/FDP.

cs.CV↗

arXiv2024

Fractional AC Josephson effect in a topological insulator proximitized by a self-formed superconductor

A lateral Josephson junction in which the surface of a 3D topological insulator serves as the weak link should support topologically protected excitations related to Majorana fermions. The resulting $4π$-periodic current-phase relationship could be detected under high-frequency excitation by the suppression of odd Shapiro steps. Here, we demonstrate such devices through the self-formation of a Pd-Te superconducting layer from a telluride topological insulator, and observe suppressed first and third Shapiro steps. Other devices, including those where the Pd-Te layer is bolstered by an additional Al layer, show no suppression of Shapiro steps, a difference supported by simulations. Though we rule out the known trivial causes of suppressed Shapiro steps in our devices, we nevertheless argue that corroborating measurements and disorder-aware theoretical descriptions of these systems are needed before confidently claiming the observation of Majorana states.

cond-mat.mes-hall↗

arXiv2024

QQQ: Quality Quattuor-Bit Quantization for Large Language Models

Quantization is a proven effective method for compressing large language models. Although popular techniques like W8A8 and W4A16 effectively maintain model performance, they often fail to concurrently speed up the prefill and decoding stages of inference. W4A8 is a promising strategy to accelerate both of them while usually leads to a significant performance degradation. To address these issues, we present QQQ, a Quality Quattuor-bit Quantization method with 4-bit weights and 8-bit activations. QQQ employs adaptive smoothing and Hessian-based compensation, significantly enhancing the performance of quantized models without extensive training. Furthermore, we meticulously engineer W4A8 GEMM kernels to increase inference speed. Our specialized per-channel W4A8 GEMM and per-group W4A8 GEMM achieve impressive speed increases of 3.67$\times$ and 3.29 $\times$ over FP16 GEMM. Our extensive experiments show that QQQ achieves performance on par with existing state-of-the-art LLM quantization methods while significantly accelerating inference, achieving speed boosts up to 2.24 $\times$, 2.10$\times$, and 1.25$\times$ compared to FP16, W8A8, and W4A16, respectively.

cs.LG↗

arXiv2024

ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools

We introduce ChatGLM, an evolving family of large language models that we have been developing over time. This report primarily focuses on the GLM-4 language series, which includes GLM-4, GLM-4-Air, and GLM-4-9B. They represent our most capable models that are trained with all the insights and lessons gained from the preceding three generations of ChatGLM. To date, the GLM-4 models are pre-trained on ten trillions of tokens mostly in Chinese and English, along with a small set of corpus from 24 languages, and aligned primarily for Chinese and English usage. The high-quality alignment is achieved via a multi-stage post-training process, which involves supervised fine-tuning and learning from human feedback. Evaluations show that GLM-4 1) closely rivals or outperforms GPT-4 in terms of general metrics such as MMLU, GSM8K, MATH, BBH, GPQA, and HumanEval, 2) gets close to GPT-4-Turbo in instruction following as measured by IFEval, 3) matches GPT-4 Turbo (128K) and Claude 3 for long context tasks, and 4) outperforms GPT-4 in Chinese alignments as measured by AlignBench. The GLM-4 All Tools model is further aligned to understand user intent and autonomously decide when and which tool(s) touse -- including web browser, Python interpreter, text-to-image model, and user-defined functions -- to effectively complete complex tasks. In practical applications, it matches and even surpasses GPT-4 All Tools in tasks like accessing online information via web browsing and solving math problems using Python interpreter. Over the course, we have open-sourced a series of models, including ChatGLM-6B (three generations), GLM-4-9B (128K, 1M), GLM-4V-9B, WebGLM, and CodeGeeX, attracting over 10 million downloads on Hugging face in the year 2023 alone. The open models can be accessed through https://github.com/THUDM and https://huggingface.co/THUDM.

cs.CL↗

arXiv2024

Computational Investigation on the formation of liquid-fueled oblique detonation waves

Utilizing a two-phase supersonic chemically reacting flow solver with the Eulerian-Lagrangian method implemented in OpenFOAM, this study computationally investigates the formation of liquid-fueled oblique detonation waves (ODWs) within a pre-injection oblique detonation wave engine operating at an altitude of 30 km and a velocity of Mach 9. The inflow undergoes two-stage compression, followed by uniform mixing with randomly distributed n-heptane droplets before entering the combustor. The study examines the effects of droplet breakup models, gas-liquid ratios, and on-wedge strips on the ODW formation. Results indicate that under the pure-droplet condition, the ODW fails to form within the combustor, irrespective of the breakup models used. However, increasing the proportion of n-heptane vapor in the fuel/air mixture facilitates the ODW formation, because the n-heptane vapor rapidly participates in the gaseous reactions, producing heat and accelerating the transition from low- to intermediate-temperature chemistry. Additionally, the presence of on-wedge strips enhances ODW formation by inducing a bow shock wave within the combustor, which significantly increases the temperature, directly triggering intermediate-temperature chemistry and subsequent heat-release reactions, thereby facilitating the formation of ODW.

physics.flu-dyn↗

arXiv2024

Matter Asymmetries in the $Z_N$ Dark matter -companion Models

A class of $Z_{N\geq 3}$-symmetric WIMP dark matter models that are characterized by the semi-annihilation into the companion of dark matter has been proposed in Ref.~\cite{Guo:2021rre}, providing a mechanism to evade the stringent direct detection constraint. In this work, we point out that such models naturally provide the three Sakharov elements necessary for dark matter asymmetry, and moreover this asymmetry can be transferred to the visible sector with a proper link to the leptonic or quark sector. In our minimal $Z_3$ example, the migration to the leptonic sector is via the asymmetric companion decay into neutrinos, and the lepton asymmetry can be further transferred to the quark sector. The CP violation parameter is restrained in this model. Thus, we explore the thermal motion effect of dark matter and find that it gives an enhancement to the CP violation parameter, which is studied for the first time. A preliminary numerical analysis based on the Boltzmann equations shows that both correct relic density of dark matter and baryon asymmetry can be accommodated.

hep-ph↗

arXiv2024

Eikonal Approximation for Floquet Scattering

The eikonal approximation (EA) is widely used in various high-energy scattering problems. In this work we generalize this approximation from the scattering problems with time-independent Hamiltonian to the ones with periodical Hamiltonians, {\it i.e.}, the Floquet scattering problems. We further illustrate the applicability of our generalized EA via the scattering problem with respect to a shaking spherical square-well potential, by comparing the results given by this approximation and the exact ones. The generalized EA we developed is helpful for the research of manipulation of high-energy scattering processes with external field, {\it e.g.}, the manipulation of atom, molecule or nuclear collisions or reactions via strong laser fields.

quant-ph↗

arXiv2024

Enhanced Safety in Autonomous Driving: Integrating Latent State Diffusion Model for End-to-End Navigation

With the advancement of autonomous driving, ensuring safety during motion planning and navigation is becoming more and more important. However, most end-to-end planning methods suffer from a lack of safety. This research addresses the safety issue in the control optimization problem of autonomous driving, formulated as Constrained Markov Decision Processes (CMDPs). We propose a novel, model-based approach for policy optimization, utilizing a conditional Value-at-Risk based Soft Actor Critic to manage constraints in complex, high-dimensional state spaces effectively. Our method introduces a worst-case actor to guide safe exploration, ensuring rigorous adherence to safety requirements even in unpredictable scenarios. The policy optimization employs the Augmented Lagrangian method and leverages latent diffusion models to predict and simulate future trajectories. This dual approach not only aids in navigating environments safely but also refines the policy's performance by integrating distribution modeling to account for environmental uncertainties. Empirical evaluations conducted in both simulated and real environment demonstrate that our approach outperforms existing methods in terms of safety, efficiency, and decision-making capabilities.

cs.AI↗

arXiv2024

The Peculiar Precursor of a Gamma-Ray Burst from a Binary Merger Involving a Magnetar

The milestone discovery of GW 170817-GRB 170817A-AT 2017gfo has shown that gravitational wave (GW) could be produced during the merger of neutron star-neutron star/black hole and that in electromagnetic (EM) wave a gamma-ray burst (GRB) and a kilonova (KN) are generated in sequence after the merger. Observationally, however, EM property before the merger phase is still unclear. Here we report a peculiar precursor in a KN-associated long-duration GRB 211211A, providing evidence of the EM before the merger. This precursor lasts $\sim$ 0.2 s, and the waiting time between the precursor and the main burst is $\sim$ 1 s, comparable to that between GW 170817 and GRB 170817A. The spectrum of the precursor could be well fit with a non-thermal cutoff power-law model instead of a blackbody. Especially, a $\sim$22 Hz Quasi-Periodic Oscillation candidate ($\sim 3σ$) is detected in the precursor. These temporal and spectral properties indicate that this precursor is probably produced by a catastrophic flare accompanying with magnetoelastic or crustal oscillations of a magnetar in binary compact merger. The strong magnetic field of the magnetar can also account for the prolonged duration of GRB 211211A. However, it poses a challenge to reconcile the rather short lifetime of a magnetar with the rather long spiraling time of a binary neutron star system only by the GW radiation before merger.

astro-ph.HE↗

arXiv2024

Hybrid Deep Learning Framework for Enhanced Melanoma Detection

Cancer is a leading cause of death worldwide, necessitating advancements in early detection and treatment technologies. In this paper, we present a novel and highly efficient melanoma detection framework that synergistically combines the strengths of U-Net for segmentation and EfficientNet for the classification of skin images. The primary objective of our study is to enhance the accuracy and efficiency of melanoma detection through an innovative hybrid approach. We utilized the HAM10000 dataset to meticulously train the U-Net model, enabling it to precisely segment cancerous regions. Concurrently, we employed the ISIC 2020 dataset to train the EfficientNet model, optimizing it for the binary classification of skin cancer. Our hybrid model demonstrates a significant improvement in performance, achieving a remarkable accuracy of 99.01% on the ISIC 2020 dataset. This exceptional result underscores the superiority of our approach compared to existing model structures. By integrating the precise segmentation capabilities of U-Net with the advanced classification prowess of EfficientNet, our framework offers a comprehensive solution for melanoma detection. The results of our extensive experiments highlight the high accuracy and reliability of our method in both segmentation and classification tasks. This indicates the potential of our hybrid approach to significantly enhance cancer detection, providing a robust tool for medical professionals in the early diagnosis and treatment of melanoma. We believe that our framework can set a new benchmark in the field of automated skin cancer detection, encouraging further research and development in this crucial area of medical imaging.

eess.IV↗

arXiv2024

Urban Waterlogging Detection: A Challenging Benchmark and Large-Small Model Co-Adapter

Urban waterlogging poses a major risk to public safety and infrastructure. Conventional methods using water-level sensors need high-maintenance to hardly achieve full coverage. Recent advances employ surveillance camera imagery and deep learning for detection, yet these struggle amidst scarce data and adverse environmental conditions. In this paper, we establish a challenging Urban Waterlogging Benchmark (UW-Bench) under diverse adverse conditions to advance real-world applications. We propose a Large-Small Model co-adapter paradigm (LSM-adapter), which harnesses the substantial generic segmentation potential of large model and the specific task-directed guidance of small model. Specifically, a Triple-S Prompt Adapter module alongside a Dynamic Prompt Combiner are proposed to generate then merge multiple prompts for mask decoder adaptation. Meanwhile, a Histogram Equalization Adap-ter module is designed to infuse the image specific information for image encoder adaptation. Results and analysis show the challenge and superiority of our developed benchmark and algorithm. Project page: \url{https://github.com/zhang-chenxu/LSM-Adapter}

cs.CV↗

arXiv2024

Physics-Informed AI Inverter

This letter devises an AI-Inverter that pilots the use of a physics-informed neural network (PINN) to enable AI-based electromagnetic transient simulations (EMT) of grid-forming inverters. The contributions are threefold: (1) A PINN-enabled AI-Inverter is formulated; (2) An enhanced learning strategy, balanced-adaptive PINN, is devised; (3) extensive validations and comparative analysis of the accuracy and efficiency of AI-Inverter are made to show its superiority over the classical electromagnetic transient programs (EMTP).

eess.SY↗

arXiv2024

Modulation of the Octahedral Structure and Potential Superconductivity of La$_3$Ni$_2$O$_7$ through Strain Engineering

The recent transport measurement of La$_3$Ni$_2$O$_7$ uncover a "right-triangle" shape of the superconducting dome in the pressure-temperature (P-T) phase diagram. Motivated by this, we perform theoretical first-principles studies of La$_3$Ni$_2$O$_7$ with the pressure ranging from 0 to 100 GPa. Notably, we reveal a pressure dependence of the Ni-$d_{z^2}$ electron density at the Fermi energy ($n_z^{EF}$) that highly coincides with such shape. On this basis, we further explore the electronic structure under uniaxial stress. By tracking the stress response of $n_z^{EF}$, we propose that superconductivity can be achieved by applying only about 2 GPa of compression along the c axis. The idea is further exemplified from the perspectives of lattice distortion, band structure, Fermi surface and superconducting phase coherence. We also discuss the possible charge modulation under the stress and provide an insight to the relation between n_z^EF and the superconducting Tc in La$_3$Ni$_2$O$_7$ system. Our study provides a helpful guide to the future experiment.

cond-mat.supr-con↗

arXiv2024

MaTe3D: Mask-guided Text-based 3D-aware Portrait Editing

3D-aware portrait editing has a wide range of applications in multiple fields. However, current approaches are limited due that they can only perform mask-guided or text-based editing. Even by fusing the two procedures into a model, the editing quality and stability cannot be ensured. To address this limitation, we propose \textbf{MaTe3D}: mask-guided text-based 3D-aware portrait editing. In this framework, first, we introduce a new SDF-based 3D generator which learns local and global representations with proposed SDF and density consistency losses. This enhances masked-based editing in local areas; second, we present a novel distillation strategy: Conditional Distillation on Geometry and Texture (CDGT). Compared to exiting distillation strategies, it mitigates visual ambiguity and avoids mismatch between texture and geometry, thereby producing stable texture and convincing geometry while editing. Additionally, we create the CatMask-HQ dataset, a large-scale high-resolution cat face annotation for exploration of model generalization and expansion. We perform expensive experiments on both the FFHQ and CatMask-HQ datasets to demonstrate the editing quality and stability of the proposed method. Our method faithfully generates a 3D-aware edited face image based on a modified mask and a text prompt. Our code and models will be publicly released.

cs.CV↗

arXiv2024

Inducing superconductivity in quantum anomalous Hall regime

Interfacing the quantum anomalous Hall insulator with a conventional superconductor is known to be a promising manner for realizing a topological superconductor, which has been continuously pursued for years. Such a proximity route depends to a great extent on the control of the delicate interfacial coupling of the two constituents. However, a recent experiment reported the failure to reproduce such a topological superconductor, which is ascribed to the negligence of the electrical short by the superconductor in the theoretical proposal. Here, we reproduce this topological superconductor with attention to the interface control. The resulted conductance matrix under a wide magnetic field range agrees with the fingerprint of this topological superconductor. This allows us to develop a phase diagram that unveils three regions parameterized by various coupling limits, which not only supports the feasibility to fabricate the topological superconductor by proximity but also fully explains the origin of the previous debate. The present work provides a comprehensible guide on fabricating the topological superconductor.

cond-mat.mes-hall↗