arXiv subjects

Dan Zhao

Publications and source records attributed to Dan Zhao.

At least 73 records · Page 4Linked to original sources

arXiv2023

Magnetic-field-induced electronic instability of Weyl-like fermions in compressed black phosphorus

Revealing the role of Coulomb interaction in topological semimetals with Dirac/Weyl-like band dispersion shapes a new frontier in condensed matter physics. Topological node-line semimetals (TNLSMs), anticipated as a fertile ground for exploring electronic correlation effects due to the anisotropy associated with their node-line structure, have recently attracted considerable attention. In this study, we report an experimental observation for correlation effects in TNLSMs realized by black phosphorus (BP) under hydrostatic pressure. By performing a combination of nuclear magnetic resonance measurements and band calculations on compressed BP, a magnetic-field-induced electronic instability of Weyl-like fermions is identified under an external magnetic field parallel to the so-called nodal ring in the reciprocal space. Anomalous spin fluctuations serving as the fingerprint of electronic instability are observed at low temperatures, and they are observed to maximize at approximately 1.0 GPa. This study presents compressed BP as a realistic material platform for exploring the rich physics in strongly coupled Weyl-like fermions.

cond-mat.str-el↗

arXiv2023

From Words to Watts: Benchmarking the Energy Costs of Large Language Model Inference

Large language models (LLMs) have exploded in popularity due to their new generative capabilities that go far beyond prior state-of-the-art. These technologies are increasingly being leveraged in various domains such as law, finance, and medicine. However, these models carry significant computational challenges, especially the compute and energy costs required for inference. Inference energy costs already receive less attention than the energy costs of training LLMs -- despite how often these large models are called on to conduct inference in reality (e.g., ChatGPT). As these state-of-the-art LLMs see increasing usage and deployment in various domains, a better understanding of their resource utilization is crucial for cost-savings, scaling performance, efficient hardware usage, and optimal inference strategies. In this paper, we describe experiments conducted to study the computational and energy utilization of inference with LLMs. We benchmark and conduct a preliminary analysis of the inference performance and inference energy costs of different sizes of LLaMA -- a recent state-of-the-art LLM -- developed by Meta AI on two generations of popular GPUs (NVIDIA V100 \& A100) and two datasets (Alpaca and GSM8K) to reflect the diverse set of tasks/benchmarks for LLMs in research and practice. We present the results of multi-node, multi-GPU inference using model sharding across up to 32 GPUs. To our knowledge, our work is the one of the first to study LLM inference performance from the perspective of computational and energy resources at this scale.

cs.CL↗

arXiv2023

Combining Explicit and Implicit Regularization for Efficient Learning in Deep Networks

Works on implicit regularization have studied gradient trajectories during the optimization process to explain why deep networks favor certain kinds of solutions over others. In deep linear networks, it has been shown that gradient descent implicitly regularizes toward low-rank solutions on matrix completion/factorization tasks. Adding depth not only improves performance on these tasks but also acts as an accelerative pre-conditioning that further enhances this bias towards low-rankedness. Inspired by this, we propose an explicit penalty to mirror this implicit bias which only takes effect with certain adaptive gradient optimizers (e.g. Adam). This combination can enable a degenerate single-layer network to achieve low-rank approximations with generalization error comparable to deep linear networks, making depth no longer necessary for learning. The single-layer network also performs competitively or out-performs various approaches for matrix completion over a range of parameter and data regimes despite its simplicity. Together with an optimizer's inductive bias, our findings suggest that explicit regularization can play a role in designing different, desirable forms of regularization and that a more nuanced understanding of this interplay may be necessary.

cs.LG↗

arXiv2023

A Heterogeneous Parallel Non-von Neumann Architecture System for Accurate and Efficient Machine Learning Molecular Dynamics

This paper proposes a special-purpose system to achieve high-accuracy and high-efficiency machine learning (ML) molecular dynamics (MD) calculations. The system consists of field programmable gate array (FPGA) and application specific integrated circuit (ASIC) working in heterogeneous parallelization. To be specific, a multiplication-less neural network (NN) is deployed on the non-von Neumann (NvN)-based ASIC (SilTerra 180 nm process) to evaluate atomic forces, which is the most computationally expensive part of MD. All other calculations of MD are done using FPGA (Xilinx XC7Z100). It is shown that, to achieve similar-level accuracy, the proposed NvN-based system based on low-end fabrication technologies (180 nm) is 1.6x faster and 10^2-10^3x more energy efficiency than state-of-the-art vN based MLMD using graphics processing units (GPUs) based on much more advanced technologies (12 nm), indicating superiority of the proposed NvN-based heterogeneous parallel architecture.

cs.LG↗

arXiv2023

Energy-Efficient Blockchain-enabled User-Centric Mobile Edge Computing

In the traditional mobile edge computing (MEC) system, the availability of MEC services is greatly limited for the edge users of the cell due to serious signal attenuation and inter-cell interference. User-centric MEC (UC-MEC) can be seen as a promising solution to address this issue. In UC-MEC, each user is served by a dedicated access point (AP) cluster enabled with MEC capability instead of a single MEC server, however, at the expense of more energy consumption and greater privacy risks. To achieve efficient and reliable resource utilization with user-centric services, we propose an energy efficient blockchain-enabled UC-MEC system where blockchain operations and resource optimization are jointly performed. Firstly, we design a resource-aware, reliable, replicated, redundant, and fault-tolerant (R-RAFT) consensus mechanism to implement secure and reliable resource trading. Then, an optimization framework based on alternating direction method of multipliers (ADMM) is proposed to minimize the total energy consumed by wireless transmission, consensus and task computing, where APs clustering, computing resource allocation and bandwidth allocation are jointly considered. Simulation results show superiority of the proposed UC-MEC system over reference schemes, at most 33.96% reduction in the total delay and 48.77% reduction in the total energy consumption.

eess.SP↗

arXiv2023

A Green(er) World for A.I

As research and practice in artificial intelligence (A.I.) grow in leaps and bounds, the resources necessary to sustain and support their operations also grow at an increasing pace. While innovations and applications from A.I. have brought significant advances, from applications to vision and natural language to improvements to fields like medical imaging and materials engineering, their costs should not be neglected. As we embrace a world with ever-increasing amounts of data as well as research and development of A.I. applications, we are sure to face an ever-mounting energy footprint to sustain these computational budgets, data storage needs, and more. But, is this sustainable and, more importantly, what kind of setting is best positioned to nurture such sustainable A.I. in both research and practice? In this paper, we outline our outlook for Green A.I. -- a more sustainable, energy-efficient and energy-aware ecosystem for developing A.I. across the research, computing, and practitioner communities alike -- and the steps required to arrive there. We present a bird's eye view of various areas for potential changes and improvements from the ground floor of AI's operational and hardware optimizations for datacenters/HPCs to the current incentive structures in the world of A.I. research and practice, and more. We hope these points will spur further discussion, and action, on some of these issues and their potential solutions.

cs.AI↗

arXiv2022

Probing complex stacking in a layered material via electron-nuclear quadrupolar coupling

For layered materials, the interlayer stacking is a critical degree of freedom tuning electronic properties, while its microscopic characterization faces great challenges. The transition-metal dichalcogenide 1T-TaS$_2$ represents a novel example, in which the stacking pattern is not only enriched by the spontaneous occurrence of the intralayer charge density wave, but also recognized as a key to understand the nature of the low-temperature insulating phase. We exploit the $^{33}\rm{S}$ nuclei in a 1T-TaS$_2$ single crystal as sensitive probes of the local stacking pattern via quadrupolar coupling to the electron density distribution nearby, by combining nuclear magnetic resonance (NMR) measurements with the state-of-the-art first-principles electric-field gradient calculations. The applicability of our proposal is analyzed through temperature, magnetic-field, and angle dependent NMR spectra. Systematic simulations of a single 1T-TaS$_2$ layer, bilayers with different stacking patterns, and typical stacking orders in three-dimensional (3D) structures unravel distinct NMR characteristics. Particularly, one 3D structure achieves a quantitative agreement with the experimental spectrum, which clearly rationalizes the coexistence of two types of interfacial environments. Our method may find general applications in the studies of layered materials.

cond-mat.mtrl-sci↗

arXiv2022

AI Enabled Maneuver Identification via the Maneuver Identification Challenge

Artificial intelligence (AI) has enormous potential to improve Air Force pilot training by providing actionable feedback to pilot trainees on the quality of their maneuvers and enabling instructor-less flying familiarization for early-stage trainees in low-cost simulators. Historically, AI challenges consisting of data, problem descriptions, and example code have been critical to fueling AI breakthroughs. The Department of the Air Force-Massachusetts Institute of Technology AI Accelerator (DAF-MIT AI Accelerator) developed such an AI challenge using real-world Air Force flight simulator data. The Maneuver ID challenge assembled thousands of virtual reality simulator flight recordings collected by actual Air Force student pilots at Pilot Training Next (PTN). This dataset has been publicly released at Maneuver-ID.mit.edu and represents the first of its kind public release of USAF flight training data. Using this dataset, we have applied a variety of AI methods to separate "good" vs "bad" simulator data and categorize and characterize maneuvers. These data, algorithms, and software are being released as baselines of model performance for others to build upon to enable the AI ecosystem for flight simulator training.

cs.AI↗

arXiv2022

Emergent charge order and unconventional superconductivity in pressurized kagome superconductor CsV3Sb5

The discovery of multiple electronic orders in kagome superconductors AV3Sb5 (A = K, Rb, Cs) provides a promising platform for exploring unprecedented emergent physics. Under moderate pressure (< 2.2 GPa), the triple-Q charge density wave (CDW) order is monotonically suppressed by pressure, while the superconductivity displays a two-dome-like behavior, suggesting an unusual interplay between superconductivity and CDW order. Given that time-reversal symmetry breaking and electronic nematicity have been revealed inside the triple-Q CDW phase, understanding this CDW order and its interplay with superconductivity becomes one of the core questions in AV3Sb5. Here, we report the evolution of CDW and superconductivity with pressure in CsV3Sb5 by 51V nuclear magnetic resonance measurements. An emergent CDW phase, ascribed to a possible stripe-like CDW order with a unidirectional 4a0 modulation, is observed between Pc1 ~ 0.58 GPa and Pc2 ~ 2.0 GPa, which explains the two-dome-like superconducting behavior under pressure. Furthermore, the nuclear spin-lattice relaxation measurement reveals evidence for pressure-independent charge fluctuations above the CDW transition temperature and unconventional superconducting pairing above Pc2. Our results not only shed new light on the interplay of superconductivity and CDW but also reveal novel electronic correlation effects in kagome superconductors AV3Sb5.

cond-mat.supr-con↗

arXiv2022

KPGT: Knowledge-Guided Pre-training of Graph Transformer for Molecular Property Prediction

Designing accurate deep learning models for molecular property prediction plays an increasingly essential role in drug and material discovery. Recently, due to the scarcity of labeled molecules, self-supervised learning methods for learning generalizable and transferable representations of molecular graphs have attracted lots of attention. In this paper, we argue that there exist two major issues hindering current self-supervised learning methods from obtaining desired performance on molecular property prediction, that is, the ill-defined pre-training tasks and the limited model capacity. To this end, we introduce Knowledge-guided Pre-training of Graph Transformer (KPGT), a novel self-supervised learning framework for molecular graph representation learning, to alleviate the aforementioned issues and improve the performance on the downstream molecular property prediction tasks. More specifically, we first introduce a high-capacity model, named Line Graph Transformer (LiGhT), which emphasizes the importance of chemical bonds and is mainly designed to model the structural information of molecular graphs. Then, a knowledge-guided pre-training strategy is proposed to exploit the additional knowledge of molecules to guide the model to capture the abundant structural and semantic information from large-scale unlabeled molecular graphs. Extensive computational tests demonstrated that KPGT can offer superior performance over current state-of-the-art methods on several molecular property prediction tasks.

q-bio.BM↗

arXiv2022

Bridging Walking and Slithering -- Stokesian Locomotion

Both legged locomotion and slithering motions typically utilize periodic gaits -- repeating cycles of body shape change that produce a net motion through the world. Legged locomotion can be viewed from the perspective of piece-wise contact constraint formation and removal. Slithering and low Reynolds number swimming operate under continuous constraints of force balance, wherein dissipation removes the ability to accumulate momentum. Here we discuss how to bridge the gap between these domains of motion, thereby, among other benefits, producing models for the space of legged locomotion with slipping. The connective fabric is the use of a "Stokesian", or "local connection" model.

cs.RO↗

arXiv2022

FLDP: Flexible strategy for local differential privacy

Local differential privacy (LDP), a technique applying unbiased statistical estimations instead of real data, is often adopted in data collection. In particular, this technique is used with frequency oracles (FO) because it can protect each user's privacy and prevent leakage of sensitive information. However, the definition of LDP is so conservative that it requires all inputs to be indistinguishable after perturbation. Indeed, LDP protects each value; however, it is rarely used in practical scenarios owing to its cost in terms of accuracy. In this paper, we address the challenge of providing weakened but flexible protection where each value only needs to be indistinguishable from part of the domain after perturbation. First, we present this weakened but flexible LDP (FLDP) notion. We then prove the association with LDP and DP. Second, we design an FHR approach for the common FO issue while satisfying FLDP. The proposed approach balances communication cost, computational complexity, and estimation accuracy. Finally, experimental results using practical and synthetic datasets verify the effectiveness and efficiency of our approach.

cs.CR↗

arXiv2022

Benchmarking Resource Usage for Efficient Distributed Deep Learning

Deep learning (DL) workflows demand an ever-increasing budget of compute and energy in order to achieve outsized gains. Neural architecture searches, hyperparameter sweeps, and rapid prototyping consume immense resources that can prevent resource-constrained researchers from experimenting with large models and carry considerable environmental impact. As such, it becomes essential to understand how different deep neural networks (DNNs) and training leverage increasing compute and energy resources -- especially specialized computationally-intensive models across different domains and applications. In this paper, we conduct over 3,400 experiments training an array of deep networks representing various domains/tasks -- natural language processing, computer vision, and chemistry -- on up to 424 graphics processing units (GPUs). During training, our experiments systematically vary compute resource characteristics and energy-saving mechanisms such as power utilization and GPU clock rate limits to capture and illustrate the different trade-offs and scaling behaviors each representative model exhibits under various resource and energy-constrained regimes. We fit power law models that describe how training time scales with available compute resources and energy constraints. We anticipate that these findings will help inform and guide high-performance computing providers in optimizing resource utilization, by selectively reducing energy consumption for different deep learning tasks/workflows with minimal impact on training.

cs.LG↗

arXiv2021

Superconducting fluctuations in overdoped Bi$_2$Sr$_2$CaCu$_2$O$_{8+δ}$

Fluctuating superconductivity - vestigial Cooper pairing in the resistive state of a material - is usually associated with low dimensionality, strong disorder or low carrier density. Here, we report single particle spectroscopic, thermodynamic and magnetic evidence for persistent superconducting fluctuations in heavily hole-doped cuprate superconductor Bi$_2$Sr$_2$CaCu$_2$O$_{8+δ}$ ($T_c$ = 66~K) despite the high carrier density. With a sign-problem free quantum Monte Carlo calculation, we show how a partially flat band at ($π$,0) can help enhance superconducting phase fluctuations. Finally, we discuss the implications of an anisotropic band structure on the phase-coherence-limited superconductivity in overdoped cuprates and other superconductors.

cond-mat.supr-con↗

arXiv2021

Ionic thermoelectric materials and devices

The tremendous amount of wasted heat from solar radiation and industry dissipation has motivated the development of thermoelectric concepts that directly convert heat into electricity. The main challenge in practical applications for thermoelectrics is the high cost from both materials and manufacturing. Recently, breakthrough progresses in ionic thermoelectrics open up new possibilities to charge energy storage devices when submitted to a temperature gradient. The charging voltage is internally from the ionic Seebeck effect of the electrolyte between two electrodes. Hence electrolytes with high thermoelectric figure of merit are classified as ionic thermoelectric materials. Most ionic thermoelectric materials are composed of abundant elements, and they can generate hundreds of times larger thermal voltage than that of electronic materials. This emerging thermoelectric category brings new hope to fabricate low cost and large area heat-to-energy conversion devices, and triggers a renewed interest for ionic thermodiffusion. In this review, we summarize the state of the art in the new field of ionic thermoelectrics, from the driving force of the ionic thermodiffusion to material and application developments. We present a general map of ionic thermoelectric materials, discuss the unique characters of each type of the reported electrolytes, and propose potential optimization and future topics of ionic thermoelectrics.

physics.app-ph↗

arXiv2020

Joint Power Allocation and User Association Optimization for IRS-Assisted mmWave Systems

Intelligent reflect surface (IRS) is a potential technology to build programmable wireless environment in future communication systems. In this paper, we consider an IRS-assisted multi-base station (multi-BS) multi-user millimeter wave (mmWave) downlink communication system, exploiting IRS to extend mmWave signal coverage to blind spots. Considering the impact of IRS on user association in multi-BS mmWave systems, we formulate a sum rate maximization problem by jointly optimizing passive beamforming at IRS, power allocation and user association. This leads to an intractable nonconvex problem, for which to tackle we propose a computationally affordable iterative algorithm, capitalizing on alternating optimization, sequential fractional programming (SFP) and forward-reverse auction (FRA). In particular, passive beamforming at IRS is optimized by utilizing the SFP method, power allocation is solved through means of standard convex optimization method, and user association is handled by the network optimization based FRA algorithm. Simulation results demonstrate that the proposed algorithm can achieve significant performance gains, e.g., it can provide up to 175% higher sum rate compared with the benchmark and 140% higher energy efficiency compared with amplify-and-forward relay.

cs.IT↗

arXiv2020

Deep Learning in Computer-Aided Diagnosis and Treatment of Tumors: A Survey

Computer-Aided Diagnosis and Treatment of Tumors is a hot topic of deep learning in recent years, which constitutes a series of medical tasks, such as detection of tumor markers, the outline of tumor leisures, subtypes and stages of tumors, prediction of therapeutic effect, and drug development. Meanwhile, there are some deep learning models with precise positioning and excellent performance produced in mainstream task scenarios. Thus follow to introduce deep learning methods from task-orient, mainly focus on the improvements for medical tasks. Then to summarize the recent progress in four stages of tumor diagnosis and treatment, which named In-Vitro Diagnosis (IVD), Imaging Diagnosis (ID), Pathological Diagnosis (PD), and Treatment Planning (TP). According to the specific data types and medical tasks of each stage, we present the applications of deep learning in the Computer-Aided Diagnosis and Treatment of Tumors and analyzing the excellent works therein. This survey concludes by discussing research issues and suggesting challenges for future improvement.

eess.IV↗

arXiv2020

Joint Passive Beamforming and User Association Optimization for IRS-assisted mmWave Systems

In this paper, we investigate an intelligent reflect surface (IRS) assisted multi-user millimeter wave (mmWave) downlink communication system, exploiting IRS to alleviate the blockage effect and enhance the performance of the mmWave system. Considering the impact of IRS on user association, we formulate a sum rate maximization problem by jointly optimizing the passive beamforming at IRS and user association, which is an intractable non-convex problem. Then an alternating optimization algorithm is proposed to solve the problem efficiently. In the proposed algorithm, passive beamforming at IRS is optimized by utilizing the fractional programming method and user association is solved through the network optimization based auction algorithm. We provide numerical comparisons between the proposed algorithm and different reference algorithms. Simulation results demonstrate that the proposed algorithm can achieve significant gains in the sum rate of all users.

eess.SP↗