arXiv subjects

Zhenyu Zhang

Publications and source records attributed to Zhenyu Zhang.

At least 253 records · Page 14Linked to original sources

arXiv2022

Layout-Aware Information Extraction for Document-Grounded Dialogue: Dataset, Method and Demonstration

Building document-grounded dialogue systems have received growing interest as documents convey a wealth of human knowledge and commonly exist in enterprises. Wherein, how to comprehend and retrieve information from documents is a challenging research problem. Previous work ignores the visual property of documents and treats them as plain text, resulting in incomplete modality. In this paper, we propose a Layout-aware document-level Information Extraction dataset, LIE, to facilitate the study of extracting both structural and semantic knowledge from visually rich documents (VRDs), so as to generate accurate responses in dialogue systems. LIE contains 62k annotations of three extraction tasks from 4,061 pages in product and official documents, becoming the largest VRD-based information extraction dataset to the best of our knowledge. We also develop benchmark methods that extend the token-based language model to consider layout features like humans. Empirical results show that layout is critical for VRD-based extraction, and system demonstration also verifies that the extracted knowledge can help locate the answers that users care about.

cs.CL↗

arXiv2022

RigNet: Repetitive Image Guided Network for Depth Completion

Depth completion deals with the problem of recovering dense depth maps from sparse ones, where color images are often used to facilitate this task. Recent approaches mainly focus on image guided learning frameworks to predict dense depth. However, blurry guidance in the image and unclear structure in the depth still impede the performance of the image guided frameworks. To tackle these problems, we explore a repetitive design in our image guided network to gradually and sufficiently recover depth values. Specifically, the repetition is embodied in both the image guidance branch and depth generation branch. In the former branch, we design a repetitive hourglass network to extract discriminative image features of complex environments, which can provide powerful contextual instruction for depth prediction. In the latter branch, we introduce a repetitive guidance module based on dynamic convolution, in which an efficient convolution factorization is proposed to simultaneously reduce its complexity and progressively model high-frequency structures. Extensive experiments show that our method achieves superior or competitive results on KITTI benchmark and NYUv2 dataset.

cs.CV↗

arXiv2022

Multi-Modal Masked Pre-Training for Monocular Panoramic Depth Completion

In this paper, we formulate a potentially valuable panoramic depth completion (PDC) task as panoramic 3D cameras often produce 360° depth with missing data in complex scenes. Its goal is to recover dense panoramic depths from raw sparse ones and panoramic RGB images. To deal with the PDC task, we train a deep network that takes both depth and image as inputs for the dense panoramic depth recovery. However, it needs to face a challenging optimization problem of the network parameters due to its non-convex objective function. To address this problem, we propose a simple yet effective approach termed M{^3}PT: multi-modal masked pre-training. Specifically, during pre-training, we simultaneously cover up patches of the panoramic RGB image and sparse depth by shared random mask, then reconstruct the sparse depth in the masked regions. To our best knowledge, it is the first time that we show the effectiveness of masked pre-training in a multi-modal vision task, instead of the single-modal task resolved by masked autoencoders (MAE). Different from MAE where fine-tuning completely discards the decoder part of pre-training, there is no architectural difference between the pre-training and fine-tuning stages in our M$^{3}$PT as they only differ in the prediction density, which potentially makes the transfer learning more convenient and effective. Extensive experiments verify the effectiveness of M{^3}PT on three panoramic datasets. Notably, we improve the state-of-the-art baselines by averagely 26.2% in RMSE, 51.7% in MRE, 49.7% in MAE, and 37.5% in RMSElog on three benchmark datasets.

cs.CV↗

arXiv2022

Capacity Bounds for the Two-User IM/DD Interference Channel

This paper studies the capacity of the two-user intensity-modulation/direct-detection (IM/DD) interference channel (IC), which is relevant in the context of multi-user optical wireless communications. Despite some known single-letter capacity characterizations for general discrete-memoryless ICs, a computable capacity expression for the IM/DD IC is missing. In this paper, we provide tight and easily computable inner and outer bounds for a general two-user IM/DD IC under peak and average optical intensity constraints. The bounds enable characterizing the asymptotic sum-rate capacity in the strong and weak interference regimes, as well as the generalized degrees of freedom (GDoF) in the symmetric case. Using the obtained bounds, the GDoF of the IM/DD IC is shown to have a `W' shape similar to the Gaussian IC with power constraints. The obtained bounds are also evaluated numerically in different interference regimes to show their tightness, and used to study the performance of on-chip and indoor OWC systems.

cs.IT↗

arXiv2022

Linearity Grafting: Relaxed Neuron Pruning Helps Certifiable Robustness

Certifiable robustness is a highly desirable property for adopting deep neural networks (DNNs) in safety-critical scenarios, but often demands tedious computations to establish. The main hurdle lies in the massive amount of non-linearity in large DNNs. To trade off the DNN expressiveness (which calls for more non-linearity) and robustness certification scalability (which prefers more linearity), we propose a novel solution to strategically manipulate neurons, by "grafting" appropriate levels of linearity. The core of our proposal is to first linearize insignificant ReLU neurons, to eliminate the non-linear components that are both redundant for DNN performance and harmful to its certification. We then optimize the associated slopes and intercepts of the replaced linear activations for restoring model performance while maintaining certifiability. Hence, typical neuron pruning could be viewed as a special case of grafting a linear function of the fixed zero slopes and intercept, that might overly restrict the network flexibility and sacrifice its performance. Extensive experiments on multiple datasets and network backbones show that our linearity grafting can (1) effectively tighten certified bounds; (2) achieve competitive certifiable robustness without certified robust training (i.e., over 30% improvements on CIFAR-10 models); and (3) scale up complete verification to large adversarially trained models with 17M parameters. Codes are available at https://github.com/VITA-Group/Linearity-Grafting.

cs.LG↗

arXiv2022

Data-Efficient Double-Win Lottery Tickets from Robust Pre-training

Pre-training serves as a broadly adopted starting point for transfer learning on various downstream tasks. Recent investigations of lottery tickets hypothesis (LTH) demonstrate such enormous pre-trained models can be replaced by extremely sparse subnetworks (a.k.a. matching subnetworks) without sacrificing transferability. However, practical security-crucial applications usually pose more challenging requirements beyond standard transfer, which also demand these subnetworks to overcome adversarial vulnerability. In this paper, we formulate a more rigorous concept, Double-Win Lottery Tickets, in which a located subnetwork from a pre-trained model can be independently transferred on diverse downstream tasks, to reach BOTH the same standard and robust generalization, under BOTH standard and adversarial training regimes, as the full pre-trained model can do. We comprehensively examine various pre-training mechanisms and find that robust pre-training tends to craft sparser double-win lottery tickets with superior performance over the standard counterparts. For example, on downstream CIFAR-10/100 datasets, we identify double-win matching subnetworks with the standard, fast adversarial, and adversarial pre-training from ImageNet, at 89.26%/73.79%, 89.26%/79.03%, and 91.41%/83.22% sparsity, respectively. Furthermore, we observe the obtained double-win lottery tickets can be more data-efficient to transfer, under practical data-limited (e.g., 1% and 10%) downstream schemes. Our results show that the benefits from robust pre-training are amplified by the lottery ticket scheme, as well as the data-limited transfer setting. Codes are available at https://github.com/VITA-Group/Double-Win-LTH.

cs.LG↗

arXiv2022

Quarantine: Sparsity Can Uncover the Trojan Attack Trigger for Free

Trojan attacks threaten deep neural networks (DNNs) by poisoning them to behave normally on most samples, yet to produce manipulated results for inputs attached with a particular trigger. Several works attempt to detect whether a given DNN has been injected with a specific trigger during the training. In a parallel line of research, the lottery ticket hypothesis reveals the existence of sparse subnetworks which are capable of reaching competitive performance as the dense network after independent training. Connecting these two dots, we investigate the problem of Trojan DNN detection from the brand new lens of sparsity, even when no clean training data is available. Our crucial observation is that the Trojan features are significantly more stable to network pruning than benign features. Leveraging that, we propose a novel Trojan network detection regime: first locating a "winning Trojan lottery ticket" which preserves nearly full Trojan information yet only chance-level performance on clean inputs; then recovering the trigger embedded in this already isolated subnetwork. Extensive experiments on various datasets, i.e., CIFAR-10, CIFAR-100, and ImageNet, with different network architectures, i.e., VGG-16, ResNet-18, ResNet-20s, and DenseNet-100 demonstrate the effectiveness of our proposal. Codes are available at https://github.com/VITA-Group/Backdoor-LTH.

cs.LG↗

arXiv2022

Label Anchored Contrastive Learning for Language Understanding

Contrastive learning (CL) has achieved astonishing progress in computer vision, speech, and natural language processing fields recently with self-supervised learning. However, CL approach to the supervised setting is not fully explored, especially for the natural language understanding classification task. Intuitively, the class label itself has the intrinsic ability to perform hard positive/negative mining, which is crucial for CL. Motivated by this, we propose a novel label anchored contrastive learning approach (denoted as LaCon) for language understanding. Specifically, three contrastive objectives are devised, including a multi-head instance-centered contrastive loss (ICL), a label-centered contrastive loss (LCL), and a label embedding regularizer (LER). Our approach does not require any specialized network architecture or any extra data augmentation, thus it can be easily plugged into existing powerful pre-trained language models. Compared to the state-of-the-art baselines, LaCon obtains up to 4.1% improvement on the popular datasets of GLUE and CLUE benchmarks. Besides, LaCon also demonstrates significant advantages under the few-shot and data imbalance settings, which obtains up to 9.4% improvement on the FewGLUE and FewCLUE benchmarking tasks.

cs.CL↗

arXiv2022

Study of exotic hadrons with machine learning

We analyzed the invariant mass spectrum of near-threshold exotic states for one-channel candidates with a deep neural network. It can extract the scattering length and effective range, which would shed light on the nature of given states, from the experimental mass spectrum. As an application, the mass spectrum of the $X(3872)$ and the $T_{cc}^+$ are studied. The obtained scattering lengths, effective ranges, and most relevant thresholds are consistent with those from fitting to the experimental data. The advantage of the neural network is that it is more stable than the fitting, especially for low-statistic data. The network, which provides another way to analyze the experimental data, can also be applied to other one-channel near-threshold exotic candidates.

hep-ph↗

arXiv2022

Large Rashba Spin-Orbit Coupling and High-Temperature Quantum Anomalous Hall Effect in Re-Intercalated Graphene/CrI$_3$ Heterostructure

In 2010, quantum anomalous Hall effect (QAHE) in graphene was proposed in the presence of Rashba spin-orbit coupling and ferromagnetic exchange field. After a decade's experimental exploration, the anomalous Hall conductance can only reach about 0.25 in the units of $2e^2/h$, which was attributed to the tiny Rashba spin-orbit coupling. Here, we theoretically show that Re-intercalation in graphene/CrI$_3$ heterostructure can not only induce sizeable Rashba spin-orbit coupling ($>$ 40~meV), but also open up large band gaps at valleys $K$ (22.2 meV) and $K' $ (30.3 meV), and a global band gap over 5.5 meV (19.5 meV with random Re distribution) hosting QAHE. A low-energy continuum model is constructed to explain the underlying physical mechanism. We find that Rashba spin-orbit coupling is robust against external stress whereas a tensile strain can increase the global bulk gap. Furthermore, we also show that Re-intercalated graphene with hexagonal boron-nitride can also realize QAHE with bulk gap over 40~meV, indicating the tunability of $5d$-intercalated graphene-based heterostructure. Our finding makes a great leap towards the experimental realization of graphene-based QAHE, and will definitely accelerate the practical application of graphene-based low-power electronics.

cond-mat.mes-hall↗

arXiv2022

The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancy

Vision transformers (ViTs) have gained increasing popularity as they are commonly believed to own higher modeling capacity and representation flexibility, than traditional convolutional networks. However, it is questionable whether such potential has been fully unleashed in practice, as the learned ViTs often suffer from over-smoothening, yielding likely redundant models. Recent works made preliminary attempts to identify and alleviate such redundancy, e.g., via regularizing embedding similarity or re-injecting convolution-like structures. However, a "head-to-toe assessment" regarding the extent of redundancy in ViTs, and how much we could gain by thoroughly mitigating such, has been absent for this field. This paper, for the first time, systematically studies the ubiquitous existence of redundancy at all three levels: patch embedding, attention map, and weight space. In view of them, we advocate a principle of diversity for training ViTs, by presenting corresponding regularizers that encourage the representation diversity and coverage at each of those levels, that enabling capturing more discriminative information. Extensive experiments on ImageNet with a number of ViT backbones validate the effectiveness of our proposals, largely eliminating the observed ViT redundancy and significantly boosting the model generalization. For example, our diversified DeiT obtains 0.70%~1.76% accuracy boosts on ImageNet with highly reduced similarity. Our codes are fully available in https://github.com/VITA-Group/Diverse-ViT.

cs.LG↗

arXiv2022

Measurements of Rayleigh Ratios in Linear Alkylbenzene

In present work an experiment has been designed to measure the Rayleigh ratio directly at $405$ nm and $432$ nm for linear alkylbenzene which is a common solvent used in liquid scintillator detectors of neutrino experiments. The perpendicularly polarized Rayleigh ratio is determined to be $(4.52\pm 0.28)\times 10^{-6}$ m$^{-1}\cdot$ sr$^{-1}$ at $405$ nm and $(3.82\pm 0.24)\times 10^{-6}$ m$^{-1}\cdot$ sr$^{-1}$ at $432$ nm, and the corresponding Rayleigh scattering length is $L_{Ray} = 22.9\pm 0.3(\mathrm{stat.})\pm 1.7(\mathrm{sys.})$ m at $405$ nm and $L_{Ray}= 27.0\pm 0.9(\mathrm{stat.})\pm 1.8(\mathrm{sys.})$ m at $432$ nm.

physics.ins-det↗

arXiv2022

Ferroelectric tuning of superconductivity and band topology in a two-dimensional heterobilayer

Realization of tunable superconductivity with concomitant nontrivial band topology is conceptually intriguing and highly desirable for superconducting devices and topological quantum computation. Based on first-principles calculations, here we present the first prediction of simultaneously tunable superconducting transition temperature (Tc) and band topology in a superconducting IrTe2 overlayer on a ferroelectric In2Se3 monolayer. We first demonstrate that the Tc is substantially enhanced from that of IrTe2 nanoflakes (Tc ~3 K) due to significant charge repartitioning around the Fermi level. More importantly, the Tc is shown to sensitively depend on the In2Se3 polarization, with the higher Tc of ~(8-10) K attributed to enhanced interlayer electron-phonon coupling when the polarization is downward. The band topology is also switched from trivial to nontrivial as the polarization is reversed from upward to downward. These findings provide physically realistic platforms for simultaneously tuning superconductivity and band topology in two-dimensional heterobilayers and related heterostructures using a reversible and nonvolatile approach.

cond-mat.supr-con↗

arXiv2022

Sparsity Winning Twice: Better Robust Generalization from More Efficient Training

Recent studies demonstrate that deep networks, even robustified by the state-of-the-art adversarial training (AT), still suffer from large robust generalization gaps, in addition to the much more expensive training costs than standard training. In this paper, we investigate this intriguing problem from a new perspective, i.e., injecting appropriate forms of sparsity during adversarial training. We introduce two alternatives for sparse adversarial training: (i) static sparsity, by leveraging recent results from the lottery ticket hypothesis to identify critical sparse subnetworks arising from the early training; (ii) dynamic sparsity, by allowing the sparse subnetwork to adaptively adjust its connectivity pattern (while sticking to the same sparsity ratio) throughout training. We find both static and dynamic sparse methods to yield win-win: substantially shrinking the robust generalization gap and alleviating the robust overfitting, meanwhile significantly saving training and inference FLOPs. Extensive experiments validate our proposals with multiple network architectures on diverse datasets, including CIFAR-10/100 and Tiny-ImageNet. For example, our methods reduce robust generalization gap and overfitting by 34.44% and 4.02%, with comparable robust/standard accuracy boosts and 87.83%/87.82% training/inference FLOPs savings on CIFAR-100 with ResNet-18. Besides, our approaches can be organically combined with existing regularizers, establishing new state-of-the-art results in AT. Codes are available in https://github.com/VITA-Group/Sparsity-Win-Robust-Generalization.

cs.CV↗

arXiv2022

A Monte Carlo Method for Rayleigh Scattering in Liquid Detectors

A new Monte Carlo method has been implemented to describe the angular and polarization distributions of anisotropic liquids, like water and linear alkylbenzene, by considering orientational fluctuations of polarizability tensors. The scattered light of anisotropic liquids is depolarized with an angular distribution of $1+(1-ρ_v)/(1+3ρ_v)\cos^2θ$, which is modified by the depolarization ratio $ρ_v$. A standalone experiment has validated the simulation results of LAB. The new method can provide more accurate knowledge on light propagation in large liquid detectors, which is beneficial to the developments of reconstruction for detectors.

physics.ins-det↗

arXiv2022

Josephson-Coulomb drag effect between graphene and LaAlO3/SrTiO3 interfacial superconductor

Coulomb drag refers to the phenomenon that a charge current in one electronic circuit induces a responsive current in a neighboring circuit solely through Coulomb interactions. For conventional interactions between fermionic particles such as electrons, the as-induced drag current in the passive layer is orders of magnitude weaker than the active current due to strong dielectric screening effect between the two. Here we propose a 'super' Coulomb drag effect between an active normal conductor and a passive superconductor of Josephson junction arrays, whereby the passive current can greatly exceed the active. The drag force originates from the interactions between the substantially enhanced dynamical quantum fluctuations of the superconducting phases in the passive layer and the normal electrons in the active layer. We demonstrate this effect in the devices composed of monolayer graphene and LaAlO3/SrTiO3 heterointerface, an inherently non-uniform superconductor described by Josephson junction arrays. Remarkable drag signal is observed in the superconducting transition regime of the LaAlO3/SrTiO3 interface, with its sign independent of the carrier type in the graphene layer. The estimated passive-to-active ratio can reach about 0.3 at the optimal gate voltage and the temperature dependence follows that of the typical Josephson energy between superconducting puddles. Strikingly, the ratio ought to be as large as 10^5 at zero temperature by theoretical extrapolation. From engineering perspective, our device may work as current or voltage transformers, and the drag mechanism lays the foundation for synchronizing Josephson-junction-array-based terahertz radiators.

cond-mat.mes-hall↗

arXiv2021

A family tree of two-dimensional magnetic materials with tunable topological properties

Two-dimensional (2D) magnetic materials empowered with nontrivial band topology may lead to the emergence of exotic quantum states with significant application potentials. Here we predict a family tree of 2D magnetic materials with tunable topological properties, starting from the parental materials of CrI3 and CrBr3. The underlying design principle is that, by substituting the alternating sites of the Cr honeycomb lattice sandwiched between the halogen layers with V or Mn, the parental materials of trivial ferromagnetic insulators are ripe to be converted into topological systems. Specifically, our first-principles calculations show that, due to the elegant interplay between bandgap narrowing and spin-orbital coupling, CrI3 branches into high-temperature quantum anomalous Hall insulators of CrVI6 and CrMnI6 with different topological invariants, while CrBr3 branches into topological half-metals of CrVBr6 and CrMnBr6. Those novel 2D magnets are also shown to be easily exfoliated from their bulk counterparts. The present study is thus geared to advance the field of 2D magnetic materials into the topologically nontrivial realm.

cond-mat.mtrl-sci↗

arXiv2021

Giant Dzyaloshinskii-Moriya interaction and ratchet motion of bimeronic excitations in two-dimensional magnets

Topological magnetic excitations rooted in Dzyaloshinskii-Moriya interaction (DMI) are promising information carriers for next-generation memory and spintronic devices. The recently discovered two-dimensional (2D) magnets provide fertile new platforms for revealing rich physical phenomena with atomic-scale precision, as exemplified by the enhanced DMI associated with an effective electric field that breaks the inversion symmetry. Here we use first-principles calculations to establish a conceptually difierent compositional engineering approach to induce giant DMI in a representative CrMnI$_6$ 2D magnet, and the underlying mechanism is rooted in the spontaneous inversion symmetry breaking in the bipartite system. Using atomistic magnetics simulations, we further reveal that bimeronic excitations can emerge upon cooling a spin random state or perturbing a ferromagnetic state. Strikingly, the bimeron can exhibit unidirectional soliton-like propagation, and such a ratchet phenomenon is highly desirable for racetrack memories. These findings may prove instrumental in developing novel devices for quantum information based on 2D magnets.

cond-mat.mtrl-sci↗