arXiv subjects

Hui Zhang

Publications and source records attributed to Hui Zhang.

At least 307 records · Page 17Linked to original sources

arXiv2023

0/1 Deep Neural Networks via Block Coordinate Descent

The step function is one of the simplest and most natural activation functions for deep neural networks (DNNs). As it counts 1 for positive variables and 0 for others, its intrinsic characteristics (e.g., discontinuity and no viable information of subgradients) impede its development for several decades. Even if there is an impressive body of work on designing DNNs with continuous activation functions that can be deemed as surrogates of the step function, it is still in the possession of some advantageous properties, such as complete robustness to outliers and being capable of attaining the best learning-theoretic guarantee of predictive accuracy. Hence, in this paper, we aim to train DNNs with the step function used as an activation function (dubbed as 0/1 DNNs). We first reformulate 0/1 DNNs as an unconstrained optimization problem and then solve it by a block coordinate descend (BCD) method. Moreover, we acquire closed-form solutions for sub-problems of BCD as well as its convergence properties. Furthermore, we also integrate $\ell_{2,0}$-regularization into 0/1 DNN to accelerate the training process and compress the network scale. As a result, the proposed algorithm has a high performance on classifying MNIST and Fashion-MNIST datasets. As a result, the proposed algorithm has a desirable performance on classifying MNIST, FashionMNIST, Cifar10, and Cifar100 datasets.

cs.LG↗

arXiv2023

Collaborative Perception in Autonomous Driving: Methods, Datasets and Challenges

Collaborative perception is essential to address occlusion and sensor failure issues in autonomous driving. In recent years, theoretical and experimental investigations of novel works for collaborative perception have increased tremendously. So far, however, few reviews have focused on systematical collaboration modules and large-scale collaborative perception datasets. This work reviews recent achievements in this field to bridge this gap and motivate future research. We start with a brief overview of collaboration schemes. After that, we systematically summarize the collaborative perception methods for ideal scenarios and real-world issues. The former focuses on collaboration modules and efficiency, and the latter is devoted to addressing the problems in actual application. Furthermore, we present large-scale public datasets and summarize quantitative results on these benchmarks. Finally, we highlight gaps and overlook challenges between current academic research and real-world applications. The project page is https://github.com/CatOneTwo/Collaborative-Perception-in-Autonomous-Driving

cs.CV↗

arXiv2023

Rigorous noise reduction with quantum autoencoders

Reducing noise in quantum systems is a major challenge towards the application of quantum technologies. Here, we propose and demonstrate a scheme to reduce noise using a quantum autoencoder with rigorous performance guarantees. The quantum autoencoder learns to compresses noisy quantum states into a latent subspace and removes noise via projective measurements. We find various noise models where we can perfectly reconstruct the original state even for high noise levels. We apply the autoencoder to cool thermal states to the ground state and reduce the cost of magic state distillation by several orders of magnitude. Our autoencoder can be implemented using only unitary transformations without ancillas, making it immediately compatible with the state of the art. We experimentally demonstrate our methods to reduce noise in a photonic integrated circuit. Our results can be directly applied to make quantum technologies more robust to noise.

quant-ph↗

arXiv2023

Giant orbit-to-charge conversion induced via the inverse orbital Hall effect

We investigate the orbit-to-charge conversion in YIG/Pt/nonmagnetic material (NM) trilayer heterostructures. With the additional Ru layer on the top of YIG/Pt stacks, the charge current signal increases nearly an order of magnitude in both longitudinal spin Seebeck effect (SSE) and spin pumping (SP) measurements. Through thickness dependence studies of the Ru metal layer and theoretical model, we quantitatively clarify different contributions of the increased SSE signal that mainly comes from the inverse orbital Hall effect (IOHE) of Ru, and partially comes from the orbital sink effect in the Ru layer. A similar enhancement of SSE(SP) signals is also observed when Ru is replaced by other materials (Ta, W, and Cu), implying the universality of the IOHE in transition metals. Our findings not only suggest a more efficient generation of the charge current via the orbital angular moment channel but also provides crucial insights into the interplay among charge, spin, and orbit.

cond-mat.mtrl-sci↗

arXiv2023

Normalized solutions for critical Choquard systems

In this paper, we consider the critical Choquard system with prescribed mass \begin{equation*} \begin{aligned} \left\{ \begin{array}{lll} -Δu+λ_1u=(I_μ\ast |u|^{2^*_μ})|u|^{2^*_μ-2}u+νp(I_μ\ast |v|^q)|u|^{p-2}u\ & \text{in}\quad \mathbb{R}^N,\\ -Δv+λ_2v=(I_μ\ast |v|^{2^*_μ})|v|^{2^*_μ-2}v+νq(I_μ\ast |u|^p)|v|^{q-2}v\ & \text{in}\quad \mathbb{R}^N,\\ \int_{\mathbb{R}^N}u^2=a^2,\quad\int_{\mathbb{R}^N}v^2=b^2, \end{array}\right.\end{aligned} \end{equation*} where $N\geq3$, $0<μ 0$, we study the existence, non-existence and asymptotic behavior of normalized solutions by distinguishing three cases: $L^2$-subcritical case: $p+q<4+\frac{4-2μ}{N}$; $L^2$-critical case: $p+q=4+\frac{4-2μ}{N}$; $L^2$-supercritical case: $p+q>4+\frac{4-2μ}{N}$. In particular, in $L^2$-subcritical case, and either $N\in\{3,4\}$ or $N\geq5$ with $(\frac N2-1)p+\frac {N}{2}q\leq 2N-μ$ and $(\frac N2-1)q+\frac {N}{2}p\leq 2N-μ$, we prove that there exists $ν_0>0$ such that the system has a positive radial normalized ground state for $0<ν<ν_0$. In $L^2$-critical case and $N\in\{3,4\}$, we show there is $ν'_0>0$ such that the system has a positive radial normalized ground state for $0<ν<ν'_0$. In $L^2$-supercritical case and $N\in\{3,4\}$, there are two thresholds $ν_2\geqν_1\geq0$ such that a positive radial normalized solution exists if $ν>ν_2$, and no normalized ground state exists for $ν<ν_1$.

math.AP↗

arXiv2023

Development of 15kA/cm$^2$ Fabrication Process for Superconducting Integrated Digital Circuits

A new fabrication process for superconducting integrated digital circuits is reported. We have developed the "SIMIT Nb04" fabrication technique for superconducting integrated circuits with Nb-based Josephson junctions based on the validated "SIMIT Nb03" process and Chemical Mechanical Planarization (CMP) technology. Seven Nb superconducting layers and one Mo resistor layer are included in the "SIMIT Nb04" process with 19 mask levels. The device structure is composed of active layers including junctions at the bottom, two passive transmission line (PTL) layers in the middle and a DC power layer at the top. The circuit fabrication started with the fabrication of Mo resistors with a target sheet resistance Rsh of 3 $Ω$, followed by the deposition of Nb/Al-AlO$_x$/Nb trilayer Josephson-junction with a target critical current density Jc at 15 kA/cm$^2$. To increase the Al-AlO$_x$ barrier layer etching's repeatability, an additional barrier protection layer was applied. To accomplish high-quality planarization, we created a planarization procedure coupled with dummy filling. To assess the process dependability and controllability, a set of process control monitors (PCMs) for monitoring fabrication and design parameters was designed and monitored. The successful manufacturing and testing of a few small-scale circuits, like our standard library cells, further attests to the viability of our fabrication process for superconducting integrated circuits.

physics.app-ph↗

arXiv2023

Retro-FPN: Retrospective Feature Pyramid Network for Point Cloud Semantic Segmentation

Learning per-point semantic features from the hierarchical feature pyramid is essential for point cloud semantic segmentation. However, most previous methods suffered from ambiguous region features or failed to refine per-point features effectively, which leads to information loss and ambiguous semantic identification. To resolve this, we propose Retro-FPN to model the per-point feature prediction as an explicit and retrospective refining process, which goes through all the pyramid layers to extract semantic features explicitly for each point. Its key novelty is a retro-transformer for summarizing semantic contexts from the previous layer and accordingly refining the features in the current stage. In this way, the categorization of each point is conditioned on its local semantic pattern. Specifically, the retro-transformer consists of a local cross-attention block and a semantic gate unit. The cross-attention serves to summarize the semantic pattern retrospectively from the previous layer. And the gate unit carefully incorporates the summarized contexts and refines the current semantic features. Retro-FPN is a pluggable neural network that applies to hierarchical decoders. By integrating Retro-FPN with three representative backbones, including both point-based and voxel-based methods, we show that Retro-FPN can significantly improve performance over state-of-the-art backbones. Comprehensive experiments on widely used benchmarks can justify the effectiveness of our design. The source is available at https://github.com/AllenXiangX/Retro-FPN

cs.CV↗

arXiv2023

Deformations and extensions of modified $λ$-differential $3$-Lie Algebras

In this paper, we introduce the representation of modified $λ$-differential $3$-Lie algebras and define the cohomology of modified $λ$-differential $3$-Lie algebras with coefficients in a representation. As applications of the proposed cohomology theory, we study linear deformations, abelian extensions and $T^*$-extensions of modified $λ$-differential $3$-Lie algebras.

math.RA↗

arXiv2023

Efficient option pricing with unary-based photonic computing chip and generative adversarial learning

In the modern financial industry system, the structure of products has become more and more complex, and the bottleneck constraint of classical computing power has already restricted the development of the financial industry. Here, we present a photonic chip that implements the unary approach to European option pricing, in combination with the quantum amplitude estimation algorithm, to achieve a quadratic speedup compared to classical Monte Carlo methods. The circuit consists of three modules: a module loading the distribution of asset prices, a module computing the expected payoff, and a module performing the quantum amplitude estimation algorithm to introduce speed-ups. In the distribution module, a generative adversarial network is embedded for efficient learning and loading of asset distributions, which precisely capture the market trends. This work is a step forward in the development of specialized photonic processors for applications in finance, with the potential to improve the efficiency and quality of financial services.

quant-ph↗

arXiv2023

Rician likelihood loss for quantitative MRI using self-supervised deep learning

Purpose: Previous quantitative MR imaging studies using self-supervised deep learning have reported biased parameter estimates at low SNR. Such systematic errors arise from the choice of Mean Squared Error (MSE) loss function for network training, which is incompatible with Rician-distributed MR magnitude signals. To address this issue, we introduce the negative log Rician likelihood (NLR) loss. Methods: A numerically stable and accurate implementation of the NLR loss was developed to estimate quantitative parameters of the apparent diffusion coefficient (ADC) model and intra-voxel incoherent motion (IVIM) model. Parameter estimation accuracy, precision and overall error were evaluated in terms of bias, variance and root mean squared error and compared against the MSE loss over a range of SNRs (5 - 30). Results: Networks trained with NLR loss show higher estimation accuracy than MSE for the ADC and IVIM diffusion coefficients as SNR decreases, with minimal loss of precision or total error. At high effective SNR (high SNR and small diffusion coefficients), both losses show comparable accuracy and precision for all parameters of both models. Conclusion: The proposed NLR loss is numerically stable and accurate across the full range of tested SNRs and improves parameter estimation accuracy of diffusion coefficients using self-supervised deep learning. We expect the development to benefit quantitative MR imaging techniques broadly, enabling more accurate parameter estimation from noisy data.

cs.LG↗

arXiv2023

DRMC: A Generalist Model with Dynamic Routing for Multi-Center PET Image Synthesis

Multi-center positron emission tomography (PET) image synthesis aims at recovering low-dose PET images from multiple different centers. The generalizability of existing methods can still be suboptimal for a multi-center study due to domain shifts, which result from non-identical data distribution among centers with different imaging systems/protocols. While some approaches address domain shifts by training specialized models for each center, they are parameter inefficient and do not well exploit the shared knowledge across centers. To address this, we develop a generalist model that shares architecture and parameters across centers to utilize the shared knowledge. However, the generalist model can suffer from the center interference issue, \textit{i.e.} the gradient directions of different centers can be inconsistent or even opposite owing to the non-identical data distribution. To mitigate such interference, we introduce a novel dynamic routing strategy with cross-layer connections that routes data from different centers to different experts. Experiments show that our generalist model with dynamic routing (DRMC) exhibits excellent generalizability across centers. Code and data are available at: https://github.com/Yaziwel/Multi-Center-PET-Image-Synthesis.

eess.IV↗

arXiv2023

SSC3OD: Sparsely Supervised Collaborative 3D Object Detection from LiDAR Point Clouds

Collaborative 3D object detection, with its improved interaction advantage among multiple agents, has been widely explored in autonomous driving. However, existing collaborative 3D object detectors in a fully supervised paradigm heavily rely on large-scale annotated 3D bounding boxes, which is labor-intensive and time-consuming. To tackle this issue, we propose a sparsely supervised collaborative 3D object detection framework SSC3OD, which only requires each agent to randomly label one object in the scene. Specifically, this model consists of two novel components, i.e., the pillar-based masked autoencoder (Pillar-MAE) and the instance mining module. The Pillar-MAE module aims to reason over high-level semantics in a self-supervised manner, and the instance mining module generates high-quality pseudo labels for collaborative detectors online. By introducing these simple yet effective mechanisms, the proposed SSC3OD can alleviate the adverse impacts of incomplete annotations. We generate sparse labels based on collaborative perception datasets to evaluate our method. Extensive experiments on three large-scale datasets reveal that our proposed SSC3OD can effectively improve the performance of sparsely supervised collaborative 3D object detectors.

cs.CV↗

arXiv2023

Patch-CNN: Training data-efficient deep learning for high-fidelity diffusion tensor estimation from minimal diffusion protocols

We propose a new method, Patch-CNN, for diffusion tensor (DT) estimation from only six-direction diffusion weighted images (DWI). Deep learning-based methods have been recently proposed for dMRI parameter estimation, using either voxel-wise fully-connected neural networks (FCN) or image-wise convolutional neural networks (CNN). In the acute clinical context -- where pressure of time limits the number of imaged directions to a minimum -- existing approaches either require an infeasible number of training images volumes (image-wise CNNs), or do not estimate the fibre orientations (voxel-wise FCNs) required for tractogram estimation. To overcome these limitations, we propose Patch-CNN, a neural network with a minimal (non-voxel-wise) convolutional kernel (3$\times$3$\times$3). Compared with voxel-wise FCNs, this has the advantage of allowing the network to leverage local anatomical information. Compared with image-wise CNNs, the minimal kernel vastly reduces training data demand. Evaluated against both conventional model fitting and a voxel-wise FCN, Patch-CNN, trained with a single subject is shown to improve the estimation of both scalar dMRI parameters and fibre orientation from six-direction DWIs. The improved fibre orientation estimation is shown to produce improved tractogram.

cs.CV↗

arXiv2023

Combining Self-Supervised and Supervised Learning with Noisy Labels

Since convolutional neural networks (CNNs) can easily overfit noisy labels, which are ubiquitous in visual classification tasks, it has been a great challenge to train CNNs against them robustly. Various methods have been proposed for this challenge. However, none of them pay attention to the difference between representation and classifier learning of CNNs. Thus, inspired by the observation that classifier is more robust to noisy labels while representation is much more fragile, and by the recent advances of self-supervised representation learning (SSRL) technologies, we design a new method, i.e., CS$^3$NL, to obtain representation by SSRL without labels and train the classifier directly with noisy labels. Extensive experiments are performed on both synthetic and real benchmark datasets. Results demonstrate that the proposed method can beat the state-of-the-art ones by a large margin, especially under a high noisy level.

cs.CV↗

arXiv2023

A Weighted Randomized Sparse Kaczmarz Method for Solving Linear Systems

The randomized sparse Kaczmarz method, designed for seeking the sparse solutions of the linear systems $Ax=b$, selects the $i$-th projection hyperplane with likelihood proportional to $\|a_{i}\|_2^2$, where $a_{i}^T$ is $i$-th row of $A$. In this work, we propose a weighted randomized sparse Kaczmarz method, which selects the $i$-th projection hyperplane with probability proportional to $\lvert\langle a_{i},x_{k}\rangle-b_{i}\rvert^p$, where $0<p<\infty$, for possible acceleration. It bridges the randomized Kaczmarz and greedy Kaczmarz by parameter $p$. Theoretically, we show its linear convergence rate in expectation with respect to the Bregman distance in the noiseless and noisy cases, which is at least as efficient as the randomized sparse Kaczmarz method. The superiority of the proposed method is demonstrated via a group of numerical experiments.

math.NA↗

arXiv2023

When SAM Meets Shadow Detection

As a promptable generic object segmentation model, segment anything model (SAM) has recently attracted significant attention, and also demonstrates its powerful performance. Nevertheless, it still meets its Waterloo when encountering several tasks, e.g., medical image segmentation, camouflaged object detection, etc. In this report, we try SAM on an unexplored popular task: shadow detection. Specifically, four benchmarks were chosen and evaluated with widely used metrics. The experimental results show that the performance for shadow detection using SAM is not satisfactory, especially when comparing with the elaborate models. Code is available at https://github.com/LeipingJie/SAMSh.

cs.CV↗

arXiv2023

Three-dimensional structural and compositional inhomogeneity in zeolites unraveled by low-dose electron ptychography

Structural and compositional inhomogeneity is common in zeolites and considerably affects their properties. Conventional transmission electron microscopy (TEM) cannot provide sufficient information on local structures in zeolites due to the thickness-limited lateral resolution, lack of depth resolution, and electron dose-constrained focusing. We demonstrate that a multislice ptychography method based on four-dimensional scanning TEM (4D-STEM) data can overcome these limitations. The 4D-STEM ptychography image obtained from a ~40-nm thick MFI zeolite exhibits an ultrahigh lateral resolution of ~0.85 Å, enabling the unambiguous identification of individual framework oxygen (O) atoms and the precise determination of the orientations of adsorbed molecules. Furthermore, a depth resolution of ~6.6 nm is achieved, allowing the three-dimensional probing of O vacancies and phase boundaries in zeolites. The results of the 4D-STEM ptychography provide an unprecedented complete understanding of the intergrowth of MFI and MEL zeolites in three dimensions. The 4D-STEM ptychography can be generally applied to various zeolites and other materials with similar electron beam sensitivity.

cond-mat.mtrl-sci↗

arXiv2023

Prototypical Residual Networks for Anomaly Detection and Localization

Anomaly detection and localization are widely used in industrial manufacturing for its efficiency and effectiveness. Anomalies are rare and hard to collect and supervised models easily over-fit to these seen anomalies with a handful of abnormal samples, producing unsatisfactory performance. On the other hand, anomalies are typically subtle, hard to discern, and of various appearance, making it difficult to detect anomalies and let alone locate anomalous regions. To address these issues, we propose a framework called Prototypical Residual Network (PRN), which learns feature residuals of varying scales and sizes between anomalous and normal patterns to accurately reconstruct the segmentation maps of anomalous regions. PRN mainly consists of two parts: multi-scale prototypes that explicitly represent the residual features of anomalies to normal patterns; a multisize self-attention mechanism that enables variable-sized anomalous feature learning. Besides, we present a variety of anomaly generation strategies that consider both seen and unseen appearance variance to enlarge and diversify anomalies. Extensive experiments on the challenging and widely used MVTec AD benchmark show that PRN outperforms current state-of-the-art unsupervised and supervised methods. We further report SOTA results on three additional datasets to demonstrate the effectiveness and generalizability of PRN.

cs.CV↗