arXiv subjects

Zi Wang

Publications and source records attributed to Zi Wang.

At least 109 records · Page 6Linked to original sources

arXiv2024

Rethinking Diversity in Deep Neural Network Testing

Motivated by the success of traditional software testing, numerous diversity measures have been proposed for testing deep neural networks (DNNs). In this study, we propose a shift in perspective, advocating for the consideration of DNN testing as directed testing problems rather than diversity-based testing tasks. We note that the objective of testing DNNs is specific and well-defined: identifying inputs that lead to misclassifications. Consequently, a more precise testing approach is to prioritize inputs with a higher potential to induce misclassifications, as opposed to emphasizing inputs that enhance "diversity." We derive six directed metrics for DNN testing. Furthermore, we conduct a careful analysis of the appropriate scope for each metric, as applying metrics beyond their intended scope could significantly diminish their effectiveness. Our evaluation demonstrates that (1) diversity metrics are particularly weak indicators for identifying buggy inputs resulting from small input perturbations, and (2) our directed metrics consistently outperform diversity metrics in revealing erroneous behaviors of DNNs across all scenarios.

cs.SE↗

arXiv2024

Coherence-enhanced thermodynamic performance in a periodically-driven inelastic heat engine

Quantum thermodynamics with microscopic inelastic scattering processes has been intensively investigated in recent years. Here, we apply quantum master equation combined with full counting statistics approach to investigate the role of quantum coherence on the periodically-driven inelastic heat engine. We demonstrate that the inelastic quantum heat engine exhibits dramatic advantage of thermodynamic performance compared to their elastic counterpart. Moreover, it is found that inelastic currents, {output work}, and the efficiency can be enhanced by quantum coherence. In particular, the geometric effect proves crucial in achieving maximal values of generated output work and energy conversion efficiency. The Berry curvature boosted by quantum coherence unveils the underlying mechanism of periodically-driven inelastic heat engine. Our findings may provide some insights for further understanding and optimizing periodically-driven heat engines via quantum coherence resource and inelastic scattering processes.

cond-mat.mes-hall↗

arXiv2024

Transfer Learning for Bayesian Optimization on Heterogeneous Search Spaces

Bayesian optimization (BO) is a popular black-box function optimization method, which makes sequential decisions based on a Bayesian model, typically a Gaussian process (GP), of the function. To ensure the quality of the model, transfer learning approaches have been developed to automatically design GP priors by learning from observations on "training" functions. These training functions are typically required to have the same domain as the "test" function (black-box function to be optimized). In this paper, we introduce MPHD, a model pre-training method on heterogeneous domains, which uses a neural net mapping from domain-specific contexts to specifications of hierarchical GPs. MPHD can be seamlessly integrated with BO to transfer knowledge across heterogeneous search spaces. Our theoretical and empirical results demonstrate the validity of MPHD and its superior performance on challenging black-box function optimization tasks.

cs.LG↗

arXiv2024

Parallel Augmentation and Dual Enhancement for Occluded Person Re-identification

Occluded person re-identification (Re-ID), the task of searching for the same person's images in occluded environments, has attracted lots of attention in the past decades. Recent approaches concentrate on improving performance on occluded data by data/feature augmentation or using extra models to predict occlusions. However, they ignore the imbalance problem in this task and can not fully utilize the information from the training data. To alleviate these two issues, we propose a simple yet effective method with Parallel Augmentation and Dual Enhancement (PADE), which is robust on both occluded and non-occluded data and does not require any auxiliary clues. First, we design a parallel augmentation mechanism (PAM) to generate more suitable occluded data to mitigate the negative effects of unbalanced data. Second, we propose the global and local dual enhancement strategy (DES) to promote the context information and details. Experimental results on three widely used occluded datasets and two non-occluded datasets validate the effectiveness of our method. The code is available at https://github.com/littleprince1121/PADE_Parallel_Augmentation_and_Dual_Enhancement_for_Occluded_Person_ReID

cs.CV↗

arXiv2024

Machine Learning of Knot Topology in Non-Hermitian Band Braids

The deep connection among braids, knots and topological physics has provided valuable insights into studying topological states in various physical systems. However, identifying distinct braid groups and knot topology embedded in non-Hermitian systems is challenging and requires significant efforts. Here, we demonstrate that an unsupervised learning with the representation basis of $su(n)$ Lie algebra on $n$-fold extended non-Hermitian bands can fully classify braid group and knot topology therein, without requiring any prior mathematical knowledge or any pre-defined topological invariants. We demonstrate that the approach successfully identifies different topological elements, such as unlink, unknot, Hopf link, Solomon ring, trefoil, and so on, by employing generalized Gell-Mann matrices in non-Hermitian models with $n$=2 and $n$=3 energy bands. Moreover, since eigenstate information of non-Hermitian bands is incorporated in addition to eigenvalues, the approach distinguishes the different parity-time symmetry and breaking phases, recognizes the opposite chirality of braids and knots, and identifies out distinct topological phases that were overlooked before. Our study shows significant potential of machine learning in classification of knots, braid groups, and non-Hermitian topological phases.

cond-mat.mes-hall↗

arXiv2023

A Faithful Deep Sensitivity Estimation for Accelerated Magnetic Resonance Imaging

Magnetic resonance imaging (MRI) is an essential diagnostic tool that suffers from prolonged scan time. To alleviate this limitation, advanced fast MRI technology attracts extensive research interests. Recent deep learning has shown its great potential in improving image quality and reconstruction speed. Faithful coil sensitivity estimation is vital for MRI reconstruction. However, most deep learning methods still rely on pre-estimated sensitivity maps and ignore their inaccuracy, resulting in the significant quality degradation of reconstructed images. In this work, we propose a Joint Deep Sensitivity estimation and Image reconstruction network, called JDSI. During the image artifacts removal, it gradually provides more faithful sensitivity maps with high-frequency information, leading to improved image reconstructions. To understand the behavior of the network, the mutual promotion of sensitivity estimation and image reconstruction is revealed through the visualization of network intermediate results. Results on in vivo datasets and radiologist reader study demonstrate that, for both calibration-based and calibrationless reconstruction, the proposed JDSI achieves the state-of-the-art performance visually and quantitatively, especially when the acceleration factor is high. Additionally, JDSI owns nice robustness to patients and autocalibration signals.

eess.IV↗

arXiv2023

Gaussian Process Probes (GPP) for Uncertainty-Aware Probing

Understanding which concepts models can and cannot represent has been fundamental to many tasks: from effective and responsible use of models to detecting out of distribution data. We introduce Gaussian process probes (GPP), a unified and simple framework for probing and measuring uncertainty about concepts represented by models. As a Bayesian extension of linear probing methods, GPP asks what kind of distribution over classifiers (of concepts) is induced by the model. This distribution can be used to measure both what the model represents and how confident the probe is about what the model represents. GPP can be applied to any pre-trained model with vector representations of inputs (e.g., activations). It does not require access to training data, gradients, or the architecture. We validate GPP on datasets containing both synthetic and real images. Our experiments show it can (1) probe a model's representations of concepts even with a very small number of examples, (2) accurately measure both epistemic uncertainty (how confident the probe is) and aleatory uncertainty (how fuzzy the concepts are to the model), and (3) detect out of distribution data using those uncertainty measures as well as classic methods do. By using Gaussian processes to expand what probing can offer, GPP provides a data-efficient, versatile and uncertainty-aware tool for understanding and evaluating the capabilities of machine learning models.

cs.LG↗

arXiv2023

Grammar Prompting for Domain-Specific Language Generation with Large Language Models

Large language models (LLMs) can learn to perform a wide range of natural language tasks from just a handful of in-context examples. However, for generating strings from highly structured languages (e.g., semantic parsing to complex domain-specific languages), it is challenging for the LLM to generalize from just a few exemplars. We propose \emph{grammar prompting}, a simple approach to enable LLMs to use external knowledge and domain-specific constraints, expressed through a grammar in Backus--Naur Form (BNF), during in-context learning. Grammar prompting augments each demonstration example with a specialized grammar that is minimally sufficient for generating the particular output example, where the specialized grammar is a subset of the full DSL grammar. For inference, the LLM first predicts a BNF grammar given a test input, and then generates the output according to the rules of the grammar. Experiments demonstrate that grammar prompting can enable LLMs to perform competitively on a diverse set of DSL generation tasks, including semantic parsing (SMCalFlow, Overnight, GeoQuery), PDDL planning, and SMILES-based molecule generation.

cs.CL↗

arXiv2023

XCloud-VIP: Virtual Peak Enables Highly Accelerated NMR Spectroscopy and Faithful Quantitative Measures

Nuclear Magnetic Resonance (NMR) spectroscopy is an important bio-engineering tool to determine the metabolic concentrations, molecule structures and so on. The data acquisition time, however, is very long in multi-dimensional NMR. To accelerate data acquisition, non-uniformly sampling is an effective way but may encounter severe spectral distortions and unfaithful quantitative measures when the acceleration factor is high. By modelling the acquired signal as the superimposed exponentials, we proposed a virtual peak (VIP) approach to selfadapt the prior spectral information, such as the resonance frequency and peak lineshape, and then feed these information into the reconstruction. The proposed method is further implemented with cloud computing to facilitate online, open, and easy access. Results on simulated and experimental data demonstrate that, compared with the low-rank Hankel matrix method, the new approach reconstructs high-fidelity NMR spectra from highly undersampled data and achieves more accurate quantification. The maximum quantitative errors of distances between nuclear pairs and concentrations of metabolites in mixtures have been reduced by 61.1% and 57.7%, respectively.

physics.med-ph↗

arXiv2023

A plug-and-play synthetic data deep learning for undersampled magnetic resonance image reconstruction

Magnetic resonance imaging (MRI) plays an important role in modern medical diagnostic but suffers from prolonged scan time. Current deep learning methods for undersampled MRI reconstruction exhibit good performance in image de-aliasing which can be tailored to the specific k-space undersampling scenario. But it is very troublesome to configure different deep networks when the sampling setting changes. In this work, we propose a deep plug-and-play method for undersampled MRI reconstruction, which effectively adapts to different sampling settings. Specifically, the image de-aliasing prior is first learned by a deep denoiser trained to remove general white Gaussian noise from synthetic data. Then the learned deep denoiser is plugged into an iterative algorithm for image reconstruction. Results on in vivo data demonstrate that the proposed method provides nice and robust accelerated image reconstruction performance under different undersampling patterns and sampling rates, both visually and quantitatively.

eess.IV↗

arXiv2023

HyperBO+: Pre-training a universal prior for Bayesian optimization with hierarchical Gaussian processes

Bayesian optimization (BO), while proved highly effective for many black-box function optimization tasks, requires practitioners to carefully select priors that well model their functions of interest. Rather than specifying by hand, researchers have investigated transfer learning based methods to automatically learn the priors, e.g. multi-task BO (Swersky et al., 2013), few-shot BO (Wistuba and Grabocka, 2021) and HyperBO (Wang et al., 2022). However, those prior learning methods typically assume that the input domains are the same for all tasks, weakening their ability to use observations on functions with different domains or generalize the learned priors to BO on different search spaces. In this work, we present HyperBO+: a pre-training approach for hierarchical Gaussian processes that enables the same prior to work universally for Bayesian optimization on functions with different domains. We propose a two-step pre-training method and analyze its appealing asymptotic properties and benefits to BO both theoretically and empirically. On real-world hyperparameter tuning tasks that involve multiple search spaces, we demonstrate that HyperBO+ is able to generalize to unseen search spaces and achieves lower regrets than competitive baselines.

cs.LG↗

arXiv2023

DialogueNeRF: Towards Realistic Avatar Face-to-Face Conversation Video Generation

Conversation is an essential component of virtual avatar activities in the metaverse. With the development of natural language processing, textual and vocal conversation generation has achieved a significant breakthrough. However, face-to-face conversations account for the vast majority of daily conversations, while most existing methods focused on single-person talking head generation. In this work, we take a step further and consider generating realistic face-to-face conversation videos. Conversation generation is more challenging than single-person talking head generation, since it not only requires generating photo-realistic individual talking heads but also demands the listener to respond to the speaker. In this paper, we propose a novel unified framework based on neural radiance field (NeRF) to address this task. Specifically, we model both the speaker and listener with a NeRF framework, with different conditions to control individual expressions. The speaker is driven by the audio signal, while the response of the listener depends on both visual and acoustic information. In this way, face-to-face conversation videos are generated between human avatars, with all the interlocutors modeled within the same network. Moreover, to facilitate future research on this task, we collect a new human conversation dataset containing 34 clips of videos. Quantitative and qualitative experiments evaluate our method in different aspects, e.g., image quality, pose sequence trend, and naturalness of the rendering videos. Experimental results demonstrate that the avatars in the resulting videos are able to perform a realistic conversation, and maintain individual styles. All the code, data, and models will be made publicly available.

cs.CV↗

arXiv2023

Flare-Aware Cross-modal Enhancement Network for Multi-spectral Vehicle Re-identification

Multi-spectral vehicle re-identification aims to address the challenge of identifying vehicles in complex lighting conditions by incorporating complementary visible and infrared information. However, in harsh environments, the discriminative cues in RGB and NIR modalities are often lost due to strong flares from vehicle lamps or sunlight, and existing multi-modal fusion methods are limited in their ability to recover these important cues. To address this problem, we propose a Flare-Aware Cross-modal Enhancement Network that adaptively restores flare-corrupted RGB and NIR features with guidance from the flare-immunized thermal infrared spectrum. First, to reduce the influence of locally degraded appearance due to intense flare, we propose a Mutual Flare Mask Prediction module to jointly obtain flare-corrupted masks in RGB and NIR modalities in a self-supervised manner. Second, to use the flare-immunized TI information to enhance the masked RGB and NIR, we propose a Flare-Aware Cross-modal Enhancement module that adaptively guides feature extraction of masked RGB and NIR spectra with prior flare-immunized knowledge from the TI spectrum. Third, to extract common informative semantic information from RGB and NIR, we propose an Inter-modality Consistency loss that enforces semantic consistency between the two modalities. Finally, to evaluate the proposed FACENet in handling intense flare, we introduce a new multi-spectral vehicle re-ID dataset, called WMVEID863, with additional challenges such as motion blur, significant background changes, and particularly intense flare degradation. Comprehensive experiments on both the newly collected dataset and public benchmark multi-spectral vehicle re-ID datasets demonstrate the superior performance of the proposed FACENet compared to state-of-the-art methods, especially in handling strong flares. The code and dataset will be released at this link.

cs.CV↗

arXiv2023

On Evaluating Multilingual Compositional Generalization with Translated Datasets

Compositional generalization allows efficient learning and human-like inductive biases. Since most research investigating compositional generalization in NLP is done on English, important questions remain underexplored. Do the necessary compositional generalization abilities differ across languages? Can models compositionally generalize cross-lingually? As a first step to answering these questions, recent work used neural machine translation to translate datasets for evaluating compositional generalization in semantic parsing. However, we show that this entails critical semantic distortion. To address this limitation, we craft a faithful rule-based translation of the MCWQ dataset from English to Chinese and Japanese. Even with the resulting robust benchmark, which we call MCWQ-R, we show that the distribution of compositions still suffers due to linguistic divergences, and that multilingual models still struggle with cross-lingual compositional generalization. Our dataset and methodology will be useful resources for the study of cross-lingual compositional generalization in other tasks.

cs.CL↗

arXiv2023

Single-pixel p-graded-n junction spectrometers

Ultra-compact spectrometers are becoming increasingly popular for their promising applications in biomedical analysis, environmental monitoring, and food safety. In this work, we report a novel single-pixel-photodetector spectrometer with a spectral range from 480 nm to 820 nm, based on the AlGaAs/GaAs p-graded-n junction with a voltage-tunable optical response. To reconstruct the optical spectrum, we propose a tailored method called Neural Spectral Fields (NSF) that leverages the unique wavelength and bias-dependent responsivity matrix. Our spectrometer achieves a high spectral wavelength accuracy of up to 0.30 nm and a spectral resolution of up to 10 nm. Additionally, we demonstrate the high spectral imaging performance of the device. The compatibility of our demonstration with the standard III-V process greatly accelerates the commercialization of miniaturized spectrometers.

physics.app-ph↗

arXiv2023

Dynamic Enhancement Network for Partial Multi-modality Person Re-identification

Many existing multi-modality studies are based on the assumption of modality integrity. However, the problem of missing arbitrary modalities is very common in real life, and this problem is less studied, but actually important in the task of multi-modality person re-identification (Re-ID). To this end, we design a novel dynamic enhancement network (DENet), which allows missing arbitrary modalities while maintaining the representation ability of multiple modalities, for partial multi-modality person Re-ID. To be specific, the multi-modal representation of the RGB, near-infrared (NIR) and thermal-infrared (TIR) images is learned by three branches, in which the information of missing modalities is recovered by the feature transformation module. Since the missing state might be changeable, we design a dynamic enhancement module, which dynamically enhances modality features according to the missing state in an adaptive manner, to improve the multi-modality representation. Extensive experiments on multi-modality person Re-ID dataset RGBNT201 and vehicle Re-ID dataset RGBNT100 comparing to the state-of-the-art methods verify the effectiveness of our method in complex and changeable environments.

cs.CV↗

arXiv2023

Scalable photonic crystal waveguides with 2 dB component loss

Periodic or gradient subwavelength structures are basic configurations of photonic crystals and metamaterials. The measured linear losses of those nanophotonic devices are well-beyond theoretical predictions. Nanofabrication related geometric inhomogeneity is considered as the primary cause of the deleterious performance. The deep-UV photolithography in CMOS foundry is a large-scale parallel processing, which can significantly suppress the random offsets and thus the optical linear loss. Here we demonstrate ultra-low loss photonic crystal waveguides with a multi-project wafer run through AIM photonics and post-processing. For sub-millimeter long photonic crystal waveguides, 2 dB total loss and 40 dB extinction ratio are observed across dies.

physics.optics↗

arXiv2023

Twisted Heating-Cooling Transition of Near-field Radiation in Drifted Metasurfaces

The magic angle twisted bilayer systems give rise to many exotic phenomena in two-dimensional electronic or photonic platforms. Here, we study the twisted near-field energy radiation between graphene metasurfaces with nonequilibrium drifted Dirac electrons. Anomalously, we find unconventional radiative flux that directs heat from cold to hot. This far-from-equilibrium phenomenon leads to a heating-cooling transition beyond a thermal magic twist angle, facilitated by twist-induced photonic topological transitions. The underlying mechanism is related to the spectrum match and mismatch caused by the cooperation between the non-reciprocal nature of drifted plasmon polaritons and their topological features. Furthermore, we report the unintuitive distance dependence of radiative energy flux under large twist angles. The near-field radiation becomes thermal insulating when increasing to a critical distance, and subsequently reverses the radiative flux to increase the cooling power as the distance increases further. Our results indicate the promising future of nonequilibrium drifted and twisted devices and pave the way towards tunable radiative thermal management.

cond-mat.mes-hall↗