arXiv subjects

Li Zhang

Publications and source records attributed to Li Zhang.

At least 595 records · Page 33Linked to original sources

arXiv2023

Second harmonic and cascaded third harmonic generation in generalized quasi-periodic poled lithium niobate waveguides

Lithium niobate (LN) thin film has recently emerged as an important platform for nonlinear optical investigations for its large $χ^{(2)}$ nonlinear coefficients and ability of light localization. In this paper, we report the first fabrication of LN on insulator (LNOI) ridge waveguides with generalized quasi-periodic poled superlattices using the electric field polarization technique and microfabrication techniques. Benefiting from the abundant reciprocal vectors, we observed efficient second-harmonic and cascaded third-harmonic signals in the same device, with the normalized conversion efficiency 1735% W$^{-1}$cm$^{-2}$ and 0.41% W$^{-2}$cm$^{-4}$, respectively. This work opens a new direction of nonlinear integrated photonics based on LN thin film.

physics.optics↗

arXiv2023

Is It Enough to Recommend Tasks to Newcomers? Understanding Mentoring on Good First Issues

Newcomers are critical for the success and continuity of open source software (OSS) projects. To attract newcomers and facilitate their onboarding, many OSS projects recommend tasks for newcomers, such as good first issues (GFIs). Previous studies have preliminarily investigated the effects of GFIs and techniques to identify suitable GFIs. However, it is still unclear whether just recommending tasks is enough and how significant mentoring is for newcomers. To better understand mentoring in OSS communities, we analyze the resolution process of 48,402 GFIs from 964 repositories through a mix-method approach. We investigate the extent, the mentorship structures, the discussed topics, and the relevance of expert involvement. We find that $\sim$70\% of GFIs have expert participation, with each GFI usually having one expert who makes two comments. Half of GFIs will receive their first expert comment within 8.5 hours after a newcomer comment. Through analysis of the collaboration networks of newcomers and experts, we observe that community mentorship presents four types of structure: centralized mentoring, decentralized mentoring, collaborative mentoring, and distributed mentoring. As for discussed topics, we identify 14 newcomer challenges and 18 expert mentoring content. By fitting the generalized linear models, we find that expert involvement positively correlates with newcomers' successful contributions but negatively correlates with newcomers' retention. Our study manifests the status and significance of mentoring in the OSS projects, which provides rich practical implications for optimizing the mentoring process and helping newcomers contribute smoothly and successfully.

cs.SE↗

arXiv2023

Moiré-induced bandgap tuning by varying electric dipole in InSe/CuSe vertical heterostructure

The stacked two layered materials with a lattice constant mismatch and/or with twist angle relative to each other can create a moiré pattern, modulating the electronic properties of the pristine materials. Here, we combine scanning tunneling microscopy/spectroscopy and density functional theory calculations to investigate the moiré potential induced bandgap tuning in InSe/CuSe vertical heterostructure synthesized by a two-step of molecular beam epitaxy. Scanning tunneling microscopy measurements demonstrate the heterostructure with a superlattice periodicity of ~3.48nm and a twist angle of about 11° between the monolayers. Scanning tunneling spectroscopy record on the different stacking sites of the heterostructure reveals the bandgap of the InSe is location-dependent and a variation of 400 meV is observed. Density functional theory calculations reveal that the moiré-induce electric dipole in the monolayer InSe is the key factor for tuning the bandgap. Besides, charge transfer between CuSe and InSe also contributes to the bandgap variation due to its stacking related. We also show that the moiré potential not only can tune the bandgap of InSe but also can vanish the Dirac nodal line of CuSe in some stackings. Our explorations provide valuable information in understanding the electronic properties of the twodimensional moiré materials.

cond-mat.mtrl-sci↗

arXiv2023

PolarFormer: Multi-camera 3D Object Detection with Polar Transformer

3D object detection in autonomous driving aims to reason "what" and "where" the objects of interest present in a 3D world. Following the conventional wisdom of previous 2D object detection, existing methods often adopt the canonical Cartesian coordinate system with perpendicular axis. However, we conjugate that this does not fit the nature of the ego car's perspective, as each onboard camera perceives the world in shape of wedge intrinsic to the imaging geometry with radical (non-perpendicular) axis. Hence, in this paper we advocate the exploitation of the Polar coordinate system and propose a new Polar Transformer (PolarFormer) for more accurate 3D object detection in the bird's-eye-view (BEV) taking as input only multi-camera 2D images. Specifically, we design a cross attention based Polar detection head without restriction to the shape of input structure to deal with irregular Polar grids. For tackling the unconstrained object scale variations along Polar's distance dimension, we further introduce a multi-scalePolar representation learning strategy. As a result, our model can make best use of the Polar representation rasterized via attending to the corresponding image observation in a sequence-to-sequence fashion subject to the geometric constraints. Thorough experiments on the nuScenes dataset demonstrate that our PolarFormer outperforms significantly state-of-the-art 3D object detection alternatives.

cs.CV↗

arXiv2023

PMP: Privacy-Aware Matrix Profile against Sensitive Pattern Inference for Time Series

Recent rapid development of sensor technology has allowed massive fine-grained time series (TS) data to be collected and set the foundation for the development of data-driven services and applications. During the process, data sharing is often involved to allow the third-party modelers to perform specific time series data mining (TSDM) tasks based on the need of data owner. The high resolution of TS brings new challenges in protecting privacy. While meaningful information in high-resolution TS shifts from concrete point values to local shape-based segments, numerous research have found that long shape-based patterns could contain more sensitive information and may potentially be extracted and misused by a malicious third party. However, the privacy issue for TS patterns is surprisingly seldom explored in privacy-preserving literature. In this work, we consider a new privacy-preserving problem: preventing malicious inference on long shape-based patterns while preserving short segment information for the utility task performance. To mitigate the challenge, we investigate an alternative approach by sharing Matrix Profile (MP), which is a non-linear transformation of original data and a versatile data structure that supports many data mining tasks. We found that while MP can prevent concrete shape leakage, the canonical correlation in MP index can still reveal the location of sensitive long pattern. Based on this observation, we design two attacks named Location Attack and Entropy Attack to extract the pattern location from MP. To further protect MP from these two attacks, we propose a Privacy-Aware Matrix Profile (PMP) via perturbing the local correlation and breaking the canonical correlation in MP index vector. We evaluate our proposed PMP against baseline noise-adding methods through quantitative analysis and real-world case studies to show the effectiveness of the proposed method.

cs.LG↗

arXiv2023

Language Models are Drummers: Drum Composition with Natural Language Pre-Training

Automatic music generation with artificial intelligence typically requires a large amount of data which is hard to obtain for many less common genres and musical instruments. To tackle this issue, we present ongoing work and preliminary findings on the possibility for deep models to transfer knowledge from language to music, by finetuning large language models pre-trained on a massive text corpus on only hundreds of MIDI files of drum performances. We show that by doing so, one of the largest, state-of-the-art models (GPT3) is capable of generating reasonable drum grooves, while models that are not pre-trained (Transformer) shows no such ability beyond naive repetition. Evaluating generated music is a challenging task, more so is evaluating drum grooves with little precedence in literature. Hence, we propose a tailored structural evaluation method and analyze drum grooves produced by GPT3 compared to those played by human professionals, exposing the strengths and weaknesses of such generation by language-to-music transfer. Our findings suggest that language-to-music transfer learning with large language models is viable and promising.

cs.SD↗

arXiv2023

Gaussian Process Modeling Blazar Multiwavelength Variability: Indirectly Resolving Jet Structure

Blazar jet structure can be indirectly resolved by analyzing the multiwavelength variability. In this work, we analyze the long-term variability of blazars in radio, optical and X-ray energies with the Gaussian process (GP) method. The multiwavelength variability can be successfully characterized by the damped-random walk (DRW) model. The nonthermal optical characteristic timescales of 38 blazars are statistically consistent with the $γ$-ray characteristic timescales of 22 blazars. For three individuals (3C 273, PKS 1510-089, and BL Lac), the nonthermal optical, X-ray, and $γ$-ray characteristic timescales are also consistent within the measured 95$\%$ errors, but the radio timescale of 3C 273 is too large to be constrained by the decade-long light curve. The synchrotron and inverse-Compton emissions have the same power spectral density, suggesting that the long-term jet variability is irrelevant to the emission mechanism. In the plot of the rest-frame timescale versus black hole mass, the optical-$γ$-ray timescales of the jet variability occupy almost the same space with the timescales of accretion disk emission from normal quasars, which may imply that the long-term variabilities of the jet and accretion disk are driven by the same physical process. It is suggested that the nonthermal optical-X-ray and $γ$-ray emissions are produced in the same region, while the radio core which can be resolved by very-long-baseline interferometry locates at a far more distant region from the black hole. Our study suggests a new methodology for comparing thermal and nonthermal emissions, which is achieved by using the standard GP method.

astro-ph.HE↗

arXiv2022

CholecTriplet2021: A benchmark challenge for surgical action triplet recognition

Context-aware decision support in the operating room can foster surgical safety and efficiency by leveraging real-time feedback from surgical workflow analysis. Most existing works recognize surgical activities at a coarse-grained level, such as phases, steps or events, leaving out fine-grained interaction details about the surgical activity; yet those are needed for more helpful AI assistance in the operating room. Recognizing surgical actions as triplets of combination delivers comprehensive details about the activities taking place in surgical videos. This paper presents CholecTriplet2021: an endoscopic vision challenge organized at MICCAI 2021 for the recognition of surgical action triplets in laparoscopic videos. The challenge granted private access to the large-scale CholecT50 dataset, which is annotated with action triplet information. In this paper, we present the challenge setup and assessment of the state-of-the-art deep learning methods proposed by the participants during the challenge. A total of 4 baseline methods from the challenge organizers and 19 new deep learning algorithms by competing teams are presented to recognize surgical action triplets directly from surgical videos, achieving mean average precision (mAP) ranging from 4.2% to 38.1%. This study also analyzes the significance of the results obtained by the presented approaches, performs a thorough methodological comparison between them, in-depth result analysis, and proposes a novel ensemble method for enhanced recognition. Our analysis shows that surgical workflow analysis is not yet solved, and also highlights interesting directions for future research on fine-grained surgical activity recognition which is of utmost importance for the development of AI in surgery.

cs.CV↗

arXiv2022

Generalized-Equiangular Geometry CT: Concept and Shift-Invariant FBP Algorithms

With advanced X-ray source and detector technologies being continuously developed, non-traditional CT geometries have been widely explored. Generalized-Equiangular Geometry CT (GEGCT) architecture, in which an X-ray source might be positioned radially far away from the focus of arced detector array that is equiangularly spaced, is of importance in many novel CT systems and designs. GEGCT, unfortunately, has no theoretically exact and shift-invariant analytical image reconstruction algorithm in general. In this study, to obtain fast and accurate reconstruction from GEGCT and to promote its system design and optimization, an in-depth investigation on a group of approximate Filtered BackProjection (FBP) algorithms with a variety of weighting strategies has been conducted. The architecture of GEGCT is first presented and characterized by using a normalized-radial-offset distance (NROD). Next, shift-invariant weighted FBP-type algorithms are derived in a unified framework, with pre-filtering, filtering, and post-filtering weights. Three viable weighting strategies are then presented including a classic one developed by Besson in the literature and two new ones generated from a curvature fitting and from an empirical formula, where all of the three weights can be expressed as certain functions of NROD. After that, an analysis of reconstruction accuracy is conducted with a wide range of NROD. We further stretch the weighted FBP-type algorithms to GEGCT with dynamic NROD. Finally, the weighted FBP algorithm for GEGCT is extended to a three-dimensional form in the case of cone-beam scan with a cylindrical detector array.

physics.med-ph↗

arXiv2022

Unsupervised Time-Aware Sampling Network with Deep Reinforcement Learning for EEG-Based Emotion Recognition

Recognizing human emotions from complex, multivariate, and non-stationary electroencephalography (EEG) time series is essential in affective brain-computer interface. However, because continuous labeling of ever-changing emotional states is not feasible in practice, existing methods can only assign a fixed label to all EEG timepoints in a continuous emotion-evoking trial, which overlooks the highly dynamic emotional states and highly non-stationary EEG signals. To solve the problems of high reliance on fixed labels and ignorance of time-changing information, in this paper we propose a time-aware sampling network (TAS-Net) using deep reinforcement learning (DRL) for unsupervised emotion recognition, which is able to detect key emotion fragments and disregard irrelevant and misleading parts. Extensive experiments are conducted on three public datasets (SEED, DEAP, and MAHNOB-HCI) for emotion recognition using leave-one-subject-out cross-validation, and the results demonstrate the superiority of the proposed method against previous unsupervised emotion recognition methods.

cs.HC↗

arXiv2022

CrossMoDA 2021 challenge: Benchmark of Cross-Modality Domain Adaptation techniques for Vestibular Schwannoma and Cochlea Segmentation

Domain Adaptation (DA) has recently raised strong interests in the medical imaging community. While a large variety of DA techniques has been proposed for image segmentation, most of these techniques have been validated either on private datasets or on small publicly available datasets. Moreover, these datasets mostly addressed single-class problems. To tackle these limitations, the Cross-Modality Domain Adaptation (crossMoDA) challenge was organised in conjunction with the 24th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2021). CrossMoDA is the first large and multi-class benchmark for unsupervised cross-modality DA. The challenge's goal is to segment two key brain structures involved in the follow-up and treatment planning of vestibular schwannoma (VS): the VS and the cochleas. Currently, the diagnosis and surveillance in patients with VS are performed using contrast-enhanced T1 (ceT1) MRI. However, there is growing interest in using non-contrast sequences such as high-resolution T2 (hrT2) MRI. Therefore, we created an unsupervised cross-modality segmentation benchmark. The training set provides annotated ceT1 (N=105) and unpaired non-annotated hrT2 (N=105). The aim was to automatically perform unilateral VS and bilateral cochlea segmentation on hrT2 as provided in the testing set (N=137). A total of 16 teams submitted their algorithm for the evaluation phase. The level of performance reached by the top-performing teams is strikingly high (best median Dice - VS:88.4%; Cochleas:85.7%) and close to full supervision (median Dice - VS:92.5%; Cochleas:87.7%). All top-performing methods made use of an image-to-image translation approach to transform the source-domain images into pseudo-target-domain images. A segmentation network was then trained using these generated images and the manual annotations provided for the source image.

eess.IV↗

arXiv2022

DeepInteraction: 3D Object Detection via Modality Interaction

Existing top-performance 3D object detectors typically rely on the multi-modal fusion strategy. This design is however fundamentally restricted due to overlooking the modality-specific useful information and finally hampering the model performance. To address this limitation, in this work we introduce a novel modality interaction strategy where individual per-modality representations are learned and maintained throughout for enabling their unique characteristics to be exploited during object detection. To realize this proposed strategy, we design a DeepInteraction architecture characterized by a multi-modal representational interaction encoder and a multi-modal predictive interaction decoder. Experiments on the large-scale nuScenes dataset show that our proposed method surpasses all prior arts often by a large margin. Crucially, our method is ranked at the first position at the highly competitive nuScenes object detection leaderboard.

cs.CV↗

arXiv2022

Swimming of the midge larva: principles and tricks of locomotion at intermediate Reynolds number

At the millimeter scale and in the intermediate Reynolds number (Re) regime, the midge and mosquito larvae can reach swimming speeds of more than one body length per cycle performing a "figure-of-8" gait, in which their elongated bodies periodically bend nearly into circles and then fully unfold. To elucidate the propulsion mechanism of this cycle of motion, we conducted a 3D numerical study which investigates the hydrodynamics of undergoing the prescribed kinematics. Novel propulsion mechanisms, such as modulating the body deformation rate to dynamically increase the maximum net propulsion force, using asymmetric kinematics to generate torque and the appropriate rotation, and controlling the radius of the curled body to manipulate the moment of inertia. The figure-of-8 gait is found to achieve propulsion at a wide range of Re, but is most effective at intermediate Re. The results were further validated experimentally, via the development of a soft millimeter-sized robot that can reach comparable speeds using the figure-of-8 gait.

physics.flu-dyn↗

arXiv2022

Accelerating Score-based Generative Models with Preconditioned Diffusion Sampling

Score-based generative models (SGMs) have recently emerged as a promising class of generative models. However, a fundamental limitation is that their inference is very slow due to a need for many (e.g., 2000) iterations of sequential computations. An intuitive acceleration method is to reduce the sampling iterations which however causes severe performance degradation. We investigate this problem by viewing the diffusion sampling process as a Metropolis adjusted Langevin algorithm, which helps reveal the underlying cause to be ill-conditioned curvature. Under this insight, we propose a model-agnostic preconditioned diffusion sampling (PDS) method that leverages matrix preconditioning to alleviate the aforementioned problem. Crucially, PDS is proven theoretically to converge to the original target distribution of a SGM, no need for retraining. Extensive experiments on three image datasets with a variety of resolutions and diversity validate that PDS consistently accelerates off-the-shelf SGMs whilst maintaining the synthesis quality. In particular, PDS can accelerate by up to 29x on more challenging high resolution (1024x1024) image generation.

cs.CV↗

arXiv2022

MSV Challenge 2022: NPU-HC Speaker Verification System for Low-resource Indian Languages

This report describes the NPU-HC speaker verification system submitted to the O-COCOSDA Multi-lingual Speaker Verification (MSV) Challenge 2022, which focuses on developing speaker verification systems for low-resource Asian languages. We participate in the I-MSV track, which aims to develop speaker verification systems for various Indian languages. In this challenge, we first explore different neural network frameworks for low-resource speaker verification. Then we leverage vanilla fine-tuning and weight transfer fine-tuning to transfer the out-domain pre-trained models to the in-domain Indian dataset. Specifically, the weight transfer fine-tuning aims to constrain the distance of the weights between the pre-trained model and the fine-tuned model, which takes advantage of the previously acquired discriminative ability from the large-scale out-domain datasets and avoids catastrophic forgetting and overfitting at the same time. Finally, score fusion is adopted to further improve performance. Together with the above contributions, we obtain 0.223% EER on the public evaluation set, ranking 2nd place on the leaderboard. On the private evaluation set, the EER of our submitted system is 2.123% and 0.630% for the constrained and unconstrained sub-tasks of the I-MSV track, leading to the 1st and 3rd place in the ranking, respectively.

eess.AS↗

arXiv2022

Automated Generating Natural Language Requirements based on Domain Ontology

Software requirements specification is undoubtedly critical for the whole software life-cycle. Nowadays, writing software requirements specifications primarily depends on human work. Although massive studies have been proposed to fasten the process via proposing advanced elicitation and analysis techniques, it is still a time-consuming and error-prone task that needs to take domain knowledge and business information into consideration. In this paper, we propose an approach, named ReqGen, which can provide recommendations by automatically generating natural language requirements specifications based on certain given keywords. Specifically, ReqGen consists of three critical steps. First, keywords-oriented knowledge is selected from domain ontology and is injected to the basic Unified pre-trained Language Model (UniLM) for domain fine-tuning. Second, a copy mechanism is integrated to ensure the occurrence of keywords in the generated statements. Finally, a requirement syntax constrained decoding is designed to close the semantic and syntax distance between the candidate and reference specifications. Experiments on two public datasets from different groups and domains show that ReqGen outperforms six popular natural language generation approaches with respect to the hard constraint of keywords(phrases) inclusion, BLEU, ROUGE and syntax compliance. We believe that ReqGen can promote the efficiency and intelligence of specifying software requirements.

cs.SE↗

arXiv2022

Panoramic Video Salient Object Detection with Ambisonic Audio Guidance

Video salient object detection (VSOD), as a fundamental computer vision problem, has been extensively discussed in the last decade. However, all existing works focus on addressing the VSOD problem in 2D scenarios. With the rapid development of VR devices, panoramic videos have been a promising alternative to 2D videos to provide immersive feelings of the real world. In this paper, we aim to tackle the video salient object detection problem for panoramic videos, with their corresponding ambisonic audios. A multimodal fusion module equipped with two pseudo-siamese audio-visual context fusion (ACF) blocks is proposed to effectively conduct audio-visual interaction. The ACF block equipped with spherical positional encoding enables the fusion in the 3D context to capture the spatial correspondence between pixels and sound sources from the equirectangular frames and ambisonic audios. Experimental results verify the effectiveness of our proposed components and demonstrate that our method achieves state-of-the-art performance on the ASOD60K dataset.

cs.CV↗

arXiv2022

Robust Time Series Chain Discovery with Incremental Nearest Neighbors

Time series motif discovery has been a fundamental task to identify meaningful repeated patterns in time series. Recently, time series chains were introduced as an expansion of time series motifs to identify the continuous evolving patterns in time series data. Informally, a time series chain (TSC) is a temporally ordered set of time series subsequences, in which every subsequence is similar to the one that precedes it, but the last and the first can be arbitrarily dissimilar. TSCs are shown to be able to reveal latent continuous evolving trends in the time series, and identify precursors of unusual events in complex systems. Despite its promising interpretability, unfortunately, we have observed that existing TSC definitions lack the ability to accurately cover the evolving part of a time series: the discovered chains can be easily cut by noise and can include non-evolving patterns, making them impractical in real-world applications. Inspired by a recent work that tracks how the nearest neighbor of a time series subsequence changes over time, we introduce a new TSC definition which is much more robust to noise in the data, in the sense that they can better locate the evolving patterns while excluding the non-evolving ones. We further propose two new quality metrics to rank the discovered chains. With extensive empirical evaluations, we demonstrate that the proposed TSC definition is significantly more robust to noise than the state of the art, and the top ranked chains discovered can reveal meaningful regularities in a variety of real world datasets.

cs.LG↗