Search arXiv⌕ Search

arXiv subjects

Tao Zhang

Publications and source records attributed to Tao Zhang.

At least 397 records · Page 22Linked to original sources
arXiv2024

Efficient fault-tolerant implementations of non-Clifford gates with reconfigurable atom arrays

To achieve scalable universal quantum computing, we need to implement a universal set of logical gates fault-tolerantly, for which the main difficulty lies with non-Clifford gates. We demonstrate that several characteristic features of the reconfigurable atom array platform are inherently well-suited for addressing this key challenge, potentially leading to significant advantages in fidelity and efficiency. Specifically, we consider a series of different strategies including magic state distillation, concatenated code array, and fault-tolerant logical multi-controlled-$Z$ gates, leveraging key platform features such as non-local connectivity, parallel gate action, collective mobility, and native multi-controlled-$Z$ gates. Our analysis provides valuable insights into the efficient experimental realization of logical gates, serving as a guide for the full-cycle demonstration of fault-tolerant quantum computation with reconfigurable atom arrays.

quant-ph↗
arXiv2024

On Practical Diversified Recommendation with Controllable Category Diversity Framework

Recommender systems have made significant strides in various industries, primarily driven by extensive efforts to enhance recommendation accuracy. However, this pursuit of accuracy has inadvertently given rise to echo chamber/filter bubble effects. Especially in industry, it could impair user's experiences and prevent user from accessing a wider range of items. One of the solutions is to take diversity into account. However, most of existing works focus on user's explicit preferences, while rarely exploring user's non-interaction preferences. These neglected non-interaction preferences are especially important for broadening user's interests in alleviating echo chamber/filter bubble effects.Therefore, in this paper, we first define diversity as two distinct definitions, i.e., user-explicit diversity (U-diversity) and user-item non-interaction diversity (N-diversity) based on user historical behaviors. Then, we propose a succinct and effective method, named as Controllable Category Diversity Framework (CCDF) to achieve both high U-diversity and N-diversity simultaneously.Specifically, CCDF consists of two stages, User-Category Matching and Constrained Item Matching. The User-Category Matching utilizes the DeepU2C model and a combined loss to capture user's preferences in categories, and then selects the top-$K$ categories with a controllable parameter $K$.These top-$K$ categories will be used as trigger information in Constrained Item Matching. Offline experimental results show that our proposed DeepU2C outperforms state-of-the-art diversity-oriented methods, especially on N-diversity task. The whole framework is validated in a real-world production environment by conducting online A/B testing.

cs.IR↗
arXiv2024

Solving separable convex optimization problems: Faster prediction-correction framework

He and Yuan's prediction-correction framework [SIAM J. Numer. Anal. 50: 700-709, 2012] is able to provide convergent algorithms for solving separable convex optimization problems at a rate of $O(1/t)$ ($t$ represents iteration times) in both ergodic (the average of iteration) and pointwise senses. This paper presents a faster prediction-correction framework at a rate of $O(1/t)$ in the non-ergodic sense (the last iteration) and $O(1/t^2)$ in the pointwise sense. Based the faster prediction-correction framework, we give three faster algorithms which enjoy $O(1/t)$ in the non-ergodic sense of primal-dual gap and $O(1/t^2)$ in the pointwise sense. The first algorithm updates dual variable twice when solving two-block separable convex optimization with equality linear constraints. The second algorithm solves multi-block separable convex optimization problems with linear equality constraints in Gauss-Seidel way. The third algorithm solves minmax problems with larger step sizes.

math.OC↗
arXiv2024

Episodic-free Task Selection for Few-shot Learning

Episodic training is a mainstream training strategy for few-shot learning. In few-shot scenarios, however, this strategy is often inferior to some non-episodic training strategy, e. g., Neighbourhood Component Analysis (NCA), which challenges the principle that training conditions must match testing conditions. Thus, a question is naturally asked: How to search for episodic-free tasks for better few-shot learning? In this work, we propose a novel meta-training framework beyond episodic training. In this framework, episodic tasks are not used directly for training, but for evaluating the effectiveness of some selected episodic-free tasks from a task set that are performed for training the meta-learners. The selection criterion is designed with the affinity, which measures the degree to which loss decreases when executing the target tasks after training with the selected tasks. In experiments, the training task set contains some promising types, e. g., contrastive learning and classification, and the target few-shot tasks are achieved with the nearest centroid classifiers on the miniImageNet, tiered-ImageNet and CIFAR-FS datasets. The experimental results demonstrate the effectiveness of our approach.

cs.LG↗
arXiv2024

New results on sparse representations in unions of orthonormal bases

The problem of sparse representation has significant applications in signal processing. The spark of a dictionary plays a crucial role in the study of sparse representation. Donoho and Elad initially explored the spark, and they provided a general lower bound. When the dictionary is a union of several orthonormal bases, Gribonval and Nielsen presented an improved lower bound for spark. In this paper, we introduce a new construction of dictionary, achieving the spark bound given by Gribonval and Nielsen. Our result extends Shen et al.' s findings [IEEE Trans. Inform. Theory, vol. 68, pp. 4230--4243, 2022].

math.CO↗
arXiv2024

A Positive-Unlabeled Metric Learning Framework for Document-Level Relation Extraction with Incomplete Labeling

The goal of document-level relation extraction (RE) is to identify relations between entities that span multiple sentences. Recently, incomplete labeling in document-level RE has received increasing attention, and some studies have used methods such as positive-unlabeled learning to tackle this issue, but there is still a lot of room for improvement. Motivated by this, we propose a positive-augmentation and positive-mixup positive-unlabeled metric learning framework (P3M). Specifically, we formulate document-level RE as a metric learning problem. We aim to pull the distance closer between entity pair embedding and their corresponding relation embedding, while pushing it farther away from the none-class relation embedding. Additionally, we adapt the positive-unlabeled learning to this loss objective. In order to improve the generalizability of the model, we use dropout to augment positive samples and propose a positive-none-class mixup method. Extensive experiments show that P3M improves the F1 score by approximately 4-10 points in document-level RE with incomplete labeling, and achieves state-of-the-art results in fully labeled scenarios. Furthermore, P3M has also demonstrated robustness to prior estimation bias in incomplete labeled scenarios.

cs.CL↗
arXiv2024

Game Rewards Vulnerabilities: Software Vulnerability Detection with Zero-Sum Game and Prototype Learning

Recent years have witnessed a growing focus on automated software vulnerability detection. Notably, deep learning (DL)-based methods, which employ source code for the implicit acquisition of vulnerability patterns, have demonstrated superior performance compared to other approaches. However, the DL-based approaches are still hard to capture the vulnerability-related information from the whole code snippet, since the vulnerable parts usually account for only a small proportion. As evidenced by our experiments, the approaches tend to excessively emphasize semantic information, potentially leading to limited vulnerability detection performance in practical scenarios. First, they cannot well distinguish between the code snippets before (i.e., vulnerable code) and after (i.e., non-vulnerable code) developers' fixes due to the minimal code changes. Besides, substituting user-defined identifiers with placeholders (e.g., "VAR1" and "FUN1") in obvious performance degradation at up to 14.53% with respect to the F1 score. To mitigate these issues, we propose to leverage the vulnerable and corresponding fixed code snippets, in which the minimal changes can provide hints about semantic-agnostic features for vulnerability detection. In this paper, we propose a software vulneRability dEteCtion framework with zerO-sum game and prototype learNing, named RECON. In RECON, we propose a zero-sum game construction module. Distinguishing the vulnerable code from the corresponding fixed code is regarded as one player (i.e. Calibrator), while the conventional vulnerability detection is another player (i.e. Detector) in the zero-sum game. The goal is to capture the semantic-agnostic features of the first player for enhancing the second player's performance for vulnerability detection. Experiments on the public benchmark dataset show that RECON outperforms the state-of-the-art baseline by 6.29% in F1 score.

cs.SE↗
arXiv2024

Training and Serving System of Foundation Models: A Comprehensive Survey

Foundation models (e.g., ChatGPT, DALL-E, PengCheng Mind, PanGu-$Σ$) have demonstrated extraordinary performance in key technological areas, such as natural language processing and visual recognition, and have become the mainstream trend of artificial general intelligence. This has led more and more major technology giants to dedicate significant human and financial resources to actively develop their foundation model systems, which drives continuous growth of these models' parameters. As a result, the training and serving of these models have posed significant challenges, including substantial computing power, memory consumption, bandwidth demands, etc. Therefore, employing efficient training and serving strategies becomes particularly crucial. Many researchers have actively explored and proposed effective methods. So, a comprehensive survey of them is essential for system developers and researchers. This paper extensively explores the methods employed in training and serving foundation models from various perspectives. It provides a detailed categorization of these state-of-the-art methods, including finer aspects such as network, computing, and storage. Additionally, the paper summarizes the challenges and presents a perspective on the future development direction of foundation model systems. Through comprehensive discussion and analysis, it hopes to provide a solid theoretical basis and practical guidance for future research and applications, promoting continuous innovation and development in foundation model systems.

cs.AI↗
arXiv2023

On the proximal point algorithms for solving the monotone inclusion problem

We consider finding a zero point of the maximally monotone operator $T$. First, instead of using the proximal point algorithm (PPA) for this purpose, we employ PPA to solve its Yosida regularization $T_λ$. Then, based on an $O(a_{k+1})$ ($a_{k+1}\geq \varepsilon>0$) resolvent index of $T$, it turns out that we can establish a convergence rate of $O (1/{\sqrt{\sum_{i=0}^{k}a_{i+1}^2}})$ for both the $\|T_λ(\cdot)\|$ and the gap function $\mathtt{Gap}(\cdot)$ in the non-ergodic sense, and $O(1/\sum_{i=0}^{k}a_{i+1})$ for $\mathtt{Gap}(\cdot)$ in the ergodic sense. Second, to enhance the convergence rate of the newly-proposed PPA, we introduce an accelerated variant called the Contracting PPA. By utilizing a resolvent index of $T$ bounded by $O(a_{k+1})$ ($a_{k+1}\geq \varepsilon>0$), we establish a convergence rate of $O(1/\sum_{i=0}^{k}a_{i+1})$ for both $\|T_λ(\cdot)\|$ and $\mathtt {Gap}(\cdot)$, considering the non-ergodic sense. Third, to mitigate the limitation that the Contracting PPA lacks a convergence guarantee, we propose two additional versions of the algorithm. These novel approaches not only ensure guaranteed convergence but also provide sublinear and linear convergence rates for both $\|T_λ(\cdot)\|$ and $\mathtt {Gap}(\cdot)$, respectively, in the non-ergodic sense.

math.OC↗
arXiv2023

Decoupling Representation and Knowledge for Few-Shot Intent Classification and Slot Filling

Few-shot intent classification and slot filling are important but challenging tasks due to the scarcity of finely labeled data. Therefore, current works first train a model on source domains with sufficiently labeled data, and then transfer the model to target domains where only rarely labeled data is available. However, experience transferring as a whole usually suffers from gaps that exist among source domains and target domains. For instance, transferring domain-specific-knowledge-related experience is difficult. To tackle this problem, we propose a new method that explicitly decouples the transferring of general-semantic-representation-related experience and the domain-specific-knowledge-related experience. Specifically, for domain-specific-knowledge-related experience, we design two modules to capture intent-slot relation and slot-slot relation respectively. Extensive experiments on Snips and FewJoint datasets show that our method achieves state-of-the-art performance. The method improves the joint accuracy metric from 27.72% to 42.20% in the 1-shot setting, and from 46.54% to 60.79% in the 5-shot setting.

cs.CL↗
arXiv2023

DVIS++: Improved Decoupled Framework for Universal Video Segmentation

We present the \textbf{D}ecoupled \textbf{VI}deo \textbf{S}egmentation (DVIS) framework, a novel approach for the challenging task of universal video segmentation, including video instance segmentation (VIS), video semantic segmentation (VSS), and video panoptic segmentation (VPS). Unlike previous methods that model video segmentation in an end-to-end manner, our approach decouples video segmentation into three cascaded sub-tasks: segmentation, tracking, and refinement. This decoupling design allows for simpler and more effective modeling of the spatio-temporal representations of objects, especially in complex scenes and long videos. Accordingly, we introduce two novel components: the referring tracker and the temporal refiner. These components track objects frame by frame and model spatio-temporal representations based on pre-aligned features. To improve the tracking capability of DVIS, we propose a denoising training strategy and introduce contrastive learning, resulting in a more robust framework named DVIS++. Furthermore, we evaluate DVIS++ in various settings, including open vocabulary and using a frozen pre-trained backbone. By integrating CLIP with DVIS++, we present OV-DVIS++, the first open-vocabulary universal video segmentation framework. We conduct extensive experiments on six mainstream benchmarks, including the VIS, VSS, and VPS datasets. Using a unified architecture, DVIS++ significantly outperforms state-of-the-art specialized methods on these benchmarks in both close- and open-vocabulary settings. Code:~\url{https://github.com/zhang-tao-whu/DVIS_Plus}.

cs.CV↗
arXiv2023

Towards Accelerating Particle-Resolved Direct Numerical Simulation with Neural Operators

We present our ongoing work aimed at accelerating a particle-resolved direct numerical simulation model designed to study aerosol-cloud-turbulence interactions. The dynamical model consists of two main components - a set of fluid dynamics equations for air velocity, temperature, and humidity, coupled with a set of equations for particle (i.e., cloud droplet) tracing. Rather than attempting to replace the original numerical solution method in its entirety with a machine learning (ML) method, we consider developing a hybrid approach. We exploit the potential of neural operator learning to yield fast and accurate surrogate models and, in this study, develop such surrogates for the velocity and vorticity fields. We discuss results from numerical experiments designed to assess the performance of ML architectures under consideration as well as their suitability for capturing the behavior of relevant dynamical systems.

physics.flu-dyn↗
arXiv2023

Search for Dark-Matter-Nucleon Interactions with a Dark Mediator in PandaX-4T

We report results of a search for dark-matter-nucleon interactions via a dark mediator using optimized low-energy data from the PandaX-4T liquid xenon experiment. With the ionization-signal-only data and utilizing the Migdal effect, we set the most stringent limits on the cross section for dark matter masses ranging from 30~$\rm{MeV/c^2}$ to 2~$\rm{GeV/c^2}$. Under the assumption that the dark mediator is a dark photon that decays into scalar dark matter pairs in the early Universe, we rule out significant parameter space of such thermal relic dark-matter model.

hep-ex↗
arXiv2023

A family of Barzilai-Borwein steplengths from the viewpoint of scaled total least squares

The Barzilai-Borwein (BB) steplengths play great roles in practical gradient methods for solving unconstrained optimization problems. Motivated by the observation that the two well-known BB steplengths correspond to the ordinary and the data least squares, respectively, we present a family of BB steplengths from the viewpoint of scaled total least squares. Numerical experiments demonstrate that a high performance can be received by a carefully-selected BB steplength in the new family.

math.OC↗
arXiv2023

PAD: Self-Supervised Pre-Training with Patchwise-Scale Adapter for Infrared Images

Self-supervised learning (SSL) for RGB images has achieved significant success, yet there is still limited research on SSL for infrared images, primarily due to three prominent challenges: 1) the lack of a suitable large-scale infrared pre-training dataset, 2) the distinctiveness of non-iconic infrared images rendering common pre-training tasks like masked image modeling (MIM) less effective, and 3) the scarcity of fine-grained textures making it particularly challenging to learn general image features. To address these issues, we construct a Multi-Scene Infrared Pre-training (MSIP) dataset comprising 178,756 images, and introduce object-sensitive random RoI cropping, an image preprocessing method, to tackle the challenge posed by non-iconic images. To alleviate the impact of weak textures on feature learning, we propose a pre-training paradigm called Pre-training with ADapter (PAD), which uses adapters to learn domain-specific features while freezing parameters pre-trained on ImageNet to retain the general feature extraction capability. This new paradigm is applicable to any transformer-based SSL method. Furthermore, to achieve more flexible coordination between pre-trained and newly-learned features in different layers and patches, a patchwise-scale adapter with dynamically learnable scale factors is introduced. Extensive experiments on three downstream tasks show that PAD, with only 1.23M pre-trainable parameters, outperforms other baseline paradigms including continual full pre-training on MSIP. Our code and dataset are available at https://github.com/casiatao/PAD.

cs.CV↗
arXiv2023

Technical Report for Argoverse Challenges on Unified Sensor-based Detection, Tracking, and Forecasting

This report presents our Le3DE2E solution for unified sensor-based detection, tracking, and forecasting in Argoverse Challenges at CVPR 2023 Workshop on Autonomous Driving (WAD). We propose a unified network that incorporates three tasks, including detection, tracking, and forecasting. This solution adopts a strong Bird's Eye View (BEV) encoder with spatial and temporal fusion and generates unified representations for multi-tasks. The solution was tested in the Argoverse 2 sensor dataset to evaluate the detection, tracking, and forecasting of 26 object categories. We achieved 1st place in Detection, Tracking, and Forecasting on the E2E Forecasting track in Argoverse Challenges at CVPR 2023 WAD.

cs.CV↗
arXiv2023

JPAVE: A Generation and Classification-based Model for Joint Product Attribute Prediction and Value Extraction

Product attribute value extraction is an important task in e-Commerce which can help several downstream applications such as product search and recommendation. Most previous models handle this task using sequence labeling or question answering method which rely on the sequential position information of values in the product text and are vulnerable to data discrepancy between training and testing. This limits their generalization ability to real-world scenario in which each product can have multiple descriptions across various shopping platforms with different composition of text and style. They also have limited zero-shot ability to new values. In this paper, we propose a multi-task learning model with value generation/classification and attribute prediction called JPAVE to predict values without the necessity of position information of values in the text. Furthermore, the copy mechanism in value generator and the value attention module in value classifier help our model address the data discrepancy issue by only focusing on the relevant part of input text and ignoring other information which causes the discrepancy issue such as sentence structure in the text. Besides, two variants of our model are designed for open-world and closed-world scenarios. In addition, copy mechanism introduced in the first variant based on value generation can improve its zero-shot ability for identifying unseen values. Experimental results on a public dataset demonstrate the superiority of our model compared with strong baselines and its generalization ability of predicting new values.

cs.CL↗