arXiv subjects

Shuo Zhang

Publications and source records attributed to Shuo Zhang.

At least 199 records · Page 11Linked to original sources

arXiv2024

On Truthful Item-Acquiring Mechanisms for Reward Maximization

In this research, we study the problem that a collector acquires items from the owner based on the item qualities the owner declares and an independent appraiser's assessments. The owner is interested in maximizing the probability that the collector acquires the items and is the only one who knows the items' factual quality. The appraiser performs her duties with impartiality, but her assessment may be subject to random noises, so it may not accurately reflect the factual quality of the items. The main challenge lies in devising mechanisms that prompt the owner to reveal accurate information, thereby optimizing the collector's expected reward. We consider the menu size of mechanisms as a measure of their practicability and study its impact on the attainable expected reward. For the single-item setting, we design optimal mechanisms with a monotone increasing menu size. Although the reward gap between the simplest and optimal mechanisms is bounded, we show that simple mechanisms with a small menu size cannot ensure any positive fraction of the optimal reward of mechanisms with a larger menu size. For the multi-item setting, we show that an ordinal mechanism that only takes the owner's ordering of the items as input is not incentive-compatible. We then propose a set of Union mechanisms that combine single-item mechanisms. Moreover, we run experiments to examine these mechanisms' robustness against the independent appraiser's assessment accuracy and the items' acquiring rate.

cs.GT↗

arXiv2024

AFN: Adaptive Fusion Normalization via an Encoder-Decoder Framework

The success of deep learning is inseparable from normalization layers. Researchers have proposed various normalization functions, and each of them has both advantages and disadvantages. In response, efforts have been made to design a unified normalization function that combines all normalization procedures and mitigates their weaknesses. We also proposed a new normalization function called Adaptive Fusion Normalization. Through experiments, we demonstrate AFN outperforms the previous normalization techniques in domain generalization and image classification tasks.

cs.LG↗

arXiv2024

CatNorth: An Improved Gaia DR3 Quasar Candidate Catalog with Pan-STARRS1 and CatWISE

A complete and pure sample of quasars with accurate redshifts is crucial for quasar studies and cosmology. In this paper, we present CatNorth, an improved Gaia DR3 quasar candidate catalog with more than 1.5 million sources in the 3$π$ sky built with data from Gaia, Pan-STARRS1, and CatWISE2020. The XGBoost algorithm is used to reclassify the original Gaia DR3 quasar candidates as stars, galaxies, and quasars. To construct training/validation datasets for the classification, we carefully built two different master stellar samples in addition to the spectroscopic galaxy and quasar samples. An ensemble classification model is obtained by averaging two XGBoost classifiers trained with different master stellar samples. Using a probability threshold of $p_{\mathrm{QSO\_mean}}>0.95$ in our ensemble classification model and an additional cut on the logarithmic probability density of zero proper motion, we retrieved 1,545,514 reliable quasar candidates from the parent Gaia DR3 quasar candidate catalog. We provide photometric redshifts for all candidates with an ensemble regression model. For a subset of 89,100 candidates, accurate spectroscopic redshifts are estimated with the Convolutional Neural Network from the Gaia BP/RP spectra. The CatNorth catalog has a high purity of ~ 90% while maintaining high completeness, which is an ideal sample to understand the quasar population and its statistical properties. The CatNorth catalog is used as the main source of input catalog for the LAMOST phase III quasar survey, which is expected to build a highly complete sample of bright quasars with $i < 19.5$.

astro-ph.GA↗

arXiv2024

Enhancing Adversarial Attacks: The Similar Target Method

Deep neural networks are vulnerable to adversarial examples, posing a threat to the models' applications and raising security concerns. An intriguing property of adversarial examples is their strong transferability. Several methods have been proposed to enhance transferability, including ensemble attacks which have demonstrated their efficacy. However, prior approaches simply average logits, probabilities, or losses for model ensembling, lacking a comprehensive analysis of how and why model ensembling significantly improves transferability. In this paper, we propose a similar targeted attack method named Similar Target~(ST). By promoting cosine similarity between the gradients of each model, our method regularizes the optimization direction to simultaneously attack all surrogate models. This strategy has been proven to enhance generalization ability. Experimental results on ImageNet validate the effectiveness of our approach in improving adversarial transferability. Our method outperforms state-of-the-art attackers on 18 discriminative classifiers and adversarially trained models.

cs.CV↗

arXiv2023

Quantum State Compression Shadow

Quantum state readout serves as the cornerstone of quantum information processing, exerting profound influence on quantum communication, computation, and metrology. In this study, we introduce an innovative readout architecture called Compression Shadow (CompShadow), which transforms the conventional readout paradigm by compressing multi-qubit states into single-qubit shadows before measurement. Compared to direct measurements of the initial quantum states, CompShadow achieves comparable accuracy in amplitude and observable expectation estimation while consuming similar measurement resources. Furthermore, its implementation on near-term quantum hardware with nearest-neighbor coupling architectures is straightforward. Significantly, CompShadow brings forth novel features, including the complete suppression of correlated readout noise, fundamentally reducing the quantum hardware demands for readout. It also facilitates the exploration of multi-body system properties through single-qubit probes and opens the door to designing quantum communication protocols with exponential loss suppression. Our findings mark the emergence of a new era in quantum state readout, setting the stage for a revolutionary leap in quantum information processing capabilities.

quant-ph↗

arXiv2023

MERGE: Fast Private Text Generation

The drastic increase in language models' parameters has led to a new trend of deploying models in cloud servers, raising growing concerns about private inference for Transformer-based models. Existing two-party privacy-preserving techniques, however, only take into account natural language understanding (NLU) scenarios. Private inference in natural language generation (NLG), crucial for applications like translation and code completion, remains underexplored.In addition, previous privacy-preserving techniques suffer from convergence issues during model training and exhibit poor inference speed when used with NLG models due to the neglect of time-consuming operations in auto-regressive generations. To address these issues, we propose a fast private text generation framework for Transformer-based language models, namely MERGE.MERGE reuses the output hidden state as the word embedding to bypass the embedding computation and reorganize the linear operations in the Transformer module to accelerate the forward procedure. Extensive experiments show that MERGE achieves a 26.5x speedup to the vanilla encrypted model under the sequence length 512, and reduces 80\% communication cost, with an up to 10x speedup to state-of-the-art approximated models.

cs.CL↗

arXiv2023

Protein Language Model-Powered 3D Ligand Binding Site Prediction from Protein Sequence

Prediction of ligand binding sites of proteins is a fundamental and important task for understanding the function of proteins and screening potential drugs. Most existing methods require experimentally determined protein holo-structures as input. However, such structures can be unavailable on novel or less-studied proteins. To tackle this limitation, we propose LaMPSite, which only takes protein sequences and ligand molecular graphs as input for ligand binding site predictions. The protein sequences are used to retrieve residue-level embeddings and contact maps from the pre-trained ESM-2 protein language model. The ligand molecular graphs are fed into a graph neural network to compute atom-level embeddings. Then we compute and update the protein-ligand interaction embedding based on the protein residue-level embeddings and ligand atom-level embeddings, and the geometric constraints in the inferred protein contact map and ligand distance map. A final pooling on protein-ligand interaction embedding would indicate which residues belong to the binding sites. Without any 3D coordinate information of proteins, our proposed model achieves competitive performance compared to baseline methods that require 3D protein structures when predicting binding sites. Given that less than 50% of proteins have reliable structure information in the current stage, LaMPSite will provide new opportunities for drug discovery.

q-bio.QM↗

arXiv2023

CoLLiE: Collaborative Training of Large Language Models in an Efficient Way

Large language models (LLMs) are increasingly pivotal in a wide range of natural language processing tasks. Access to pre-trained models, courtesy of the open-source community, has made it possible to adapt these models to specific applications for enhanced performance. However, the substantial resources required for training these models necessitate efficient solutions. This paper introduces CoLLiE, an efficient library that facilitates collaborative training of large language models using 3D parallelism, parameter-efficient fine-tuning (PEFT) methods, and optimizers such as Lion, Adan, Sophia, LOMO and AdaLomo. With its modular design and comprehensive functionality, CoLLiE offers a balanced blend of efficiency, ease of use, and customization. CoLLiE has proven superior training efficiency in comparison with prevalent solutions in pre-training and fine-tuning scenarios. Furthermore, we provide an empirical evaluation of the correlation between model size and GPU memory consumption under different optimization methods, as well as an analysis of the throughput. Lastly, we carry out a comprehensive comparison of various optimizers and PEFT methods within the instruction-tuning context. CoLLiE is available at https://github.com/OpenLMLab/collie.

cs.CL↗

arXiv2023

A long exact sequence on the composition of Dehn twists

We prove an exact sequence relating the Lagrangian Floer homology of a collection of Lagrangian spheres $\{L_i\}$ and the fixed-point Floer homology of iterated Dehn twists along them, making progress toward a conjecture of Seidel.

math.SG↗

arXiv2023

Efficient and Accurate Physics-aware Multiplex Graph Neural Networks for 3D Small Molecules and Macromolecule Complexes

Recent advances in applying Graph Neural Networks (GNNs) to molecular science have showcased the power of learning three-dimensional (3D) structure representations with GNNs. However, most existing GNNs suffer from the limitations of insufficient modeling of diverse interactions, computational expensive operations, and ignorance of vectorial values. Here, we tackle these limitations by proposing a novel GNN model, Physics-aware Multiplex Graph Neural Network (PaxNet), to efficiently and accurately learn the representations of 3D molecules for both small organic compounds and macromolecule complexes. PaxNet separates the modeling of local and non-local interactions inspired by molecular mechanics, and reduces the expensive angle-related computations. Besides scalar properties, PaxNet can also predict vectorial properties by learning an associated vector for each atom. To evaluate the performance of PaxNet, we compare it with state-of-the-art baselines in two tasks. On small molecule dataset for predicting quantum chemical properties, PaxNet reduces the prediction error by 15% and uses 73% less memory than the best baseline. On macromolecule dataset for predicting protein-ligand binding affinities, PaxNet outperforms the best baseline while reducing the memory consumption by 33% and the inference time by 85%. Thus, PaxNet provides a universal, robust and accurate method for large-scale machine learning of molecules. Our code is available at https://github.com/zetayue/Physics-aware-Multiplex-GNN.

q-bio.BM↗

arXiv2023

A Universal Framework for Accurate and Efficient Geometric Deep Learning of Molecular Systems

Molecular sciences address a wide range of problems involving molecules of different types and sizes and their complexes. Recently, geometric deep learning, especially Graph Neural Networks, has shown promising performance in molecular science applications. However, most existing works often impose targeted inductive biases to a specific molecular system, and are inefficient when applied to macromolecules or large-scale tasks, thereby limiting their applications to many real-world problems. To address these challenges, we present PAMNet, a universal framework for accurately and efficiently learning the representations of three-dimensional (3D) molecules of varying sizes and types in any molecular system. Inspired by molecular mechanics, PAMNet induces a physics-informed bias to explicitly model local and non-local interactions and their combined effects. As a result, PAMNet can reduce expensive operations, making it time and memory efficient. In extensive benchmark studies, PAMNet outperforms state-of-the-art baselines regarding both accuracy and efficiency in three diverse learning tasks: small molecule properties, RNA 3D structures, and protein-ligand binding affinities. Our results highlight the potential for PAMNet in a broad range of molecular science applications.

cs.LG↗

arXiv2023

TempTabQA: Temporal Question Answering for Semi-Structured Tables

Semi-structured data, such as Infobox tables, often include temporal information about entities, either implicitly or explicitly. Can current NLP systems reason about such information in semi-structured tables? To tackle this question, we introduce the task of temporal question answering on semi-structured tables. We present a dataset, TempTabQA, which comprises 11,454 question-answer pairs extracted from 1,208 Wikipedia Infobox tables spanning more than 90 distinct domains. Using this dataset, we evaluate several state-of-the-art models for temporal reasoning. We observe that even the top-performing LLMs lag behind human performance by more than 13.5 F1 points. Given these results, our dataset has the potential to serve as a challenging benchmark to improve the temporal reasoning capabilities of NLP models.

cs.CL↗

arXiv2023

Time-efficient combined morphologic and quantitative joint MRI based on clinical image contrasts -- An exploratory in-situ study of standardized cartilage defects

OBJECTIVES: Quantitative MRI techniques such as T2 and T1$ρ$ mapping are beneficial in evaluating cartilage and meniscus. We aimed to evaluate the MIXTURE (Multi-Interleaved X-prepared Turbo-Spin Echo with IntUitive RElaxometry) sequences that provide morphologic images with clinical turbo spin-echo (TSE) contrasts and additional parameter maps versus reference TSE sequences in an in-situ model of human cartilage defects. MATERIALS AND METHODS: Prospectively, standardized cartilage defects of 8mm, 5mm, and 3mm diameter were created in the lateral femora of 10 human cadaveric knee specimens (81$\pm$10 years, nine male/one female). Using a clinical 3T MRI scanner and knee coil, MIXTURE sequences combining (i) proton-density weighted fat-saturated (PD-w FS) images and T2 maps and (ii) T1-weighted images and T1$ρ$ maps were acquired before and after defect creation, alongside the corresponding 2D TSE and 3D TSE reference sequences. Defect delineability, bone texture, and cartilage relaxation times were quantified. Inter-sequence comparisons were made using appropriate parametric and non-parametric tests. RESULTS: Overall, defect delineability and texture features were not significantly different between the MIXTURE and reference sequences. After defect creation, relaxation times increased significantly in the central femur (for T2) and all regions combined (for T1$ρ$). CONCLUSION: MIXTURE sequences permit time-efficient simultaneous morphologic and quantitative joint assessment based on clinical image contrasts. While providing T2 or T1$ρ$ maps in clinically feasible scan time, morphologic image features, i.e., cartilage defect delineability and bone texture, were comparable between MIXTURE and corresponding reference sequences.

physics.med-ph↗

arXiv2023

The High Energy X-ray Probe (HEX-P): resolving the nature of Sgr A* flares, compact object binaries and diffuse X-ray emission in the Galactic Center and beyond

HEX-P is a probe-class mission concept that will combine high spatial resolution X-ray imaging ($<10"$ FWHM) and broad spectral coverage (0.2-80 keV) with an effective area far superior to current facilities' (including XMM-Newton and NuSTAR). These capabilities will enable revolutionary new insights into a variety of important astrophysical problems. We present scientific objectives and simulations of HEX-P observations of the Galactic Center (GC) and Bulge. We demonstrate the unique and powerful capabilities of the HEX-P observatory for studying both X-ray point sources and diffuse X-ray emission. HEX-P will be uniquely equipped to explore a variety of major topics in Galactic astrophysics, allowing us to (1) investigate broad-band properties of X-ray flares emitted from the supermassive black hole (BH) at Sgr A* and probe the associated particle acceleration and emission mechanisms; (2) identify hard X-ray sources detected by NuSTAR and determine X-ray point source populations in different regions and luminosity ranges; (3) determine the distribution of compact object binaries in the nuclear star cluster and the composition of the Galactic Ridge X-ray emission; (4) identify X-ray transients and measure fundamental parameters such as BH spin; (5) find hidden pulsars in the GC; (6) search for BH-OB binaries and hard X-ray flares from young stellar objects in young massive clusters; (7) measure white dwarf (WD) masses of magnetic CVs to deepen our understanding of CV evolution and the origin of WD magnetic fields; (8) explore primary particle accelerators in the GC in synergy with future TeV and neutrino observatories; (9) map out cosmic-ray distributions by observing non-thermal X-ray filaments; (10) explore past X-ray outbursts from Sgr A* through X-ray reflection components from giant molecular clouds.

astro-ph.HE↗

arXiv2023

Improved Deterministic Streaming Algorithms for Non-monotone Submodular Maximization

Submodular maximization is one of the central topics in combinatorial optimization. It has found numerous applications in the real world. Streaming algorithms for submodule maximization have gained attention in recent years, allowing for real-time processing of large data sets by looking at each piece of data only once. However, most of the state-of-the-art algorithms are subject to monotone cardinality constraint. There remain non-negligible gaps with respect to approximation ratios between cardinality and other constraints like $d$-knapsack in non-monotone submodular maximization. In this paper, we propose deterministic algorithms with improved approximation ratios for non-monotone submodular maximization. Specifically, for the cardinality constraint, we provide a deterministic $1/6-ε$ approximation algorithm with $O(\frac{k\log k}ε)$ memory and sublinear query time, while the previous best algorithm is randomized with a $0.1921$ approximation ratio. To the best of our knowledge, this is the first deterministic streaming algorithm for the cardinality constraint. For the $d$-knapsack constraint, we provide a deterministic $\frac{1}{4(d+1)}-ε$ approximation algorithm with $O(\frac{b\log b}ε)$ memory and $O(\frac{\log b}ε)$ query time per element. To the best of our knowledge, there is currently no streaming algorithm for this constraint.

cs.DS↗

arXiv2023

Metaphorical User Simulators for Evaluating Task-oriented Dialogue Systems

Task-oriented dialogue systems (TDSs) are assessed mainly in an offline setting or through human evaluation. The evaluation is often limited to single-turn or is very time-intensive. As an alternative, user simulators that mimic user behavior allow us to consider a broad set of user goals to generate human-like conversations for simulated evaluation. Employing existing user simulators to evaluate TDSs is challenging as user simulators are primarily designed to optimize dialogue policies for TDSs and have limited evaluation capabilities. Moreover, the evaluation of user simulators is an open challenge. In this work, we propose a metaphorical user simulator for end-to-end TDS evaluation, where we define a simulator to be metaphorical if it simulates user's analogical thinking in interactions with systems. We also propose a tester-based evaluation framework to generate variants, i.e., dialogue systems with different capabilities. Our user simulator constructs a metaphorical user model that assists the simulator in reasoning by referring to prior knowledge when encountering new items. We estimate the quality of simulators by checking the simulated interactions between simulators and variants. Our experiments are conducted using three TDS datasets. The proposed user simulator demonstrates better consistency with manual evaluation than an agenda-based simulator and a seq2seq model on three datasets; our tester framework demonstrates efficiency and has been tested on multiple tasks, such as conversational recommendation and e-commerce dialogues.

cs.CL↗

arXiv2023

Two for One -- Combined Morphologic and Quantitative Knee Joint MRI Using a Versatile Turbo Spin-Echo Platform

Introduction: Quantitative MRI techniques such as T2 and T1\r{ho} mapping are beneficial in evaluating knee joint pathologies; however, long acquisition times limit their clinical adoption. MIXTURE (Multi-Interleaved X-prepared Turbo-Spin Echo with IntUitive RElaxometry) provides a versatile turbo spin-echo (TSE) sequence platform for simultaneous morphologic and quantitative joint imaging yet lacks comparative evaluation in basic and translational research contexts. Methods: Two MIXTURE sequences were designed along clinical requirements: (i) MIX1, combining proton density (PD)-weighted fat-saturated (FS) images and quantitative T2 mapping (acquisition time: 4:59 min), and (ii) MIX2, combining T1-weighted images with quantitative T1\r{ho} mapping (6:38 min). MIXTURE sequences and their reference 2D and 3D TSE counterparts were acquired from ten human cadaveric knee joints using a clinical 3T MRI scanner and knee coil. Contrast, contrast-to-noise ratios, and coefficients of variation were comparatively evaluated using parametric tests. Clinical radiologists (n=3) assessed diagnostic quality as a function of sequence and anatomic structure using 5-point Likert scales and ordinal regression. The significance level was set to α=0.01. Results: MIX1 and MIX2 had at least equal diagnostic quality compared to the 2D and 3D TSE sequences of the same image weighting. Contrast, contrast-to-noise ratios, and coefficients of variation were largely similar for the PD-weighted FS and T1-weighted images. Discussion: In clinically feasible scan times, the MIXTURE sequence platform yields (i) morphologic images of diagnostic quality and adjustable TSE-based contrasts and (ii) quantitative parameter mapping with additional insights on soft tissue composition and ultrastructure.

physics.med-ph↗

arXiv2023

RADE: Reference-Assisted Dialogue Evaluation for Open-Domain Dialogue

Evaluating open-domain dialogue systems is challenging for reasons such as the one-to-many problem, i.e., many appropriate responses other than just the golden response. As of now, automatic evaluation methods need better consistency with humans, while reliable human evaluation can be time- and cost-intensive. To this end, we propose the Reference-Assisted Dialogue Evaluation (RADE) approach under the multi-task learning framework, which leverages the pre-created utterance as reference other than the gold response to relief the one-to-many problem. Specifically, RADE explicitly compares reference and the candidate response to predict their overall scores. Moreover, an auxiliary response generation task enhances prediction via a shared encoder. To support RADE, we extend three datasets with additional rated responses other than just a golden response by human annotation. Experiments on our three datasets and two existing benchmarks demonstrate the effectiveness of our method, where Pearson, Spearman, and Kendall correlations with human evaluation outperform state-of-the-art baselines.

cs.CL↗