arXiv subjects

Kai Wang

Publications and source records attributed to Kai Wang.

At least 505 records · Page 28Linked to original sources

arXiv2024

Must: Maximizing Latent Capacity of Spatial Transcriptomics Data

Spatial transcriptomics (ST) technologies have revolutionized the study of gene expression patterns in tissues by providing multimodality data in transcriptomic, spatial, and morphological, offering opportunities for understanding tissue biology beyond transcriptomics. However, we identify the modality bias phenomenon in ST data species, i.e., the inconsistent contribution of different modalities to the labels leads to a tendency for the analysis methods to retain the information of the dominant modality. How to mitigate the adverse effects of modality bias to satisfy various downstream tasks remains a fundamental challenge. This paper introduces Multiple-modality Structure Transformation, named MuST, a novel methodology to tackle the challenge. MuST integrates the multi-modality information contained in the ST data effectively into a uniform latent space to provide a foundation for all the downstream tasks. It learns intrinsic local structures by topology discovery strategy and topology fusion loss function to solve the inconsistencies among different modalities. Thus, these topology-based and deep learning techniques provide a solid foundation for a variety of analytical tasks while coordinating different modalities. The effectiveness of MuST is assessed by performance metrics and biological significance. The results show that it outperforms existing state-of-the-art methods with clear advantages in the precision of identifying and preserving structures of tissues and biomarkers. MuST offers a versatile toolkit for the intricate analysis of complex biological systems.

cs.CE↗

arXiv2024

Understanding YTHDF2-mediated mRNA Degradation By m6A-BERT-Deg

N6-methyladenosine (m6A) is the most abundant mRNA modification within mammalian cells, holding pivotal significance in the regulation of mRNA stability, translation, and splicing. Furthermore, it plays a critical role in the regulation of RNA degradation by primarily recruiting the YTHDF2 reader protein. However, the selective regulation of mRNA decay of the m6A-methylated mRNA through YTHDF2 binding is poorly understood. To improve our understanding, we developed m6A-BERT-Deg, a BERT model adapted for predicting YTHDF2-mediated degradation of m6A-methylated mRNAs. We meticulously assembled a high-quality training dataset by integrating multiple data sources for the HeLa cell line. To overcome the limitation of small training samples, we employed a pre-training-fine-tuning strategy by first performing a self-supervised pre-training of the model on 427,760 unlabeled m6A site sequences. The test results demonstrated the importance of this pre-training strategy in enabling m6A-BERT-Deg to outperform other benchmark models. We further conducted a comprehensive model interpretation and revealed a surprising finding that the presence of co-factors in proximity to m6A sites may disrupt YTHDF2-mediated mRNA degradation, subsequently enhancing mRNA stability. We also extended our analyses to the HEK293 cell line, shedding light on the context-dependent YTHDF2-mediated mRNA degradation.

q-bio.MN↗

arXiv2024

Characterizing the assembly of dark matter halos with protohalo size histories: I. Redshift evolution, relation to descendant halos, and halo assembly bias

We propose a novel method to quantify the assembly histories of dark matter halos with the redshift evolution of the mass-weighted spatial variance of their progenitor halos, i.e. the protohalo size history. We find that the protohalo size history for each individual halo at z~0 can be described by a double power-law function. The amplitude of the fitting function strongly correlates to the central-to-total stellar mass ratios of descendant halos. The variation of the amplitude of the protohalo size history can induce a strong halo assembly bias effect for massive halos. This effect is detectable in observation using the central-to-total stellar mass ratio as a proxy of the protohalo size. The correlation to the descendant central-to-total stellar mass ratio and the halo assembly bias effect seen in the protohalo size are much stronger than that seen in the commonly adopted half-mass formation time derived from the mass accretion history. This indicates that the information loss caused by the compression of halo merger trees to mass accretion histories can be captured by the protohalo size history. Protohalo size thus provides a useful quantity to connect protoclusters across cosmic time and to link protoclusters with their descendant clusters in observations.

astro-ph.GA↗

arXiv2024

Deep Lossy Plus Residual Coding for Lossless and Near-lossless Image Compression

Lossless and near-lossless image compression is of paramount importance to professional users in many technical fields, such as medicine, remote sensing, precision engineering and scientific research. But despite rapidly growing research interests in learning-based image compression, no published method offers both lossless and near-lossless modes. In this paper, we propose a unified and powerful deep lossy plus residual (DLPR) coding framework for both lossless and near-lossless image compression. In the lossless mode, the DLPR coding system first performs lossy compression and then lossless coding of residuals. We solve the joint lossy and residual compression problem in the approach of VAEs, and add autoregressive context modeling of the residuals to enhance lossless compression performance. In the near-lossless mode, we quantize the original residuals to satisfy a given $\ell_\infty$ error bound, and propose a scalable near-lossless compression scheme that works for variable $\ell_\infty$ bounds instead of training multiple networks. To expedite the DLPR coding, we increase the degree of algorithm parallelization by a novel design of coding context, and accelerate the entropy coding with adaptive residual interval. Experimental results demonstrate that the DLPR coding system achieves both the state-of-the-art lossless and near-lossless image compression performance with competitive coding speed.

eess.IV↗

arXiv2024

Versatile Diffusion: Text, Images and Variations All in One Diffusion Model

Recent advances in diffusion models have set an impressive milestone in many generation tasks, and trending works such as DALL-E2, Imagen, and Stable Diffusion have attracted great interest. Despite the rapid landscape changes, recent new approaches focus on extensions and performance rather than capacity, thus requiring separate models for separate tasks. In this work, we expand the existing single-flow diffusion pipeline into a multi-task multimodal network, dubbed Versatile Diffusion (VD), that handles multiple flows of text-to-image, image-to-text, and variations in one unified model. The pipeline design of VD instantiates a unified multi-flow diffusion framework, consisting of sharable and swappable layer modules that enable the crossmodal generality beyond images and text. Through extensive experiments, we demonstrate that VD successfully achieves the following: a) VD outperforms the baseline approaches and handles all its base tasks with competitive quality; b) VD enables novel extensions such as disentanglement of style and semantics, dual- and multi-context blending, etc.; c) The success of our multi-flow multimodal framework over images and text may inspire further diffusion-based universal AI research. Our code and models are open-sourced at https://github.com/SHI-Labs/Versatile-Diffusion.

cs.CV↗

arXiv2024

Summarizing Stream Data for Memory-Constrained Online Continual Learning

Replay-based methods have proved their effectiveness on online continual learning by rehearsing past samples from an auxiliary memory. With many efforts made on improving training schemes based on the memory, however, the information carried by each sample in the memory remains under-investigated. Under circumstances with restricted storage space, the informativeness of the memory becomes critical for effective replay. Although some works design specific strategies to select representative samples, by only employing a small number of original images, the storage space is still not well utilized. To this end, we propose to Summarize the knowledge from the Stream Data (SSD) into more informative samples by distilling the training characteristics of real images. Through maintaining the consistency of training gradients and relationship to the past tasks, the summarized samples are more representative for the stream data compared to the original images. Extensive experiments are conducted on multiple online continual learning benchmarks to support that the proposed SSD method significantly enhances the replay effects. We demonstrate that with limited extra computational overhead, SSD provides more than 3% accuracy boost for sequential CIFAR-100 under extremely restricted memory buffer. Code in https://github.com/vimar-gu/SSD.

cs.CV↗

arXiv2024

Efficient Temporal Butterfly Counting and Enumeration on Temporal Bipartite Graphs

Bipartite graphs characterize relationships between two different sets of entities, like actor-movie, user-item, and author-paper. The butterfly, a 4-vertices 4-edges (2,2)-biclique, is the simplest cohesive motif in a bipartite graph and is the fundamental component of higher-order substructures. Counting and enumerating the butterflies offer significant benefits across various applications, including fraud detection, graph embedding, and community search. While the corresponding motif, the triangle, in the unipartite graphs has been widely studied in both static and temporal settings, the extension of butterfly to temporal bipartite graphs remains unexplored. In this paper, we investigate the temporal butterfly counting and enumeration problem: count and enumerate the butterflies whose edges establish following a certain order within a given duration. Towards efficient computation, we devise a non-trivial baseline rooted in the state-of-the-art butterfly counting algorithm on static graphs, further, explore the intrinsic property of the temporal butterfly, and develop a new optimization framework with a compact data structure and effective priority strategy. The time complexity is proved to be significantly reduced without compromising on space efficiency. In addition, we generalize our algorithms to practical streaming settings and multi-core computing architectures. Our extensive experiments on 11 large-scale real-world datasets demonstrate the efficiency and scalability of our solutions.

cs.DS↗

arXiv2024

Mutual Information as Intrinsic Reward of Reinforcement Learning Agents for On-demand Ride Pooling

The emergence of on-demand ride pooling services allows each vehicle to serve multiple passengers at a time, thus increasing drivers' income and enabling passengers to travel at lower prices than taxi/car on-demand services (only one passenger can be assigned to a car at a time like UberX and Lyft). Although on-demand ride pooling services can bring so many benefits, ride pooling services need a well-defined matching strategy to maximize the benefits for all parties (passengers, drivers, aggregation companies and environment), in which the regional dispatching of vehicles has a significant impact on the matching and revenue. Existing algorithms often only consider revenue maximization, which makes it difficult for requests with unusual distribution to get a ride. How to increase revenue while ensuring a reasonable assignment of requests brings a challenge to ride pooling service companies (aggregation companies). In this paper, we propose a framework for vehicle dispatching for ride pooling tasks, which splits the city into discrete dispatching regions and uses the reinforcement learning (RL) algorithm to dispatch vehicles in these regions. We also consider the mutual information (MI) between vehicle and order distribution as the intrinsic reward of the RL algorithm to improve the correlation between their distributions, thus ensuring the possibility of getting a ride for unusually distributed requests. In experimental results on a real-world taxi dataset, we demonstrate that our framework can significantly increase revenue up to an average of 3\% over the existing best on-demand ride pooling method.

cs.AI↗

arXiv2023

Determining the Origin of Very-high-energy Gamma Rays from Galactic Sources by Future Neutrino Observations

Recently, the Large High Altitude Air Shower Observatory (LHAASO) identified 12 $γ$-ray sources emitting gamma rays with energies above 100 TeV, making them potential PeV cosmic-ray accelerators (PeVatrons). Neutrino observations are crucial in determining whether the gamma-ray radiation process is of hadronic or leptonic origin. In this paper, we study three detected sources, LHAASO J1908+0621, LHAASO J2018+3651, and LHAASO J2032+4102, which are also the most promising galactic high-energy neutrino candidate sources with the lowest pre-trial p-value based on the stacking searches testing for excess neutrino emission by IceCube Neutrino Observatory. We study the lepto-hadronic scenario for the observed multiband spectra of these LHAASO sources considering the possible counterpart source of the LHAASO sources. The very-high-energy gamma rays are entirely attributed to the hadronic contribution, therefore the most optimistic neutrino flux can be derived. Then, we evaluate the statistical significance (p-value) as a function of the observation time of IceCube and the next-generation IceCube-Gen2 neutrino observatory respectively. Our results tend to disfavor that all gamma rays above $100\,\rm GeV$ from LHAASO J1908+0621 are of purely hadronic origin based on current IceCube observations, but the purely hadronic origin of gamma rays above $100\,\rm TeV$ is still possible. By IceCube-Gen2, the origin of gamma rays above $100\,\rm TeV$ from LHAASO J1908+0621 can be further determined at a $5σ$ significance level within a running time of $\sim 3$ years. For LHAASO J2018+3651 and LHAASO J2032+4102, the required running time of IceCube-Gen2 is $\sim 10$ years ($3σ$) and $\sim 10$ years ($5σ$), respectively. Future observations by the next-generation neutrino telescope will be crucial to understanding the particle acceleration and radiation processes inside the sources.

astro-ph.HE↗

arXiv2023

An efficient and robust method to estimate halo concentration

We propose an efficient and robust method to estimate the halo concentration based on the first moment of the density distribution, which is $R_1\equiv \int_0^{r_{\rm vir}}4πr^3ρ(r)dr/M_{\rm vir}/r_{\rm vir}$. We find that $R_1$ has a monotonic relation with the concentration parameter of the NFW profile, and that a cubic polynomial function can fit the relation with an error $\lesssim 3\%$. Tests on ideal NFW halos show that the conventional NFW profile fitting method and the $V_{\rm max}/V_{\rm vir}$ method produce biased halo concentration estimation by $\approx 10\%$ and $\approx 30\%$, respectively, for halos with 100 particles. In contrast, the systematic error for our $R_1$ method is smaller than $0.5\%$ even for halos containing only 100 particles. Convergence tests on realistic halos in $N$-body simulations show that the NFW profile fitting method underestimates the concentration parameter for halos with $\lesssim 300$ particles by $\gtrsim 20\%$, while the error for the $R_1$ method is $\lesssim 8\%$. We also show other applications of $R_1$, including estimating $V_{\rm max}$ and the Einasto concentration $c_{\rm e}\equiv r_{\rm vir}/r_{-2}$. The calculation of $R_1$ is efficient and robust, and we recommend including it as one of the halo properties in halo catalogs of cosmological simulations.

astro-ph.CO↗

arXiv2023

Koszul duality, minimal model and $L_\infty$-structure for differential algebras with weight

A differential algebra with weight is an abstraction of both the derivation (weight zero) and the forward and backward difference operators (weight $\pm 1$). In 2010 Loday established the Koszul duality for the operad of differential algebras of weight zero. He did not treat the case of nonzero weight, noting that new techniques are needed since the operad is no longer quadratic. This paper continues Loday's work and establishes the Koszul duality in the case of nonzero weight. In the process, the minimal model and the Koszul dual homotopy cooperad of the operad governing differential algebras with weight are determined. As a consequence, a notion of homotopy differential algebras with weight is obtained and the deformation complex as well as its $L_\infty$-algebra structure for differential algebras with weight are deduced.

math.RA↗

arXiv2023

Scenario Diffusion: Controllable Driving Scenario Generation With Diffusion

Automated creation of synthetic traffic scenarios is a key part of validating the safety of autonomous vehicles (AVs). In this paper, we propose Scenario Diffusion, a novel diffusion-based architecture for generating traffic scenarios that enables controllable scenario generation. We combine latent diffusion, object detection and trajectory regression to generate distributions of synthetic agent poses, orientations and trajectories simultaneously. To provide additional control over the generated scenario, this distribution is conditioned on a map and sets of tokens describing the desired scenario. We show that our approach has sufficient expressive capacity to model diverse traffic patterns and generalizes to different geographical regions.

cs.LG↗

arXiv2023

Towards Long-term Annotators: A Supervised Label Aggregation Baseline

Relying on crowdsourced workers, data crowdsourcing platforms are able to efficiently provide vast amounts of labeled data. Due to the variability in the annotation quality of crowd workers, modern techniques resort to redundant annotations and subsequent label aggregation to infer true labels. However, these methods require model updating during the inference, posing challenges in real-world implementation. Meanwhile, in recent years, many data labeling tasks have begun to require skilled and experienced annotators, leading to an increasing demand for long-term annotators. These annotators could leave substantial historical annotation records on the crowdsourcing platforms, which can benefit label aggregation, but are ignored by previous works. Hereby, in this paper, we propose a novel label aggregation technique, which does not need any model updating during inference and can extensively explore the historical annotation records. We call it SuperLA, a Supervised Label Aggregation method. Inside this model, we design three types of input features and a straightforward neural network structure to merge all the information together and subsequently produce aggregated labels. Based on comparison experiments conducted on 22 public datasets and 11 baseline methods, we find that SuperLA not only outperforms all those baselines in inference performance but also offers significant advantages in terms of efficiency.

cs.HC↗

arXiv2023

PRIOR: Personalized Prior for Reactivating the Information Overlooked in Federated Learning

Classical federated learning (FL) enables training machine learning models without sharing data for privacy preservation, but heterogeneous data characteristic degrades the performance of the localized model. Personalized FL (PFL) addresses this by synthesizing personalized models from a global model via training on local data. Such a global model may overlook the specific information that the clients have been sampled. In this paper, we propose a novel scheme to inject personalized prior knowledge into the global model in each client, which attempts to mitigate the introduced incomplete information problem in PFL. At the heart of our proposed approach is a framework, the PFL with Bregman Divergence (pFedBreD), decoupling the personalized prior from the local objective function regularized by Bregman divergence for greater adaptability in personalized scenarios. We also relax the mirror descent (RMD) to extract the prior explicitly to provide optional strategies. Additionally, our pFedBreD is backed up by a convergence analysis. Sufficient experiments demonstrate that our method reaches the state-of-the-art performances on 5 datasets and outperforms other methods by up to 3.5% across 8 benchmarks. Extensive analyses verify the robustness and necessity of proposed designs.

cs.LG↗

arXiv2023

Enhancing Phenotype Recognition in Clinical Notes Using Large Language Models: PhenoBCBERT and PhenoGPT

We hypothesize that large language models (LLMs) based on the transformer architecture can enable automated detection of clinical phenotype terms, including terms not documented in the HPO. In this study, we developed two types of models: PhenoBCBERT, a BERT-based model, utilizing Bio+Clinical BERT as its pre-trained model, and PhenoGPT, a GPT-based model that can be initialized from diverse GPT models, including open-source versions such as GPT-J, Falcon, and LLaMA, as well as closed-source versions such as GPT-3 and GPT-3.5. We compared our methods with PhenoTagger, a recently developed HPO recognition tool that combines rule-based and deep learning methods. We found that our methods can extract more phenotype concepts, including novel ones not characterized by HPO. We also performed case studies on biomedical literature to illustrate how new phenotype information can be recognized and extracted. We compared current BERT-based versus GPT-based models for phenotype tagging, in multiple aspects including model architecture, memory usage, speed, accuracy, and privacy protection. We also discussed the addition of a negation step and an HPO normalization layer to the transformer models for improved HPO term tagging. In conclusion, PhenoBCBERT and PhenoGPT enable the automated discovery of phenotype terms from clinical notes and biomedical literature, facilitating automated downstream tasks to derive new biological insights on human diseases.

q-bio.QM↗

arXiv2023

High-energy Neutrino Emission Associated with GWs from Binary Black Hole Mergers in AGN Accretion Discs

The search for multi-messenger signals of binary black hole (BBH) mergers is crucial to understanding the merger process of BBH and the relative astrophysical environment. Considering BBH mergers occurring in the active galactic nuclei (AGN) accretion disks, we focus on the accompanying high-energy neutrino production from the interaction between the jet launched by the post-merger remnant BH and disk materials. Particles can be accelerated by the shocks generated from the jet-disk interaction and subsequently interact with the disk gas and radiations to produce high-energy neutrinos through hadronic processes. We demonstrate that the identification of the high-energy neutrino signal from BBH merger in AGN disks is feasible. In addition, the joint BBH gravitational wave (GW) and neutrino detection rate is derived, which can be used to constrain the BBH merger rate and the accretion rate of the remnant BH based on the future associated detections of GWs and neutrinos. To date, an upper limit of BBH merger rate density in AGN disks of $R_0 \lesssim 3\,\rm Gpc^{-3} yr^{-1}$ is derived for the fiducial parameter values based on the current null association of GWs and neutrinos.

astro-ph.HE↗

arXiv2023

Branch-and-Price for Prescriptive Contagion Analytics

Predictive contagion models are ubiquitous in epidemiology, social sciences, engineering, and management. This paper formulates a prescriptive contagion analytics model where a decision-maker allocates shared resources across multiple segments of a population, each governed by continuous-time dynamics. We define four real-world problems under this umbrella: vaccine distribution, vaccination centers deployment, content promotion, and congestion mitigation. These problems feature a large-scale mixed-integer non-convex optimization structure with constraints governed by ordinary differential equations, combining the challenges of discrete optimization, non-linear optimization, and continuous-time system dynamics. This paper develops a branch-and-price methodology for prescriptive contagion analytics based on: (i) a set partitioning reformulation; (ii) a column generation decomposition; (iii) a state-clustering algorithm for discrete-decision continuous-state dynamic programming; and (iv) a tri-partite branching scheme to circumvent non-linearities. Extensive experiments show that the algorithm scales to very large and otherwise-intractable instances, outperforming state-of-the-art benchmarks. Our methodology provides practical benefits in contagion systems; in particular, it can increase the effectiveness of a vaccination campaign by an estimated 12-70%, resulting in 7,000 to 12,000 extra saved lives over a three-month horizon mirroring the COVID-19 pandemic. We provide an open-source implementation of the methodology in an online repository to enable replication.

math.OC↗

arXiv2023

DREAM+: Efficient Dataset Distillation by Bidirectional Representative Matching

Dataset distillation plays a crucial role in creating compact datasets with similar training performance compared with original large-scale ones. This is essential for addressing the challenges of data storage and training costs. Prevalent methods facilitate knowledge transfer by matching the gradients, embedding distributions, or training trajectories of synthetic images with those of the sampled original images. Although there are various matching objectives, currently the strategy for selecting original images is limited to naive random sampling. We argue that random sampling overlooks the evenness of the selected sample distribution, which may result in noisy or biased matching targets. Besides, the sample diversity is also not constrained by random sampling. Additionally, current methods predominantly focus on single-dimensional matching, where information is not fully utilized. To address these challenges, we propose a novel matching strategy called Dataset Distillation by Bidirectional REpresentAtive Matching (DREAM+), which selects representative original images for bidirectional matching. DREAM+ is applicable to a variety of mainstream dataset distillation frameworks and significantly reduces the number of distillation iterations by more than 15 times without affecting performance. Given sufficient training time, DREAM+ can further improve the performance and achieve state-of-the-art results. We have released the code at github.com/NUS-HPC-AI-Lab/DREAM+.

cs.CV↗