arXiv subjects

Zihao Li

Publications and source records attributed to Zihao Li.

At least 181 records · Page 10Linked to original sources

arXiv2024

Large Language Models for Biomedical Text Simplification: Promising But Not There Yet

In this system report, we describe the models and methods we used for our participation in the PLABA2023 task on biomedical abstract simplification, part of the TAC 2023 tracks. The system outputs we submitted come from the following three categories: 1) domain fine-tuned T5-like models including Biomedical-T5 and Lay-SciFive; 2) fine-tuned BARTLarge model with controllable attributes (via tokens) BART-w-CTs; 3) ChatGPTprompting. We also present the work we carried out for this task on BioGPT finetuning. In the official automatic evaluation using SARI scores, BeeManc ranks 2nd among all teams and our model LaySciFive ranks 3rd among all 13 evaluated systems. In the official human evaluation, our model BART-w-CTs ranks 2nd on Sentence-Simplicity (score 92.84), 3rd on Term-Simplicity (score 82.33) among all 7 evaluated systems; It also produced a high score 91.57 on Fluency in comparison to the highest score 93.53. In the second round of submissions, our team using ChatGPT-prompting ranks the 2nd in several categories including simplified term accuracy score 92.26 and completeness score 96.58, and a very similar score on faithfulness score 95.3 to re-evaluated PLABA-base-1 (95.73) via human evaluations. Our codes, fine-tuned models, prompts, and data splits from the system development stage will be available at https://github.com/ HECTA-UoM/PLABA-MU

cs.CL↗

arXiv2024

A SPectroscopic survey of biased halos In the Reionization Era (ASPIRE): Broad-line AGN at $z=4-5$ revealed by JWST/NIRCam WFSS

Low-luminosity AGNs with low-mass black holes (BHs) in the early universe are fundamental to understanding the BH growth and their co-evolution with the host galaxies. Utilizing JWST NIRCam Wide Field Slitless Spectroscopy (WFSS), we perform a systematic search for broad-line ${\rm Hα}$ emitters (BHAEs) at $z\approx 4-5$ in 25 fields of the ASPIRE (A SPectroscopic survey of biased halos In the Reionization Era) project, covering a total area of 275 arcmin$^2$. We identify 16 BHAEs with FWHM of the broad components spanning from $\sim$ 1000 km s$^{-1}$ to 3000 km s$^{-1}$. Assuming the broad linewidths arise due to Doppler broadening around BHs, the implied BH masses range from $10^7$ to $10^{8}~M_\odot$, with broad ${\rm Hα}$-converted bolometric luminosity of $10^{44.5}-10^{45.5}$ erg s$^{-1}$ and Eddington ratios of $0.07-0.47$. The spatially extended structure of the F200W stacked image may trace the stellar light from the host galaxies. The ${\rm Hα}$ luminosity function indicates an increasing AGN fraction towards the higher ${\rm Hα}$ luminosities. We find possible evidence for clustering of BHAEs: two sources are at the same redshift with a projected separation of 519 kpc; one BHAE appears as a composite system residing in an overdense region with three close companion ${\rm Hα}$ emitters. Three BHAEs exhibit blueshifted absorption troughs indicative of the presence of high-column-density gas. We find the broad-line and photometrically selected BHAE samples exhibit different distributions in the optical continuum slopes, which can be attributed to their different selection methods. The ASPIRE broad-line ${\rm Hα}$ sample provides a good database for future studies of faint AGN populations at high redshift.

astro-ph.GA↗

arXiv2024

Unlocking the Potential: Benchmarking Large Language Models in Water Engineering and Research

Recent advancements in Large Language Models (LLMs) have sparked interest in their potential applications across various fields. This paper embarked on a pivotal inquiry: Can existing LLMs effectively serve as "water expert models" for water engineering and research tasks? This study was the first to evaluate LLMs' contributions across various water engineering and research tasks by establishing a domain-specific benchmark suite, namely, WaterER. Herein, we prepared 983 tasks related to water engineering and research, categorized into "wastewater treatment", "environmental restoration", "drinking water treatment and distribution", "sanitation", "anaerobic digestion" and "contaminants assessment". We evaluated the performance of seven LLMs (i.e., GPT-4, GPT-3.5, Gemini, GLM-4, ERNIE, QWEN and Llama3) on these tasks. We highlighted the strengths of GPT-4 in handling diverse and complex tasks of water engineering and water research, the specialized capabilities of Gemini in academic contexts, Llama3's strongest capacity to answer Chinese water engineering questions and the competitive performance of Chinese-oriented models like GLM-4, ERNIE and QWEN in some water engineering tasks. More specifically, current LLMs excelled particularly in generating precise research gaps for papers on "contaminants and related water quality monitoring and assessment". Additionally, they were more adept at creating appropriate titles for research papers on "treatment processes for wastewaters", "environmental restoration", and "drinking water treatment". Overall, this study pioneered evaluating LLMs in water engineering and research by introducing the WaterER benchmark to assess the trustworthiness of their predictions. This standardized evaluation framework would also drive future advancements in LLM technology by using targeting datasets, propelling these models towards becoming true "water expert".

cs.CL↗

arXiv2024

Mapping the Empirical Evidence of the GDPR (In-)Effectiveness: A Systematic Review

In the realm of data protection, a striking disconnect prevails between traditional domains of doctrinal, legal, theoretical, and policy-based inquiries and a burgeoning body of empirical evidence. Much of the scholarly and regulatory discourse remains entrenched in abstract legal principles or normative frameworks, leaving the empirical landscape uncharted or minimally engaged. Since the birth of EU data protection law, a modest body of empirical evidence has been generated but remains widely scattered and unexamined. Such evidence offers vital insights into the perception, impact, clarity, and effects of data protection measures but languishes on the periphery, inadequately integrated into the broader conversation. To make a meaningful connection, we conduct a comprehensive review and synthesis of empirical research spanning nearly three decades (1995- March 2022), advocating for a more robust integration of empirical evidence into the evaluation and review of the GDPR, while laying a methodological foundation for future empirical research.

cs.CY↗

arXiv2024

Mapping the Scholarship of Dark Pattern Regulation: A Systematic Review of Concepts, Regulatory Paradigms, and Solutions from an Interdisciplinary Perspective

Dark patterns, design tricks used on online interfaces to manipulate users decision-making process, have raised public concerns. However, research on regulation of dark pattern remains underdeveloped and scattered, particularly regarding scholars views on the concept, regulatory paradigms, and solutions. Following PRISMA guidelines, this paper systematically reviews the formats and content of regulatory discussions on dark patterns from the interdisciplinary scholarship of Law and Human-Computer Interaction. A total of 65 studies were analysed through content and thematic analysis. This study synthesises the unique trends and characteristics of legal scholarship on dark patterns, identifying five root problems and triple layered harms. It critiques current regulations in terms of legal theories and sectoral legislations, highlighting their inadequacies in addressing dark patterns. The paper also critically examines existing proposed solutions, including paradigmatic shifts in legal doctrines, refinements to existing frameworks, technical design-embedded solutions, and accountability measures for design practices. This research critically discusses the current barriers to effective dark pattern regulations and explores promising regulatory solutions. The difficulty in identifying the normative nature of various forms of dark patterns, in identifying evident and actionable harm, and the expanding scope of dark patterns connotation inherently hinders effective regulation. However, technical design-embedded solutions, accountability frameworks, and practical design guidelines offer potential routes for more proactive regulation, while legal pluralism stands as a promising macro-level change in regulatory paradigms for dark pattern regulation.

cs.CY↗

arXiv2024

Global Attention-Guided Dual-Domain Point Cloud Feature Learning for Classification and Segmentation

Previous studies have demonstrated the effectiveness of point-based neural models on the point cloud analysis task. However, there remains a crucial issue on producing the efficient input embedding for raw point coordinates. Moreover, another issue lies in the limited efficiency of neighboring aggregations, which is a critical component in the network stem. In this paper, we propose a Global Attention-guided Dual-domain Feature Learning network (GAD) to address the above-mentioned issues. We first devise the Contextual Position-enhanced Transformer (CPT) module, which is armed with an improved global attention mechanism, to produce a global-aware input embedding that serves as the guidance to subsequent aggregations. Then, the Dual-domain K-nearest neighbor Feature Fusion (DKFF) is cascaded to conduct effective feature aggregation through novel dual-domain feature learning which appreciates both local geometric relations and long-distance semantic connections. Extensive experiments on multiple point cloud analysis tasks (e.g., classification, part segmentation, and scene semantic segmentation) demonstrate the superior performance of the proposed method and the efficacy of the devised modules.

cs.CV↗

arXiv2024

ShiftAddAug: Augment Multiplication-Free Tiny Neural Network with Hybrid Computation

Operators devoid of multiplication, such as Shift and Add, have gained prominence for their compatibility with hardware. However, neural networks (NNs) employing these operators typically exhibit lower accuracy compared to conventional NNs with identical structures. ShiftAddAug uses costly multiplication to augment efficient but less powerful multiplication-free operators, improving performance without any inference overhead. It puts a ShiftAdd tiny NN into a large multiplicative model and encourages it to be trained as a sub-model to obtain additional supervision. In order to solve the weight discrepancy problem between hybrid operators, a new weight sharing method is proposed. Additionally, a novel two stage neural architecture search is used to obtain better augmentation effects for smaller but stronger multiplication-free tiny neural networks. The superiority of ShiftAddAug is validated through experiments in image classification and semantic segmentation, consistently delivering noteworthy enhancements. Remarkably, it secures up to a 4.95% increase in accuracy on the CIFAR100 compared to its directly trained counterparts, even surpassing the performance of multiplicative NNs.

cs.LG↗

arXiv2024

Efficient Sparse Attention needs Adaptive Token Release

In recent years, Large Language Models (LLMs) have demonstrated remarkable capabilities across a wide array of text-centric tasks. However, their `large' scale introduces significant computational and storage challenges, particularly in managing the key-value states of the transformer, which limits their wider applicability. Therefore, we propose to adaptively release resources from caches and rebuild the necessary key-value states. Particularly, we accomplish this by a lightweight controller module to approximate an ideal top-$K$ sparse attention. This module retains the tokens with the highest top-$K$ attention weights and simultaneously rebuilds the discarded but necessary tokens, which may become essential for future decoding. Comprehensive experiments in natural language generation and modeling reveal that our method is not only competitive with full attention in terms of performance but also achieves a significant throughput improvement of up to 221.8%. The code for replication is available on the https://github.com/WHUIR/ADORE.

cs.CL↗

arXiv2024

Nurgle: Exacerbating Resource Consumption in Blockchain State Storage via MPT Manipulation

Blockchains, with intricate architectures, encompass various components, e.g., consensus network, smart contracts, decentralized applications, and auxiliary services. While offering numerous advantages, these components expose various attack surfaces, leading to severe threats to blockchains. In this study, we unveil a novel attack surface, i.e., the state storage, in blockchains. The state storage, based on the Merkle Patricia Trie, plays a crucial role in maintaining blockchain state. Besides, we design Nurgle, the first Denial-of-Service attack targeting the state storage. By proliferating intermediate nodes within the state storage, Nurgle forces blockchains to expend additional resources on state maintenance and verification, impairing their performance. We conduct a comprehensive and systematic evaluation of Nurgle, including the factors affecting it, its impact on blockchains, its financial cost, and practically demonstrating the resulting damage to blockchains. The implications of Nurgle extend beyond the performance degradation of blockchains, potentially reducing trust in them and the value of their cryptocurrencies. Additionally, we further discuss three feasible mitigations against Nurgle. At the time of writing, the vulnerability exploited by Nurgle has been confirmed by six mainstream blockchains, and we received thousands of USD bounty from them.

cs.CR↗

arXiv2024

Enhancing Counterfactual Image Generation Using Mahalanobis Distance with Distribution Preferences in Feature Space

In the realm of Artificial Intelligence (AI), the importance of Explainable Artificial Intelligence (XAI) is increasingly recognized, particularly as AI models become more integral to our lives. One notable single-instance XAI approach is counterfactual explanation, which aids users in comprehending a model's decisions and offers guidance on altering these decisions. Specifically in the context of image classification models, effective image counterfactual explanations can significantly enhance user understanding. This paper introduces a novel method for computing feature importance within the feature space of a black-box model. By employing information fusion techniques, our method maximizes the use of data to address feature counterfactual explanations in the feature space. Subsequently, we utilize an image generation model to transform these feature counterfactual explanations into image counterfactual explanations. Our experiments demonstrate that the counterfactual explanations generated by our method closely resemble the original images in both pixel and feature spaces. Additionally, our method outperforms established baselines, achieving impressive experimental results.

cs.LG↗

arXiv2024

Accuracy of training data and model outputs in Generative AI: CREATe Response to the Information Commissioner Office Consultation

The accuracy of Generative AI is increasingly critical as Large Language Models become more widely adopted. Due to potential flaws in training data and hallucination in outputs, inaccuracy can significantly impact individuals interests by distorting perceptions and leading to decisions based on flawed information. Therefore, ensuring these models accuracy is not only a technical necessity but also a regulatory imperative. ICO call for evidence on the accuracy of Generative AI marks a timely effort in ensuring responsible Generative AI development and use. CREATe, as the Centre for Regulation of the Creative Economy based at the University of Glasgow, has conducted relevant research involving intellectual property, competition, information and technology law. We welcome the ICO call for evidence on the accuracy of Generative AI, and we are happy to highlight aspects of data protection law and AI regulation that we believe should receive attention.

cs.CY↗

arXiv2024

SpherE: Expressive and Interpretable Knowledge Graph Embedding for Set Retrieval

Knowledge graphs (KGs), which store an extensive number of relational facts (head, relation, tail), serve various applications. While many downstream tasks highly rely on the expressive modeling and predictive embedding of KGs, most of the current KG representation learning methods, where each entity is embedded as a vector in the Euclidean space and each relation is embedded as a transformation, follow an entity ranking protocol. On one hand, such an embedding design cannot capture many-to-many relations. On the other hand, in many retrieval cases, the users wish to get an exact set of answers without any ranking, especially when the results are expected to be precise, e.g., which genes cause an illness. Such scenarios are commonly referred to as "set retrieval". This work presents a pioneering study on the KG set retrieval problem. We show that the set retrieval highly depends on expressive modeling of many-to-many relations, and propose a new KG embedding model SpherE to address this problem. SpherE is based on rotational embedding methods, but each entity is embedded as a sphere instead of a vector. While inheriting the high interpretability of rotational-based models, our SpherE can more expressively model one-to-many, many-to-one, and many-to-many relations. Through extensive experiments, we show that our SpherE can well address the set retrieval problem while still having a good predictive ability to infer missing facts. The code is available at https://github.com/Violet24K/SpherE.

cs.IR↗

arXiv2024

Allocating Mixed Goods with Customized Fairness and Indivisibility Ratio

We consider the problem of fairly allocating a combination of divisible and indivisible goods. While fairness criteria like envy-freeness (EF) and proportionality (PROP) can always be achieved for divisible goods, only their relaxed versions, such as the ''up to one'' relaxations EF1 and PROP1, can be satisfied when the goods are indivisible. The ''up to one'' relaxations require the fairness conditions to be satisfied provided that one good can be completely eliminated or added in the comparison. In this work, we bridge the gap between the two extremes and propose ''up to a fraction'' relaxations for the allocation of mixed divisible and indivisible goods. The fraction is determined based on the proportion of indivisible goods, which we call the indivisibility ratio. The new concepts also introduce asymmetric conditions that are customized for individuals with varying indivisibility ratios. We provide both upper and lower bounds on the fractions of the modified item in order to satisfy the fairness criterion. Our results are tight up to a constant for EF and asymptotically tight for PROP.

cs.GT↗

arXiv2024

Quantifying the escape of Ly$α$ at $z\approx 5-6$: a census of Ly$α$ escape fraction with H$α$ emitting galaxies spectroscopically confirmed by JWST and VLT/MUSE

JWST provides an unprecedented opportunity for unbiased surveys of H$α$-emitting galaxies at $z>4$ with the NIRCam wide-field slitless spectroscopy (WFSS). In this work, we present a census of Ly$α$ escape fraction ($f_{esc, Lyα}$) of 165 star-forming galaxies at $z=4.9-6.3$ using their H$α$ emission directly measured from FRESCO NIRCam/WFSS data. We search for Ly$α$ emission of each H$α$-emitting galaxy in VLT/MUSE data. The overall $f_{esc, Lyα}$ measured by stacking is $f_{esc, Lyα}$ is $0.090\pm0.006$. We find that $f_{esc, Lyα}$ displays a strong dependence on the observed UV slope ($β_{\rm obs}$) and E(B-V), such that the bluest galaxies ($β_{\rm obs}\sim-2.5$) have the largest escape fractions ($f_{\rm esc, Lyα}\approx0.6$), indicative of the crucial role of dust and gas in modulating the escape of Ly$α$ photons. $f_{esc, Lyα}$ is less well related to other parameters, including the UV luminosity and stellar mass, and the variation in $f_{esc, Lyα}$ with them can be explained by their underlying coupling with E(B-V) or $β_{\rm obs}$. Our results suggest a tentative decline in $f_{esc, Lyα}$ at $z\gtrsim 5$, implying increasing intergalactic medium attenuation towards higher redshift. Furthermore, the dependence of $f_{esc, Lyα}$ on $β_{\rm obs}$ is proportional to that of the ionizing photon escape fraction ($f_{\rm esc, LyC}$), indicating the escape of Ly$α$ and ionizing photon may be regulated by similar physical processes. With $f_{esc, Lyα}$ as a proxy to $f_{\rm esc, LyC}$, we infer that UV-faint ($M_{\rm UV}>-16$) galaxies contribute $>70\%$ of the total ionizing emissivity at $z=5-6$. If these relations hold during the epoch of reionization, UV-faint galaxies can contribute the majority of UV photon budget to reionize the Universe.

astro-ph.GA↗

arXiv2024

Multi-level Graph Subspace Contrastive Learning for Hyperspectral Image Clustering

Hyperspectral image (HSI) clustering is a challenging task due to its high complexity. Despite subspace clustering shows impressive performance for HSI, traditional methods tend to ignore the global-local interaction in HSI data. In this study, we proposed a multi-level graph subspace contrastive learning (MLGSC) for HSI clustering. The model is divided into the following main parts. Graph convolution subspace construction: utilizing spectral and texture feautures to construct two graph convolution views. Local-global graph representation: local graph representations were obtained by step-by-step convolutions and a more representative global graph representation was obtained using an attention-based pooling strategy. Multi-level graph subspace contrastive learning: multi-level contrastive learning was conducted to obtain local-global joint graph representations, to improve the consistency of the positive samples between views, and to obtain more robust graph embeddings. Specifically, graph-level contrastive learning is used to better learn global representations of HSI data. Node-level intra-view and inter-view contrastive learning is designed to learn joint representations of local regions of HSI. The proposed model is evaluated on four popular HSI datasets: Indian Pines, Pavia University, Houston, and Xu Zhou. The overall accuracies are 97.75%, 99.96%, 92.28%, and 95.73%, which significantly outperforms the current state-of-the-art clustering methods.

cs.CV↗

arXiv2024

S2RC-GCN: A Spatial-Spectral Reliable Contrastive Graph Convolutional Network for Complex Land Cover Classification Using Hyperspectral Images

Spatial correlations between different ground objects are an important feature of mining land cover research. Graph Convolutional Networks (GCNs) can effectively capture such spatial feature representations and have demonstrated promising results in performing hyperspectral imagery (HSI) classification tasks of complex land. However, the existing GCN-based HSI classification methods are prone to interference from redundant information when extracting complex features. To classify complex scenes more effectively, this study proposes a novel spatial-spectral reliable contrastive graph convolutional classification framework named S2RC-GCN. Specifically, we fused the spectral and spatial features extracted by the 1D- and 2D-encoder, and the 2D-encoder includes an attention model to automatically extract important information. We then leveraged the fused high-level features to construct graphs and fed the resulting graphs into the GCNs to determine more effective graph representations. Furthermore, a novel reliable contrastive graph convolution was proposed for reliable contrastive learning to learn and fuse robust features. Finally, to test the performance of the model on complex object classification, we used imagery taken by Gaofen-5 in the Jiang Xia area to construct complex land cover datasets. The test results show that compared with other models, our model achieved the best results and effectively improved the classification performance of complex remote sensing imagery.

cs.CV↗

arXiv2024

Heterogeneous Contrastive Learning for Foundation Models and Beyond

In the era of big data and Artificial Intelligence, an emerging paradigm is to utilize contrastive self-supervised learning to model large-scale heterogeneous data. Many existing foundation models benefit from the generalization capability of contrastive self-supervised learning by learning compact and high-quality representations without relying on any label information. Amidst the explosive advancements in foundation models across multiple domains, including natural language processing and computer vision, a thorough survey on heterogeneous contrastive learning for the foundation model is urgently needed. In response, this survey critically evaluates the current landscape of heterogeneous contrastive learning for foundation models, highlighting the open challenges and future trends of contrastive learning. In particular, we first present how the recent advanced contrastive learning-based methods deal with view heterogeneity and how contrastive learning is applied to train and fine-tune the multi-view foundation models. Then, we move to contrastive learning methods for task heterogeneity, including pretraining tasks and downstream tasks, and show how different tasks are combined with contrastive learning loss for different purposes. Finally, we conclude this survey by discussing the open challenges and shedding light on the future directions of contrastive learning.

cs.LG↗

arXiv2024

Interpretable Machine Learning for Weather and Climate Prediction: A Survey

Advanced machine learning models have recently achieved high predictive accuracy for weather and climate prediction. However, these complex models often lack inherent transparency and interpretability, acting as "black boxes" that impede user trust and hinder further model improvements. As such, interpretable machine learning techniques have become crucial in enhancing the credibility and utility of weather and climate modeling. In this survey, we review current interpretable machine learning approaches applied to meteorological predictions. We categorize methods into two major paradigms: 1) Post-hoc interpretability techniques that explain pre-trained models, such as perturbation-based, game theory based, and gradient-based attribution methods. 2) Designing inherently interpretable models from scratch using architectures like tree ensembles and explainable neural networks. We summarize how each technique provides insights into the predictions, uncovering novel meteorological relationships captured by machine learning. Lastly, we discuss research challenges around achieving deeper mechanistic interpretations aligned with physical principles, developing standardized evaluation benchmarks, integrating interpretability into iterative model development workflows, and providing explainability for large foundation models.

physics.ao-ph↗