arXiv subjects

Yi Cai

Publications and source records attributed to Yi Cai.

At least 91 records · Page 5Linked to original sources

arXiv2023

Joint Multimodal Entity-Relation Extraction Based on Edge-enhanced Graph Alignment Network and Word-pair Relation Tagging

Multimodal named entity recognition (MNER) and multimodal relation extraction (MRE) are two fundamental subtasks in the multimodal knowledge graph construction task. However, the existing methods usually handle two tasks independently, which ignores the bidirectional interaction between them. This paper is the first to propose jointly performing MNER and MRE as a joint multimodal entity-relation extraction task (JMERE). Besides, the current MNER and MRE models only consider aligning the visual objects with textual entities in visual and textual graphs but ignore the entity-entity relationships and object-object relationships. To address the above challenges, we propose an edge-enhanced graph alignment network and a word-pair relation tagging (EEGA) for JMERE task. Specifically, we first design a word-pair relation tagging to exploit the bidirectional interaction between MNER and MRE and avoid the error propagation. Then, we propose an edge-enhanced graph alignment network to enhance the JMERE task by aligning nodes and edges in the cross-graph. Compared with previous methods, the proposed method can leverage the edge information to auxiliary alignment between objects and entities and find the correlations between entity-entity relationships and object-object relationships. Experiments are conducted to show the effectiveness of our model.

cs.CL↗

arXiv2023

Explaining text classifiers through progressive neighborhood approximation with realistic samples

The importance of neighborhood construction in local explanation methods has been already highlighted in the literature. And several attempts have been made to improve neighborhood quality for high-dimensional data, for example, texts, by adopting generative models. Although the generators produce more realistic samples, the intuitive sampling approaches in the existing solutions leave the latent space underexplored. To overcome this problem, our work, focusing on local model-agnostic explanations for text classifiers, proposes a progressive approximation approach that refines the neighborhood of a to-be-explained decision with a careful two-stage interpolation using counterfactuals as landmarks. We explicitly specify the two properties that should be satisfied by generative models, the reconstruction ability and the locality-preserving property, to guide the selection of generators for local explanation methods. Moreover, noticing the opacity of generative models during the study, we propose another method that implements progressive neighborhood approximation with probability-based editions as an alternative to the generator-based solution. The explanation results from both methods consist of word-level and instance-level explanations benefiting from the realistic neighborhood. Through exhaustive experiments, we qualitatively and quantitatively demonstrate the effectiveness of the two proposed methods.

cs.CL↗

arXiv2022

Power of Explanations: Towards automatic debiasing in hate speech detection

Hate speech detection is a common downstream application of natural language processing (NLP) in the real world. In spite of the increasing accuracy, current data-driven approaches could easily learn biases from the imbalanced data distributions originating from humans. The deployment of biased models could further enhance the existing social biases. But unlike handling tabular data, defining and mitigating biases in text classifiers, which deal with unstructured data, are more challenging. A popular solution for improving machine learning fairness in NLP is to conduct the debiasing process with a list of potentially discriminated words given by human annotators. In addition to suffering from the risks of overlooking the biased terms, exhaustively identifying bias with human annotators are unsustainable since discrimination is variable among different datasets and may evolve over time. To this end, we propose an automatic misuse detector (MiD) relying on an explanation method for detecting potential bias. And built upon that, an end-to-end debiasing framework with the proposed staged correction is designed for text classifiers without any external resources required.

cs.CL↗

arXiv2022

CLOSE: Curriculum Learning On the Sharing Extent Towards Better One-shot NAS

One-shot Neural Architecture Search (NAS) has been widely used to discover architectures due to its efficiency. However, previous studies reveal that one-shot performance estimations of architectures might not be well correlated with their performances in stand-alone training because of the excessive sharing of operation parameters (i.e., large sharing extent) between architectures. Thus, recent methods construct even more over-parameterized supernets to reduce the sharing extent. But these improved methods introduce a large number of extra parameters and thus cause an undesirable trade-off between the training costs and the ranking quality. To alleviate the above issues, we propose to apply Curriculum Learning On Sharing Extent (CLOSE) to train the supernet both efficiently and effectively. Specifically, we train the supernet with a large sharing extent (an easier curriculum) at the beginning and gradually decrease the sharing extent of the supernet (a harder curriculum). To support this training strategy, we design a novel supernet (CLOSENet) that decouples the parameters from operations to realize a flexible sharing scheme and adjustable sharing extent. Extensive experiments demonstrate that CLOSE can obtain a better ranking quality across different computational budget constraints than other one-shot supernets, and is able to discover superior architectures when combined with various search strategies. Code is available at https://github.com/walkerning/aw_nas.

cs.CV↗

arXiv2022

Bases which admit exactly two expansions

For a positive integer $m$ let $Ω_m=\{0,1, \cdots , m\}$ and \begin{align*} \mathcal B_2(m)=&\left \{q\in(1,m+1]: \text{$\exists\; x\in [0, m/(q-1)]$ has exactly }\right. \\ &\left. \text{two different $q$-expansions w.r.t. $Ω_m$}\right \}. \end{align*} Sidorov \cite{S} firstly studied the set $\mathcal B_2(1)$ and raised some questions. Komornik and Kong \cite{KK} further studied the set $\mathcal B_2(1)$ and answered partial Sidorov's questions. In the present paper, we consider the set $\mathcal B_2(m)$ for general positive integer $m$ and generalise the results obtained by Komornik and Kong.

math.NT↗

arXiv2021

Learning Large-scale Network Embedding from Representative Subgraph

We study the problem of large-scale network embedding, which aims to learn low-dimensional latent representations for network mining applications. Recent research in the field of network embedding has led to significant progress such as DeepWalk, LINE, NetMF, NetSMF. However, the huge size of many real-world networks makes it computationally expensive to learn network embedding from the entire network. In this work, we present a novel network embedding method called "NES", which learns network embedding from a small representative subgraph. NES leverages theories from graph sampling to efficiently construct representative subgraph with smaller size which can be used to make inferences about the full network, enabling significantly improved efficiency in embedding learning. Then, NES computes the network embedding from this representative subgraph, efficiently. Compared with well-known methods, extensive experiments on networks of various scales and types demonstrate that NES achieves comparable performance and significant efficiency superiority.

cs.SI↗

arXiv2021

Quadratic fractional solitons

We introduce a system combining the quadratic self-attractive or composite quadratic-cubic nonlinearity, acting in the combination with the fractional diffraction, which is characterized by its Lévy index $α$. The model applies to a gas of quantum particles moving by Lévy flights, with the quadratic term representing the Lee-Huang-Yang correction to the mean-field interactions. A family of fundamental solitons is constructed in a numerical form, while the dependence of its norm on the chemical potential characteristic is obtained in an exact analytical form. The family of \textit{quasi-Townes solitons}, appearing in the limit case of $α=1/2$, is investigated by means of a variational approximation. A nonlinear lattice, represented by spatially periodical modulation of the quadratic term, is briefly addressed too. The consideration of the interplay of competing quadratic (attractive) and cubic (repulsive) terms with a lattice potential reveals families of single-, double-, and triple-peak gap solitons (GSs) in two finite bandgaps. The competing nonlinearity gives rise to alternating regions of stability and instability of the GS, the stability intervals shrinking with the increase of the number of peaks in the GS.

cond-mat.quant-gas↗

arXiv2021

Single-shot framing integration photography with high spatial resolution at 5.3*1012 frames per second by an inversed 4f system

We present a framing integration ultrafast photography (FIP), which integrates the framing structure, codes a dynamic event by an inversed 4f system (I4F) and decodes regionally with high spatial resolution. In the experiment about laser-induced plasma, FIP achieved a framing rate of 5.3*1012 frames per second (fps) and an intrinsic spatial resolution of 110.4 lp/mm. It has an excellent spatio-temporal resolution and a compact and flexible structure. Hence, it can probe unrepeatable ultrafast intra- and inter-atomic/molecular dynamics, different in size and duration with high-quality. Besides, FIP can lay a foundation for integrating and simplifying ultrafast photography instruments. Here the minimum framing time (temporal resolution) is limited by only the laser pulse duration; be sure, attosecond laser technology may further increase framing rates by several orders of magnitude.

physics.optics↗

arXiv2021

XPROAX-Local explanations for text classification with progressive neighborhood approximation

The importance of the neighborhood for training a local surrogate model to approximate the local decision boundary of a black box classifier has been already highlighted in the literature. Several attempts have been made to construct a better neighborhood for high dimensional data, like texts, by using generative autoencoders. However, existing approaches mainly generate neighbors by selecting purely at random from the latent space and struggle under the curse of dimensionality to learn a good local decision boundary. To overcome this problem, we propose a progressive approximation of the neighborhood using counterfactual instances as initial landmarks and a careful 2-stage sampling approach to refine counterfactuals and generate factuals in the neighborhood of the input instance to be explained. Our work focuses on textual data and our explanations consist of both word-level explanations from the original instance (intrinsic) and the neighborhood (extrinsic) and factual- and counterfactual-instances discovered during the neighborhood generation process that further reveal the effect of altering certain parts in the input text. Our experiments on real-world datasets demonstrate that our method outperforms the competitors in terms of usefulness and stability (for the qualitative part) and completeness, compactness and correctness (for the quantitative part).

cs.LG↗

arXiv2021

Fast Extraction of Word Embedding from Q-contexts

The notion of word embedding plays a fundamental role in natural language processing (NLP). However, pre-training word embedding for very large-scale vocabulary is computationally challenging for most existing methods. In this work, we show that with merely a small fraction of contexts (Q-contexts)which are typical in the whole corpus (and their mutual information with words), one can construct high-quality word embedding with negligible errors. Mutual information between contexts and words can be encoded canonically as a sampling state, thus, Q-contexts can be fast constructed. Furthermore, we present an efficient and effective WEQ method, which is capable of extracting word embedding directly from these typical contexts. In practical scenarios, our algorithm runs 11$\sim$13 times faster than well-established methods. By comparing with well-known methods such as matrix factorization, word2vec, GloVeand fasttext, we demonstrate that our method achieves comparable performance on a variety of downstream NLP tasks, and in the meanwhile maintains run-time and resource advantages over all these baselines.

cs.CL↗

arXiv2021

Flat-floor bubbles, dark solitons, and vortices stabilized by inhomogeneous nonlinear media

We consider one- and two-dimensional (1D and 2D) optical or matter-wave media with a maximum of the local self-repulsion strength at the center, and a minimum at periphery. If the central area is broad enough, it supports ground states in the form of flat-floor \textquotedblleft bubbles", and topological excitations, in the form of dark solitons in 1D and vortices with winding number $m$ in 2D. Unlike bright solitons, delocalized bubbles and dark modes were not previously considered in this setting. The ground and excited states are accurately approximated by the Thomas-Fermi expressions. The 1D and 2D bubbles, as well as vortices with $m=1$, are completely stable, while the dark solitons and vortices with $m=2$ have nontrivial stability boundaries in their existence areas. Unstable dark solitons are expelled to the periphery, while unstable double vortices split in rotating pairs of unitary ones. Displaced stable vortices precess around the central point.

nlin.PS↗

arXiv2021

Localized modes and dark solitons sustained by nonlinear defects

Dark solitons and localized defect modes against periodic backgrounds are considered in arrays of waveguides with defocusing Kerr nonlinearity constituting a nonlinear lattice. Bright defect modes are supported by local increase of the nonlinearity, while dark defect modes are supported by a local decrease of the nonlinearity. Dark solitons exist for both types of the defect, although in the case of weak nonlinearity they feature side bright humps making the total energy propagating through the system larger than the energy transferred by the constant background. All considered defect modes are found stable. Dark solitons are characterized by relatively narrow windows of stability. Interactions of unstable dark solitons with bright and dark modes are described.

nlin.PS↗

arXiv2021

Stable and oscillating solitons of $\mathcal{PT}$-symmetric couplers with gain and loss in fractional dimension

Families of coupled solitons of $\mathcal{PT}$-symmetric physical models with gain and loss in fractional dimension and in settings with and without cross-interactions modulation (CIM), are reported. Profiles, powers, stability areas, and propagation dynamics of the obtained $\mathcal{PT}$-symmetric coupled solitons are investigated. By comparing the results of the models with and without CIM, we find that the stability area of the model with CIM is much broader than the one without CIM. Remarkably, oscillating $\mathcal{PT}$-symmetric coupled solitons can also exist in the model of CIM with the same coefficients of the self- and cross-interactions modulations. In addition, the period of these oscillating coupled solitons can be controlled by the linear coupling coefficient.

nlin.PS↗

arXiv2021

Bubbles and W-shaped solitons in Kerr media with fractional diffraction

We demonstrate that, with the help of a Gaussian potential barrier, dark modes in the form of a local depression ("bubbles") can be supported by the repulsive Kerr nonlinearity in combination with fractional dimension. Similarly, W-shaped modes are supported by a double potential barrier. Families of the modes are constructed in a numerical form, and also by means of the Thomas-Fermi and variational approximations. All these modes are stable, which is predicted by computation of eigenvalues for small perturbations and confirmed by direct numerical simulations.

nlin.PS↗

arXiv2021

Families of fundamental and multipole solitons in a cubic-quintic nonlinear lattice in fractional dimension

We construct families of fundamental, dipole, and tripole solitons in the fractional Schrödinger equation (FSE)\ incorporating self-focusing cubic and defocusing quintic terms modulated by factors $\cos ^{2}x$ and $\sin^{2}x$, respectively. While the fundamental solitons are similar to those in the model with the uniform nonlinearity, the multipole complexes exist only in the presence of the nonlinear lattice. The shapes and stability of all the solitons strongly depend on the Lévy index (LI)\ that determines the FSE fractionality. Stability areas are identified in the plane of LI and propagation constant by means of numerical methods, and some results are explained with the help of an analytical approximation. The stability areas are broadest for the fundamental solitons and narrowest for the tripoles.

nlin.PS↗

arXiv2021

Ensemble-in-One: Learning Ensemble within Random Gated Networks for Enhanced Adversarial Robustness

Adversarial attacks have rendered high security risks on modern deep learning systems. Adversarial training can significantly enhance the robustness of neural network models by suppressing the non-robust features. However, the models often suffer from significant accuracy loss on clean data. Ensemble training methods have emerged as promising solutions for defending against adversarial attacks by diversifying the vulnerabilities among the sub-models, simultaneously maintaining comparable accuracy as standard training. However, existing ensemble methods are with poor scalability, owing to the rapid complexity increase when including more sub-models in the ensemble. Moreover, in real-world applications, it is difficult to deploy an ensemble with multiple sub-models, owing to the tight hardware resource budget and latency requirement. In this work, we propose ensemble-in-one (EIO), a simple but efficient way to train an ensemble within one random gated network (RGN). EIO augments the original model by replacing the parameterized layers with multi-path random gated blocks (RGBs) to construct a RGN. By diversifying the vulnerability of the numerous paths within the RGN, better robustness can be achieved. It provides high scalability because the paths within an EIO network exponentially increase with the network depth. Our experiments demonstrate that EIO consistently outperforms previous ensemble training methods with even less computational overhead.

cs.LG↗

arXiv2021

Enabling Lower-Power Charge-Domain Nonvolatile In-Memory Computing with Ferroelectric FETs

Compute-in-memory (CiM) is a promising approach to alleviating the memory wall problem for domain-specific applications. Compared to current-domain CiM solutions, charge-domain CiM shows the opportunity for higher energy efficiency and resistance to device variations. However, the area occupation and standby leakage power of existing SRAMbased charge-domain CiM (CD-CiM) are high. This paper proposes the first concept and analysis of CD-CiM using nonvolatile memory (NVM) devices. The design implementation and performance evaluation are based on a proposed 2-transistor-1-capacitor (2T1C) CiM macro using ferroelectric field-effect-transistors (FeFETs), which is free from leakage power and much denser than the SRAM solution. With the supply voltage between 0.45V and 0.90V, operating frequency between 100MHz to 1.0GHz, binary neural network application simulations show over 47%, 60%, and 64% energy consumption reduction from existing SRAM-based CD-CiM, SRAM-based current-domain CiM, and RRAM-based current-domain CiM, respectively. For classifications in MNIST and CIFAR-10 data sets, the proposed FeFETbased CD-CiM achieves an accuracy over 95% and 80%, respectively.

cs.ET↗

arXiv2020

A Q-values Sharing Framework for Multiagent Reinforcement Learning under Budget Constraint

In teacher-student framework, a more experienced agent (teacher) helps accelerate the learning of another agent (student) by suggesting actions to take in certain states. In cooperative multiagent reinforcement learning (MARL), where agents need to cooperate with one another, a student may fail to cooperate well with others even by following the teachers' suggested actions, as the polices of all agents are ever changing before convergence. When the number of times that agents communicate with one another is limited (i.e., there is budget constraint), the advising strategy that uses actions as advices may not be good enough. We propose a partaker-sharer advising framework (PSAF) for cooperative MARL agents learning with budget constraint. In PSAF, each Q-learner can decide when to ask for Q-values and share its Q-values. We perform experiments in three typical multiagent learning problems. Evaluation results show that our approach PSAF outperforms existing advising methods under both unlimited and limited budget, and we give an analysis of the impact of advising actions and sharing Q-values on agents' learning.

cs.MA↗