arXiv subjects

Muhao Chen

Publications and source records attributed to Muhao Chen.

At least 181 records · Page 10Linked to original sources

arXiv2021

Structure-Aware Label Smoothing for Graph Neural Networks

Representing a label distribution as a one-hot vector is a common practice in training node classification models. However, the one-hot representation may not adequately reflect the semantic characteristics of a node in different classes, as some nodes may be semantically close to their neighbors in other classes. It would cause over-confidence since the models are encouraged to assign full probabilities when classifying every node. While training models with label smoothing can ease this problem to some degree, it still fails to capture the nodes' semantic characteristics implied by the graph structures. In this work, we propose a novel SALS (\textit{Structure-Aware Label Smoothing}) method as an enhancement component to popular node classification models. SALS leverages the graph structures to capture the semantic correlations between the connected nodes and generate the structure-aware label distribution to replace the original one-hot label vectors, thus improving the node classification performance without inference costs. Extensive experiments on seven node classification benchmark datasets reveal the effectiveness of our SALS on improving both transductive and inductive node classification. Empirical results show that SALS is superior to the label smoothing method and enhances the node classification models to outperform the baseline methods.

cs.LG↗

arXiv2021

Dynamics and control of clustered tensegrity systems

This paper presents the formulations of nonlinear and linearized statics, dynamics, and control for any clustered tensegrity system (CTS). Based on the Lagrangian method and FEM assumptions, the nonlinear clustered tensegrity dynamics with and without constraints are first derived. It is shown that the traditional tensegrity system (TTS), whose node to node strings are individual ones, yield to be a special case of the CTS. Then, equilibrium equations of the CTS in three standard forms (in terms of nodal coordinate, force density, and force vector) and the compatibility equation are given. Moreover, the linearized dynamics and modal analysis of the CTS with and without constraints are also derived. We also present a nonlinear shape control law for the control of any CTS. The control turns out to be a linear algebra problem in terms of the control variable, which is the force densities in the strings. The statics, dynamics, and control examples are carefully selected to demonstrate the developed principles. The presented approaches can boost the comprehensive studies of the statics, dynamics, and control for any CTS, as well as promoting the integration of structure and control design.

eess.SY↗

arXiv2021

Salience-Aware Event Chain Modeling for Narrative Understanding

Storytelling, whether via fables, news reports, documentaries, or memoirs, can be thought of as the communication of interesting and related events that, taken together, form a concrete process. It is desirable to extract the event chains that represent such processes. However, this extraction remains a challenging problem. We posit that this is due to the nature of the texts from which chains are discovered. Natural language text interleaves a narrative of concrete, salient events with background information, contextualization, opinion, and other elements that are important for a variety of necessary discourse and pragmatics acts but are not part of the principal chain of events being communicated. We introduce methods for extracting this principal chain from natural language text, by filtering away non-salient events and supportive sentences. We demonstrate the effectiveness of our methods at isolating critical event chains by comparing their effect on downstream tasks. We show that by pre-training large language models on our extracted chains, we obtain improvements in two tasks that benefit from a clear understanding of event chains: narrative prediction and event-based temporal question answering. The demonstrated improvements and ablative studies confirm that our extraction method isolates critical event chains.

cs.CL↗

arXiv2021

HyperExpan: Taxonomy Expansion with Hyperbolic Representation Learning

Taxonomies are valuable resources for many applications, but the limited coverage due to the expensive manual curation process hinders their general applicability. Prior works attempt to automatically expand existing taxonomies to improve their coverage by learning concept embeddings in Euclidean space, while taxonomies, inherently hierarchical, more naturally align with the geometric properties of a hyperbolic space. In this paper, we present HyperExpan, a taxonomy expansion algorithm that seeks to preserve the structure of a taxonomy in a more expressive hyperbolic embedding space and learn to represent concepts and their relations with a Hyperbolic Graph Neural Network (HGNN). Specifically, HyperExpan leverages position embeddings to exploit the structure of the existing taxonomies, and characterizes the concept profile information to support the inference on unseen concepts during training. Experiments show that our proposed HyperExpan outperforms baseline models with representation learning in a Euclidean feature space and achieves state-of-the-art performance on the taxonomy expansion benchmarks.

cs.CL↗

arXiv2021

Learning Constraints and Descriptive Segmentation for Subevent Detection

Event mentions in text correspond to real-world events of varying degrees of granularity. The task of subevent detection aims to resolve this granularity issue, recognizing the membership of multi-granular events in event complexes. Since knowing the span of descriptive contexts of event complexes helps infer the membership of events, we propose the task of event-based text segmentation (EventSeg) as an auxiliary task to improve the learning for subevent detection. To bridge the two tasks together, we propose an approach to learning and enforcing constraints that capture dependencies between subevent detection and EventSeg prediction, as well as guiding the model to make globally consistent inference. Specifically, we adopt Rectifier Networks for constraint learning and then convert the learned constraints to a regularization term in the loss function of the neural model. Experimental results show that the proposed method outperforms baseline methods by 2.3% and 2.5% on benchmark datasets for subevent detection, HiEve and IC, respectively, while achieving a decent performance on EventSeg prediction.

cs.CL↗

arXiv2021

Table-based Fact Verification with Salience-aware Learning

Tables provide valuable knowledge that can be used to verify textual statements. While a number of works have considered table-based fact verification, direct alignments of tabular data with tokens in textual statements are rarely available. Moreover, training a generalized fact verification model requires abundant labeled training data. In this paper, we propose a novel system to address these problems. Inspired by counterfactual causality, our system identifies token-level salience in the statement with probing-based salience estimation. Salience estimation allows enhanced learning of fact verification from two perspectives. From one perspective, our system conducts masked salient token prediction to enhance the model for alignment and reasoning between the table and the statement. From the other perspective, our system applies salience-aware data augmentation to generate a more diverse set of training instances by replacing non-salient terms. Experimental results on TabFact show the effective improvement by the proposed salience-aware learning techniques, leading to the new SOTA performance on the benchmark. Our code is publicly available at https://github.com/luka-group/Salience-aware-Learning .

cs.CL↗

arXiv2021

Contrastive Out-of-Distribution Detection for Pretrained Transformers

Pretrained Transformers achieve remarkable performance when training and test data are from the same distribution. However, in real-world scenarios, the model often faces out-of-distribution (OOD) instances that can cause severe semantic shift problems at inference time. Therefore, in practice, a reliable model should identify such instances, and then either reject them during inference or pass them over to models that handle another distribution. In this paper, we develop an unsupervised OOD detection method, in which only the in-distribution (ID) data are used in training. We propose to fine-tune the Transformers with a contrastive loss, which improves the compactness of representations, such that OOD instances can be better differentiated from ID ones. These OOD instances can then be accurately detected using the Mahalanobis distance in the model's penultimate layer. We experiment with comprehensive settings and achieve near-perfect OOD detection performance, outperforming baselines drastically. We further investigate the rationales behind the improvement, finding that more compact representations through margin-based contrastive learning bring the improvement. We release our code to the community for future research.

cs.CL↗

arXiv2021

Learning from Noisy Labels for Entity-Centric Information Extraction

Recent information extraction approaches have relied on training deep neural models. However, such models can easily overfit noisy labels and suffer from performance degradation. While it is very costly to filter noisy labels in large learning resources, recent studies show that such labels take more training steps to be memorized and are more frequently forgotten than clean labels, therefore are identifiable in training. Motivated by such properties, we propose a simple co-regularization framework for entity-centric information extraction, which consists of several neural models with identical structures but different parameter initialization. These models are jointly optimized with the task-specific losses and are regularized to generate similar predictions based on an agreement loss, which prevents overfitting on noisy labels. Extensive experiments on two widely used but noisy benchmarks for information extraction, TACRED and CoNLL03, demonstrate the effectiveness of our framework. We release our code to the community for future research.

cs.CL↗

arXiv2021

Do Language Models Perform Generalizable Commonsense Inference?

Inspired by evidence that pretrained language models (LMs) encode commonsense knowledge, recent work has applied LMs to automatically populate commonsense knowledge graphs (CKGs). However, there is a lack of understanding on their generalization to multiple CKGs, unseen relations, and novel entities. This paper analyzes the ability of LMs to perform generalizable commonsense inference, in terms of knowledge capacity, transferability, and induction. Our experiments with these three aspects show that: (1) LMs can adapt to different schemas defined by multiple CKGs but fail to reuse the knowledge to generalize to new relations. (2) Adapted LMs generalize well to unseen subjects, but less so on novel objects. Future work should investigate how to improve the transferability and induction of commonsense mining from LMs.

cs.CL↗

arXiv2021

Minimal mass design of a tensegrity tower for lunar electromagnetic launching

Lunar explorations have provided us with information about its abundant resources that can be utilized in orbiting-resource depots as lunar-derived commodities. To reduce the energy requirements of a launcher to send these commodities from the lunar surface to the space depots, this paper explores the application of the electromagnetic acceleration principle and provides an assessment of the actual technical characteristics of the launcher's installation to ensure the acceleration of a payload with a mass of 1,500 kg to a speed of 2,200 m/s (circumlunar orbit speed). To fulfill a lightweight (fewer materials and less energy) support structure for the electromagnetic launcher with strength requirements, the tensegrity structure minimum mass principle without global buckling has been developed and applied to support the electromagnetic acceleration device. Therefore, this paper proposes and develops a minimal mass electromagnetic tensegrity lunar launcher. We first demonstrate the mechanics of launcher and payload, how a payload can be accelerated to a specific velocity, and how a payload carrier can be recycled for another launch. Then, a detailed discussion on the lunar launch system, procedures of propulsion, the required mass, and energy of the launch barrel are given. The governing equations of tensegrity minimal mass tensegrity design algorithm with gravity and without global buckling. Finally, a case study is conducted to show a feasible structure design, the required mass, and energy. The principles developed in this paper are also applicable to the rocket launch system, space elevator, space train transportation, interstellar payload package delivery, etc.

physics.acc-ph↗

arXiv2021

Design and analysis of deployable clustered tensegrity cable domes

This study presents the design and analysis of deployable cable domes based on the clustered tensegrity structures (CTS). In this paper, the statics and dynamics equations of the CTS are first given. Using a traditional Levy cable dome as an example, we show the approach to modify the Levy dome to a deployable CTS one. The strings to be clustered are determined by the requirement of prestress mode and global stability. The deployment trajectory is proposed by changing the deployment ratio (the ratio between the radius of the inner and outer rings of the cable dome). Then, the quasi-static and dynamic deployment of clustered tensegrity dome is studied. Results show that the proposed CTS cable dome always has one prestress mode and is globally stable in its deployment trajectory. In the deployment process analysis, the dynamics show that the system's dynamic response differs from the quasi-static simulation as the actuation speed increases. That is, for a fast deployment process, quasi-static simulation is not accurate enough. The dynamics effects of the deployment must be considered. The developed approaches can also be used for the design and analysis of various kinds of CTS.

eess.SY↗

arXiv2021

DREAMS: Drilling and Extraction Automated System

Drilling and Extraction Automated System (DREAMS) is a fully automated prototype-drilling rig that can drill, extract water and assess subsurface density profiles from simulated lunar and Martian subsurface ice. DREAMS system is developed by the Texas A&M drilling automation team and composed of four main components: 1- tensegrity rig structure, 2- drilling system, 3- water extracting and heating system, and 4- electronic hardware, controls, and machine algorithm. The vertical and rotational movements are controlled by using an Acme rod, stepper, and rotary motor. DREAMS is a unique system and different from other systems presented before in the NASA Rascal-Al competition because 1- It uses the tensegrity structure concept to decrease the system weight, improve mobility, and easier installation in space. 2- It cuts rock layers by using a short bit length connected to drill pipes. This drilling methodology is expected to drill hundreds and thousands of meters below the moon and Martian surfaces without any anticipated problems (not only 1 m.). 3- Drilling, heating, and extraction systems are integrated into one system that can work simultaneously or individually to save time and cost.

cs.RO↗

arXiv2021

Knowing the No-match: Entity Alignment with Dangling Cases

This paper studies a new problem setting of entity alignment for knowledge graphs (KGs). Since KGs possess different sets of entities, there could be entities that cannot find alignment across them, leading to the problem of dangling entities. As the first attempt to this problem, we construct a new dataset and design a multi-task learning framework for both entity alignment and dangling entity detection. The framework can opt to abstain from predicting alignment for the detected dangling entities. We propose three techniques for dangling entity detection that are based on the distribution of nearest-neighbor distances, i.e., nearest neighbor classification, marginal ranking and background ranking. After detecting and removing dangling entities, an incorporated entity alignment model in our framework can provide more robust alignment for remaining entities. Comprehensive experiments and analyses demonstrate the effectiveness of our framework. We further discover that the dangling entity detection module can, in turn, improve alignment learning and the final performance. The contributed resource is publicly available to foster further research.

cs.CL↗

arXiv2021

Tensegrity system dynamics based on finite element method

This study presents a finite element analysis approach to non-linear and linearized tensegrity dynamics based on the Lagrangian method with nodal coordinate vectors as the generalized coordinates. In this paper, nonlinear tensegrity dynamics with and without constraints are first derived. The equilibrium equations in three standard forms (in terms of nodal coordinate, force density, and force vectors) and the compatibility equation are also given. Then, we present the linearized dynamics and modal analysis equations with and without constraints. The developed approach is capable of conducting the following comprehensive dynamics studies for any tensegrity structures accurately: 1. Performing rigid body dynamics with acceptable errors, which is achieved by setting relatively high stiffness for bars in the simulation. 2. Simulating FEM dynamics accurately, where bars and strings can have elastic or plastic deformations. 3. Dealing with various kinds of boundary conditions, for example, fixing or applying static/dynamic loads at any nodes in any direction (i.e., gravitational force, some specified forces, or arbitrary seismic vibrations). 4. Conducting accurate modal analysis, including natural frequency and corresponding modes. Three examples, a double pendulum, a cantilever truss with external force, and a double prism tensegrity tower, are carefully selected and studied. The results are compared with rigid body dynamics and FEM software ANSYS. This study provides a deep insight into structures, materials, performances, as well as an interface towards integrating control theories.

physics.app-ph↗

arXiv2021

Retrieving Complex Tables with Multi-Granular Graph Representation Learning

The task of natural language table retrieval (NLTR) seeks to retrieve semantically relevant tables based on natural language queries. Existing learning systems for this task often treat tables as plain text based on the assumption that tables are structured as dataframes. However, tables can have complex layouts which indicate diverse dependencies between subtable structures, such as nested headers. As a result, queries may refer to different spans of relevant content that is distributed across these structures. Moreover, such systems fail to generalize to novel scenarios beyond those seen in the training set. Prior methods are still distant from a generalizable solution to the NLTR problem, as they fall short in handling complex table layouts or queries over multiple granularities. To address these issues, we propose Graph-based Table Retrieval (GTR), a generalizable NLTR framework with multi-granular graph representation learning. In our framework, a table is first converted into a tabular graph, with cell nodes, row nodes and column nodes to capture content at different granularities. Then the tabular graph is input to a Graph Transformer model that can capture both table cell content and the layout structures. To enhance the robustness and generalizability of the model, we further incorporate a self-supervised pre-training task based on graph-context matching. Experimental results on two benchmarks show that our method leads to significant improvements over the current state-of-the-art systems. Further experiments demonstrate promising performance of our method on cross-dataset generalization, and enhanced capability of handling complex tables and fulfilling diverse query intents. Code and data are available at https://github.com/FeiWang96/GTR.

cs.IR↗

arXiv2021

Joint Constrained Learning for Event-Event Relation Extraction

Understanding natural language involves recognizing how multiple event mentions structurally and temporally interact with each other. In this process, one can induce event complexes that organize multi-granular events with temporal order and membership relations interweaving among them. Due to the lack of jointly labeled data for these relational phenomena and the restriction on the structures they articulate, we propose a joint constrained learning framework for modeling event-event relations. Specifically, the framework enforces logical constraints within and across multiple temporal and subevent relations by converting these constraints into differentiable learning objectives. We show that our joint constrained learning approach effectively compensates for the lack of jointly labeled data, and outperforms SOTA methods on benchmarks for both temporal relation extraction and event hierarchy construction, replacing a commonly used but more expensive global inference process. We also present a promising case study showing the effectiveness of our approach in inducing event complexes on an external corpus.

cs.CL↗

arXiv2021

Probabilistic Box Embeddings for Uncertain Knowledge Graph Reasoning

Knowledge bases often consist of facts which are harvested from a variety of sources, many of which are noisy and some of which conflict, resulting in a level of uncertainty for each triple. Knowledge bases are also often incomplete, prompting the use of embedding methods to generalize from known facts, however, existing embedding methods only model triple-level uncertainty, and reasoning results lack global consistency. To address these shortcomings, we propose BEUrRE, a novel uncertain knowledge graph embedding method with calibrated probabilistic semantics. BEUrRE models each entity as a box (i.e. axis-aligned hyperrectangle) and relations between two entities as affine transforms on the head and tail entity boxes. The geometry of the boxes allows for efficient calculation of intersections and volumes, endowing the model with calibrated probabilistic semantics and facilitating the incorporation of relational constraints. Extensive experiments on two benchmark datasets show that BEUrRE consistently outperforms baselines on confidence prediction and fact ranking due to its probabilistic calibration and ability to capture high-order dependencies among facts.

cs.AI↗

arXiv2021

Universal Representation Learning of Knowledge Bases by Jointly Embedding Instances and Ontological Concepts

Many large-scale knowledge bases simultaneously represent two views of knowledge graphs (KGs): an ontology view for abstract and commonsense concepts, and an instance view for specific entities that are instantiated from ontological concepts. Existing KG embedding models, however, merely focus on representing one of the two views alone. In this paper, we propose a novel two-view KG embedding model, JOIE, with the goal to produce better knowledge embedding and enable new applications that rely on multi-view knowledge. JOIE employs both cross-view and intra-view modeling that learn on multiple facets of the knowledge base. The cross-view association model is learned to bridge the embeddings of ontological concepts and their corresponding instance-view entities. The intra-view models are trained to capture the structured knowledge of instance and ontology views in separate embedding spaces, with a hierarchy-aware encoding technique enabled for ontologies with hierarchies. We explore multiple representation techniques for the two model components and investigate with nine variants of JOIE. Our model is trained on large-scale knowledge bases that consist of massive instances and their corresponding ontological concepts connected via a (small) set of cross-view links. Experimental results on public datasets show that the best variant of JOIE significantly outperforms previous models on instance-view triple prediction task as well as ontology population on ontologyview KG. In addition, our model successfully extends the use of KG embeddings to entity typing with promising performance.

cs.AI↗