arXiv subjects

Ge Zhang

Publications and source records attributed to Ge Zhang.

At least 253 records · Page 14Linked to original sources

arXiv2022

HERB: Measuring Hierarchical Regional Bias in Pre-trained Language Models

Fairness has become a trending topic in natural language processing (NLP), which addresses biases targeting certain social groups such as genders and religions. However, regional bias in language models (LMs), a long-standing global discrimination problem, still remains unexplored. This paper bridges the gap by analysing the regional bias learned by the pre-trained language models that are broadly used in NLP tasks. In addition to verifying the existence of regional bias in LMs, we find that the biases on regional groups can be strongly influenced by the geographical clustering of the groups. We accordingly propose a HiErarchical Regional Bias evaluation method (HERB) utilising the information from the sub-region clusters to quantify the bias in pre-trained LMs. Experiments show that our hierarchical metric can effectively evaluate the regional bias with respect to comprehensive topics and measure the potential regional bias that can be propagated to downstream tasks. Our codes are available at https://github.com/Bernard-Yang/HERB.

cs.CL↗

arXiv2022

1Cademy @ Causal News Corpus 2022: Leveraging Self-Training in Causality Classification of Socio-Political Event Data

This paper details our participation in the Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE) workshop @ EMNLP 2022, where we take part in Subtask 1 of Shared Task 3. We approach the given task of event causality detection by proposing a self-training pipeline that follows a teacher-student classifier method. More specifically, we initially train a teacher model on the true, original task data, and use that teacher model to self-label data to be used in the training of a separate student model for the final task prediction. We test how restricting the number of positive or negative self-labeled examples in the self-training process affects classification performance. Our final results show that using self-training produces a comprehensive performance improvement across all models and self-labeled training sets tested within the task of event causality sequence classification. On top of that, we find that self-training performance did not diminish even when restricting either positive/negative examples used in training. Our code is be publicly available at https://github.com/Gzhang-umich/1CademyTeamOfCASE.

cs.CL↗

arXiv2022

1Cademy @ Causal News Corpus 2022: Enhance Causal Span Detection via Beam-Search-based Position Selector

In this paper, we present our approach and empirical observations for Cause-Effect Signal Span Detection -- Subtask 2 of Shared task 3~\cite{tan-etal-2022-event} at CASE 2022. The shared task aims to extract the cause, effect, and signal spans from a given causal sentence. We model the task as a reading comprehension (RC) problem and apply a token-level RC-based span prediction paradigm to the task as the baseline. We explore different training objectives to fine-tune the model, as well as data augmentation (DA) tricks based on the language model (LM) for performance improvement. Additionally, we propose an efficient beam-search post-processing strategy to due with the drawbacks of span detection to obtain a further performance gain. Our approach achieves an average $F_1$ score of 54.15 and ranks \textbf{$1^{st}$} in the CASE competition. Our code is available at \url{https://github.com/Gzhang-umich/1CademyTeamOfCASE}.

cs.CL↗

arXiv2022

Emergent Microrobotic Oscillators via Asymmetry-Induced Order

Spontaneous low-frequency oscillations on the order of several hertz are the drivers of many crucial processes in nature. From bacterial swimming to mammal gaits, the conversion of static energy inputs into slowly oscillating electrical and mechanical power is key to the autonomy of organisms across scales. However, the fabrication of slow artificial oscillators at micrometre scales remains a major roadblock towards the development of fully-autonomous microrobots. Here, we report the emergence of a low-frequency relaxation oscillator from a simple collective of active microparticles interacting at the air-liquid interface of a peroxide drop. Their collective oscillations form chemomechanical and electrochemical limit cycles that enable the transduction of ambient chemical energy into periodic mechanical motion and on-board electrical currents. Surprisingly, the collective can oscillate robustly even as more particles are introduced, but only when we add a single particle with modified reactivity to intentionally break the system's permutation symmetry. We explain such emergent order through a novel thermodynamic mechanism for asymmetry-induced order. The energy harvested from the stabilized system oscillations enables the use of on-board electronic components, which we demonstrate by cyclically and synchronously driving microrobotic arms. This work highlights a new strategy for achieving low-frequency oscillations at the microscale that are otherwise difficult to observe outside of natural systems, paving the way for future microrobotic autonomy.

nlin.AO↗

arXiv2022

Correlation between entropy and generalizability in a neural network

Although neural networks can solve very complex machine-learning problems, the theoretical reason for their generalizability is still not fully understood. Here we use Wang-Landau Mote Carlo algorithm to calculate the entropy (logarithm of the volume of a part of the parameter space) at a given test accuracy, and a given training loss function value or training accuracy. Our results show that entropical forces help generalizability. Although our study is on a very simple application of neural networks (a spiral dataset and a small, fully-connected neural network), our approach should be useful in explaining the generalizability of more complicated neural networks in future works.

cond-mat.stat-mech↗

arXiv2022

1Cademy at Semeval-2022 Task 1: Investigating the Effectiveness of Multilingual, Multitask, and Language-Agnostic Tricks for the Reverse Dictionary Task

This paper describes our system for the SemEval2022 task of matching dictionary glosses to word embeddings. We focus on the Reverse Dictionary Track of the competition, which maps multilingual glosses to reconstructed vector representations. More specifically, models convert the input of sentences to three types of embeddings: SGNS, Char, and Electra. We propose several experiments for applying neural network cells, general multilingual and multitask structures, and language-agnostic tricks to the task. We also provide comparisons over different types of word embeddings and ablation studies to suggest helpful strategies. Our initial transformer-based model achieves relatively low performance. However, trials on different retokenization methodologies indicate improved performance. Our proposed Elmobased monolingual model achieves the highest outcome, and its multitask, and multilingual varieties show competitive results as well.

cs.CL↗

arXiv2022

Graph-level Neural Networks: Current Progress and Future Directions

Graph-structured data consisting of objects (i.e., nodes) and relationships among objects (i.e., edges) are ubiquitous. Graph-level learning is a matter of studying a collection of graphs instead of a single graph. Traditional graph-level learning methods used to be the mainstream. However, with the increasing scale and complexity of graphs, Graph-level Neural Networks (GLNNs, deep learning-based graph-level learning methods) have been attractive due to their superiority in modeling high-dimensional data. Thus, a survey on GLNNs is necessary. To frame this survey, we propose a systematic taxonomy covering GLNNs upon deep neural networks, graph neural networks, and graph pooling. The representative and state-of-the-art models in each category are focused on this survey. We also investigate the reproducibility, benchmarks, and new graph datasets of GLNNs. Finally, we conclude future directions to further push forward GLNNs. The repository of this survey is available at https://github.com/GeZhangMQ/Awesome-Graph-level-Neural-Networks.

cs.LG↗

arXiv2022

Rayleigh-Taylor instability under multi-mode perturbation: discrete Boltzmann modeling with tracers

The Rayleigh-Taylor Instability (RTI) under multi-mode perturbation in compressible flow is probed via the Discrete Boltzmann Modeling (DBM) with tracers. The distribution of tracers provides clear boundaries between light and heavy fluids in the position space. Besides, the position-velocity phase space offers a new perspective for understanding the flow behavior of RTI with intuitive geometrical correspondence. The effects of viscosity, acceleration, compressibility, and Atwood number on the mixing of material and momentum and the mean non-equilibrium strength at the interfaces are investigated separately based on both the mixedness defined by the tracers and the non-equilibrium strength defined by the DBM. The mixedness increases with viscosity during early stage but decreases with viscosity at the later stage. Acceleration, compressibility, and Atwood number show enhancement effects on mixing based on different mechanisms. After the system relaxes from the initial state, the mean non-equilibrium strength at the interfaces presents an initially increasing and then declining trend, which is jointly determined by the interface length and the macroscopic physical quantity gradient. We conclude that the four factors investigated all significantly affect early evolution behavior of RTI system, such as the competition between interface length and macroscopic physical quantity gradient. The results contribute to the understanding of the multi-mode RTI evolutionary mechanism and the accompanied kinetic effects.

physics.flu-dyn↗

arXiv2022

Structuro-elasto-plasticity (StEP) model for plasticity in disordered solids

Elastoplastic lattice models for the response of solids to deformation typically incorporate structure only implicitly via a local yield strain that is assigned to each site. However, the local yield strain can change in response to a nearby or even distant plastic event in the system. This interplay is key to understanding phenomena such as avalanches in which one plastic event can trigger another, leading to a cascade of events, but typically is neglected in elastoplastic models. To include the interplay one could calculate the local yield strain for a given particulate system and follow its evolution, but this is expensive and requires knowledge of particle interactions, which is often hard to extract from experiments. Instead, we introduce a structural quantity, "softness," obtained using machine learning to correlate with imminent plastic rearrangements. We show that softness also correlates with local yield strain. We incorporate softness to construct a "structuro-elasto-plasticity" model that reproduces particle simulation results quantitatively for several observable quantities, confirming that we capture the influence of the interplay of local structure, plasticity, and elasticity on material response.

cond-mat.soft↗

arXiv2021

Spectral Variability Augmented Sparse Unmixing of Hyperspectral Images

Spectral unmixing (SU) expresses the mixed pixels existed in hyperspectral images as the product of endmember and abundance, which has been widely used in hyperspectral imagery analysis. However, the influence of light, acquisition conditions and the inherent properties of materials, results in that the identified endmembers can vary spectrally within a given image (construed as spectral variability). To address this issue, recent methods usually use a priori obtained spectral library to represent multiple characteristic spectra of the same object, but few of them extracted the spectral variability explicitly. In this paper, a spectral variability augmented sparse unmixing model (SVASU) is proposed, in which the spectral variability is extracted for the first time. The variable spectra are divided into two parts of intrinsic spectrum and spectral variability for spectral reconstruction, and modeled synchronously in the SU model adding the regular terms restricting the sparsity of abundance and the generalization of the variability coefficient. It is noted that the spectral variability library and the intrinsic spectral library are all constructed from the In-situ observed image. Experimental results over both synthetic and real-world data sets demonstrate that the augmented decomposition by spectral variability significantly improves the unmixing performance than the decomposition only by spectral library, as well as compared to state-of-the-art algorithms.

eess.IV↗

arXiv2021

Tilting the playing field: Dynamical loss functions for machine learning

We show that learning can be improved by using loss functions that evolve cyclically during training to emphasize one class at a time. In underparameterized networks, such dynamical loss functions can lead to successful training for networks that fail to find a deep minima of the standard cross-entropy loss. In overparameterized networks, dynamical loss functions can lead to better generalization. Improvement arises from the interplay of the changing loss landscape with the dynamics of the system as it evolves to minimize the loss. In particular, as the loss function oscillates, instabilities develop in the form of bifurcation cascades, which we study using the Hessian and Neural Tangent Kernel. Valleys in the landscape widen and deepen, and then narrow and rise as the loss landscape changes during a cycle. As the landscape narrows, the learning rate becomes too large and the network becomes unstable and bounces around the valley. This process ultimately pushes the system into deeper and wider regions of the loss landscape and is characterized by decreasing eigenvalues of the Hessian. This results in better regularized models with improved generalization performance.

cs.LG↗

arXiv2021

StrobeNet: Category-Level Multiview Reconstruction of Articulated Objects

We present StrobeNet, a method for category-level 3D reconstruction of articulating objects from one or more unposed RGB images. Reconstructing general articulating object categories % has important applications, but is challenging since objects can have wide variation in shape, articulation, appearance and topology. We address this by building on the idea of category-level articulation canonicalization -- mapping observations to a canonical articulation which enables correspondence-free multiview aggregation. Our end-to-end trainable neural network estimates feature-enriched canonical 3D point clouds, articulation joints, and part segmentation from one or more unposed images of an object. These intermediate estimates are used to generate a final implicit 3D reconstruction.Our approach reconstructs objects even when they are observed in different articulations in images with large baselines, and animation of reconstructed shapes. Quantitative and qualitative evaluations on different object categories show that our method is able to achieve high reconstruction accuracy, especially as more views are added.

cs.CV↗

arXiv2021

An Irreversible Synthetic Route to an Ultra-Strong Two-Dimensional Polymer

Polymers that extend covalently in two dimensions have attracted recent attention as a means of combining the mechanical strength and in-plane energy conduction of conventional two-dimensional (2D) materials with the low densities, synthetic processability, and organic composition of their one-dimensional counterparts. Efforts to date have proven successful in forms that do not allow full realization of these properties, such as polymerization at flat interfaces or fixation of monomers in immobilized lattices. A frequently employed synthetic approach is to introduce microscopic reversibility, at the cost of bond stability, to achieve 2D crystals after extensive error correction. Herein we demonstrate a synthetic route to 2D irreversible polycondensation directly in the solution phase, resulting in covalently bonded 2D polymer platelets that are chemically stable and highly processable. Further fabrication offers highly oriented, free-standing films which exhibit exceptional 2D elastic modulus and yield strength at 50.9 +- 15.0 GPa and 0.976 +- 0.113 GPa, respectively. Platelet alignment is evidenced by polarized photoluminescence centered at 580 and 680 nm from different dipole transitions. This new synthetic route provides opportunities for 2D polymers in applications ranging from composite structures to molecular sieving membranes.

cond-mat.soft↗

arXiv2021

Delineation of the flow and mixing induced by Rayleigh-Taylor instability through tracers

Rayleigh-Taylor-instability(RTI) induced flow and mixing are of great importance in both nature and engineering scenarios. To capture the underpinning physics, tracers are introduced to make a supplement to discrete Boltzmann simulation of RTI in compressible flows. Via marking two types of tracers with different colors, the tracer distribution provides a clear boundary of two fluids during the RTI evolution. Fine structures of the flow and thermodynamic nonequilibrium behavior around the interface in a miscible two-fluid system are delineated. Distribution of tracers in its velocity phase space makes a charming pattern showing quite dense information on the flow behavior, which opens a new perspective for analyzing and accessing significantly deep insights into the flow system. RTI mixing is further investigated via tracer defined local mixedness. The appearance of Kelvin-Helmholtz instability is quantitatively captured by mixedness averaged align the direction of the pressure gradient. The role of compressibility and viscosity on mixing are investigated separately, both of which show two-stage effect. The underlying mechanism of the two-stage effect is interpreted as the development of large structures at the initial stage and the generation of small structures at the late stage. At the late stage, for a fixed time, a saturation phenomenon of viscosity is found that further increase of viscosity cannot see an evident decline in mixedness. The mixing statues of heavy and light fluids are not synchronous and the mixing of a RTI system is heterogenous. The results are helpful for understanding the mechanism of flow and mixing induced by RTI.

physics.flu-dyn↗

arXiv2020

Diverse Melody Generation from Chinese Lyrics via Mutual Information Maximization

In this paper, we propose to adapt the method of mutual information maximization into the task of Chinese lyrics conditioned melody generation to improve the generation quality and diversity. We employ scheduled sampling and force decoding techniques to improve the alignment between lyrics and melodies. With our method, which we called Diverse Melody Generation (DMG), a sequence-to-sequence model learns to generate diverse melodies heavily depending on the input style ids, while keeping the tonality and improving the alignment. The experimental results of subjective tests show that DMG can generate more pleasing and coherent tunes than baseline methods.

cs.SD↗

arXiv2020

Interplay of rearrangements, strain, and local structure during avalanche propagation

Jammed soft disks exhibit avalanches of particle rearrangements under quasistatic shear. We follow the avalanches using steepest descent to decompose them into individual localized rearrangements. We characterize the local structural environment of each particle by a machine-learned quantity, softness, designed to be highly correlated with rearrangements, and analyze the interplay between softness, rearrangements and strain. Our findings form the foundation of an augmented elastoplastic model that includes local structure.

cond-mat.soft↗

arXiv2020

CORAL: COde RepresentAtion Learning with Weakly-Supervised Transformers for Analyzing Data Analysis

Large scale analysis of source code, and in particular scientific source code, holds the promise of better understanding the data science process, identifying analytical best practices, and providing insights to the builders of scientific toolkits. However, large corpora have remained unanalyzed in depth, as descriptive labels are absent and require expert domain knowledge to generate. We propose a novel weakly supervised transformer-based architecture for computing joint representations of code from both abstract syntax trees and surrounding natural language comments. We then evaluate the model on a new classification task for labeling computational notebook cells as stages in the data analysis process from data import to wrangling, exploration, modeling, and evaluation. We show that our model, leveraging only easily-available weak supervision, achieves a 38% increase in accuracy over expert-supplied heuristics and outperforms a suite of baselines. Our model enables us to examine a set of 118,000 Jupyter Notebooks to uncover common data analysis patterns. Focusing on notebooks with relationships to academic articles, we conduct the largest ever study of scientific code and find that notebook composition correlates with the citation count of corresponding papers.

cs.LG↗

arXiv2020

Self-Supervised Joint Learning Framework of Depth Estimation via Implicit Cues

In self-supervised monocular depth estimation, the depth discontinuity and motion objects' artifacts are still challenging problems. Existing self-supervised methods usually utilize a single view to train the depth estimation network. Compared with static views, abundant dynamic properties between video frames are beneficial to refined depth estimation, especially for dynamic objects. In this work, we propose a novel self-supervised joint learning framework for depth estimation using consecutive frames from monocular and stereo videos. The main idea is using an implicit depth cue extractor which leverages dynamic and static cues to generate useful depth proposals. These cues can predict distinguishable motion contours and geometric scene structures. Furthermore, a new high-dimensional attention module is introduced to extract clear global transformation, which effectively suppresses uncertainty of local descriptors in high-dimensional space, resulting in a more reliable optimization in learning framework. Experiments demonstrate that the proposed framework outperforms the state-of-the-art(SOTA) on KITTI and Make3D datasets.

cs.CV↗