arXiv subjects

Chen Zhang

Publications and source records attributed to Chen Zhang.

At least 469 records · Page 26Linked to original sources

arXiv2022

SDMuse: Stochastic Differential Music Editing and Generation via Hybrid Representation

While deep generative models have empowered music generation, it remains a challenging and under-explored problem to edit an existing musical piece at fine granularity. In this paper, we propose SDMuse, a unified Stochastic Differential Music editing and generation framework, which can not only compose a whole musical piece from scratch, but also modify existing musical pieces in many ways, such as combination, continuation, inpainting, and style transferring. The proposed SDMuse follows a two-stage pipeline to achieve music generation and editing on top of a hybrid representation including pianoroll and MIDI-event. In particular, SDMuse first generates/edits pianoroll by iteratively denoising through a stochastic differential equation (SDE) based on a diffusion model generative prior, and then refines the generated pianoroll and predicts MIDI-event tokens auto-regressively. We evaluate the generated music of our method on ailabs1k7 pop music dataset in terms of quality and controllability on various music editing and generation tasks. Experimental results demonstrate the effectiveness of our proposed stochastic differential music editing and generation process, as well as the hybrid representations.

cs.SD↗

arXiv2022

FineD-Eval: Fine-grained Automatic Dialogue-Level Evaluation

Recent model-based reference-free metrics for open-domain dialogue evaluation exhibit promising correlations with human judgment. However, they either perform turn-level evaluation or look at a single dialogue quality dimension. One would expect a good evaluation metric to assess multiple quality dimensions at the dialogue level. To this end, we are motivated to propose a multi-dimensional dialogue-level metric, which consists of three sub-metrics with each targeting a specific dimension. The sub-metrics are trained with novel self-supervised objectives and exhibit strong correlations with human judgment for their respective dimensions. Moreover, we explore two approaches to combine the sub-metrics: metric ensemble and multitask learning. Both approaches yield a holistic metric that significantly outperforms individual sub-metrics. Compared to the existing state-of-the-art metric, the combined metrics achieve around 16% relative improvement on average across three high-quality dialogue-level evaluation benchmarks.

cs.CL↗

arXiv2022

A Focused Study on Sequence Length for Dialogue Summarization

Output length is critical to dialogue summarization systems. The dialogue summary length is determined by multiple factors, including dialogue complexity, summary objective, and personal preferences. In this work, we approach dialogue summary length from three perspectives. First, we analyze the length differences between existing models' outputs and the corresponding human references and find that summarization models tend to produce more verbose summaries due to their pretraining objectives. Second, we identify salient features for summary length prediction by comparing different model settings. Third, we experiment with a length-aware summarizer and show notable improvement on existing models if summary length can be well incorporated. Analysis and experiments are conducted on popular DialogSum and SAMSum datasets to validate our findings.

cs.CL↗

arXiv2022

A Little Excitement Across the Horizon

We analyse numerically the transitions in an Unruh-DeWitt detector, coupled linearly to a massless scalar field, in radial infall in (3+1)-dimensional Schwarzschild spacetime. In the Hartle-Hawking and Unruh states, the transition probability attains a small local extremum near the horizon-crossing and is then moderately enhanced on approaching the singularity. In the Boulware state, the transition probability drops on approaching the horizon. The unexpected near-horizon extremum arises numerically from angular momentum superpositions, with a deeper physical explanation to be found.

gr-qc↗

arXiv2022

Analyzing and Evaluating Faithfulness in Dialogue Summarization

Dialogue summarization is abstractive in nature, making it suffer from factual errors. The factual correctness of summaries has the highest priority before practical applications. Many efforts have been made to improve faithfulness in text summarization. However, there is a lack of systematic study on dialogue summarization systems. In this work, we first perform the fine-grained human analysis on the faithfulness of dialogue summaries and observe that over 35% of generated summaries are faithfully inconsistent respective the source dialogues. Furthermore, we present a new model-level faithfulness evaluation method. It examines generation models with multi-choice questions created by rule-based transformations. Experimental results show that our evaluation schema is a strong proxy for the factual correctness of summarization models. The human-annotated faithfulness samples and the evaluation toolkit are released to facilitate future research toward faithful dialogue summarization.

cs.CL↗

arXiv2022

Sparse Teachers Can Be Dense with Knowledge

Recent advances in distilling pretrained language models have discovered that, besides the expressiveness of knowledge, the student-friendliness should be taken into consideration to realize a truly knowledgable teacher. Based on a pilot study, we find that over-parameterized teachers can produce expressive yet student-unfriendly knowledge and are thus limited in overall knowledgableness. To remove the parameters that result in student-unfriendliness, we propose a sparse teacher trick under the guidance of an overall knowledgable score for each teacher parameter. The knowledgable score is essentially an interpolation of the expressiveness and student-friendliness scores. The aim is to ensure that the expressive parameters are retained while the student-unfriendly ones are removed. Extensive experiments on the GLUE benchmark show that the proposed sparse teachers can be dense with knowledge and lead to students with compelling performance in comparison with a series of competitive baselines.

cs.CL↗

arXiv2022

SongDriver: Real-time Music Accompaniment Generation without Logical Latency nor Exposure Bias

Real-time music accompaniment generation has a wide range of applications in the music industry, such as music education and live performances. However, automatic real-time music accompaniment generation is still understudied and often faces a trade-off between logical latency and exposure bias. In this paper, we propose SongDriver, a real-time music accompaniment generation system without logical latency nor exposure bias. Specifically, SongDriver divides one accompaniment generation task into two phases: 1) The arrangement phase, where a Transformer model first arranges chords for input melodies in real-time, and caches the chords for the next phase instead of playing them out. 2) The prediction phase, where a CRF model generates playable multi-track accompaniments for the coming melodies based on previously cached chords. With this two-phase strategy, SongDriver directly generates the accompaniment for the upcoming melody, achieving zero logical latency. Furthermore, when predicting chords for a timestep, SongDriver refers to the cached chords from the first phase rather than its previous predictions, which avoids the exposure bias problem. Since the input length is often constrained under real-time conditions, another potential problem is the loss of long-term sequential information. To make up for this disadvantage, we extract four musical features from a long-term music piece before the current time step as global information. In the experiment, we train SongDriver on some open-source datasets and an original àiSong Dataset built from Chinese-style modern pop music scores. The results show that SongDriver outperforms existing SOTA (state-of-the-art) models on both objective and subjective metrics, meanwhile significantly reducing the physical latency.

cs.SD↗

arXiv2022

Making Pretrained Language Models Good Long-tailed Learners

Prompt-tuning has shown appealing performance in few-shot classification by virtue of its capability in effectively exploiting pre-trained knowledge. This motivates us to check the hypothesis that prompt-tuning is also a promising choice for long-tailed classification, since the tail classes are intuitively few-shot ones. To achieve this aim, we conduct empirical studies to examine the hypothesis. The results demonstrate that prompt-tuning makes pretrained language models at least good long-tailed learners. For intuitions on why prompt-tuning can achieve good performance in long-tailed classification, we carry out in-depth analyses by progressively bridging the gap between prompt-tuning and commonly used finetuning. The summary is that the classifier structure and parameterization form the key to making good long-tailed learners, in comparison with the less important input structure. Finally, we verify the applicability of our finding to few-shot classification. Good long-tailed learners can be abbreviated as Glee.

cs.CL↗

arXiv2022

XPrompt: Exploring the Extreme of Prompt Tuning

Prompt tuning learns soft prompts to condition frozen Pre-trained Language Models (PLMs) for performing downstream tasks in a parameter-efficient manner. While prompt tuning has gradually reached the performance level of fine-tuning as the model scale increases, there is still a large performance gap between prompt tuning and fine-tuning for models of moderate and small scales (typically less than 11B parameters). In this paper, we empirically show that the trained prompt tokens can have a negative impact on a downstream task and thus degrade its performance. To bridge the gap, we propose a novel Prompt tuning model with an eXtremely small scale (XPrompt) under the regime of lottery tickets hypothesis. Specifically, XPrompt eliminates the negative prompt tokens at different granularity levels through a hierarchical structured pruning, yielding a more parameter-efficient prompt yet with a competitive performance. Comprehensive experiments are carried out on SuperGLUE tasks, and the extensive results indicate that XPrompt is able to close the performance gap at smaller model scales.

cs.CL↗

arXiv2022

Does Thermal Really Always Matter for RGB-T Salient Object Detection?

In recent years, RGB-T salient object detection (SOD) has attracted continuous attention, which makes it possible to identify salient objects in environments such as low light by introducing thermal image. However, most of the existing RGB-T SOD models focus on how to perform cross-modality feature fusion, ignoring whether thermal image is really always matter in SOD task. Starting from the definition and nature of this task, this paper rethinks the connotation of thermal modality, and proposes a network named TNet to solve the RGB-T SOD task. In this paper, we introduce a global illumination estimation module to predict the global illuminance score of the image, so as to regulate the role played by the two modalities. In addition, considering the role of thermal modality, we set up different cross-modality interaction mechanisms in the encoding phase and the decoding phase. On the one hand, we introduce a semantic constraint provider to enrich the semantics of thermal images in the encoding phase, which makes thermal modality more suitable for the SOD task. On the other hand, we introduce a two-stage localization and complementation module in the decoding phase to transfer object localization cue and internal integrity cue in thermal features to the RGB modality. Extensive experiments on three datasets show that the proposed TNet achieves competitive performance compared with 20 state-of-the-art methods.

cs.CV↗

arXiv2022

CIR-Net: Cross-modality Interaction and Refinement for RGB-D Salient Object Detection

Focusing on the issue of how to effectively capture and utilize cross-modality information in RGB-D salient object detection (SOD) task, we present a convolutional neural network (CNN) model, named CIR-Net, based on the novel cross-modality interaction and refinement. For the cross-modality interaction, 1) a progressive attention guided integration unit is proposed to sufficiently integrate RGB-D feature representations in the encoder stage, and 2) a convergence aggregation structure is proposed, which flows the RGB and depth decoding features into the corresponding RGB-D decoding streams via an importance gated fusion unit in the decoder stage. For the cross-modality refinement, we insert a refinement middleware structure between the encoder and the decoder, in which the RGB, depth, and RGB-D encoder features are further refined by successively using a self-modality attention refinement unit and a cross-modality weighting refinement unit. At last, with the gradually refined features, we predict the saliency map in the decoder stage. Extensive experiments on six popular RGB-D SOD benchmarks demonstrate that our network outperforms the state-of-the-art saliency detectors both qualitatively and quantitatively.

cs.CV↗

arXiv2022

SoccerNet 2022 Challenges Results

The SoccerNet 2022 challenges were the second annual video understanding challenges organized by the SoccerNet team. In 2022, the challenges were composed of 6 vision-based tasks: (1) action spotting, focusing on retrieving action timestamps in long untrimmed videos, (2) replay grounding, focusing on retrieving the live moment of an action shown in a replay, (3) pitch localization, focusing on detecting line and goal part elements, (4) camera calibration, dedicated to retrieving the intrinsic and extrinsic camera parameters, (5) player re-identification, focusing on retrieving the same players across multiple views, and (6) multiple object tracking, focusing on tracking players and the ball through unedited video streams. Compared to last year's challenges, tasks (1-2) had their evaluation metrics redefined to consider tighter temporal accuracies, and tasks (3-6) were novel, including their underlying data and annotations. More information on the tasks, challenges and leaderboards are available on https://www.soccer-net.org. Baselines and development kits are available on https://github.com/SoccerNet.

cs.CV↗

arXiv2022

Data-mining of In-Situ TEM Experiments: on the Dynamics of Dislocations in CoCrFeMnNi Alloys

High entropy alloys are a class of materials with many significant improvements in terms of mechanical properties as compared to ``classical'' alloys. The corresponding structure-property relations are not yet entirely clear, but it is commonly believed that the good mechanical performance is strongly related to dislocation interactions with the complex energy landscape formed due to alloying. Although in-situ Transmission Electron Microscopy (TEM) allows high-resolution studies of the structure and dynamics of moving dislocations and makes the local obstacle/energy ``landscape'' directly visible in the geometry of dislocations; such observation, however, are merely qualitative, and detailed three-dimensional analyses of the interaction between dislocations and the energy landscape is still missing. In this work, we utilized dislocations as ``probes'' for the local energy maxima which play the role of pinning points for the dislocation movement. To this end, we developed a unique data-mining approach that can perform coarse-grained spatio-temporal analysis, making ensemble averaging of a considerable number of snapshots possible. We investigate the effect of pinning points on the dislocation gliding behavior of CoCrFeMnNi alloy during in-situ TEM straining and find that (i) the pinning point strength changes when dislocations glide through and (ii) the pinning point moves along the direction close to the Burgers vector direction. Our data-mining method can be applied to dislocation motion in general, making it a useful tool for dislocation research.

cond-mat.mtrl-sci↗

arXiv2022

X-ray performance of a customized large-format scientifc CMOS detector

In recent years, the performance of Scientifc Complementary Metal Oxide Semiconductor (sCMOS) sensors has been improved signifcantly. Compared with CCD sensors, sCMOS sensors have various advantages, making them potentially better devices for optical and X-ray detection, especially in time-domain astronomy. After a series of tests of sCMOS sensors, we proposed a new dedicated high-speed, large-format X-ray detector in 2016 cooperating with Gpixel Inc. This new sCMOS sensor has a physical size of 6 cm by 6 cm, with an array of 4096 by 4096 pixels and a pixel size of 15 um. The frame rate is 20.1 fps under current condition and can be boosted to a maximum value around 100 fps. The epitaxial thickness is increased to 10 um compared to the previous sCMOS product. We show the results of its frst taped-out product in this work. The dark current of this sCMOS is lower than 10 e/pixel/s at 20C, and lower than 0.02 e/pixel/s at -30C. The Fixed Pattern Noise (FPN) and the readout noise are lower than 5 e in high-gain situation and show a small increase at low temperature. The energy resolution reaches 180.1 eV (3.1%) at 5.90 keV for single-pixel events and 212.3 eV (3.6%) for all split events. The continuous X-ray spectrum measurement shows that this sensor is able to response to X-ray photons from 500 eV to 37 keV. The excellent performance, as demonstrated from these test results, makes sCMOS sensor an ideal detector for X-ray imaging and spectroscopic application.

astro-ph.IM↗

arXiv2022

Design and test results of scientific X-ray CMOS cameras

In recent years, scientific CMOS (sCMOS) sensors have found increasing applications to X-ray detection, including X-ray astronomical observations. In order to examine the performance of sCMOS sensors, we have developed X-ray cameras based on sCMOS sensors. Two cameras, CNX22 and CNX 66, have been developed using sCMOS sensors with a photosensitive area of 2 cm * 2 cm and 6 cm * 6 cm, respectively. The designs of the cameras are presented in this paper. The CNX22 camera has a frame rate of 48 fps, whereas CNX66 has a frame rate of currently 20 fps, that can be boosted to 100 fps in the future. The operating temperature of the sCMOS sensor can reach to -20C for CNX22 and -30C for CNX66 with a peltier cooler device. In addition to the commonly used mode of saving original images, the cameras provide a mode of real-time extraction of X-ray events and storage their information, which significantly reduces the requirement for data storage and offline analysis work. For both cameras, the energy resolutions can reach less than 200 eV at 5.9 keV using single-pixel events. These cameras are suitable for X-ray spectroscopy applications in laboratories and calibration for the space X-ray telescopes.

astro-ph.IM↗

arXiv2022

Asset Pricing and Deep Learning

Traditional machine learning methods have been widely studied in financial innovation. My study focuses on the application of deep learning methods on asset pricing. I investigate various deep learning methods for asset pricing, especially for risk premia measurement. All models take the same set of predictive signals (firm characteristics, systematic risks and macroeconomics). I demonstrate high performance of all kinds of state-of-the-art (SOTA) deep learning methods, and figure out that RNNs with memory mechanism and attention have the best performance in terms of predictivity. Furthermore, I demonstrate large economic gains to investors using deep learning forecasts. The results of my comparative experiments highlight the importance of domain knowledge and financial theory when designing deep learning models. I also show return prediction tasks bring new challenges to deep learning. The time varying distribution causes distribution shift problem, which is essential for financial time series prediction. I demonstrate that deep learning methods can improve asset risk premium measurement. Due to the booming deep learning studies, they can constantly promote the study of underlying financial mechanisms behind asset pricing. I also propose a promising research method that learning from data and figuring out the underlying economic mechanisms through explainable artificial intelligence (AI) methods. My findings not only justify the value of deep learning in blooming fintech development, but also highlight their prospects and advantages over traditional machine learning methods.

q-fin.ST↗

arXiv2022

Nesting Forward Automatic Differentiation for Memory-Efficient Deep Neural Network Training

An activation function is an element-wise mathematical function and plays a crucial role in deep neural networks (DNN). Many novel and sophisticated activation functions have been proposed to improve the DNN accuracy but also consume massive memory in the training process with back-propagation. In this study, we propose the nested forward automatic differentiation (Forward-AD), specifically for the element-wise activation function for memory-efficient DNN training. We deploy nested Forward-AD in two widely-used deep learning frameworks, TensorFlow and PyTorch, which support the static and dynamic computation graph, respectively. Our evaluation shows that nested Forward-AD reduces the memory footprint by up to 1.97x than the baseline model and outperforms the recomputation by 20% under the same memory reduction ratio.

cs.LG↗

arXiv2022

The Einstein Probe Mission

The Einstein Probe (EP) is a mission designed to monitor the sky in the soft X-ray band. It will perform systematic surveys and characterisation of high-energy transients and monitoring of variable objects at unprecedented sensitivity and monitoring cadences. It has a large instantaneous field-of-view (3,600 sq. deg.), that is realised via the lobster-eye micro-pore X-ray focusing optics. EP also carries a conventional X-ray focusing telescope with a larger effective area to perform followup observations and precise positioning of newly-discovered transients. Alerts for transient objects will be issued publicly and timely. The scientific goals of EP are concerned with discovering faint, distant or rare types of high-energy transients and variable sources. Within the confines of a modest-sized mission, EP will cover a wide range of scientific topics, from the nearby to high-redshift Universe. The Einstein Probe is a mission of the Chinese Academy of Sciences, and also an international collaborative project. This paper presents the background, scientific objectives, and the mission design including the micro-pore optics and CMOS technologies adopted, the instruments and their expected performance, and the mission profile. The development status of the project is also presented.

astro-ph.HE↗