arXiv subjects

Kun Zhou

Publications and source records attributed to Kun Zhou.

At least 199 records · Page 11Linked to original sources

arXiv2023

Mixed-EVC: Mixed Emotion Synthesis and Control in Voice Conversion

Emotional voice conversion (EVC) traditionally targets the transformation of spoken utterances from one emotional state to another, with previous research mainly focusing on discrete emotion categories. This paper departs from the norm by introducing a novel perspective: a nuanced rendering of mixed emotions and enhancing control over emotional expression. To achieve this, we propose a novel EVC framework, Mixed-EVC, which only leverages discrete emotion training labels. We construct an attribute vector that encodes the relationships among these discrete emotions, which is predicted using a ranking-based support vector machine and then integrated into a sequence-to-sequence (seq2seq) EVC framework. Mixed-EVC not only learns to characterize the input emotional style but also quantifies its relevance to other emotions during training. As a result, users have the ability to assign these attributes to achieve their desired rendering of mixed emotions. Objective and subjective evaluations confirm the effectiveness of our approach in terms of mixed emotion synthesis and control while surpassing traditional baselines in the conversion of discrete emotions from one to another.

eess.AS↗

arXiv2023

A Locality-based Neural Solver for Optical Motion Capture

We present a novel locality-based learning method for cleaning and solving optical motion capture data. Given noisy marker data, we propose a new heterogeneous graph neural network which treats markers and joints as different types of nodes, and uses graph convolution operations to extract the local features of markers and joints and transform them to clean motions. To deal with anomaly markers (e.g. occluded or with big tracking errors), the key insight is that a marker's motion shows strong correlations with the motions of its immediate neighboring markers but less so with other markers, a.k.a. locality, which enables us to efficiently fill missing markers (e.g. due to occlusion). Additionally, we also identify marker outliers due to tracking errors by investigating their acceleration profiles. Finally, we propose a training regime based on representation learning and data augmentation, by training the model on data with masking. The masking schemes aim to mimic the occluded and noisy markers often observed in the real data. Finally, we show that our method achieves high accuracy on multiple metrics across various datasets. Extensive comparison shows our method outperforms state-of-the-art methods in terms of prediction accuracy of occluded marker position error by approximately 20%, which leads to a further error reduction on the reconstructed joint rotations and positions by 30%. The code and data for this paper are available at https://github.com/non-void/LocalMoCap.

cs.GR↗

arXiv2023

Fermi level depinning via insertion of a graphene buffer layer at the gold-2D tin monoxide contact

Two-dimensional (2D) tin monoxide (SnO) has attracted much attention owing to its distinctive electronic and optical properties, which render itself suitable as a channel material in field effect transistors (FETs). However, upon contact with metals for such applications, the Fermi level pinning effect may occur, where states are induced in its band gap by the metal, hindering its intrinsic semiconducting properties. We propose the insertion of graphene at the contact interface to alleviate the metal-induced gap states. By using gold (Au) as the electrode material and monolayer SnO (mSnO) as the channel material, the geometry, bonding strength, charge transfer and tunnel barriers of charges, and electronic properties including the work function, band structure, density of states, and Schottky barriers are thoroughly investigated using first-principles calculations for the structures with and without graphene to reveal the contact behaviours and Fermi level depinning mechanism. It has been demonstrated that strong covalent bonding is formed between gold and mSnO, while the graphene interlayer forms weak van der Waals interaction with both materials, which minimises the perturbance to the band structure of mSnO. The effects of out-of-plane compression are also analysed to assess the performance of the contact under mechanical deformation, and a feasible fabrication route for the heterostructure with graphene is proposed. This work systematically explores the properties of the Au-mSnO contact for applications in FETs and provides thorough guidance for future exploitation of 2D materials in various electronic applications and for selection of buffer layers to improve metal-semiconductor contact.

cond-mat.mtrl-sci↗

arXiv2023

Abnormal behavior of preferred formation of cationic vacancy from the interior in γ-GeSe monolayer with the stereo-chemical antibonding lone-pair state

Two-dimensional (2D) materials tend to have the preferably formation of vacancies at the outer surface. Here, contrary to the normal notion, we reveal a type of vacancy that thermodynamically initiates from the interior part of the 2D backbone of germanium selenide (γ-GeSe). Interestingly, the Ge-vacancy (VGe) in the interior part of γ-GeSe possesses the lowest formation energy amongst the various types of defects considered. We also find a low diffusion barrier (1.04 eV) of VGe which is a half of those of sulfur vacancy in MoS2. The facile formation of mobile VGe is rooted in the antibonding coupling of the lone-pair Ge 4s and Se 4p states near the valence band maximum, which also exists in other gamma-phase MX (M=Sn, Ge; X=S, Te). The VGe is accompanied by a shallow acceptor level in the band gap and induces strong infrared light absorption and p-type conductivity. The VGe located in the middle cationic Ge sublattice is well protected by the surface Se layers-a feature that is absent in other atomically thin materials. Our work suggests that the unique well-buried inner VGe, with the potential of forming structurally protected ultrathin conducting filaments, may render the GeSe layer an ideal platform for quantum emitting, memristive, and neuromorphic applications.

cond-mat.mtrl-sci↗

arXiv2023

NeuralReshaper: Single-image Human-body Retouching with Deep Neural Networks

In this paper, we present NeuralReshaper, a novel method for semantic reshaping of human bodies in single images using deep generative networks. To achieve globally coherent reshaping effects, our approach follows a fit-then-reshape pipeline, which first fits a parametric 3D human model to a source human image and then reshapes the fitted 3D model with respect to user-specified semantic attributes. Previous methods rely on image warping to transfer 3D reshaping effects to the entire image domain and thus often cause distortions in both foreground and background. In contrast, we resort to generative adversarial nets conditioned on the source image and a 2D warping field induced by the reshaped 3D model, to achieve more realistic reshaping results. Specifically, we separately encode the foreground and background information in the source image using a two-headed UNet-like generator, and guide the information flow from the foreground branch to the background branch via feature space warping. Furthermore, to deal with the lack-of-data problem that no paired data exist (i.e., the same human bodies in varying shapes), we introduce a novel self-supervised strategy to train our network. Unlike previous methods that often require manual efforts to correct undesirable artifacts caused by incorrect body-to-image fitting, our method is fully automatic. Extensive experiments on both indoor and outdoor datasets demonstrate the superiority of our method over previous approaches.

cs.CV↗

arXiv2023

Alleviating the Long-Tail Problem in Conversational Recommender Systems

Conversational recommender systems (CRS) aim to provide the recommendation service via natural language conversations. To develop an effective CRS, high-quality CRS datasets are very crucial. However, existing CRS datasets suffer from the long-tail issue, \ie a large proportion of items are rarely (or even never) mentioned in the conversations, which are called long-tail items. As a result, the CRSs trained on these datasets tend to recommend frequent items, and the diversity of the recommended items would be largely reduced, making users easier to get bored. To address this issue, this paper presents \textbf{LOT-CRS}, a novel framework that focuses on simulating and utilizing a balanced CRS dataset (\ie covering all the items evenly) for improving \textbf{LO}ng-\textbf{T}ail recommendation performance of CRSs. In our approach, we design two pre-training tasks to enhance the understanding of simulated conversation for long-tail items, and adopt retrieval-augmented fine-tuning with label smoothness strategy to further improve the recommendation of long-tail items. Extensive experiments on two public CRS datasets have demonstrated the effectiveness and extensibility of our approach, especially on long-tail recommendation.

cs.IR↗

arXiv2023

Adaptive Local Basis Functions for Shape Completion

In this paper, we focus on the task of 3D shape completion from partial point clouds using deep implicit functions. Existing methods seek to use voxelized basis functions or the ones from a certain family of functions (e.g., Gaussians), which leads to high computational costs or limited shape expressivity. On the contrary, our method employs adaptive local basis functions, which are learned end-to-end and not restricted in certain forms. Based on those basis functions, a local-to-local shape completion framework is presented. Our algorithm learns sparse parameterization with a small number of basis functions while preserving local geometric details during completion. Quantitative and qualitative experiments demonstrate that our method outperforms the state-of-the-art methods in shape completion, detail preservation, generalization to unseen geometries, and computational cost. Code and data are at https://github.com/yinghdb/Adaptive-Local-Basis-Functions.

cs.CV↗

arXiv2023

Detecting anisotropies of the stochastic gravitational wave background with TianQin

The investigation of the anisotropy of the stochastic gravitational wave background (SGWB) using the TianQin detector plays a crucial role in studying the early universe and astrophysics. In this work, we examine the response of the $AET$ channel of the TianQin Time Delay Interferometry (TDI) to the anisotropy of the SGWB. We calculate the corresponding angular sensitivity curves and find that TianQin is capable of detecting the anisotropy of the SGWB, with an angular sensitivity reaching $10^{-10}$ for quadrupoles. Due to the fixed $z$-axis of TianQin pointing towards J0806, its overlap reduction functions (ORFs) exhibit specific symmetries, enabling the resolution of different multipole moments $\ell m$. The detection sensitivity is optimal for the $(2, 0)$ mode, with a sensitivity reaching $10^{-10}$. Using the Fisher matrix approach, we estimate the parameters and find that in the power-law spectrum model, higher logarithmic amplitudes lead to more effective reconstruction of the spectral index for all multipole moments. Under the optimal scenario with a signal amplitude of $Ω_{\mathrm{GW}} (f = f_{\mathrm{c}}) h^2 = 10^{-9}$, the spectral indices can be reconstructed with uncertainties of $10^{-3}$, $10$, and $10^{-3}$ for $\ell = 0$, $1$, and $2$ multipole moments, respectively. For the cases of $(\ell, m) = (0, 0)$, $(1, 1)$, $(2, 0)$, and $(2, 2)$, the spectral indices can be reconstructed with uncertainties of $10^{-3}$, $10$, $10^{-3}$, and $10$, respectively.

gr-qc↗

arXiv2023

MASTER: Multi-task Pre-trained Bottlenecked Masked Autoencoders are Better Dense Retrievers

Pre-trained Transformers (\eg BERT) have been commonly used in existing dense retrieval methods for parameter initialization, and recent studies are exploring more effective pre-training tasks for further improving the quality of dense vectors. Although various novel and effective tasks have been proposed, their different input formats and learning objectives make them hard to be integrated for jointly improving the model performance. In this work, we aim to unify a variety of pre-training tasks into the bottlenecked masked autoencoder manner, and integrate them into a multi-task pre-trained model, namely MASTER. Concretely, MASTER utilizes a shared-encoder multi-decoder architecture that can construct a representation bottleneck to compress the abundant semantic information across tasks into dense vectors. Based on it, we integrate three types of representative pre-training tasks: corrupted passages recovering, related passages recovering and PLMs outputs recovering, to characterize the inner-passage information, inter-passage relations and PLMs knowledge. Extensive experiments have shown that our approach outperforms competitive dense retrieval methods. Our code and data are publicly released in \url{https://github.com/microsoft/SimXNS}.

cs.CL↗

arXiv2023

JiuZhang 2.0: A Unified Chinese Pre-trained Language Model for Multi-task Mathematical Problem Solving

Although pre-trained language models~(PLMs) have recently advanced the research progress in mathematical reasoning, they are not specially designed as a capable multi-task solver, suffering from high cost for multi-task deployment (\eg a model copy for a task) and inferior performance on complex mathematical problems in practical applications. To address these issues, in this paper, we propose \textbf{JiuZhang~2.0}, a unified Chinese PLM specially for multi-task mathematical problem solving. Our idea is to maintain a moderate-sized model and employ the \emph{cross-task knowledge sharing} to improve the model capacity in a multi-task setting. Specially, we construct a Mixture-of-Experts~(MoE) architecture for modeling mathematical text, so as to capture the common mathematical knowledge across tasks. For optimizing the MoE architecture, we design \emph{multi-task continual pre-training} and \emph{multi-task fine-tuning} strategies for multi-task adaptation. These training strategies can effectively decompose the knowledge from the task data and establish the cross-task sharing via expert networks. In order to further improve the general capacity of solving different complex tasks, we leverage large language models~(LLMs) as complementary models to iteratively refine the generated solution by our PLM, via in-context learning. Extensive experiments have demonstrated the effectiveness of our model.

cs.CL↗

arXiv2023

Improving Conversational Recommendation Systems via Counterfactual Data Simulation

Conversational recommender systems (CRSs) aim to provide recommendation services via natural language conversations. Although a number of approaches have been proposed for developing capable CRSs, they typically rely on sufficient training data for training. Since it is difficult to annotate recommendation-oriented dialogue datasets, existing CRS approaches often suffer from the issue of insufficient training due to the scarcity of training data. To address this issue, in this paper, we propose a CounterFactual data simulation approach for CRS, named CFCRS, to alleviate the issue of data scarcity in CRSs. Our approach is developed based on the framework of counterfactual data augmentation, which gradually incorporates the rewriting to the user preference from a real dialogue without interfering with the entire conversation flow. To develop our approach, we characterize user preference and organize the conversation flow by the entities involved in the dialogue, and design a multi-stage recommendation dialogue simulator based on a conversation flow language model. Under the guidance of the learned user preference and dialogue schema, the flow language model can produce reasonable, coherent conversation flows, which can be further realized into complete dialogues. Based on the simulator, we perform the intervention at the representations of the interacted entities of target users, and design an adversarial training method with a curriculum schedule that can gradually optimize the data augmentation strategy. Extensive experiments show that our approach can consistently boost the performance of several competitive CRSs, and outperform other data augmentation methods, especially when the training data is limited. Our code is publicly available at https://github.com/RUCAIBox/CFCRS.

cs.CL↗

arXiv2023

Evaluating and Improving Tool-Augmented Computation-Intensive Math Reasoning

Chain-of-thought prompting~(CoT) and tool augmentation have been validated in recent work as effective practices for improving large language models~(LLMs) to perform step-by-step reasoning on complex math-related tasks. However, most existing math reasoning datasets may be not able to fully evaluate and analyze the ability of LLMs in manipulating tools and performing reasoning, as they may only require very few invocations of tools or miss annotations for evaluating intermediate reasoning steps. To address the issue, we construct \textbf{CARP}, a new Chinese dataset consisting of 4,886 computation-intensive algebra problems with formulated annotations on intermediate steps. In CARP, we test four LLMs with CoT prompting, and find that they are all prone to make mistakes at the early steps of the solution, leading to wrong answers. Based on this finding, we propose a new approach that can deliberate the reasoning steps with tool interfaces, namely \textbf{DELI}. In DELI, we first initialize a step-by-step solution based on retrieved exemplars, then iterate two deliberation procedures that check and refine the intermediate steps of the generated solution, from the perspectives of tool manipulation and natural language reasoning, until obtaining converged solutions or reaching the maximum turn. Experimental results on CARP and six other datasets show that the proposed DELI mostly outperforms competitive baselines, and can further boost the performance of existing CoT methods. Our data and code are available in \url{https://github.com/RUCAIBox/CARP}.

cs.CL↗

arXiv2023

BEDRF: Bidirectional Edge Diffraction Response Function for Interactive Sound Propagation

We introduce bidirectional edge diffraction response function (BEDRF), a new approach to model wave diffraction around edges with path tracing. The diffraction part of the wave is expressed as an integration on path space, and the wave-edge interaction is expressed using only the localized information around points on the edge similar to a bidirectional scattering distribution function (BSDF) for visual rendering. For an infinite single wedge, our model generates the same result as the analytic solution. Our approach can be easily integrated into interactive geometric sound propagation algorithms that use path tracing to compute specular and diffuse reflections. Our resulting propagation algorithm can approximate complex wave propagation phenomena involving high-order diffraction, and is able to handle dynamic, deformable objects and moving sources and listeners. We highlight the performance of our approach in different scenarios to generate smooth auralization.

cs.SD↗

arXiv2023

C2-CRS: Coarse-to-Fine Contrastive Learning for Conversational Recommender System

Conversational recommender systems (CRS) aim to recommend suitable items to users through natural language conversations. For developing effective CRSs, a major technical issue is how to accurately infer user preference from very limited conversation context. To address issue, a promising solution is to incorporate external data for enriching the context information. However, prior studies mainly focus on designing fusion models tailored for some specific type of external data, which is not general to model and utilize multi-type external data. To effectively leverage multi-type external data, we propose a novel coarse-to-fine contrastive learning framework to improve data semantic fusion for CRS. In our approach, we first extract and represent multi-grained semantic units from different data signals, and then align the associated multi-type semantic units in a coarse-to-fine way. To implement this framework, we design both coarse-grained and fine-grained procedures for modeling user preference, where the former focuses on more general, coarse-grained semantic fusion and the latter focuses on more specific, fine-grained semantic fusion. Such an approach can be extended to incorporate more kinds of external data. Extensive experiments on two public CRS datasets have demonstrated the effectiveness of our approach in both recommendation and conversation tasks.

cs.CL↗

arXiv2023

Visually-augmented pretrained language models for NLP tasks without images

Although pre-trained language models~(PLMs) have shown impressive performance by text-only self-supervised training, they are found lack of visual semantics or commonsense. Existing solutions often rely on explicit images for visual knowledge augmentation (requiring time-consuming retrieval or generation), and they also conduct the augmentation for the whole input text, without considering whether it is actually needed in specific inputs or tasks. To address these issues, we propose a novel \textbf{V}isually-\textbf{A}ugmented fine-tuning approach that can be generally applied to various PLMs or NLP tasks, \textbf{W}ithout using any retrieved or generated \textbf{I}mages, namely \textbf{VAWI}. Experimental results show that our approach can consistently improve the performance of BERT, RoBERTa, BART, and T5 at different scales, and outperform several competitive baselines on ten tasks. Our codes and data are publicly available at~\url{https://github.com/RUCAIBox/VAWI}.

cs.CL↗

arXiv2023

Promoted Electronic Coupling of Acoustic Phonon Modes in Doped Semimetallic MoTe2

As a prototype of the Weyl superconductor, layered molybdenum telluride (MoTe2) encompasses two semimetallic phases (1T_prime and Td) which differentiate from each other via a slight tilting of the out-of-plane lattice. Both phases are subjected to serious phase mixing which complicates the analysis of its origin of superconductivity. Herein, we explore the electron-phonon coupling (EPC) of the monolayer semimetallic MoTe2, without phase ambiguity under this thickness limit. Apart from the hardening or softening of phonon modes, the strength of the EPC can be strongly modulated by doping. Specifically, longitudinal and out-of-plane acoustic modes are significantly activated for electron doped MoTe2. This is ascribed to the presence of rich valley states and equispaced nesting bands which are dynamically populated under charge doping. Through comparing the monolayer and bilayer MoTe2, the strength of EPC is found to be less likely to depend on thickness for neutral samples but clearly promoted for thinner samples with electron doping, while for hole doping, the strength alters more significantly with the thickness than doping. Our work explains the puzzling issue of the doping sensitivity of the superconductivity in semimetallic MoTe2 and establishes the critical role of activating acoustic phonons in such low-dimensional materials.

cond-mat.supr-con↗

arXiv2023

Diffusion Models for Non-autoregressive Text Generation: A Survey

Non-autoregressive (NAR) text generation has attracted much attention in the field of natural language processing, which greatly reduces the inference latency but has to sacrifice the generation accuracy. Recently, diffusion models, a class of latent variable generative models, have been introduced into NAR text generation, showing an improved text generation quality. In this survey, we review the recent progress in diffusion models for NAR text generation. As the background, we first present the general definition of diffusion models and the text diffusion models, and then discuss their merits for NAR generation. As the core content, we further introduce two mainstream diffusion models in existing work of text diffusion, and review the key designs of the diffusion process. Moreover, we discuss the utilization of pre-trained language models (PLMs) for text diffusion models and introduce optimization techniques for text data. Finally, we discuss several promising directions and conclude this paper. Our survey aims to provide researchers with a systematic reference of related research on text diffusion models for NAR generation. We present our collection of text diffusion models at https://github.com/RUCAIBox/Awesome-Text-Diffusion-Models.

cs.CL↗

arXiv2023

Diffusion-NAT: Self-Prompting Discrete Diffusion for Non-Autoregressive Text Generation

Recently, continuous diffusion models (CDM) have been introduced into non-autoregressive (NAR) text-to-text generation. However, the discrete nature of text increases the difficulty of CDM to generate coherent and fluent texts, and also causes the incompatibility problem between CDM and advanced NLP techniques, especially the popular pre-trained language models~(PLMs). To solve it, we propose Diffusion-NAT, which introduces discrete diffusion models~(DDM) into NAR text-to-text generation and integrates BART to improve the performance. By revising the decoding process of BART and the typical settings of DDM, we unify the inference process of BART and the denoising process of DDM into the same NAR masked tokens recovering task. In this way, DDM can rely on BART to perform denoising, which can benefit from both the rich pre-learned knowledge of BART and the iterative refining paradigm of DDM. Besides, we also propose the iterative self-prompting strategy to further improve the generation quality. Experimental results on 7 datasets show that our approach can outperform competitive NAR methods, and even surpass autoregressive methods. Our code and data will be publicly released.

cs.CL↗