arXiv subjects

Bin Li

Publications and source records attributed to Bin Li.

At least 469 records · Page 26Linked to original sources

arXiv2022

A Systematic Survey of Regularization and Normalization in GANs

Generative Adversarial Networks (GANs) have been widely applied in different scenarios thanks to the development of deep neural networks. The original GAN was proposed based on the non-parametric assumption of the infinite capacity of networks. However, it is still unknown whether GANs can fit the target distribution without any prior information. Due to the overconfident assumption, many issues remain unaddressed in GANs' training, such as non-convergence, mode collapses, gradient vanishing. Regularization and normalization are common methods of introducing prior information to stabilize training and improve discrimination. Although a handful number of regularization and normalization methods have been proposed for GANs, to the best of our knowledge, there exists no comprehensive survey that primarily focuses on objectives and development of these methods, apart from some in-comprehensive and limited scope studies. In this work, we conduct a comprehensive survey on the regularization and normalization techniques from different perspectives of GANs training. First, we systematically describe different perspectives of GANs training and thus obtain the different objectives of regularization and normalization. Based on these objectives, we propose a new taxonomy. Furthermore, we compare the performance of the mainstream methods on different datasets and investigate the applications of regularization and normalization techniques that have been frequently employed in state-of-the-art GANs. Finally, we highlight potential future directions of research in this domain. Code and studies related to the regularization and normalization of GANs in this work is summarized on https://github.com/iceli1007/GANs-Regularization-Review.

cs.LG↗

arXiv2022

Slippage-robust Gaze Tracking for Near-eye Display

In recent years, head-mounted near-eye display devices have become the key hardware foundation for virtual reality and augmented reality. Thus head-mounted gaze tracking technology has received attention as an essential part of human-computer interaction. However, unavoidable slippage of head-mounted devices (HMD) often results higher gaze tracking errors and hinders the practical usage of HMD. To tackle this problem, we propose a slippage-robust gaze tracking for near-eye display method based on the aspheric eyeball model and accurately compute the eyeball optical axis and rotation center. We tested several methods on datasets with slippage and the experimental results show that the proposed method significantly outperforms the previous method (almost double the suboptimal method).

cs.CV↗

arXiv2022

Superconductivity in CeBeH$_{8}$ and CeBH$_{8}$ at moderate pressures

High-pressure structural searches of superhydrides CeBeH$_8$ and CeBH$_8$ were performed under ambient pressure up to 300 GPa. We identify $Fm\overline{3}m$-CeBeH$_8$ with a superconducting transition temperature $T_{c}$ of 56 K at 10 GPa. Two more phases with spacegroup $R\overline{3}m$ and $C2/m$, were investigated within the increasing pressures. CeBH$_8$ shows a similar phase transition process as CeBeH$_8$ but with higher transition pressures and higher $T_c$. $Fm\overline{3}m$-CeBH$_8$ is predicted to be superconducting above 120 GPa with a maximum $T_{c}$ of 118 K at 150 GPa. $R\overline{3}m$-CeBH$_8$ and $C2/m$-CeBH$_8$ are dynamically stable above 120 GPa and 100 GPa, respectively. The maximum $T_{c}$ is 123 K at 195 GPa for $R\overline{3}m$-CeBH$_8$, and 115 K at 350 GPa for $C2/m$-CeBH$_8$. Our work enriches the family of ternary hydrides and may provide a useful guideline for further search for superconducting hydrides at low and moderate pressures.

cond-mat.supr-con↗

arXiv2022

Visual Answer Localization with Cross-modal Mutual Knowledge Transfer

The goal of visual answering localization (VAL) in the video is to obtain a relevant and concise time clip from a video as the answer to the given natural language question. Early methods are based on the interaction modelling between video and text to predict the visual answer by the visual predictor. Later, using the textual predictor with subtitles for the VAL proves to be more precise. However, these existing methods still have cross-modal knowledge deviations from visual frames or textual subtitles. In this paper, we propose a cross-modal mutual knowledge transfer span localization (MutualSL) method to reduce the knowledge deviation. MutualSL has both visual predictor and textual predictor, where we expect the prediction results of these both to be consistent, so as to promote semantic knowledge understanding between cross-modalities. On this basis, we design a one-way dynamic loss function to dynamically adjust the proportion of knowledge transfer. We have conducted extensive experiments on three public datasets for evaluation. The experimental results show that our method outperforms other competitive state-of-the-art (SOTA) methods, demonstrating its effectiveness.

cs.CV↗

arXiv2022

A Comprehensive Survey on Data-Efficient GANs in Image Generation

Generative Adversarial Networks (GANs) have achieved remarkable achievements in image synthesis. These successes of GANs rely on large scale datasets, requiring too much cost. With limited training data, how to stable the training process of GANs and generate realistic images have attracted more attention. The challenges of Data-Efficient GANs (DE-GANs) mainly arise from three aspects: (i) Mismatch Between Training and Target Distributions, (ii) Overfitting of the Discriminator, and (iii) Imbalance Between Latent and Data Spaces. Although many augmentation and pre-training strategies have been proposed to alleviate these issues, there lacks a systematic survey to summarize the properties, challenges, and solutions of DE-GANs. In this paper, we revisit and define DE-GANs from the perspective of distribution optimization. We conclude and analyze the challenges of DE-GANs. Meanwhile, we propose a taxonomy, which classifies the existing methods into three categories: Data Selection, GANs Optimization, and Knowledge Sharing. Last but not the least, we attempt to highlight the current problems and the future directions.

cs.CV↗

arXiv2022

Domain Discrepancy Aware Distillation for Model Aggregation in Federated Learning

Knowledge distillation has recently become popular as a method of model aggregation on the server for federated learning. It is generally assumed that there are abundant public unlabeled data on the server. However, in reality, there exists a domain discrepancy between the datasets of the server domain and a client domain, which limits the performance of knowledge distillation. How to improve the aggregation under such a domain discrepancy setting is still an open problem. In this paper, we first analyze the generalization bound of the aggregation model produced from knowledge distillation for the client domains, and then describe two challenges, server-to-client discrepancy and client-to-client discrepancy, brought to the aggregation model by the domain discrepancies. Following our analysis, we propose an adaptive knowledge aggregation algorithm FedD3A based on domain discrepancy aware distillation to lower the bound. FedD3A performs adaptive weighting at the sample level in each round of FL. For each sample in the server domain, only the client models of its similar domains will be selected for playing the teacher role. To achieve this, we show that the discrepancy between the server-side sample and the client domain can be approximately measured using a subspace projection matrix calculated on each client without accessing its raw data. The server can thus leverage the projection matrices from multiple clients to assign weights to the corresponding teacher models for each server-side sample. We validate FedD3A on two popular cross-domain datasets and show that it outperforms the compared competitors in both cross-silo and cross-device FL settings.

cs.LG↗

arXiv2022

Domain-Unified Prompt Representations for Source-Free Domain Generalization

Domain generalization (DG), aiming to make models work on unseen domains, is a surefire way toward general artificial intelligence. Limited by the scale and diversity of current DG datasets, it is difficult for existing methods to scale to diverse domains in open-world scenarios (e.g., science fiction and pixelate style). Therefore, the source-free domain generalization (SFDG) task is necessary and challenging. To address this issue, we propose an approach based on large-scale vision-language pretraining models (e.g., CLIP), which exploits the extensive domain information embedded in it. The proposed scheme generates diverse prompts from a domain bank that contains many more diverse domains than existing DG datasets. Furthermore, our method yields domain-unified representations from these prompts, thus being able to cope with samples from open-world domains. Extensive experiments on mainstream DG datasets, namely PACS, VLCS, OfficeHome, and DomainNet, show that the proposed method achieves competitive performance compared to state-of-the-art (SOTA) DG methods that require source domain data for training. Besides, we collect a small datasets consists of two domains to evaluate the open-world domain generalization ability of the proposed method. The source code and the dataset will be made publicly available at https://github.com/muse1998/Source-Free-Domain-Generalization

cs.CV↗

arXiv2022

Shape model and rotation acceleration of (1685) Toro and (85989) 1999 JD6 from optical observations

The Yarkovsky-O'Keefe-Radzievskii-Paddack (YORP) effect is a net torque caused by solar radiation directly reflected and thermally re-emitted from the surface of small asteroids and is considered to be crucial in their dynamical evolution. By long-term photometric observations of selected near-Earth asteroids, it's hoped to enlarge asteroid samples with a detected YORP effect to facilitate the development of a theoretical framework. Archived light-curve data are collected and photometric observations are made for (1685) Toro and (85989) 1999 JD6, which enables measurement of their YORP effect by inverting the light curve to fit observations from a convex shape model. For (1685) Toro, a YORP acceleration $\upsilon=(3.2\pm0.3)\times10^{-9}\ \rm{rad\cdot d^{-2}}(1σ error)$ is updated, which is consistent with previous YORP detection based on different light-curve data; for (85989) 1999 JD6, it is determined that the sidereal period is $7.667749\pm 0.000009$ h, the rotation pole direction locates is at $λ=232\pm 2^{\circ},\ β= -59\pm 1^{\circ}$, the acceleration is detected to be $\upsilon = (2.4\pm0.3)\times10^{-8}\ \rm{rad\cdot d^{-2}}(1σ error)$ and in addition to obtaining an excellent agreement between the observations and model. YORP should produce both spin-up and spin-down cases. However, including (85989) 1999 JD6, the $\rm{d}ω/\rm{d}t$ values of eleven near-Earth asteroids are positive totally, which suggests that there is either a bias in the sample of YORP detections or a real feature needs to be explained.

astro-ph.EP↗

arXiv2022

Clustering-Based Representation Learning through Output Translation and Its Application to Remote--Sensing Images

In supervised deep learning, learning good representations for remote--sensing images (RSI) relies on manual annotations. However, in the area of remote sensing, it is hard to obtain huge amounts of labeled data. Recently, self--supervised learning shows its outstanding capability to learn representations of images, especially the methods of instance discrimination. Comparing methods of instance discrimination, clustering--based methods not only view the transformations of the same image as ``positive" samples but also similar images. In this paper, we propose a new clustering-based method for representation learning. We first introduce a quantity to measure representations' discriminativeness and from which we show that even distribution requires the most discriminative representations. This provides a theoretical insight into why evenly distributing the images works well. We notice that only the even distributions that preserve representations' neighborhood relations are desirable. Therefore, we develop an algorithm that translates the outputs of a neural network to achieve the goal of evenly distributing the samples while preserving outputs' neighborhood relations. Extensive experiments have demonstrated that our method can learn representations that are as good as or better than the state of the art approaches, and that our method performs computationally efficiently and robustly on various RSI datasets.

cs.LG↗

arXiv2022

TODE-Trans: Transparent Object Depth Estimation with Transformer

Transparent objects are widely used in industrial automation and daily life. However, robust visual recognition and perception of transparent objects have always been a major challenge. Currently, most commercial-grade depth cameras are still not good at sensing the surfaces of transparent objects due to the refraction and reflection of light. In this work, we present a transformer-based transparent object depth estimation approach from a single RGB-D input. We observe that the global characteristics of the transformer make it easier to extract contextual information to perform depth estimation of transparent areas. In addition, to better enhance the fine-grained features, a feature fusion module (FFM) is designed to assist coherent prediction. Our empirical evidence demonstrates that our model delivers significant improvements in recent popular datasets, e.g., 25% gain on RMSE and 21% gain on REL compared to previous state-of-the-art convolutional-based counterparts in ClearGrasp dataset. Extensive results show that our transformer-based model enables better aggregation of the object's RGB and inaccurate depth information to obtain a better depth representation. Our code and the pre-trained model will be available at https://github.com/yuchendoudou/TODE.

cs.CV↗

arXiv2022

Phase Transitions and Superconductivity in Ternary Hydride Li$_2$SiH$_6$ at High Pressures

We predicted a new ternary hydride Li$_2$SiH$_6$ at high pressures. A systematic structure search in Li$_2$SiH$_6$ compound reveals novel stable phases with intriguing electronic and phonon properties. It is found that Li$_2$SiH$_6$ is dynamically stable from ambient pressure up to 400 GPa with three novel phases: P312, P$\bar{3}$, and P$\bar{6}$2m. The calculation of electron-phonon coupling combined with Bardeen-Cooper-Schrieffer's argument indicates that this compound may be a candidate for high $T_c$ superconductors under high pressures. In particular, the maximum $T_c$ of $P\bar{6}2m$-Li$_2$SiH$_6$ at 400 GPa reaches 56 K. These findings may pave the way for obtaining room temperature superconductors in dense hydrogen-rich compounds.

cond-mat.mtrl-sci↗

arXiv2022

A Q-band line survey towards Orion KL using the Tianma radio telescope

We have conducted a line survey towards Orion KL using the Q-band receiver of Tianma 65 m radio telescope (TMRT), covering 34.8--50 GHz with a velocity resolution between 0.79 km s$^{-1}$ and 0.55 km s$^{-1}$ respectively. The observations reach a sensitivity on the level of 1-8 mK, proving that the TMRT is sensitive for conducting deep line surveys. In total, 597 Gaussian features are extracted. Among them, 177 radio recombination lines (RRLs) are identified, including 126, 40 and 11 RRLs of hydrogen, helium and carbon, with a maximum $Δn$ of 16, 7, and 3, respectively. The carbon RRLs are confirmed to originate from photodissociation regions with a $V_{\rm LSR}\sim$9 km s$^{-1}$. In addition, 371 molecular transitions of 53 molecular species are identified. Twenty-one molecular species of this survey were not firmly detected in the Q band by Rizzo et al. (2017), including species such as H$_2$CS, HCOOH, C$_2$H$_5$OH, H$_2^{13}$CO, H$_2$CCO, CH$_3$CHO, CH$_2$OCH$_2$, HCN $v_2=1$, and CH$_3$OCHO $v_t=1$. In particular, the vibrationally excited states of ethyl cyanide (C$_2$H$_5$CN $v$13/$v$21) are for the first time firmly detected in the Q band. NH$_3$ (15,15) and (16,16) are identified, and they are so far the highest transitions of the NH$_3$ inversion lines detected towards Orion KL. All the identified lines can be reproduced by a radiative transfer model.

astro-ph.GA↗

arXiv2022

Combinatorial Procurement Auction in Social Networks

This paper studies one emerging procurement auction scenario where the market is constructed over the social networks. In a social network composed of many agents, smartphones or computers, one requester releases her requirement for goods or tasks to suppliers, then suppliers who have entered the market are also encouraged to invite some other suppliers to join and all the suppliers in the network could compete for the business. The key problem for this networked auction is about how to incentivize each node who have entered the sell not only to truthfully use her full ability, but also to forward the task to her neighbours. Auctions conducting over social networks have attracted considerable interests in recent years. However, most of the existing works focus on classic forward auctions. Moreover, there is no existing valid networked auction considering multiple goods/tasks. This work is the first to explore procurement auction for both homogeneous and heterogeneous goods or tasks in social networks. From both theoretical proof and experimental simulation, we proved that the proposed mechanisms are proved to be individual-rational and incentive-compatible, also both the cost of the system and the requester could get decreased.

cs.GT↗

arXiv2022

Charactering instrumental noises and stochastic gravitational wave signals from combined time-delay interferometry

LISA will detect gravitational waves (GWs) in the milli-Hz frequency band in space. Time-delay interferometry (TDI) is developed to suppress laser frequency noise beneath the acceleration noise and optical metrology noise. To identify stochastic GW signals, it would be required to characterize these noise components entangled in TDI data streams. In this work, we investigate noises characterization by combining the first-generation TDI channels from Michelson and Relay configurations. The Michelson channels are helpful to characterize acceleration noises in the lower frequency band, and the Relay configuration could effectively resolve optical path noises in the higher frequencies. Synergy could be achieved from their combination to determine these instrumental noises. Based on the characterized noises, we further reconstruct the power spectrum of noise in the selected TDI channel. Two cases are performed to characterize the spectrum shape of a stochastic GW signal. For a modeled signal, its parameter(s) could be directly estimated from the TDI data, and its spectrum could be recovered from the inferred values. And for an unexpected signal, its spectrum may be recognized and retrieved from noise-subtracted residual in which its power spectral density surpasses the noise level.

gr-qc↗

arXiv2022

Style Spectroscope: Improve Interpretability and Controllability through Fourier Analysis

Universal style transfer (UST) infuses styles from arbitrary reference images into content images. Existing methods, while enjoying many practical successes, are unable of explaining experimental observations, including different performances of UST algorithms in preserving the spatial structure of content images. In addition, methods are limited to cumbersome global controls on stylization, so that they require additional spatial masks for desired stylization. In this work, we provide a systematic Fourier analysis on a general framework for UST. We present an equivalent form of the framework in the frequency domain. The form implies that existing algorithms treat all frequency components and pixels of feature maps equally, except for the zero-frequency component. We connect Fourier amplitude and phase with Gram matrices and a content reconstruction loss in style transfer, respectively. Based on such equivalence and connections, we can thus interpret different structure preservation behaviors between algorithms with Fourier phase. Given the interpretations we have, we propose two manipulations in practice for structure preservation and desired stylization. Both qualitative and quantitative experiments demonstrate the competitive performance of our method against the state-of-the-art methods. We also conduct experiments to demonstrate (1) the abovementioned equivalence, (2) the interpretability based on Fourier amplitude and phase and (3) the controllability associated with frequency components.

cs.CV↗

arXiv2022

Overlooked Poses Actually Make Sense: Distilling Privileged Knowledge for Human Motion Prediction

Previous works on human motion prediction follow the pattern of building a mapping relation between the sequence observed and the one to be predicted. However, due to the inherent complexity of multivariate time series data, it still remains a challenge to find the extrapolation relation between motion sequences. In this paper, we present a new prediction pattern, which introduces previously overlooked human poses, to implement the prediction task from the view of interpolation. These poses exist after the predicted sequence, and form the privileged sequence. To be specific, we first propose an InTerPolation learning Network (ITP-Network) that encodes both the observed sequence and the privileged sequence to interpolate the in-between predicted sequence, wherein the embedded Privileged-sequence-Encoder (Priv-Encoder) learns the privileged knowledge (PK) simultaneously. Then, we propose a Final Prediction Network (FP-Network) for which the privileged sequence is not observable, but is equipped with a novel PK-Simulator that distills PK learned from the previous network. This simulator takes as input the observed sequence, but approximates the behavior of Priv-Encoder, enabling FP-Network to imitate the interpolation process. Extensive experimental results demonstrate that our prediction pattern achieves state-of-the-art performance on benchmarked H3.6M, CMU-Mocap and 3DPW datasets in both short-term and long-term predictions.

cs.CV↗

arXiv2022

A New Perspective on Stabilizing GANs training: Direct Adversarial Training

Generative Adversarial Networks (GANs) are the most popular image generation models that have achieved remarkable progress on various computer vision tasks. However, training instability is still one of the open problems for all GAN-based algorithms. Quite a number of methods have been proposed to stabilize the training of GANs, the focuses of which were respectively put on the loss functions, regularization and normalization technologies, training algorithms, and model architectures. Different from the above methods, in this paper, a new perspective on stabilizing GANs training is presented. It is found that sometimes the images produced by the generator act like adversarial examples of the discriminator during the training process, which may be part of the reason causing the unstable training of GANs. With this finding, we propose the Direct Adversarial Training (DAT) method to stabilize the training process of GANs. Furthermore, we prove that the DAT method is able to minimize the Lipschitz constant of the discriminator adaptively. The advanced performance of DAT is verified on multiple loss functions, network architectures, hyper-parameters, and datasets. Specifically, DAT achieves significant improvements of 11.5% FID on CIFAR-100 unconditional generation based on SSGAN, 10.5% FID on STL-10 unconditional generation based on SSGAN, and 13.2% FID on LSUN-Bedroom unconditional generation based on SSGAN. Code will be available at https://github.com/iceli1007/DAT-GAN

eess.IV↗

arXiv2022

FakeCLR: Exploring Contrastive Learning for Solving Latent Discontinuity in Data-Efficient GANs

Data-Efficient GANs (DE-GANs), which aim to learn generative models with a limited amount of training data, encounter several challenges for generating high-quality samples. Since data augmentation strategies have largely alleviated the training instability, how to further improve the generative performance of DE-GANs becomes a hotspot. Recently, contrastive learning has shown the great potential of increasing the synthesis quality of DE-GANs, yet related principles are not well explored. In this paper, we revisit and compare different contrastive learning strategies in DE-GANs, and identify (i) the current bottleneck of generative performance is the discontinuity of latent space; (ii) compared to other contrastive learning strategies, Instance-perturbation works towards latent space continuity, which brings the major improvement to DE-GANs. Based on these observations, we propose FakeCLR, which only applies contrastive learning on perturbed fake samples, and devises three related training techniques: Noise-related Latent Augmentation, Diversity-aware Queue, and Forgetting Factor of Queue. Our experimental results manifest the new state of the arts on both few-shot generation and limited-data generation. On multiple datasets, FakeCLR acquires more than 15% FID improvement compared to existing DE-GANs. Code is available at https://github.com/iceli1007/FakeCLR.

cs.CV↗