arXiv subjects

Jia Li

Publications and source records attributed to Jia Li.

At least 487 records · Page 27Linked to original sources

arXiv2022

Coexistence of extended flat band and Kekulé order in Li-intercalated graphene

Doping graphene near the 1/4 filling to shift the extended flat band and van Hove singularity below E$_F$ has been highly desirable. Here we report the experimental observation of an extended flat band below E$_F$ in Li-intercalated graphene. Strong electron-phonon interaction is clearly identified by notable kinks in the band dispersion. Moreover, the evolution of the band structure upon Li intercalation shows that the extended flat band and the Kekulé order emerge simultaneously. Our work provides opportunities for investigating flat band related instabilities and its interplay with the Kekulé order

cond-mat.mes-hall↗

arXiv2022

Pyramid Grafting Network for One-Stage High Resolution Saliency Detection

Recent salient object detection (SOD) methods based on deep neural network have achieved remarkable performance. However, most of existing SOD models designed for low-resolution input perform poorly on high-resolution images due to the contradiction between the sampling depth and the receptive field size. Aiming at resolving this contradiction, we propose a novel one-stage framework called Pyramid Grafting Network (PGNet), using transformer and CNN backbone to extract features from different resolution images independently and then graft the features from transformer branch to CNN branch. An attention-based Cross-Model Grafting Module (CMGM) is proposed to enable CNN branch to combine broken detailed information more holistically, guided by different source feature during decoding process. Moreover, we design an Attention Guided Loss (AGL) to explicitly supervise the attention matrix generated by CMGM to help the network better interact with the attention from different models. We contribute a new Ultra-High-Resolution Saliency Detection dataset UHRSD, containing 5,920 images at 4K-8K resolutions. To our knowledge, it is the largest dataset in both quantity and resolution for high-resolution SOD task, which can be used for training and testing in future research. Sufficient experiments on UHRSD and widely-used SOD datasets demonstrate that our method achieves superior performance compared to the state-of-the-art methods.

cs.CV↗

arXiv2022

Video Demoireing with Relation-Based Temporal Consistency

Moire patterns, appearing as color distortions, severely degrade image and video qualities when filming a screen with digital cameras. Considering the increasing demands for capturing videos, we study how to remove such undesirable moire patterns in videos, namely video demoireing. To this end, we introduce the first hand-held video demoireing dataset with a dedicated data collection pipeline to ensure spatial and temporal alignments of captured data. Further, a baseline video demoireing model with implicit feature space alignment and selective feature aggregation is developed to leverage complementary information from nearby frames to improve frame-level video demoireing. More importantly, we propose a relation-based temporal consistency loss to encourage the model to learn temporal consistency priors directly from ground-truth reference videos, which facilitates producing temporally consistent predictions and effectively maintains frame-level qualities. Extensive experiments manifest the superiority of our model. Code is available at \url{https://daipengwa.github.io/VDmoire_ProjectPage/}.

cs.CV↗

arXiv2022

Automatic detection of low surface brightness galaxies from SDSS images

Low surface brightness (LSB) galaxies are galaxies with central surface brightness fainter than the night sky. Due to the faint nature of LSB galaxies and the comparable sky background, it is difficult to search LSB galaxies automatically and efficiently from large sky survey. In this study, we established the Low Surface Brightness Galaxies Auto Detect model (LSBG-AD), which is a data-driven model for end-to-end detection of LSB galaxies from Sloan Digital Sky Survey (SDSS) images. Object detection techniques based on deep learning are applied to the SDSS field images to identify LSB galaxies and estimate their coordinates at the same time. Applying LSBG-AD to 1120 SDSS images, we detected 1197 LSB galaxy candidates, of which 1081 samples are already known and 116 samples are newly found candidates. The B-band central surface brightness of the candidates searched by the model ranges from 22 mag arcsec $^ {- 2} $ to 24 mag arcsec $^ {- 2} $, quite consistent with the surface brightness distribution of the standard sample. 96.46\% of LSB galaxy candidates have an axis ratio ($b/a$) greater than 0.3, and 92.04\% of them have $fracDev\_r$\textless 0.4, which is also consistent with the standard sample. The results show that the LSBG-AD model learns the features of LSB galaxies of the training samples well, and can be used to search LSB galaxies without using photometric parameters. Next, this method will be used to develop efficient algorithms to detect LSB galaxies from massive images of the next generation observatories.

astro-ph.IM↗

arXiv2022

Improving Contrastive Learning with Model Augmentation

The sequential recommendation aims at predicting the next items in user behaviors, which can be solved by characterizing item relationships in sequences. Due to the data sparsity and noise issues in sequences, a new self-supervised learning (SSL) paradigm is proposed to improve the performance, which employs contrastive learning between positive and negative views of sequences. However, existing methods all construct views by adopting augmentation from data perspectives, while we argue that 1) optimal data augmentation methods are hard to devise, 2) data augmentation methods destroy sequential correlations, and 3) data augmentation fails to incorporate comprehensive self-supervised signals. Therefore, we investigate the possibility of model augmentation to construct view pairs. We propose three levels of model augmentation methods: neuron masking, layer dropping, and encoder complementing. This work opens up a novel direction in constructing views for contrastive SSL. Experiments verify the efficacy of model augmentation for the SSL in the sequential recommendation. Code is available\footnote{\url{https://github.com/salesforce/SRMA}}.

cs.LG↗

arXiv2022

ConTinTin: Continual Learning from Task Instructions

The mainstream machine learning paradigms for NLP often work with two underlying presumptions. First, the target task is predefined and static; a system merely needs to learn to solve it exclusively. Second, the supervision of a task mainly comes from a set of labeled examples. A question arises: how to build a system that can keep learning new tasks from their instructions? This work defines a new learning paradigm ConTinTin (Continual Learning from Task Instructions), in which a system should learn a sequence of new tasks one by one, each task is explained by a piece of textual instruction. The system is required to (i) generate the expected outputs of a new task by learning from its instruction, (ii) transfer the knowledge acquired from upstream tasks to help solve downstream tasks (i.e., forward-transfer), and (iii) retain or even improve the performance on earlier tasks after learning new tasks (i.e., backward-transfer). This new problem is studied on a stream of more than 60 tasks, each equipped with an instruction. Technically, our method InstructionSpeak contains two strategies that make full use of task instructions to improve forward-transfer and backward-transfer: one is to learn from negative outputs, the other is to re-visit instructions of previous tasks. To our knowledge, this is the first time to study ConTinTin in NLP. In addition to the problem formulation and our promising approach, this work also contributes to providing rich analyses for the community to better understand this novel learning problem.

cs.CL↗

arXiv2022

Styleverse: Towards Identity Stylization across Heterogeneous Domains

We propose a new challenging task namely IDentity Stylization (IDS) across heterogeneous domains. IDS focuses on stylizing the content identity, rather than completely swapping it using the reference identity. We use an effective heterogeneous-network-based framework $Styleverse$ that uses a single domain-aware generator to exploit the Metaverse of diverse heterogeneous faces, based on the proposed dataset FS13 with limited data. FS13 means 13 kinds of Face Styles considering diverse lighting conditions, art representations and life dimensions. Previous similar tasks, \eg, image style transfer can handle textural style transfer based on a reference image. This task usually ignores the high structure-aware facial area and high-fidelity preservation of the content. However, Styleverse intends to controllably create topology-aware faces in the Parallel Style Universe, where the source facial identity is adaptively styled via AdaIN guided by the domain-aware and reference-aware style embeddings from heterogeneous pretrained models. We first establish the IDS quantitative benchmark as well as the qualitative Styleverse matrix. Extensive experiments demonstrate that Styleverse achieves higher-fidelity identity stylization compared with other state-of-the-art methods.

cs.CV↗

arXiv2022

Intent Contrastive Learning for Sequential Recommendation

Users' interactions with items are driven by various intents (e.g., preparing for holiday gifts, shopping for fishing equipment, etc.).However, users' underlying intents are often unobserved/latent, making it challenging to leverage such latent intents forSequentialrecommendation(SR). To investigate the benefits of latent intents and leverage them effectively for recommendation, we proposeIntentContrastiveLearning(ICL), a general learning paradigm that leverages a latent intent variable into SR. The core idea is to learn users' intent distribution functions from unlabeled user behavior sequences and optimize SR models with contrastive self-supervised learning (SSL) by considering the learned intents to improve recommendation. Specifically, we introduce a latent variable to represent users' intents and learn the distribution function of the latent variable via clustering. We propose to leverage the learned intents into SR models via contrastive SSL, which maximizes the agreement between a view of sequence and its corresponding intent. The training is alternated between intent representation learning and the SR model optimization steps within the generalized expectation-maximization (EM) framework. Fusing user intent information into SR also improves model robustness. Experiments conducted on four real-world datasets demonstrate the superiority of the proposed learning paradigm, which improves performance, and robustness against data sparsity and noisy interaction issues.

cs.AI↗

arXiv2022

RGRecSys: A Toolkit for Robustness Evaluation of Recommender Systems

Robust machine learning is an increasingly important topic that focuses on developing models resilient to various forms of imperfect data. Due to the pervasiveness of recommender systems in online technologies, researchers have carried out several robustness studies focusing on data sparsity and profile injection attacks. Instead, we propose a more holistic view of robustness for recommender systems that encompasses multiple dimensions - robustness with respect to sub-populations, transformations, distributional disparity, attack, and data sparsity. While there are several libraries that allow users to compare different recommender system models, there is no software library for comprehensive robustness evaluation of recommender system models under different scenarios. As our main contribution, we present a robustness evaluation toolkit, Robustness Gym for RecSys (RGRecSys -- https://www.github.com/salesforce/RGRecSys), that allows us to quickly and uniformly evaluate the robustness of recommender system models.

cs.IR↗

arXiv2022

Interaction expansion inchworm Monte Carlo solver for lattice and impurity models

Multi-orbital quantum impurity models with general interaction and hybridization terms appear in a wide range of applications including embedding, quantum transport, and nanoscience. However, most quantum impurity solvers are restricted to a few impurity orbitals, discretized baths, diagonal hybridizations, or density-density interactions. Here, we generalize the inchworm quantum Monte Carlo method to the interaction expansion and explore its application to typical single- and multi-orbital problems encountered in investigations of impurity and lattice models. Our implementation generically outperforms bare and bold-line quantum Monte Carlo algorithms in the interaction expansion. So far, for the systems studied here, it remains inferior to the more specialized hybridization expansion and auxiliary field algorithms. The problem of convergence to unphysical fixed points, which hampers so-called bold-line methods, is not encountered in inchworm Monte Carlo.

cond-mat.str-el↗

arXiv2022

Complementary Feature Enhanced Network with Vision Transformer for Image Dehazing

Conventional CNNs-based dehazing models suffer from two essential issues: the dehazing framework (limited in interpretability) and the convolution layers (content-independent and ineffective to learn long-range dependency information). In this paper, firstly, we propose a new complementary feature enhanced framework, in which the complementary features are learned by several complementary subtasks and then together serve to boost the performance of the primary task. One of the prominent advantages of the new framework is that the purposively chosen complementary tasks can focus on learning weakly dependent complementary features, avoiding repetitive and ineffective learning of the networks. We design a new dehazing network based on such a framework. Specifically, we select the intrinsic image decomposition as the complementary tasks, where the reflectance and shading prediction subtasks are used to extract the color-wise and texture-wise complementary features. To effectively aggregate these complementary features, we propose a complementary features selection module (CFSM) to select the more useful features for image dehazing. Furthermore, we introduce a new version of vision transformer block, named Hybrid Local-Global Vision Transformer (HyLoG-ViT), and incorporate it within our dehazing networks. The HyLoG-ViT block consists of the local and the global vision transformer paths used to capture local and global dependencies. As a result, the HyLoG-ViT introduces locality in the networks and captures the global and long-range dependencies. Extensive experiments on homogeneous, non-homogeneous, and nighttime dehazing tasks reveal that the proposed dehazing network can achieve comparable or even better performance than CNNs-based dehazing models.

cs.CV↗

arXiv2021

Verification of asymptotic homogenization method developed for periodic architected materials in strain gradient continuum

Strain gradient theory is an accurate model for capturing the size effect and localization phenomena. However, the challenge in identification of corresponding constitutive parameters limits the practical application of the theory. We present and utilize asymptotic homogenization herein. All parameters in rank four, five, and six tensors are determined with the demonstrated computational approach. Examples for epoxy carbon fiber composite, metal matrix composite, and aluminum foam illustrate the effectiveness and versatility of the proposed method. The influences of volume fraction of matrix, the stack of RVEs, and the varying unit cell lengths on the identified parameters are investigated. The homogenization computational tool is applicable to a wide class materials and makes use of open-source codes in FEniCS. We make all of the codes publicly available in order to encourage a transparent scientific exchange.

cs.CE↗

arXiv2021

Combining Data-driven Supervision with Human-in-the-loop Feedback for Entity Resolution

The distribution gap between training datasets and data encountered in production is well acknowledged. Training datasets are often constructed over a fixed period of time and by carefully curating the data to be labeled. Thus, training datasets may not contain all possible variations of data that could be encountered in real-world production environments. Tasked with building an entity resolution system - a model that identifies and consolidates data points that represent the same person - our first model exhibited a clear training-production performance gap. In this case study, we discuss our human-in-the-loop enabled, data-centric solution to closing the training-production performance divergence. We conclude with takeaways that apply to data-centric learning at large.

cs.CL↗

arXiv2021

Deconvolutional Networks on Graph Data

In this paper, we consider an inverse problem in graph learning domain -- ``given the graph representations smoothed by Graph Convolutional Network (GCN), how can we reconstruct the input graph signal?" We propose Graph Deconvolutional Network (GDN) and motivate the design of GDN via a combination of inverse filters in spectral domain and de-noising layers in wavelet domain, as the inverse operation results in a high frequency amplifier and may amplify the noise. We demonstrate the effectiveness of the proposed method on several tasks including graph feature imputation and graph structure generation.

cs.LG↗

arXiv2021

Receptive Field Broadening and Boosting for Salient Object Detection

Salient object detection requires a comprehensive and scalable receptive field to locate the visually significant objects in the image. Recently, the emergence of visual transformers and multi-branch modules has significantly enhanced the ability of neural networks to perceive objects at different scales. However, compared to the traditional backbone, the calculation process of transformers is time-consuming. Moreover, different branches of the multi-branch modules could cause the same error back propagation in each training iteration, which is not conducive to extracting discriminative features. To solve these problems, we propose a bilateral network based on transformer and CNN to efficiently broaden local details and global semantic information simultaneously. Besides, a Multi-Head Boosting (MHB) strategy is proposed to enhance the specificity of different network branches. By calculating the errors of different prediction heads, each branch can separately pay more attention to the pixels that other branches predict incorrectly. Moreover, Unlike multi-path parallel training, MHB randomly selects one branch each time for gradient back propagation in a boosting way. Additionally, an Attention Feature Fusion Module (AF) is proposed to fuse two types of features according to respective characteristics. Comprehensive experiments on five benchmark datasets demonstrate that the proposed method can achieve a significant performance improvement compared with the state-of-the-art methods.

cs.CV↗

arXiv2021

Transformer-based Dual Relation Graph for Multi-label Image Recognition

The simultaneous recognition of multiple objects in one image remains a challenging task, spanning multiple events in the recognition field such as various object scales, inconsistent appearances, and confused inter-class relationships. Recent research efforts mainly resort to the statistic label co-occurrences and linguistic word embedding to enhance the unclear semantics. Different from these researches, in this paper, we propose a novel Transformer-based Dual Relation learning framework, constructing complementary relationships by exploring two aspects of correlation, i.e., structural relation graph and semantic relation graph. The structural relation graph aims to capture long-range correlations from object context, by developing a cross-scale transformer-based architecture. The semantic graph dynamically models the semantic meanings of image objects with explicit semantic-aware constraints. In addition, we also incorporate the learnt structural relationship into the semantic graph, constructing a joint relation graph for robust representations. With the collaborative learning of these two effective relation graphs, our approach achieves new state-of-the-art on two popular multi-label recognition benchmarks, i.e., MS-COCO and VOC 2007 dataset.

cs.CV↗

arXiv2021

Universal Face Restoration With Memorized Modulation

Blind face restoration (BFR) is a challenging problem because of the uncertainty of the degradation patterns. This paper proposes a Restoration with Memorized Modulation (RMM) framework for universal BFR in diverse degraded scenes and heterogeneous domains. We apply random noise as well as unsupervised wavelet memory to adaptively modulate the face-enhancement generator, considering attentional denormalization in both layer and instance levels. Specifically, in the training stage, the low-level spatial feature embedding, the wavelet memory embedding obtained by wavelet transformation of the high-resolution image, as well as the disentangled high-level noise embeddings are integrated, with the guidance of attentional maps generated from layer normalization, instance normalization, and the original feature map. These three embeddings are respectively associated with the spatial content, high-frequency texture details, and a learnable universal prior against other blind image degradation patterns. We store the spatial feature of the low-resolution image and the corresponding wavelet style code as key and value in the memory unit, respectively. In the test stage, the wavelet memory value whose corresponding spatial key is the most matching with that of the inferred image is retrieved to modulate the generator. Moreover, the universal prior learned from the random noise has been memorized by training the modulation network. Experimental results show the superiority of the proposed method compared with the state-of-the-art methods, and a good generalization in the wild.

cs.CV↗

arXiv2021

Epsilon Consistent Mixup: Structural Regularization with an Adaptive Consistency-Interpolation Tradeoff

In this paper we propose $ε$-Consistent Mixup ($ε$mu). $ε$mu is a data-based structural regularization technique that combines Mixup's linear interpolation with consistency regularization in the Mixup direction, by compelling a simple adaptive tradeoff between the two. This learnable combination of consistency and interpolation induces a more flexible structure on the evolution of the response across the feature space and is shown to improve semi-supervised classification accuracy on the SVHN and CIFAR10 benchmark datasets, yielding the largest gains in the most challenging low label-availability scenarios. Empirical studies comparing $ε$mu and Mixup are presented and provide insight into the mechanisms behind $ε$mu's effectiveness. In particular, $ε$mu is found to produce more accurate synthetic labels and more confident predictions than Mixup.

stat.ML↗