Search arXiv⌕ Search

arXiv subjects

Ping Liu

Publications and source records attributed to Ping Liu.

At least 91 records · Page 5Linked to original sources
arXiv2024

VehicleGAN: Pair-flexible Pose Guided Image Synthesis for Vehicle Re-identification

Vehicle Re-identification (Re-ID) has been broadly studied in the last decade; however, the different camera view angle leading to confused discrimination in the feature subspace for the vehicles of various poses, is still challenging for the Vehicle Re-ID models in the real world. To promote the Vehicle Re-ID models, this paper proposes to synthesize a large number of vehicle images in the target pose, whose idea is to project the vehicles of diverse poses into the unified target pose so as to enhance feature discrimination. Considering that the paired data of the same vehicles in different traffic surveillance cameras might be not available in the real world, we propose the first Pair-flexible Pose Guided Image Synthesis method for Vehicle Re-ID, named as VehicleGAN in this paper, which works for both supervised and unsupervised settings without the knowledge of geometric 3D models. Because of the feature distribution difference between real and synthetic data, simply training a traditional metric learning based Re-ID model with data-level fusion (i.e., data augmentation) is not satisfactory, therefore we propose a new Joint Metric Learning (JML) via effective feature-level fusion from both real and synthetic data. Intensive experimental results on the public VeRi-776 and VehicleID datasets prove the accuracy and effectiveness of our proposed VehicleGAN and JML.

cs.CV↗
arXiv2024

Exponentially localised interface eigenmodes in finite chains of resonators

This paper studies wave localisation in chains of finitely many resonators. There is an extensive theory predicting the existence of localised modes induced by defects in infinitely periodic systems. This work extends these principles to finite-sized systems. We consider finite systems of subwavelength resonators arranged in dimers that have a geometric defect in the structure. This is a classical wave analogue of the Su-Schrieffer-Heeger model. We prove the existence of a spectral gap for defectless finite dimer structures and find a direct relationship between eigenvalues being within the spectral gap and the localisation of their associated eigenmode. Then we show the existence and uniqueness of an eigenvalue in the gap in the defect structure, proving the existence of a unique localised interface mode. To the best of our knowledge, our method, based on Chebyshev polynomials, is the first to characterise quantitatively the localised interface modes in systems of finitely many resonators.

math-ph↗
arXiv2024

Frequency-Aware Deepfake Detection: Improving Generalizability through Frequency Space Learning

This research addresses the challenge of developing a universal deepfake detector that can effectively identify unseen deepfake images despite limited training data. Existing frequency-based paradigms have relied on frequency-level artifacts introduced during the up-sampling in GAN pipelines to detect forgeries. However, the rapid advancements in synthesis technology have led to specific artifacts for each generation model. Consequently, these detectors have exhibited a lack of proficiency in learning the frequency domain and tend to overfit to the artifacts present in the training data, leading to suboptimal performance on unseen sources. To address this issue, we introduce a novel frequency-aware approach called FreqNet, centered around frequency domain learning, specifically designed to enhance the generalizability of deepfake detectors. Our method forces the detector to continuously focus on high-frequency information, exploiting high-frequency representation of features across spatial and channel dimensions. Additionally, we incorporate a straightforward frequency domain learning module to learn source-agnostic features. It involves convolutional layers applied to both the phase spectrum and amplitude spectrum between the Fast Fourier Transform (FFT) and Inverse Fast Fourier Transform (iFFT). Extensive experimentation involving 17 GANs demonstrates the effectiveness of our proposed method, showcasing state-of-the-art performance (+9.8\%) while requiring fewer parameters. The code is available at {\cred \url{https://github.com/chuangchuangtan/FreqNet-DeepfakeDetection}}.

cs.CV↗
arXiv2024

Data-Independent Operator: A Training-Free Artifact Representation Extractor for Generalizable Deepfake Detection

Recently, the proliferation of increasingly realistic synthetic images generated by various generative adversarial networks has increased the risk of misuse. Consequently, there is a pressing need to develop a generalizable detector for accurately recognizing fake images. The conventional methods rely on generating diverse training sources or large pretrained models. In this work, we show that, on the contrary, the small and training-free filter is sufficient to capture more general artifact representations. Due to its unbias towards both the training and test sources, we define it as Data-Independent Operator (DIO) to achieve appealing improvements on unseen sources. In our framework, handcrafted filters and the randomly-initialized convolutional layer can be used as the training-free artifact representations extractor with excellent results. With the data-independent operator of a popular classifier, such as Resnet50, one could already reach a new state-of-the-art without bells and whistles. We evaluate the effectiveness of the DIO on 33 generation models, even DALLE and Midjourney. Our detector achieves a remarkable improvement of $13.3\%$, establishing a new state-of-the-art performance. The DIO and its extension can serve as strong baselines for future methods. The code is available at \url{https://github.com/chuangchuangtan/Data-Independent-Operator}.

cs.CV↗
arXiv2024

Counterfactual Co-occurring Learning for Bias Mitigation in Weakly-supervised Object Localization

Contemporary weakly-supervised object localization (WSOL) methods have primarily focused on addressing the challenge of localizing the most discriminative region while largely overlooking the relatively less explored issue of biased activation -- incorrectly spotlighting co-occurring background with the foreground feature. In this paper, we conduct a thorough causal analysis to investigate the origins of biased activation. Based on our analysis, we attribute this phenomenon to the presence of co-occurring background confounders. Building upon this profound insight, we introduce a pioneering paradigm known as Counterfactual Co-occurring Learning (CCL), meticulously engendering counterfactual representations by adeptly disentangling the foreground from the co-occurring background elements. Furthermore, we propose an innovative network architecture known as Counterfactual-CAM. This architecture seamlessly incorporates a perturbation mechanism for counterfactual representations into the vanilla CAM-based model. By training the WSOL model with these perturbed representations, we guide the model to prioritize the consistent foreground content while concurrently reducing the influence of distracting co-occurring backgrounds. To the best of our knowledge, this study represents the initial exploration of this research direction. Our extensive experiments conducted across multiple benchmarks validate the effectiveness of the proposed Counterfactual-CAM in mitigating biased activation.

cs.CV↗
arXiv2024

Learning to Retrieve for Job Matching

Web-scale search systems typically tackle the scalability challenge with a two-step paradigm: retrieval and ranking. The retrieval step, also known as candidate selection, often involves extracting standardized entities, creating an inverted index, and performing term matching for retrieval. Such traditional methods require manual and time-consuming development of query models. In this paper, we discuss applying learning-to-retrieve technology to enhance LinkedIns job search and recommendation systems. In the realm of promoted jobs, the key objective is to improve the quality of applicants, thereby delivering value to recruiter customers. To achieve this, we leverage confirmed hire data to construct a graph that evaluates a seeker's qualification for a job, and utilize learned links for retrieval. Our learned model is easy to explain, debug, and adjust. On the other hand, the focus for organic jobs is to optimize seeker engagement. We accomplished this by training embeddings for personalized retrieval, fortified by a set of rules derived from the categorization of member feedback. In addition to a solution based on a conventional inverted index, we developed an on-GPU solution capable of supporting both KNN and term matching efficiently.

cs.IR↗
arXiv2024

LinkSAGE: Optimizing Job Matching Using Graph Neural Networks

We present LinkSAGE, an innovative framework that integrates Graph Neural Networks (GNNs) into large-scale personalized job matching systems, designed to address the complex dynamics of LinkedIns extensive professional network. Our approach capitalizes on a novel job marketplace graph, the largest and most intricate of its kind in industry, with billions of nodes and edges. This graph is not merely extensive but also richly detailed, encompassing member and job nodes along with key attributes, thus creating an expansive and interwoven network. A key innovation in LinkSAGE is its training and serving methodology, which effectively combines inductive graph learning on a heterogeneous, evolving graph with an encoder-decoder GNN model. This methodology decouples the training of the GNN model from that of existing Deep Neural Nets (DNN) models, eliminating the need for frequent GNN retraining while maintaining up-to-date graph signals in near realtime, allowing for the effective integration of GNN insights through transfer learning. The subsequent nearline inference system serves the GNN encoder within a real-world setting, significantly reducing online latency and obviating the need for costly real-time GNN infrastructure. Validated across multiple online A/B tests in diverse product scenarios, LinkSAGE demonstrates marked improvements in member engagement, relevance matching, and member retention, confirming its generalizability and practical impact.

cs.LG↗
arXiv2024

LiGNN: Graph Neural Networks at LinkedIn

In this paper, we present LiGNN, a deployed large-scale Graph Neural Networks (GNNs) Framework. We share our insight on developing and deployment of GNNs at large scale at LinkedIn. We present a set of algorithmic improvements to the quality of GNN representation learning including temporal graph architectures with long term losses, effective cold start solutions via graph densification, ID embeddings and multi-hop neighbor sampling. We explain how we built and sped up by 7x our large-scale training on LinkedIn graphs with adaptive sampling of neighbors, grouping and slicing of training data batches, specialized shared-memory queue and local gradient optimization. We summarize our deployment lessons and learnings gathered from A/B test experiments. The techniques presented in this work have contributed to an approximate relative improvements of 1% of Job application hearing back rate, 2% Ads CTR lift, 0.5% of Feed engaged daily active users, 0.2% session lift and 0.1% weekly active user lift from people recommendation. We believe that this work can provide practical solutions and insights for engineers who are interested in applying Graph neural networks at large scale.

cs.LG↗
arXiv2024

Spectra and pseudo-spectra of tridiagonal $k$-Toeplitz matrices and the topological origin of the non-Hermitian skin effect

We establish new results on the spectra and pseudo-spectra of tridiagonal $k$-Toeplitz operators and matrices. In particular, we prove the connection between the winding number of the eigenvalues of the symbol function and the exponential decay of the associated eigenvectors (or pseudo-eigenvectors). Our results elucidate the topological origin of the non-Hermitian skin effect in general one-dimensional polymer systems of subwavelength resonators with imaginary gauge potentials, proving the observation and conjecture in arXiv:2307.13551. We also numerically verify our theory for these systems.

math-ph↗
arXiv2024

Unified Diffusion-Based Rigid and Non-Rigid Editing with Text and Image Guidance

Existing text-to-image editing methods tend to excel either in rigid or non-rigid editing but encounter challenges when combining both, resulting in misaligned outputs with the provided text prompts. In addition, integrating reference images for control remains challenging. To address these issues, we present a versatile image editing framework capable of executing both rigid and non-rigid edits, guided by either textual prompts or reference images. We leverage a dual-path injection scheme to handle diverse editing scenarios and introduce an integrated self-attention mechanism for fusion of appearance and structural information. To mitigate potential visual artifacts, we further employ latent fusion techniques to adjust intermediate latents. Compared to previous work, our approach represents a significant advance in achieving precise and versatile image editing. Comprehensive experiments validate the efficacy of our method, showcasing competitive or superior results in text-based editing and appearance transfer tasks, encompassing both rigid and non-rigid settings.

cs.CV↗
arXiv2023

Rethinking the Up-Sampling Operations in CNN-based Generative Network for Generalizable Deepfake Detection

Recently, the proliferation of highly realistic synthetic images, facilitated through a variety of GANs and Diffusions, has significantly heightened the susceptibility to misuse. While the primary focus of deepfake detection has traditionally centered on the design of detection algorithms, an investigative inquiry into the generator architectures has remained conspicuously absent in recent years. This paper contributes to this lacuna by rethinking the architectures of CNN-based generators, thereby establishing a generalized representation of synthetic artifacts. Our findings illuminate that the up-sampling operator can, beyond frequency-based artifacts, produce generalized forgery artifacts. In particular, the local interdependence among image pixels caused by upsampling operators is significantly demonstrated in synthetic images generated by GAN or diffusion. Building upon this observation, we introduce the concept of Neighboring Pixel Relationships(NPR) as a means to capture and characterize the generalized structural artifacts stemming from up-sampling operations. A comprehensive analysis is conducted on an open-world dataset, comprising samples generated by \tft{28 distinct generative models}. This analysis culminates in the establishment of a novel state-of-the-art performance, showcasing a remarkable \tft{11.6\%} improvement over existing methods. The code is available at https://github.com/chuangchuangtan/NPR-DeepfakeDetection.

cs.CV↗
arXiv2023

The Non-Hermitian Skin Effect With Three-Dimensional Long-Range Coupling

We study the non-Hermitian skin effect in a three-dimensional system of finitely many subwavelength resonators with an imaginary gauge potential. We introduce a discrete approximation of the eigenmodes and eigenfrequencies of the system in terms of the eigenvectors and eigenvalues of the so-called gauge capacitance matrix $\mathcal{C}_N^γ$, which is a dense matrix due to long-range interactions in the system. Based on translational invariance of this matrix and the decay of its off-diagonal entries, we prove the condensation of the eigenmodes at one edge of the structure by showing the exponential decay of its pseudo-eigenvectors. In particular, we consider a range-k approximation to keep the long-range interaction to a certain extent, thus obtaining a k-banded gauge capacitance matrix $\mathcal{C}_{N,k}^γ$ . Using techniques for Toeplitz matrices and operators, we establish the exponential decay of the pseudo-eigenvectors of $\mathcal{C}_{N,k}^γ$ and demonstrate that they approximate those of the gauge capacitance matrix $\mathcal{C}_N^γ$ well. Our results are numerically verified. In particular, we show that long-range interactions affect only the first eigenmodes in the system. As a result, a tridiagonal approximation of the gauge capacitance matrix, similar to the nearest-neighbour approximation in quantum mechanics, provides a good approximation for the higher modes. Moreover, we also illustrate numerically the behaviour of the eigenmodes and the stability of the non-Hermitian skin effect with respect to disorder in a variety of three-dimensional structures.

math.AP↗
arXiv2023

CLE Diffusion: Controllable Light Enhancement Diffusion Model

Low light enhancement has gained increasing importance with the rapid development of visual creation and editing. However, most existing enhancement algorithms are designed to homogeneously increase the brightness of images to a pre-defined extent, limiting the user experience. To address this issue, we propose Controllable Light Enhancement Diffusion Model, dubbed CLE Diffusion, a novel diffusion framework to provide users with rich controllability. Built with a conditional diffusion model, we introduce an illumination embedding to let users control their desired brightness level. Additionally, we incorporate the Segment-Anything Model (SAM) to enable user-friendly region controllability, where users can click on objects to specify the regions they wish to enhance. Extensive experiments demonstrate that CLE Diffusion achieves competitive performance regarding quantitative metrics, qualitative results, and versatile controllability. Project page: https://yuyangyin.github.io/CLEDiffusion/

cs.CV↗
arXiv2023

Stability of the non-Hermitian skin effect

This paper shows that the skin effect in systems of non-Hermitian subwavelength resonators is robust with respect to random imperfections in the system. The subwavelength resonators are highly contrasting material inclusions that resonate in a low-frequency regime. The non-Hermiticity is due to the introduction of an imaginary gauge potential, which leads to a skin effect that is manifested by the system's eigenmodes accumulating at one edge of the structure. We elucidate the topological protection of the associated (real) eigenfrequencies and illustrate the competition between the two different localisation effects present when the system is randomly perturbed: the non-Hermitian skin effect and the disorder-induced Anderson localisation. We show that, as the strength of the disorder increases, more and more eigenmodes become localised in the bulk. Our results are based on an asymptotic matrix model for subwavelength physics and can be generalised also to tight-binding models in condensed matter theory.

math-ph↗
arXiv2023

Dual Stage Stylization Modulation for Domain Generalized Semantic Segmentation

Obtaining sufficient labeled data for training deep models is often challenging in real-life applications. To address this issue, we propose a novel solution for single-source domain generalized semantic segmentation. Recent approaches have explored data diversity enhancement using hallucination techniques. However, excessive hallucination can degrade performance, particularly for imbalanced datasets. As shown in our experiments, minority classes are more susceptible to performance reduction due to hallucination compared to majority classes. To tackle this challenge, we introduce a dual-stage Feature Transform (dFT) layer within the Adversarial Semantic Hallucination+ (ASH+) framework. The ASH+ framework performs a dual-stage manipulation of hallucination strength. By leveraging semantic information for each pixel, our approach adaptively adjusts the pixel-wise hallucination strength, thus providing fine-grained control over hallucination. We validate the effectiveness of our proposed method through comprehensive experiments on publicly available semantic segmentation benchmark datasets (Cityscapes and SYNTHIA). Quantitative and qualitative comparisons demonstrate that our approach is competitive with state-of-the-art methods for the Cityscapes dataset and surpasses existing solutions for the SYNTHIA dataset. Code for our framework will be made readily available to the research community.

cs.CV↗
arXiv2023

A mathematical theory of resolution limits for super-resolution of positive sources

A priori information on the positivity of source intensities is ubiquitous in imaging fields and is also important for a multitude of super-resolution and deconvolution algorithms. However, the fundamental resolution limit of positive sources is still unknown, and research in this field is very limited indeed. In this work, we analyze the super-resolving capacity for number and location recoveries in the super-resolution of positive sources and aim to answer the resolution limit problem in a rigorous manner. Specifically, we introduce the computational resolution limit for respectively the number detection and location recovery in the one-dimensional super-resolution problem and quantitatively characterize their dependency on the cutoff frequency, signal-to-noise ratio, and the sparsity of the sources. As a direct consequence, we show that targeting at the sparest positive solution in the super-resolution already provides the optimal resolution order. These results are generalized to multi-dimensional spaces. Our estimates indicate that there exist phase transitions in the corresponding reconstructions, which are confirmed by numerical experiments. On the other hand, despite the fact that positivity plays important roles in improving the resolution of certain super-resolution algorithms, our theory has made several different but significant discoveries: i) The a priori information of positivity cannot further improve the order of the resolution limit; ii) The positivity of the source sometimes deteriorates the resolution limit instead of enhancing it. In particular, under certain signal-to-noise ratio, two point sources with different phases actually have a better resolution limit than those with the same one.

eess.IV↗
arXiv2023

Perturbed Block Toeplitz matrices and the non-Hermitian skin effect in dimer systems of subwavelength resonators

The aim of this paper is fourfold: (i) to obtain explicit formulas for the eigenpairs of perturbed tridiagonal block Toeplitz matrices; (ii) to make use of such formulas in order to provide a mathematical justification of the non-Hermitian skin effect in dimer systems by proving the condensation of the system's bulk eigenmodes at one of the edges of the system; (iii) to show the topological origin of the non-Hermitian skin effect for dimer systems and (iv) to prove localisation of the interface modes between two dimer structures with non-Hermitian gauge potentials of opposite signs based on new estimates of the decay of the entries of the eigenvectors of block matrices with mirrored blocks.

math-ph↗
arXiv2023

Text-guided Eyeglasses Manipulation with Spatial Constraints

Virtual try-on of eyeglasses involves placing eyeglasses of different shapes and styles onto a face image without physically trying them on. While existing methods have shown impressive results, the variety of eyeglasses styles is limited and the interactions are not always intuitive or efficient. To address these limitations, we propose a Text-guided Eyeglasses Manipulation method that allows for control of the eyeglasses shape and style based on a binary mask and text, respectively. Specifically, we introduce a mask encoder to extract mask conditions and a modulation module that enables simultaneous injection of text and mask conditions. This design allows for fine-grained control of the eyeglasses' appearance based on both textual descriptions and spatial constraints. Our approach includes a disentangled mapper and a decoupling strategy that preserves irrelevant areas, resulting in better local editing. We employ a two-stage training scheme to handle the different convergence speeds of the various modality conditions, successfully controlling both the shape and style of eyeglasses. Extensive comparison experiments and ablation analyses demonstrate the effectiveness of our approach in achieving diverse eyeglasses styles while preserving irrelevant areas.

cs.CV↗