Search arXiv⌕ Search

arXiv subjects

Ming Lu

Publications and source records attributed to Ming Lu.

At least 181 records · Page 10Linked to original sources
arXiv2022

NTIRE 2021 Challenge on Quality Enhancement of Compressed Video: Methods and Results

This paper reviews the first NTIRE challenge on quality enhancement of compressed video, with a focus on the proposed methods and results. In this challenge, the new Large-scale Diverse Video (LDV) dataset is employed. The challenge has three tracks. Tracks 1 and 2 aim at enhancing the videos compressed by HEVC at a fixed QP, while Track 3 is designed for enhancing the videos compressed by x265 at a fixed bit-rate. Besides, the quality enhancement of Tracks 1 and 3 targets at improving the fidelity (PSNR), and Track 2 targets at enhancing the perceptual quality. The three tracks totally attract 482 registrations. In the test phase, 12 teams, 8 teams and 11 teams submitted the final results of Tracks 1, 2 and 3, respectively. The proposed methods and solutions gauge the state-of-the-art of video quality enhancement. The homepage of the challenge: https://github.com/RenYang-home/NTIRE21_VEnh

eess.IV↗
arXiv2022

Uncertainty Guided Depth Fusion for Spike Camera

Depth estimation is essential for various important real-world applications such as autonomous driving. However, it suffers from severe performance degradation in high-velocity scenario since traditional cameras can only capture blurred images. To deal with this problem, the spike camera is designed to capture the pixel-wise luminance intensity at high frame rate. However, depth estimation with spike camera remains very challenging using traditional monocular or stereo depth estimation algorithms, which are based on the photometric consistency. In this paper, we propose a novel Uncertainty-Guided Depth Fusion (UGDF) framework to fuse the predictions of monocular and stereo depth estimation networks for spike camera. Our framework is motivated by the fact that stereo spike depth estimation achieves better results at close range while monocular spike depth estimation obtains better results at long range. Therefore, we introduce a dual-task depth estimation architecture with a joint training strategy and estimate the distributed uncertainty to fuse the monocular and stereo results. In order to demonstrate the advantage of spike depth estimation over traditional camera depth estimation, we contribute a spike-depth dataset named CitySpike20K, which contains 20K paired samples, for spike depth estimation. UGDF achieves state-of-the-art results on CitySpike20K, surpassing all monocular or stereo spike depth estimation baselines. We conduct extensive experiments to evaluate the effectiveness and generalization of our method on CitySpike20K. To the best of our knowledge, our framework is the first dual-task fusion framework for spike camera depth estimation. Code and dataset will be released.

cs.CV↗
arXiv2022

Serre-Lusztig relations for $\imath$quantum groups III

let $\widetilde{\bf U}^\imath$ be a quasi-split universal $\imath$quantum group associated to a quantum symmetric pair $(\widetilde{\bf U}, \widetilde{\bf U}^\imath)$ of Kac-Moody type with a diagram involution $τ$. We establish the Serre-Lusztig relations for $\widetilde{\bf U}^\imath$ associated to a simple root $i$ such that $i \neq τi$, complementary to the Serre-Lusztig relations associated to $i=τi$ which we obtained earlier. A conjecture on braid group symmetries on $\widetilde{\bf U}^\imath$ associated to $i$ disjoint from $τi$ is formulated.

math.QA↗
arXiv2022

Modified Generalized-Brillouin-Zone Theory with On-site Disorders

We study the characterization of the non-Hermitian skin effect (NHSE) in non-Hermitian systems with on-site disorder. We extend the application of generalized-Brillouin-zone (GBZ) theory to these systems. By proposing a modified GBZ theory, we give a faithfully description of the NHSE. For applications, we obtain a unified $β$ for system with long-range hopping, and explain the conventional-GBZ irrelevance of the magnetic suppression of the NHSE in the previous study.

cond-mat.dis-nn↗
arXiv2022

Non-Hermiticity stabilized Majorana zero modes in semiconductor-superconductor nanowires

Coupled Majorana zero modes with nonzero energies are generally detrimental to the non-Abelian statistics due to the additional dynamic phase. Nevertheless, we show that a well-connected lead can introduce a local non-Hermitian dissipation term to shift the energies of the both coupled Majorana modes to zero, and surprisingly turn the coupled Majorana mode far from the lead into a dark Majorana mode with exponentially small dissipation. This dark Majorana mode can conquer the drawback of the partially overlapped Majorana zero modes and possess the properties of true Majorana zero mode such as the perfect fractional Josephson effect and the non-Abelian statistics.

cond-mat.mes-hall↗
arXiv2022

Adaptive Patch Exiting for Scalable Single Image Super-Resolution

Since the future of computing is heterogeneous, scalability is a crucial problem for single image super-resolution. Recent works try to train one network, which can be deployed on platforms with different capacities. However, they rely on the pixel-wise sparse convolution, which is not hardware-friendly and achieves limited practical speedup. As image can be divided into patches, which have various restoration difficulties, we present a scalable method based on Adaptive Patch Exiting (APE) to achieve more practical speedup. Specifically, we propose to train a regressor to predict the incremental capacity of each layer for the patch. Once the incremental capacity is below the threshold, the patch can exit at the specific layer. Our method can easily adjust the trade-off between performance and efficiency by changing the threshold of incremental capacity. Furthermore, we propose a novel strategy to enable the network training of our method. We conduct extensive experiments across various backbones, datasets and scaling factors to demonstrate the advantages of our method. Code is available at https://github.com/littlepure2333/APE

cs.CV↗
arXiv2022

Efficient Meta-Tuning for Content-aware Neural Video Delivery

Recently, Deep Neural Networks (DNNs) are utilized to reduce the bandwidth and improve the quality of Internet video delivery. Existing methods train corresponding content-aware super-resolution (SR) model for each video chunk on the server, and stream low-resolution (LR) video chunks along with SR models to the client. Although they achieve promising results, the huge computational cost of network training limits their practical applications. In this paper, we present a method named Efficient Meta-Tuning (EMT) to reduce the computational cost. Instead of training from scratch, EMT adapts a meta-learned model to the first chunk of the input video. As for the following chunks, it fine-tunes the partial parameters selected by gradient masking of previous adapted model. In order to achieve further speedup for EMT, we propose a novel sampling strategy to extract the most challenging patches from video frames. The proposed strategy is highly efficient and brings negligible additional cost. Our method significantly reduces the computational cost and achieves even better performance, paving the way for applying neural video delivery techniques to practical applications. We conduct extensive experiments based on various efficient SR architectures, including ESPCN, SRCNN, FSRCNN and EDSR-1, demonstrating the generalization ability of our work. The code is released at \url{https://github.com/Neural-video-delivery/EMT-Pytorch-ECCV2022}.

cs.CV↗
arXiv2022

Structure-aware Editable Morphable Model for 3D Facial Detail Animation and Manipulation

Morphable models are essential for the statistical modeling of 3D faces. Previous works on morphable models mostly focus on large-scale facial geometry but ignore facial details. This paper augments morphable models in representing facial details by learning a Structure-aware Editable Morphable Model (SEMM). SEMM introduces a detail structure representation based on the distance field of wrinkle lines, jointly modeled with detail displacements to establish better correspondences and enable intuitive manipulation of wrinkle structure. Besides, SEMM introduces two transformation modules to translate expression blendshape weights and age values into changes in latent space, allowing effective semantic detail editing while maintaining identity. Extensive experiments demonstrate that the proposed model compactly represents facial details, outperforms previous methods in expression animation qualitatively and quantitatively, and achieves effective age editing and wrinkle line editing of facial details. Code and model are available at https://github.com/gerwang/facial-detail-manipulation.

cs.CV↗
arXiv2022

NFANet: A Novel Method for Weakly Supervised Water Extraction from High-Resolution Remote Sensing Imagery

The use of deep learning for water extraction requires precise pixel-level labels. However, it is very difficult to label high-resolution remote sensing images at the pixel level. Therefore, we study how to utilize point labels to extract water bodies and propose a novel method called the neighbor feature aggregation network (NFANet). Compared with pixellevel labels, point labels are much easier to obtain, but they will lose much information. In this paper, we take advantage of the similarity between the adjacent pixels of a local water-body, and propose a neighbor sampler to resample remote sensing images. Then, the sampled images are sent to the network for feature aggregation. In addition, we use an improved recursive training algorithm to further improve the extraction accuracy, making the water boundary more natural. Furthermore, our method utilizes neighboring features instead of global or local features to learn more representative features. The experimental results show that the proposed NFANet method not only outperforms other studied weakly supervised approaches, but also obtains similar results as the state-of-the-art ones.

cs.CV↗
arXiv2021

SamplingAug: On the Importance of Patch Sampling Augmentation for Single Image Super-Resolution

With the development of Deep Neural Networks (DNNs), plenty of methods based on DNNs have been proposed for Single Image Super-Resolution (SISR). However, existing methods mostly train the DNNs on uniformly sampled LR-HR patch pairs, which makes them fail to fully exploit informative patches within the image. In this paper, we present a simple yet effective data augmentation method. We first devise a heuristic metric to evaluate the informative importance of each patch pair. In order to reduce the computational cost for all patch pairs, we further propose to optimize the calculation of our metric by integral image, achieving about two orders of magnitude speedup. The training patch pairs are sampled according to their informative importance with our method. Extensive experiments show our sampling augmentation can consistently improve the convergence and boost the performance of various SISR architectures, including EDSR, RCAN, RDN, SRCNN and ESPCN across different scaling factors (x2, x3, x4). Code is available at https://github.com/littlepure2333/SamplingAug

cs.CV↗
arXiv2021

Transformer-based Image Compression

A Transformer-based Image Compression (TIC) approach is developed which reuses the canonical variational autoencoder (VAE) architecture with paired main and hyper encoder-decoders. Both main and hyper encoders are comprised of a sequence of neural transformation units (NTUs) to analyse and aggregate important information for more compact representation of input image, while the decoders mirror the encoder-side operations to generate pixel-domain image reconstruction from the compressed bitstream. Each NTU is consist of a Swin Transformer Block (STB) and a convolutional layer (Conv) to best embed both long-range and short-range information; In the meantime, a casual attention module (CAM) is devised for adaptive context modeling of latent features to utilize both hyper and autoregressive priors. The TIC rivals with state-of-the-art approaches including deep convolutional neural networks (CNNs) based learnt image coding (LIC) methods and handcrafted rules-based intra profile of recently-approved Versatile Video Coding (VVC) standard, and requires much less model parameters, e.g., up to 45% reduction to leading-performance LIC.

eess.IV↗
arXiv2021

Magnetic suppression of non-Hermitian skin effects

Skin effect, where macroscopically many bulk states are aggregated towards the system boundary, is one of the most important and distinguishing phenomena in non-Hermitian quantum systems. We discuss a new aspect of this effect whereby, despite its topological origin, applying magnetic field can largely suppress it. Skin states are pushed back into the bulk and the skin topological area, which we define, is sharply reduced. As seen from exact solutions of representative models this is fundamentally rooted in the fact that the applied magnetic field restores the validity of the low-energy description that is rendered inapplicable in the presence of non-Bloch skin states. We further study this phenomenon using rational gauge fluxes, which reveals a unique irrelevance of the generalized Brillouin zone in the standard non-Bloch band theory of non-Hermitian systems.

cond-mat.mes-hall↗
arXiv2021

Hall algebras and quantum symmetric pairs III: Quiver varieties

The $\imath$quiver algebras were introduced recently by the authors to provide a Hall algebra realization of universal $\imath$quantum groups, which is a generalization of Bridgeland's Hall algebra construction for (Drinfeld doubles of) quantum groups; here an $\imath$quantum group and a corresponding Drinfeld-Jimbo quantum group form a quantum symmetric pair. In this paper, the Dynkin $\imath$quiver algebras are shown to arise as new examples of singular Nakajima-Keller-Scherotzke categories. Then we provide a geometric construction of the universal $\imath$quantum groups and their ``dual canonical bases" with positivity, via the quantum Grothendieck rings of Nakajima-Keller-Scherotzke quiver varieties, generalizing Qin's geometric realization of quantum groups of type ADE.

math.RT↗
arXiv2021

A Drinfeld type presentation of affine $\imath$quantum groups I: split ADE type

We establish a Drinfeld type new presentation for the $\imath$quantum groups arising from quantum symmetric pairs of split affine ADE type, which includes the $q$-Onsager algebra as the rank 1 case. This presentation takes a form which can be viewed as a deformation of half an affine quantum group in the Drinfeld presentation.

math.RT↗
arXiv2021

Electrically Switchable van der Waals Magnon Valves

Van der Waals magnets have emerged as a fertile ground for the exploration of highly tunable spin physics and spin-related technology. Two-dimensional (2D) magnons in van der Waals magnets are collective excitation of spins under strong confinement. Although considerable progress has been made in understanding 2D magnons, a crucial magnon device called the van der Waals magnon valve, in which the magnon signal can be completely and repeatedly turned on and off electrically, has yet to be realized. Here we demonstrate such magnon valves based on van der Waals antiferromagnetic insulator MnPS3. By applying DC electric current through the gate electrode, we show that the second harmonic thermal magnon (SHM) signal can be tuned from positive to negative. The guaranteed zero crossing during this tuning demonstrates a complete blocking of SHM transmission, arising from the nonlinear gate dependence of the non-equilibrium magnon density in the 2D spin channel. Using the switchable magnon valves we demonstrate a magnon-based inverter. These results illustrate the potential of van der Waals anti-ferromagnets for studying highly tunable spin-wave physics and for application in magnon-base circuitry in future information technology.

cond-mat.mtrl-sci↗
arXiv2021

Overfitting the Data: Compact Neural Video Delivery via Content-aware Feature Modulation

Internet video delivery has undergone a tremendous explosion of growth over the past few years. However, the quality of video delivery system greatly depends on the Internet bandwidth. Deep Neural Networks (DNNs) are utilized to improve the quality of video delivery recently. These methods divide a video into chunks, and stream LR video chunks and corresponding content-aware models to the client. The client runs the inference of models to super-resolve the LR chunks. Consequently, a large number of models are streamed in order to deliver a video. In this paper, we first carefully study the relation between models of different chunks, then we tactfully design a joint training framework along with the Content-aware Feature Modulation (CaFM) layer to compress these models for neural video delivery. {\bf With our method, each video chunk only requires less than $1\% $ of original parameters to be streamed, achieving even better SR performance.} We conduct extensive experiments across various SR backbones, video time length, and scaling factors to demonstrate the advantages of our method. Besides, our method can be also viewed as a new approach of video coding. Our primary experiments achieve better video quality compared with the commercial H.264 and H.265 standard under the same storage cost, showing the great potential of the proposed method. Code is available at:\url{https://github.com/Neural-video-delivery/CaFM-Pytorch-ICCV2021}

eess.IV↗
arXiv2021

Learning Deep Multimodal Feature Representation with Asymmetric Multi-layer Fusion

We propose a compact and effective framework to fuse multimodal features at multiple layers in a single network. The framework consists of two innovative fusion schemes. Firstly, unlike existing multimodal methods that necessitate individual encoders for different modalities, we verify that multimodal features can be learnt within a shared single network by merely maintaining modality-specific batch normalization layers in the encoder, which also enables implicit fusion via joint feature representation learning. Secondly, we propose a bidirectional multi-layer fusion scheme, where multimodal features can be exploited progressively. To take advantage of such scheme, we introduce two asymmetric fusion operations including channel shuffle and pixel shift, which learn different fused features with respect to different fusion directions. These two operations are parameter-free and strengthen the multimodal feature interactions across channels as well as enhance the spatial feature discrimination within channels. We conduct extensive experiments on semantic segmentation and image translation tasks, based on three publicly available datasets covering diverse modalities. Results indicate that our proposed framework is general, compact and is superior to state-of-the-art fusion frameworks.

cs.CV↗