arXiv subjects

Guangyuan Li

Publications and source records attributed to Guangyuan Li.

At least 37 records · Page 2Linked to original sources

arXiv2024

3DGStream: On-the-Fly Training of 3D Gaussians for Efficient Streaming of Photo-Realistic Free-Viewpoint Videos

Constructing photo-realistic Free-Viewpoint Videos (FVVs) of dynamic scenes from multi-view videos remains a challenging endeavor. Despite the remarkable advancements achieved by current neural rendering techniques, these methods generally require complete video sequences for offline training and are not capable of real-time rendering. To address these constraints, we introduce 3DGStream, a method designed for efficient FVV streaming of real-world dynamic scenes. Our method achieves fast on-the-fly per-frame reconstruction within 12 seconds and real-time rendering at 200 FPS. Specifically, we utilize 3D Gaussians (3DGs) to represent the scene. Instead of the naïve approach of directly optimizing 3DGs per-frame, we employ a compact Neural Transformation Cache (NTC) to model the translations and rotations of 3DGs, markedly reducing the training time and storage required for each FVV frame. Furthermore, we propose an adaptive 3DG addition strategy to handle emerging objects in dynamic scenes. Experiments demonstrate that 3DGStream achieves competitive performance in terms of rendering speed, image quality, training time, and model storage when compared with state-of-the-art methods.

cs.CV↗

arXiv2024

Rethink Arbitrary Style Transfer with Transformer and Contrastive Learning

Arbitrary style transfer holds widespread attention in research and boasts numerous practical applications. The existing methods, which either employ cross-attention to incorporate deep style attributes into content attributes or use adaptive normalization to adjust content features, fail to generate high-quality stylized images. In this paper, we introduce an innovative technique to improve the quality of stylized images. Firstly, we propose Style Consistency Instance Normalization (SCIN), a method to refine the alignment between content and style features. In addition, we have developed an Instance-based Contrastive Learning (ICL) approach designed to understand the relationships among various styles, thereby enhancing the quality of the resulting stylized images. Recognizing that VGG networks are more adept at extracting classification features and need to be better suited for capturing style features, we have also introduced the Perception Encoder (PE) to capture style features. Extensive experiments demonstrate that our proposed method generates high-quality stylized images and effectively prevents artifacts compared with the existing state-of-the-art methods.

cs.CV↗

arXiv2024

Rethinking Diffusion Model for Multi-Contrast MRI Super-Resolution

Recently, diffusion models (DM) have been applied in magnetic resonance imaging (MRI) super-resolution (SR) reconstruction, exhibiting impressive performance, especially with regard to detailed reconstruction. However, the current DM-based SR reconstruction methods still face the following issues: (1) They require a large number of iterations to reconstruct the final image, which is inefficient and consumes a significant amount of computational resources. (2) The results reconstructed by these methods are often misaligned with the real high-resolution images, leading to remarkable distortion in the reconstructed MR images. To address the aforementioned issues, we propose an efficient diffusion model for multi-contrast MRI SR, named as DiffMSR. Specifically, we apply DM in a highly compact low-dimensional latent space to generate prior knowledge with high-frequency detail information. The highly compact latent space ensures that DM requires only a few simple iterations to produce accurate prior knowledge. In addition, we design the Prior-Guide Large Window Transformer (PLWformer) as the decoder for DM, which can extend the receptive field while fully utilizing the prior knowledge generated by DM to ensure that the reconstructed MR image remains undistorted. Extensive experiments on public and clinical datasets demonstrate that our DiffMSR outperforms state-of-the-art methods.

cs.CV↗

arXiv2023

ArtBank: Artistic Style Transfer with Pre-trained Diffusion Model and Implicit Style Prompt Bank

Artistic style transfer aims to repaint the content image with the learned artistic style. Existing artistic style transfer methods can be divided into two categories: small model-based approaches and pre-trained large-scale model-based approaches. Small model-based approaches can preserve the content strucuture, but fail to produce highly realistic stylized images and introduce artifacts and disharmonious patterns; Pre-trained large-scale model-based approaches can generate highly realistic stylized images but struggle with preserving the content structure. To address the above issues, we propose ArtBank, a novel artistic style transfer framework, to generate highly realistic stylized images while preserving the content structure of the content images. Specifically, to sufficiently dig out the knowledge embedded in pre-trained large-scale models, an Implicit Style Prompt Bank (ISPB), a set of trainable parameter matrices, is designed to learn and store knowledge from the collection of artworks and behave as a visual prompt to guide pre-trained large-scale models to generate highly realistic stylized images while preserving content structure. Besides, to accelerate training the above ISPB, we propose a novel Spatial-Statistical-based self-Attention Module (SSAM). The qualitative and quantitative experiments demonstrate the superiority of our proposed method over state-of-the-art artistic style transfer methods.

cs.CV↗

arXiv2023

VGOS: Voxel Grid Optimization for View Synthesis from Sparse Inputs

Neural Radiance Fields (NeRF) has shown great success in novel view synthesis due to its state-of-the-art quality and flexibility. However, NeRF requires dense input views (tens to hundreds) and a long training time (hours to days) for a single scene to generate high-fidelity images. Although using the voxel grids to represent the radiance field can significantly accelerate the optimization process, we observe that for sparse inputs, the voxel grids are more prone to overfitting to the training views and will have holes and floaters, which leads to artifacts. In this paper, we propose VGOS, an approach for fast (3-5 minutes) radiance field reconstruction from sparse inputs (3-10 views) to address these issues. To improve the performance of voxel-based radiance field in sparse input scenarios, we propose two methods: (a) We introduce an incremental voxel training strategy, which prevents overfitting by suppressing the optimization of peripheral voxels in the early stage of reconstruction. (b) We use several regularization techniques to smooth the voxels, which avoids degenerate solutions. Experiments demonstrate that VGOS achieves state-of-the-art performance for sparse inputs with super-fast convergence. Code will be available at https://github.com/SJoJoK/VGOS.

cs.CV↗

arXiv2023

Integrated motion control and energy management of series hybrid electric vehicles: A multi-objective MPC approach

This paper considers the integrated motion control and energy management problems of the series hybrid electric vehicles (SHEV) with constraints. We propose a multi-objective model predictive control (MOMPC)-based energy management approach, which is embedded with the motion control to guarantee driving comfort. In addition, due to the slow response of the engine, it may cause excessive batter power when HEVs work in different conditions (e.g., uphill or sudden acceleration) with a certain request power; this implies the discharge current is too large. A battery current constraint is designed and incorporated into the MOMPC optimization problem and hence avoids the extra high charge-discharge current. This prevents potential safety hazards and extends the battery's life. Finally, numerical experiments are performed to verify the proposed approach.

math.OC↗

arXiv2022

Polarization-independent resonant lattice Kerker effect in phase-change metasurface

Resonant lattice Kerker effect in periodic resonators is one of the most interesting generalizations of the Kerker effect that relates to various fascinating functionalities such as scattering management and Huygens metasurfaces. However, so far this effect has been shown to be sensitive to the incident polarization, restricting its applications. Here, we report, for the first time, polarization-independent resonant lattice Kerker effect in metasurfaces composed of periodic Ge$_2$Se$_2$Te$_5$ (GST) disks. For such a metasurface of square lattice, the spectrally overlap of the electric dipole and magnetic dipole surface lattice resonances can be realized by choosing an appropriate GST crystalline fraction regardless of the incident polarization. The operation wavelength and the required GST crystalline fraction can be conveniently tuned over large ranges since these parameters scale linearly with the disk size and the lattice period, greatly facilitating the design. Making use of the obtained resonant lattice Kerker effect, we realize a reconfigurable and polarization-independent lattice Huygens' metasurface with a dynamic phase modulation of close to $2π$ and high transmittance. This work will advance the engineering of the resonant lattice Kerker effect and promote its applications in phase modulation and wavefront control.

physics.optics↗

arXiv2022

Broadband switchable terahertz half-/quarter-wave plate based on a graphene-metal hybrid metasurface

Metasurfaces incorporating graphene hold great promise for dynamic manipulation of terahertz waves. However, it remains challenging to design a broadband graphene-based terahertz metasurface with switchable functionality of half-wave plate (HWP) and quarter-wave plate (QWP). Here, we propose a graphene-metal hybrid metasurface for achieving broadband switchable HWP/QWP in the terahertz regime. Simulation results show that, by varying the Fermi energy of graphene from 0 eV to 1 eV, the function of the reflective metasurface can be switched from an HWP with polarization conversion ratio exceeding 97% over a wide band ranging from 0.7 THz to 1.3 THz, to a QWP with ellipticity above 0.92 over 0.78-1.33 THz. The sharing bandwidth reaches up to 0.52 THz and the relative bandwidth is as high as 50%. We expect this broadband and dynamically switchable terahertz HWP/QWP will find applications in terahertz sensing, imaging, and telecommunications.

physics.optics↗

arXiv2022

Transformer-empowered Multi-scale Contextual Matching and Aggregation for Multi-contrast MRI Super-resolution

Magnetic resonance imaging (MRI) can present multi-contrast images of the same anatomical structures, enabling multi-contrast super-resolution (SR) techniques. Compared with SR reconstruction using a single-contrast, multi-contrast SR reconstruction is promising to yield SR images with higher quality by leveraging diverse yet complementary information embedded in different imaging modalities. However, existing methods still have two shortcomings: (1) they neglect that the multi-contrast features at different scales contain different anatomical details and hence lack effective mechanisms to match and fuse these features for better reconstruction; and (2) they are still deficient in capturing long-range dependencies, which are essential for the regions with complicated anatomical structures. We propose a novel network to comprehensively address these problems by developing a set of innovative Transformer-empowered multi-scale contextual matching and aggregation techniques; we call it McMRSR. Firstly, we tame transformers to model long-range dependencies in both reference and target images. Then, a new multi-scale contextual matching method is proposed to capture corresponding contexts from reference features at different scales. Furthermore, we introduce a multi-scale aggregation mechanism to gradually and interactively aggregate multi-scale matched features for reconstructing the target SR MR image. Extensive experiments demonstrate that our network outperforms state-of-the-art approaches and has great potential to be applied in clinical practice. Codes are available at https://github.com/XAIMI-Lab/McMRSR.

eess.IV↗

arXiv2021

Active Tuning of Resonant Lattice Kerker Effect

The Kerker effect has been generalized in nanophotonics and meta-optics, and has recently been of great interest by relating to various fascinating functionalities such as scattering management and perfect transmission, reflection or absorption. One of the most interesting generalizations is the resonant lattice Kerker effect in periodic nanostructures. However, its active tuning has not been explored yet. Here, we report, for the first time, the active control of the resonant lattice Kerker effect in periodic Ge2Se2Te5 nanodisks. By changing the crystalline fraction, we show that the electric dipole lattice resonance (ED-LR), the magnetic dipole resonance (MDR), and thus the resonant lattice Kerker effect are all red-shifted. We therefore realize the transition from the ED-LR to the resonant lattice Kerker effect, which enables multilevel tuning of reflection, transmission and absorption with modulation depths above 86%. Taking advantage of the MDR redshifts, we also observe broadband and multilevel tuning of transmission with modulation depth of 87% over a broadband range of 588 nm. Our work establishes a new path for designing high-performance active nanophotonic devices.

physics.optics↗

arXiv2021

Mirror-backed dielectric metasurface sensor with ultrahigh figure of merit based on super-narrow Rayleigh anomaly

Plasmonic nanostructures with large local field enhancement have been extensively investigated for sensing applications. However, the quality factor and thus the sensing figure of merit are limited due to relatively high ohmic loss. Here we propose a novel plasmonic sensor with ultrahigh figure of merit based on super-narrow Rayleigh anomaly (RA) in a mirror-backed dielectric metasurface. Simulation results show that the RA in such a metasurface can have a super-high quality factor of 16000 in the visible regime, which is an order of magnitude larger than the highest value of reported plasmonic nanostructures. We attribute this striking performance to the enhanced electric fields far away from the metal film. The super-high quality factor and the greatly enhanced field confined to the superstrate region make the mirror-backed dielectric metasurface an ideal platform for sensing. We show that the figure of merit of this RA-based metasurface sensor can be as high as 15930/RIU. Additionally, we reveal that RA-based plasmonic sensors share some typical characteristics, providing guidance for the structure design. We expect this work advance the development of high-performance plasmonic metasurface sensors.

physics.optics↗

arXiv2021

Multi-Modal Sarcasm Detection Based on Contrastive Attention Mechanism

In the past decade, sarcasm detection has been intensively conducted in a textual scenario. With the popularization of video communication, the analysis in multi-modal scenarios has received much attention in recent years. Therefore, multi-modal sarcasm detection, which aims at detecting sarcasm in video conversations, becomes increasingly hot in both the natural language processing community and the multi-modal analysis community. In this paper, considering that sarcasm is often conveyed through incongruity between modalities (e.g., text expressing a compliment while acoustic tone indicating a grumble), we construct a Contras-tive-Attention-based Sarcasm Detection (ConAttSD) model, which uses an inter-modality contrastive attention mechanism to extract several contrastive features for an utterance. A contrastive feature represents the incongruity of information between two modalities. Our experiments on MUStARD, a benchmark multi-modal sarcasm dataset, demonstrate the effectiveness of the proposed ConAttSD model.

cs.CL↗

arXiv2021

Narrow quadrupolar surface lattice resonances and band reversal in vertical metal-insulator-metal gratings

We report narrow quadrupolar surface lattice resonances (SLRs) under normal incidence, and the observation, for the first time, of the band reversal effect of SLRs supported by a vertical metal-insulator-metal nanograting, which is embedded in a homogeneous dielectric environment. Simulation results show that under normal incidence, quadrupolar SLR with linewidth of 1~nm and high quality factor of 979 can be excited in the near-infrared regime, and that under oblique incidence, out-of-plane dipolar SLRs of relatively large quality factors (>=150) can be launched. By varying the incidence angle, the SLR wavelength can be continuously tuned over an extremely broadband range of 750 nm, covering most of the near-infrared regime, and the quality factor decreases exponentially. Remarkably, the resonance lineshape can also be dynamically tuned from an asymmetric Fano-shaped dip to a peak, a dip/peak pair, and a perfect symmetric Lorentzian peak, suggesting the appearance of the band reversal effect. We expect the high-Q SLRs with broadband tunability and tunable lineshapes will find potential applications in enhanced nanoscale light-matter interactions in nanolasers, nonlinear optics and sensing.

physics.optics↗

arXiv2021

High-Resolution Pelvic MRI Reconstruction Using a Generative Adversarial Network with Attention and Cyclic Loss

Magnetic resonance imaging (MRI) is an important medical imaging modality, but its acquisition speed is quite slow due to the physiological limitations. Recently, super-resolution methods have shown excellent performance in accelerating MRI. In some circumstances, it is difficult to obtain high-resolution images even with prolonged scan time. Therefore, we proposed a novel super-resolution method that uses a generative adversarial network (GAN) with cyclic loss and attention mechanism to generate high-resolution MR images from low-resolution MR images by a factor of 2. We implemented our model on pelvic images from healthy subjects as training and validation data, while those data from patients were used for testing. The MR dataset was obtained using different imaging sequences, including T2, T2W SPAIR, and mDIXON-W. Four methods, i.e., BICUBIC, SRCNN, SRGAN, and EDSR were used for comparison. Structural similarity, peak signal to noise ratio, root mean square error, and variance inflation factor were used as calculation indicators to evaluate the performances of the proposed method. Various experimental results showed that our method can better restore the details of the high-resolution MR image as compared to the other methods. In addition, the reconstructed high-resolution MR image can provide better lesion textures in the tumor patients, which is promising to be used in clinical diagnosis.

eess.IV↗

arXiv2021

Broadband switchable terahertz half-/quarter-wave plate based on VO2-metal hybrid metasurface with over/underdamped transition

Dynamically switchable half-/quarter-wave plates have recently been the focus in the terahertz regime. Conventional design philosophy leads to multilayer metamaterials or narrowband metasurfaces. Here we propose a novel design philosophy and a VO2-metal hybrid metasurface for achieving broadband dynamically switchable half-/quarter-wave plate (HWP/QWP) based on the transition from the overdamped to the underdamped resonance. Results show that, by varying the VO2 conductivity by three orders of magnitude, the proposed metasurface's function can be switched between an HWP with polarization conversion ratio larger than 96% and a QWP with ellipticity close to -1 over the broad working band of 0.8-1.2 THz. We expect that the proposed design philosophy will advance the engineering of metasurfaces for dynamically switchable functionalities beyond the terahertz regime.

physics.optics↗

arXiv2021

Transfer Learning Enhanced Generative Adversarial Networks for Multi-Channel MRI Reconstruction

Deep learning based generative adversarial networks (GAN) can effectively perform image reconstruction with under-sampled MR data. In general, a large number of training samples are required to improve the reconstruction performance of a certain model. However, in real clinical applications, it is difficult to obtain tens of thousands of raw patient data to train the model since saving k-space data is not in the routine clinical flow. Therefore, enhancing the generalizability of a network based on small samples is urgently needed. In this study, three novel applications were explored based on parallel imaging combined with the GAN model (PI-GAN) and transfer learning. The model was pre-trained with public Calgary brain images and then fine-tuned for use in (1) patients with tumors in our center; (2) different anatomies, including knee and liver; (3) different k-space sampling masks with acceleration factors (AFs) of 2 and 6. As for the brain tumor dataset, the transfer learning results could remove the artifacts found in PI-GAN and yield smoother brain edges. The transfer learning results for the knee and liver were superior to those of the PI-GAN model trained with its own dataset using a smaller number of training cases. However, the learning procedure converged more slowly in the knee datasets compared to the learning in the brain tumor datasets. The reconstruction performance was improved by transfer learning both in the models with AFs of 2 and 6. Of these two models, the one with AF=2 showed better results. The results also showed that transfer learning with the pre-trained model could solve the problem of inconsistency between the training and test datasets and facilitate generalization to unseen data.

eess.IV↗

arXiv2021

Dynamically reversible and strong circular dichroism based on Babinet-invertible chiral metasurfaces

We propose a Babinet-invertible chiral metasurface for achieving dynamically reversible and strong circular dichroism (CD). The proposed metasurface is composed of VO$_2$-metal hybrid structure, and when VO$_2$ transits between the dielectric state and the metallic state, the metasurface unit cell switches between complementary structures that are designed according to the Babinet principle. This leads to a large and reversible CD tuning range between $\pm 0.5$ at 0.97~THz, which is larger than the literature. We attribute the CD effect to extrinsic chirality of the proposed metasurface. We envision that the Babinet-invertible chiral metasurface proposed here will advance the engineering of active and tunable chiro-optical devices and promote their applications.

physics.optics↗

arXiv2020

Nonvolatile, reconfigurable and narrowband mid-infrared filter based on surface lattice resonance in phase-change GeSbTe

We propose a nonvolatile, reconfigurable, and narrowband mid-infrared bandpass filter based on surface lattice resonance in phase-change material Ge2Sb2Te5 (GST). The proposed filter is composed of a two-dimensional gold nanorod array embedded in a thick GST film. Results show that when GST transits from the amorphous state to the crystalline state, the narrowband reflection spectrum of the proposed filter is tuned from 3.197 μm to 4.795 μm, covering the majority of the mid-infrared regime, the peak reflectance decreases from 72.6% to 25.8%, and the corresponding Q-factor decreases from 19.6 to 10.3. We show that the spectral tuning range can be adjusted by varying the incidence angle or the lattice period. By properly designing the gold nanorod sizes, we also show that the Q-factor can be greatly increased to 70 at the cost of relatively smaller peak reflection efficiencies, and that the peak reflection efficiency can be further increased to 80% at the cost of relatively smaller Q-factors. We expect this work will advance the engineering of GST-based nonvalatile tunable surface lattice resonances and will promote their applications especially in reconfigurable narrowband filters.

physics.optics↗