Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2103.01760

Transform Network Architectures for Deep Learning based End-to-End Image/Video Coding in Subsampled Color Spaces

Hilmi E. Egilmez·Ankitesh K. Singh·Muhammed Coban·Marta Karczewicz·Yinhao Zhu·Yang Yang·Amir Said·Taco S. Cohen
View original source ↗Read source document ↗Export metadata ↓

Abstract

Most of the existing deep learning based end-to-end image/video coding (DLEC) architectures are designed for non-subsampled RGB color format. However, in order to achieve a superior coding performance, many state-of-the-art block-based compression standards such as High Efficiency Video Coding (HEVC/H.265) and Versatile Video Coding (VVC/H.266) are designed primarily for YUV 4:2:0 format, where U and V components are subsampled by considering the human visual system. This paper investigates various DLEC designs to support YUV 4:2:0 format by comparing their performance against the main profiles of HEVC and VVC standards under a common evaluation framework. Moreover, a new transform network architecture is proposed to improve the efficiency of coding YUV 4:2:0 data. The experimental results on YUV 4:2:0 datasets show that the proposed architecture significantly outperforms naive extensions of existing architectures designed for RGB format and achieves about 10% average BD-rate improvement over the intra-frame coding in HEVC.

Explore related subjects

eess.IVcs.AIcs.CVcs.LGcs.MM

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Hilmi E. Egilmez, Ankitesh K. Singh, Muhammed Coban, Marta Karczewicz, Yinhao Zhu, Yang Yang, Amir Said, Taco S. Cohen. 2021-08-27. Transform Network Architectures for Deep Learning based End-to-End Image/Video Coding in Subsampled Color Spaces. https://doi.org/10.1109/ojsp.2021.3092257

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

StainBridge: Stain-Aware Pairwise Registration of Serial Renal Biopsy Whole-Slide Images Across Structural and Immunohistochemical Stains

Three-dimensional (3D) reconstruction of histopathology tissue requires accurate pairwise registration of serial whole-slide images (WSIs). Cross-stain benchmarks have advanced registration of differently stained histology, including structural-to-immunohistochemistry (IHC) pairs, but serial renal biopsy stacks remain difficult: they interleave several structural stains with diverse IHC markers whose expression can be sparse or absent, leaving few shared features to match. We present StainBridge, a stain-aware framework for registering serial renal biopsy WSIs across structural and IHC stains. StainBridge couples three preprocessing components, stain deconvolution, intensity normalization, and tissue-mask injection, with XFeat-based affine initialization and four nonrigid backends (VoxelMorph, ConvexAdam, FireANTs, and DeeperHistReg). We evaluate it on 23 cases comprising 338 WSIs, four structural stains, and ten IHC markers, with functional tissue units annotated on consecutive sections to give 1,468 landmark correspondences across 272 image pairs, and report tissue-mask Dice, functional-unit centroid error in micrometers, and tissue-restricted structural similarity. Nonrigid refinement improves on the affine initialization for three of four backends, VoxelMorph being the exception. DeeperHistReg, which computes its own initialization rather than relying on XFeat, gives the best pooled landmark accuracy and registers the most pairs, including every attempted structural-IHC pair. Preprocessing improves landmark accuracy for ConvexAdam and FireANTs in every stain-pairing category, and FireANTs shows both the largest single preprocessing gain on structural-IHC pairs and the best pooled tissue overlap. These results offer practical guidance for cross-stain registration and a foundation for integrated 3D analysis of renal tissue architecture and molecular expression.

eess.IV↗
arXiv2026

Quantum-Inspired Trainable and Parameter-Efficient Tensor Networks for Image Inpainting

This work introduces quantum-inspired tensor-network circuits as trainable transforms for image inpainting. Among the proposed architectures, the diagonal quantum Fourier transform (QFT) relaxation is invertible with $O(N^2 \log N)$ computational cost for $N\times N$ images, inherently preserving minimum coherence throughout training via its circuit structure and eliminating the need for explicit coherence penalties. Unconstrained gradient-based phase optimization (Riemannian-optimization free) enables efficient learning from randomly sampled training data, allowing the learned transform to generalize to test images observed through fixed sampling masks. Numerical tests show that the learned models outperform fixed transforms and per-image optimization while matching the performance of much larger unitary architectures, yet with far fewer parameters.

eess.IV↗
arXiv2026

Hyperspectral Image Restoration and Super-resolution with Physics-Aware Deep Learning for Biomedical Applications

Hyperspectral imaging is a powerful bioimaging tool which can uncover novel insights, thanks to its sensitivity to the intrinsic properties of materials. However, this enhanced contrast comes at the cost of system complexity, constrained by an inherent trade-off between spatial, spectral, and temporal resolution. To overcome this limitation, we present a self-supervised deep learning-based approach that restores and enhances pixel resolution post-acquisition without requiring external training data beyond the images to be restored. Fine-tuned using metrics aligned with the imaging model, our physics-aware method achieves a 16$\times$ pixel super-resolution enhancement and a 12$\times$ imaging speedup without the need of additional training data for transfer learning. Applied to both synthetic and experimental data from five different sample types, including healthy and diseased tissues, we demonstrate that the model preserves biological integrity, as we did not detect systematic loss of biological features or biologically consequential hallucinations in tested datasets. We also concretely demonstrate the model's ability to reveal disease-associated metabolic changes that would otherwise remain undetectable. Furthermore, we provide physical insights into the model's inner workings, paving the way for future refinements that could potentially reveal novel high resolution features in an explainable manner. All methods are available as open-source software on GitHub.

eess.IV↗