Real-World Image Restoration Using Degradation Adaptive Transformer-Based Adversarial Network

Author

Lujun Zhai, Prairie View A&M University

Date of Award

8-2023

Document Type

Dissertation

Degree Name

Doctor of Philosophy (PhD)

Degree Discipline

Electrical Engineering

Abstract

Most existing learning-based image restoration methods heavily rely on paired degraded/non-degraded training datasets that are based on simplistic handcrafted degradation assumptions. These assumptions often involve a limited set of degradations, such as Gaussian blurs, noises, and bicubic downsampling. However, when these methods are applied to real-world images, there is a significant decrease in performance due to the discrepancy between synthetic and realistic degradation. Additionally, they lack the flexibility to adapt to unknown degradations in practical scenarios, which limits their generalizability to complex and unconstrained scenes.

To address the absence of image pairs, recent studies have proposed Generative Adversarial Network (GAN)-based unpaired methods. Nevertheless, unpaired learning models based on convolution operations encounter challenges in capturing long-range pixel dependencies in real-world images. This limitation stems from their reliance on convolution operations, which offer local connectivity and translation equivariance but struggle to capture global dependencies due to their limited receptive field.

To address these challenges, this dissertation proposed an innovative unpaired image restoration basic model along with an advanced model. The proposed basic model is the DA-CycleGAN model, which is based on the CycleGAN [1] neural network and specifically designed for blind real-world Single Image Super-Resolution (SISR). The DA-CycleGAN incorporates a degradation adaptive (DA) module to learn various real-world degradations (such as noise and blur patterns) in an unpaired manner, enabling strong flexible adaptation. Additionally, an advanced model called Trans-CycleGAN was designed, which integrated the Transformer architecture into CycleGAN to leverage its global connectivity. This combination allowed for image-to-image translation using CycleGAN [1] while enabling the Transformer to model global connectivity across long-range pixels. Extensive experiments conducted on realistic images demonstrate the superior performance of the proposed method in solving real-world image restoration problems, resulting in clearer and finer details.

Overall, this dissertation presents a novel unpaired image restoration basic model and an advanced model that effectively address the limitations of existing approaches. The proposed approach achieves significant advancements in handling real-world degradations and modeling long-range pixel dependencies, thereby offering substantial improvements in image restoration tasks.

Index Terms— Cross-domain translation, generative adversarial network, image restoration, super-resolution, transformer, unpaired training.

Committee Chair/Advisor

Suxia Cui

Committee Co-Chair:

Yonghui Wang

Committee Member

Richard T. Wilkins

Committee Member

Cajetan Akujuobi

Committee Member

Justin Foreman

Publisher

Prairie View A&M University

Rights

Date of Digitization

8/29/2023

Contributing Institution

John B Coleman Library

City of Publication

Prairie View

MIME Type

Application/PDF

Recommended Citation

Zhai, L. (2023). Real-World Image Restoration Using Degradation Adaptive Transformer-Based Adversarial Network. Retrieved from https://digitalcommons.pvamu.edu/pvamu-dissertations/23