Super-Resolution by Image Enhancement Using Texture Transfer

Jose Jaena Mari Ople, Daniel Stanley Tan, Arnulfo Azcarraga, Chao-Lung Yang, Kai-Lung Hua

Research output: Chapter in Book/Report/Conference proceedingConference Article in proceedingAcademicpeer-review


Recent deep learning approaches in single image super-resolution (SISR) can generate high-definition textures for super-resolved (SR) images. However, they tend to hallucinate fake textures and even produce artifacts. An alternative to SISR, reference-based SR (RefSR) approaches use high-resolution (HR) reference (Ref) images to provide HR details that are missing in the low-resolution (LR) input image. We propose a novel framework that leverages existing SISR approaches and enhances them with RefSR. Specifically, we refine the output of SISR methods using neural texture transfer, where HR features are queried from the Ref images. The query is conducted by computing the similarity of textural and semantic features between the input image and the Ref images. The most similar HR features, patch-wise, to the LR image is used to augment the SR image through an augmentation network. In the case of dissimilar Ref images from the LR input image, we prevent performance degradation by including the similarity scores in the input features of the network. Furthermore, we use random texture patches during the training to condition our augmentation network to not always trust the queried texture features. Different from past RefSR approaches, our method can use arbitrary Ref images and its lower-bound performance is based on the SR image. We showcase that our method drastically improves the performance of the base SISR approach.
Original languageEnglish
Title of host publication2020 IEEE International Conference on Image Processing (ICIP)
Subtitle of host publicationProceedings
Number of pages5
ISBN (Electronic)978-1-7281-6395-6
ISBN (Print)978-1-7281-6396-3
Publication statusPublished - Oct 2020
EventIEEE International Conference on Image Processing: ICIP 2020 - Abu Dhabi, United Arab Emirates
Duration: 25 Oct 202028 Oct 2020


ConferenceIEEE International Conference on Image Processing
Country/TerritoryUnited Arab Emirates
CityAbu Dhabi
Internet address


  • Super-resolution
  • image enhancement
  • texture transfer


Dive into the research topics of 'Super-Resolution by Image Enhancement Using Texture Transfer'. Together they form a unique fingerprint.

Cite this