Abstract

We present SR3, an approach to image Super-Resolution via Repeated Refinement. SR3 adapts denoising diffusion probabilistic models (Ho et al. 2020), (Sohl-Dickstein et al. 2015) to image-to-image translation, and performs super-resolution through a stochastic iterative denoising process. Output images are initialized with pure Gaussian noise and iteratively refined using a U-Net architecture that is trained on denoising at various noise levels, conditioned on a low-resolution input image. SR3 exhibits strong performance on super-resolution tasks at different magnification factors, on faces and natural images. We conduct human evaluation on a standard 8× face super-resolution task on CelebA-HQ for which SR3 achieves a fool rate close to 50%, suggesting photo-realistic outputs, while GAN baselines do not exceed a fool rate of 34%. We evaluate SR3 on a 4× super-resolution task on ImageNet, where SR3 outperforms baselines in human evaluation and classification accuracy of a ResNet-50 classifier trained on high-resolution images. We further show the effectiveness of SR3 in cascaded image generation, where a generative model is chained with super-resolution models to synthesize high-resolution images with competitive FID scores on the class-conditional 256×256 ImageNet generation challenge.

Keywords

Artificial intelligenceComputer sciencePattern recognition (psychology)Image resolutionComputer visionProbabilistic logicNoise reduction

Affiliated Institutions

Related Publications

Publication Info

Year
2022
Type
article
Volume
45
Issue
4
Pages
1-14
Citations
1451
Access
Closed

Social Impact

Social media, news, blog, policy document mentions

Citation Metrics

1451
OpenAlex
184
Influential
965
CrossRef

Cite This

Chitwan Saharia, Jonathan Ho, William Chan et al. (2022). Image Super-Resolution Via Iterative Refinement. IEEE Transactions on Pattern Analysis and Machine Intelligence , 45 (4) , 1-14. https://doi.org/10.1109/tpami.2022.3204461

Identifiers

DOI
10.1109/tpami.2022.3204461
PMID
36094974
arXiv
2104.07636

Data Quality

Data completeness: 84%