Abstract

Existing deep learning based image inpainting methods use a standard convolutional network over the corrupted image, using convolutional filter responses conditioned on both valid pixels as well as the substitute values in the masked holes (typically the mean value). This often leads to artifacts such as color discrepancy and blurriness. Post-processing is usually used to reduce such artifacts, but are expensive and may fail. We propose the use of partial convolutions, where the convolution is masked and renormalized to be conditioned on only valid pixels. We further include a mechanism to automatically generate an updated mask for the next layer as part of the forward pass. Our model outperforms other methods for irregular masks. We show qualitative and quantitative comparisons with other methods to validate our approach.

Keywords

InpaintingComputer scienceConvolution (computer science)Artificial intelligencePixelImage (mathematics)Computer visionPattern recognition (psychology)Filter (signal processing)AlgorithmArtificial neural network

Affiliated Institutions

Related Publications

Publication Info

Year
2018
Type
book-chapter
Pages
89-105
Citations
2238
Access
Closed

External Links

Download PDF (Free) View on DOI.org arXiv PubMed Semantic Scholar

Social Impact

Altmetric
PlumX Metrics

Social media, news, blog, policy document mentions

Citation Metrics

2238
OpenAlex
399
Influential

Cite This

Guilin Liu, Fitsum A. Reda, Kevin J. Shih et al. (2018). Image Inpainting for Irregular Holes Using Partial Convolutions. Lecture notes in computer science , 89-105. https://doi.org/10.1007/978-3-030-01252-6_6

Identifiers

DOI
10.1007/978-3-030-01252-6_6
PMID
41376875
PMCID
PMC12686310
arXiv
1804.07723

Data Quality

Data completeness: 79%