RepVGG: Making VGG-style ConvNets Great Again

Abstract

We present a simple but powerful architecture of convolutional neural network, which has a VGG-like inference-time body composed of nothing but a stack of 3 × 3 convolution and ReLU, while the training-time model has a multi-branch topology. Such decoupling of the training-time and inference-time architecture is realized by a structural re-parameterization technique so that the model is named RepVGG. On ImageNet, RepVGG reaches over 80% top-1 accuracy, which is the first time for a plain model, to the best of our knowledge. On NVIDIA 1080Ti GPU, RepVGG models run 83% faster than ResNet-50 or 101% faster than ResNet-101 with higher accuracy and show favorable accuracy-speed trade-off compared to the state-of-the-art models like EfficientNet and RegNet. The code and trained models are available at https://github.com/megvii-model/RepVGG.

Keywords

Computer scienceInferenceResidual neural networkConvolution (computer science)Convolutional neural networkCode (set theory)Decoupling (probability)Artificial intelligenceArchitectureFLOPSSimple (philosophy)Parallel computingPattern recognition (psychology)AlgorithmComputer engineeringArtificial neural networkProgramming language

Affiliated Institutions

Related Publications

EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks

Mingxing Tan , Quoc V. Le

Convolutional Neural Networks (ConvNets) are commonly developed at a fixed resource budget, and then scaled up for better accuracy if more resources are available. In this paper...

2019 arXiv (Cornell University) 5008 citations

ResNeSt: Split-Attention Networks

Hang Zhang , Chongruo Wu , Zhongyue Zhang +9 more

The ability to learn richer network representations generally boosts the performance of deep learning models. To improve representation-learning in convolutional neural networks...

2022 2022 IEEE/CVF Conference on Computer ... 1177 citations

Is Second-Order Information Helpful for Large-Scale Visual Recognition?

Peihua Li , Jiangtao Xie , Qilong Wang +1 more

By stacking layers of convolution and nonlinearity, convolutional networks (ConvNets) effectively learn from lowlevel to high-level features and discriminative representations. ...

2017 270 citations

EfficientNetV2: Smaller Models and Faster Training

Mingxing Tan , Quoc V. Le

This paper introduces EfficientNetV2, a new family of convolutional networks that have faster training speed and better parameter efficiency than previous models. To develop thi...

2021 arXiv (Cornell University) 1103 citations

Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration

Yang He , Ping Liu , Ziwei Wang +2 more

Previous works utilized "smaller-norm-less-important" criterion to prune filters with smaller norm values in a convolutional neural network. In this paper, we analyze this norm-...

2019 1186 citations

Publication Info

Year: 2021
Type: article
Pages: 13728-13737
Citations: 2124
Access: Closed

External Links

Download PDF (Free) View on DOI.org arXiv Semantic Scholar

Social Impact

Altmetric

RepVGG: Making VGG-style ConvNets Great Again

PlumX Metrics

Social media, news, blog, policy document mentions

Citation Metrics

2124

OpenAlex

195

Influential

1929

CrossRef

Cite This

APA Style

                            
                                    Xiaohan Ding, 
                                
                                    Xiangyu Zhang, 
                                
                                    Ningning Ma
                                
                                et al.
                            
                            (2021). 
                            RepVGG: Making VGG-style ConvNets Great Again. 
                            2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
                            
                            , 13728-13737.
                            https://doi.org/10.1109/cvpr46437.2021.01352

Identifiers

DOI: 10.1109/cvpr46437.2021.01352
arXiv: 2101.03697

Data Quality

Data completeness: 88%