Aggregated Residual Transformations for Deep Neural Networks

Abstract

We present a simple, highly modularized network architecture for image classification. Our network is constructed by repeating a building block that aggregates a set of transformations with the same topology. Our simple design results in a homogeneous, multi-branch architecture that has only a few hyper-parameters to set. This strategy exposes a new dimension, which we call cardinality (the size of the set of transformations), as an essential factor in addition to the dimensions of depth and width. On the ImageNet-1K dataset, we empirically show that even under the restricted condition of maintaining complexity, increasing cardinality is able to improve classification accuracy. Moreover, increasing cardinality is more effective than going deeper or wider when we increase the capacity. Our models, named ResNeXt, are the foundations of our entry to the ILSVRC 2016 classification task in which we secured 2nd place. We further investigate ResNeXt on an ImageNet-5K set and the COCO detection set, also showing better results than its ResNet counterpart. The code and models are publicly available online.

Keywords

Cardinality (data modeling)Set (abstract data type)Computer scienceDimension (graph theory)Simple (philosophy)Block (permutation group theory)Artificial neural networkResidualCode (set theory)ArchitectureTheoretical computer scienceHomogeneousArtificial intelligenceMachine learningAlgorithmData miningMathematicsProgramming languageCombinatorics

Affiliated Institutions

Related Publications

Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning

Christian Szegedy , Sergey Ioffe , Vincent Vanhoucke +1 more

Very deep convolutional networks have been central to the largest advances in image recognition performance in recent years. One example is the Inception architecture that has b...

2017 Proceedings of the AAAI Conference on... 4483 citations

CBAM: Convolutional Block Attention Module

Sanghyun Woo , Jongchan Park , Joon‐Young Lee +1 more

We propose Convolutional Block Attention Module (CBAM), a simple yet effective attention module for feed-forward convolutional neural networks. Given an intermediate feature map...

2018 Lecture notes in computer science 20102 citations

EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks

Mingxing Tan , Quoc V. Le

Convolutional Neural Networks (ConvNets) are commonly developed at a fixed resource budget, and then scaled up for better accuracy if more resources are available. In this paper...

2019 arXiv (Cornell University) 5008 citations

GhostNet: More Features From Cheap Operations

Kai Han , Yunhe Wang , Qi Tian +3 more

Deploying convolutional neural networks (CNNs) on embedded devices is difficult due to the limited memory and computation resources. The redundancy in feature maps is an importa...

2020 3894 citations

CSPNet: A New Backbone that can Enhance Learning Capability of CNN

Chien-Yao Wang , Hong-Yuan Mark Liao , Yueh-Hua Wu +3 more

Neural networks have enabled state-of-the-art approaches to achieve incredible results on computer vision tasks such as object detection. However, such success greatly relies on...

2020 4309 citations

Publication Info

Year: 2017
Type: article
Citations: 11353
Access: Closed

External Links

Download PDF (Free) View on DOI.org arXiv Semantic Scholar

Social Impact

Altmetric

Aggregated Residual Transformations for Deep Neural Networks

PlumX Metrics

Social media, news, blog, policy document mentions

Citation Metrics

11353

OpenAlex

1338

Influential

8100

CrossRef

Cite This

APA Style

                            
                                    Saining Xie, 
                                
                                    Ross Girshick, 
                                
                                    Piotr Dollár
                                
                                et al.
                            
                            (2017). 
                            Aggregated Residual Transformations for Deep Neural Networks. 
                            2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
                            
                            .
                            https://doi.org/10.1109/cvpr.2017.634

Identifiers

DOI: 10.1109/cvpr.2017.634
arXiv: 1611.05431

Data Quality

Data completeness: 88%