Dynamic Convolution: Attention Over Convolution Kernels

Yinpeng Chen; Xiyang Dai; Mengchen Liu; Dongdong Chen; Lu Yuan; Zicheng Liu

doi:10.1109/cvpr42600.2020.01104

Abstract

Light-weight convolutional neural networks (CNNs) suffer performance degradation as their low computational budgets constrain both the depth (number of convolution layers) and the width (number of channels) of CNNs, resulting in limited representation capability. To address this issue, we present Dynamic Convolution, a new design that increases model complexity without increasing the network depth or width. Instead of using a single convolution kernel per layer, dynamic convolution aggregates multiple parallel convolution kernels dynamically based upon their attentions, which are input dependent. Assembling multiple kernels is not only computationally efficient due to the small kernel size, but also has more representation power since these kernels are aggregated in a non-linear way via attention. By simply using dynamic convolution for the state-of-the-art architecture MobileNetV3-Small, the top-1 accuracy of ImageNet classification is boosted by 2.9% with only 4% additional FLOPs and 2.9 AP gain is achieved on COCO keypoint detection.

Keywords

Convolution (computer science)FLOPSKernel (algebra)Computer scienceConvolutional neural networkRepresentation (politics)AlgorithmComputational complexity theoryArtificial intelligencePattern recognition (psychology)MathematicsArtificial neural networkParallel computingDiscrete mathematics

Affiliated Institutions

Microsoft Research (United Kingdom) GB

Related Publications

Selective Kernel Networks

Xiang Li , Wenhai Wang , Xiaolin Hu +1 more

In standard Convolutional Neural Networks (CNNs), the receptive fields of artificial neurons in each layer are designed to share the same size. It is well-known in the neuroscie...

2019 2019 IEEE/CVF Conference on Computer ... 2769 citations

ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks

Qilong Wang , Banggu Wu , Pengfei Zhu +3 more

Recently, channel attention mechanism has demonstrated to offer great potential in improving the performance of deep convolutional neural networks (CNNs). However, most existing...

2020 6942 citations

Scaling Up Your Kernels to 31×31: Revisiting Large Kernel Design in CNNs

Xiaohan Ding , Xiangyu Zhang , Jungong Han +1 more

We revisit large kernel design in modern convolutional neural networks (CNNs). Inspired by recent advances in vision transformers (ViTs), in this paper, we demonstrate that usin...

2022 2022 IEEE/CVF Conference on Computer ... 1152 citations

Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classification

Saining Xie , Chen Sun , Jonathan Huang +2 more

Despite the steady progress in video analysis led by the adoption of convolutional neural networks (CNNs), the relative improvement has been less drastic as that in 2D static im...

2018 Lecture notes in computer science 1431 citations

PointConv: Deep Convolutional Networks on 3D Point Clouds

Wenxuan Wu , Zhongang Qi , Fuxin Li

Unlike images which are represented in regular dense grids, 3D point clouds are irregular and unordered, hence applying convolution on them can be difficult. In this paper, we e...

2019 2019 IEEE/CVF Conference on Computer ... 1796 citations

Publication Info

Year: 2020
Type: article
Pages: 11027-11036
Citations: 1166
Access: Closed

External Links

View on DOI.org

Social Impact

Altmetric

Dynamic Convolution: Attention Over Convolution Kernels

PlumX Metrics

Social media, news, blog, policy document mentions

Citation Metrics

1166

OpenAlex

Cite This

APA Style

                            
                                    Yinpeng Chen, 
                                
                                    Xiyang Dai, 
                                
                                    Mengchen Liu
                                
                                et al.
                            
                            (2020). 
                            Dynamic Convolution: Attention Over Convolution Kernels. 
                            
                            , 11027-11036.
                            https://doi.org/10.1109/cvpr42600.2020.01104

Identifiers

DOI: 10.1109/cvpr42600.2020.01104