Progressive Neural Architecture Search

Chenxi Liu; Barret Zoph; Maxim Neumann; Jonathon Shlens; Hua Wei; Li-Jia Li; Li Fei-Fei; Alan Yuille; Jonathan Huang; Kevin Murphy

doi:10.1007/978-3-030-01246-5_2

Abstract

We propose a new method for learning the structure of convolutional neural networks (CNNs) that is more efficient than recent state-of-the-art methods based on reinforcement learning and evolutionary algorithms. Our approach uses a sequential model-based optimization (SMBO) strategy, in which we search for structures in order of increasing complexity, while simultaneously learning a surrogate model to guide the search through structure space. Direct comparison under the same search space shows that our method is up to 5 times more efficient than the RL method of Zoph et al. (2018) in terms of number of models evaluated, and 8 times faster in terms of total compute. The structures we discover in this way achieve state of the art classification accuracies on CIFAR-10 and ImageNet.

Keywords

Computer scienceReinforcement learningArtificial intelligenceConvolutional neural networkArtificial neural networkState (computer science)State spaceSearch algorithmMachine learningAlgorithmMathematics

Affiliated Institutions

Related Publications

AutoAugment: Learning Augmentation Strategies From Data

Ekin D. Cubuk , Barret Zoph , Dandelion Mané +2 more

Data augmentation is an effective technique for improving the accuracy of modern image classifiers. However, current data augmentation implementations are manually designed. In ...

2019 2019 IEEE/CVF Conference on Computer ... 2555 citations

A State-of-the-Art Survey on Deep Learning Theory and Architectures

Md Zahangir Alom , Tarek M. Taha , Chris Yakopcic +7 more

In recent years, deep learning has garnered tremendous success in a variety of application domains. This new field of machine learning has been growing rapidly and has been appl...

2019 Electronics 1521 citations

EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks

Mingxing Tan , Quoc V. Le

Convolutional Neural Networks (ConvNets) are commonly developed at a fixed resource budget, and then scaled up for better accuracy if more resources are available. In this paper...

2019 arXiv (Cornell University) 5008 citations

Factorized Bilinear Models for Image Recognition

Yanghao Li , Naiyan Wang , Jiaying Liu +1 more

Although Deep Convolutional Neural Networks (CNNs) have liberated their power in various computer vision tasks, the most important components of CNN, convolutional layers and fu...

2017 103 citations

A Deep Reinforced Model for Abstractive Summarization

Romain Paulus , Caiming Xiong , Richard Socher

Attentional, RNN-based encoder-decoder models for abstractive summarization have achieved good performance on short input and output sequences. For longer documents and summarie...

2017 arXiv (Cornell University) 1272 citations

Publication Info

Year: 2018
Type: book-chapter
Pages: 19-35
Citations: 1934
Access: Closed

External Links

Download PDF (Free) View on DOI.org arXiv PubMed Semantic Scholar

Social Impact

Altmetric

Progressive Neural Architecture Search

PlumX Metrics

Social media, news, blog, policy document mentions

Citation Metrics

1934

OpenAlex

297

Influential

Cite This

APA Style

                            
                                    Chenxi Liu, 
                                
                                    Barret Zoph, 
                                
                                    Maxim Neumann
                                
                                et al.
                            
                            (2018). 
                            Progressive Neural Architecture Search. 
                            Lecture notes in computer science
                            
                            , 19-35.
                            https://doi.org/10.1007/978-3-030-01246-5_2

Identifiers

DOI: 10.1007/978-3-030-01246-5_2
PMID: 41041021
PMCID: PMC12484352
arXiv: 1712.00559

Data Quality

Data completeness: 79%