Abstract

We present a compact but effective CNN model for optical flow, called PWC-Net. PWC-Net has been designed according to simple and well-established principles: pyramidal processing, warping, and the use of a cost volume. Cast in a learnable feature pyramid, PWC-Net uses the current optical flow estimate to warp the CNN features of the second image. It then uses the warped features and features of the first image to construct a cost volume, which is processed by a CNN to estimate the optical flow. PWC-Net is 17 times smaller in size and easier to train than the recent FlowNet2 model. Moreover, it outperforms all published optical flow methods on the MPI Sintel final pass and KITTI 2015 benchmarks, running at about 35 fps on Sintel resolution (1024 Ã × 436) images. Our models are available on our project website.

Keywords

Image warpingOptical flowPyramid (geometry)Volume (thermodynamics)Computer scienceFeature (linguistics)Artificial intelligenceNet (polyhedron)Flow (mathematics)Computer visionImage (mathematics)Computer graphics (images)Pattern recognition (psychology)MathematicsOpticsPhysicsGeometry

Related Publications

Pyramid Stereo Matching Network

Recent work has shown that depth estimation from a stereo pair of images can be formulated as a supervised learning task to be resolved with convolutional neural networks (CNNs)...

2018 2018 IEEE/CVF Conference on Computer ... 1686 citations

Publication Info

Year
2018
Type
preprint
Pages
8934-8943
Citations
2575
Access
Closed

Social Impact

Social media, news, blog, policy document mentions

Citation Metrics

2575
OpenAlex
457
Influential
2047
CrossRef

Cite This

Deqing Sun, Xiaodong Yang, Ming-Yu Liu et al. (2018). PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition , 8934-8943. https://doi.org/10.1109/cvpr.2018.00931

Identifiers

DOI
10.1109/cvpr.2018.00931
arXiv
1709.02371

Data Quality

Data completeness: 84%