Joint 3D Proposal Generation and Object Detection from View Aggregation

Abstract

We present AVOD, an Aggregate View Object Detection network for autonomous driving scenarios. The proposed neural network architecture uses LIDAR point clouds and RGB images to generate features that are shared by two subnetworks: a region proposal network (RPN) and a second stage detector network. The proposed RPN uses a novel architecture capable of performing multimodal feature fusion on high resolution feature maps to generate reliable 3D object proposals for multiple object classes in road scenes. Using these proposals, the second stage detection network performs accurate oriented 3D bounding box regression and category classification to predict the extents, orientation, and classification of objects in 3D space. Our proposed architecture is shown to produce state of the art results on the KITTI 3D object detection benchmark [1] while running in real time with a low memory footprint, making it a suitable candidate for deployment on autonomous vehicles. Code is available at: https://github.com/kujason/avod.

Keywords

Computer scienceObject detectionMinimum bounding boxArtificial intelligenceBenchmark (surveying)Point cloudMemory footprintComputer visionFeature (linguistics)Orientation (vector space)Feature extractionLidarObject (grammar)Code (set theory)Pattern recognition (psychology)Image (mathematics)

Affiliated Institutions

University of Waterloo CA

Related Publications

VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection

Yin Zhou , Oncel Tuzel

Accurate detection of objects in 3D point clouds is a central problem in many applications, such as autonomous navigation, housekeeping robots, and augmented/virtual reality. To...

2018 4245 citations

PointRCNN: 3D Object Proposal Generation and Detection From Point Cloud

Shaoshuai Shi , Xiaogang Wang , Hongsheng Li

In this paper, we propose PointRCNN for 3D object detection from raw point cloud. The whole framework is composed of two stages: stage-1 for the bottom-up 3D proposal generation...

2019 2019 IEEE/CVF Conference on Computer ... 2773 citations

An End-to-End Steel Surface Defect Detection Approach via Fusing Multiple Hierarchical Features

Yu He , Kechen Song , Qinggang Meng +1 more

A complete defect detection task aims to achieve the specific class and precise location of each defect in an image, which makes it still challenging for applying this task in p...

2019 IEEE Transactions on Instrumentation ... 1119 citations

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

Shaoqing Ren , Kaiming He , Ross Girshick +1 more

State-of-the-art object detection networks depend on region proposal algorithms to hypothesize object locations. Advances like SPPnet and Fast R-CNN have reduced the running tim...

2015 arXiv (Cornell University) 18214 citations

PIXOR: Real-time 3D Object Detection from Point Clouds

Bin Yang , Wenjie Luo , Raquel Urtasun

We address the problem of real-time 3D object detection from point clouds in the context of autonomous driving. Speed is critical as detection is a necessary component for safet...

2018 1284 citations

Publication Info

Year: 2018
Type: article
Citations: 1593
Access: Closed

External Links

Download PDF (Free) View on DOI.org arXiv Semantic Scholar

Social Impact

Altmetric

Joint 3D Proposal Generation and Object Detection from View Aggregation

PlumX Metrics

Social media, news, blog, policy document mentions

Citation Metrics

1593

OpenAlex

175

Influential

1196

CrossRef

Cite This

APA Style

                            
                                    Jason S. Ku, 
                                
                                    Melissa Mozifian, 
                                
                                    Jungwook Lee
                                
                                et al.
                            
                            (2018). 
                            Joint 3D Proposal Generation and Object Detection from View Aggregation. 
                            2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
                            
                            .
                            https://doi.org/10.1109/iros.2018.8594049

Identifiers

DOI: 10.1109/iros.2018.8594049
arXiv: 1712.02294

Data Quality

Data completeness: 79%