Ego4D: Around the World in 3,000 Hours of Egocentric Video

Abstract

We introduce Ego4D, a massive-scale egocentric video dataset and benchmark suite. It offers 3,670 hours of dailylife activity video spanning hundreds of scenarios (household, outdoor, workplace, leisure, etc.) captured by 931 unique camera wearers from 74 worldwide locations and 9 different countries. The approach to collection is designed to uphold rigorous privacy and ethics standards, with consenting participants and robust de-identification procedures where relevant. Ego4D dramatically expands the volume of diverse egocentric video footage publicly available to the research community. Portions of the video are accompanied by audio, 3D meshes of the environment, eye gaze, stereo, and/or synchronized videos from multiple egocentric cameras at the same event. Furthermore, we present a host of new benchmark challenges centered around understanding the first-person visual experience in the past (querying an episodic memory), present (analyzing hand-object manipulation, audio-visual conversation, and social interactions), and future (forecasting activities). By publicly sharing this massive annotated dataset and benchmark suite, we aim to push the frontier of first-person perception.

Keywords

Affiliated Institutions

• University of California, Berkeley US
• Berkeley College US
• International Institute of Information Technology, Hyderabad IN
• Carnegie Mellon University Africa RW
• Indiana University US
• University of Catania IT
• National University of Singapore SG
• Carnegie Mellon University US
• King Abdullah University of Science and Technology SA
• Georgia Institute of Technology US
• Moscow Institute of Thermal Technology RU
• The University of Tokyo JP
• Dartmouth Hospital GB
• University of Minnesota System US
• Universidad de Los Andes BO
• California University of Pennsylvania US
• META Health US
• University of Bristol GB
• The University of Texas at Austin US

Related Publications