Our NeurIPS competition is live! See details and prizes here.

Moving autonomous vehicles forward, together.

Autonomous vehicles are expected to dramatically redefine the future of transportation. When fully realized, this technology promises to unlock a myriad of societal, environmental, and economic benefits.

We’re thrilled to share a comprehensive, large-scale dataset featuring the raw sensor camera and LiDAR inputs as perceived by a fleet of multiple, high-end, autonomous vehicles in a bounded geographic area. This dataset also includes high quality, human-labelled 3D bounding boxes of traffic agents, an underlying HD spatial semantic map.

With this, we aim to empower the community, stimulate further development, and share our insights into future opportunities from the perspective of an advanced industrial Autonomous Vehicles program.

AV_Golden_Gate-2

Tutorials and Competitions

NeurIPS 2019

Level 5 is currently hosting a competition on our dataset on 3D object detection over semantic maps.

We’re offering $25,000 in prizes and inviting top contestants to join us at NeurIPS 2019 in December to present their solutions at the conference. They’ll also get the opportunity to interview with our team!

The competition opened on September 12, 2019 and runs until November 12, 2019.

lyft competition image

Enter the competition! 

 

Our competition challenges participants to build and optimize algorithms based on our large-scale dataset. Our dataset features the raw sensor camera and LiDAR inputs as perceived by a fleet of multiple, high-end, autonomous vehicles in a bounded geographic area.

 

If you’re enjoying working with our dataset, we’d love to see your solutions! Visit the competition page to learn more about the submission guidelines, evaluation metric, and more. Don't miss the public leaderboards where you can track who's in the lead and where you rank.

CVPR 2019

In June 2019, we conducted a tutorial covering the practical tips for building a perception & prediction system for autonomous driving.

CVPRR-Image

Presenters

  • Luc Vincent, EVP of Autonomous Technology
  • Peter Ondruska, Director of Engineering
  • Ashesh Jain, Head of Perception
  • Sammy Omari, Head of Prediction & Planning
  • Vinay Shet, Director of Product Management

 

Our team reviewed the challenges involved in building a system that needs to operate without a human driver and how to push state-of-the-art neural network models into production. Attendees learned about different kinds of labeled data needed for perception & prediction, and how to combine classical robotics and computer vision methods with modern deep learning approaches for perception & prediction.

 

The performance of the real-time perception, prediction and planning systems can be improved by prior knowledge of the environment, traffic patterns, expected anomalies etc. We showed how a large-scale fleet of camera-phone equipped vehicles can help generate those priors and help discover infrequent events increasing overall prediction performance. Finally, we walked the audience through a set of hands-on sessions into building basic blocks of self-driving stack, its challenges and how to use the presented dataset for its development & evaluation.

Presenters

 

  • Luc Vincent, EVP of Autonomous Technology
  • Peter Ondruska, Director of Engineering
  • Ashesh Jain, Head of Perception
  • Sammy Omari, Head of Prediction & Planning
  • Vinay Shet, Director of Product Management

 

Our team reviewed the challenges involved in building a system that needs to operate without a human driver and how to push state-of-the-art neural network models into production. Attendees learned about different kinds of labeled data needed for perception & prediction, and how to combine classical robotics and computer vision methods with modern deep learning approaches for perception & prediction.

 

The performance of the real-time perception, prediction and planning systems can be improved by prior knowledge of the environment, traffic patterns, expected anomalies etc. We showed how a large-scale fleet of camera-phone equipped vehicles can help generate those priors and help discover infrequent events increasing overall prediction performance. Finally, we walked the audience through a set of hands-on sessions into building basic blocks of self-driving stack, its challenges and how to use the presented dataset for its development & evaluation.

Explore Dataset Samples

Annotations provided by

scale_logotype_black_transparent_rgb_01

To download this data, please register or login. Once logged in, you will have access to links to download the dataset.

If you use the dataset for scientific work, please cite the following:

@misc{lyft2019,
title = {Lyft Level 5 AV Dataset 2019},
author = {Kesten, R. and Usman, M. and Houston, J. and Pandya, T. and Nadhamuni, K. and Ferreira, A. and Yuan, M. and Low, B. and Jain, A. and Ondruska, P. and Omari, S. and Shah, S. and Kulkarni, A. and Kazakova, A. and Tao, C. and Platinsky, L. and Jiang, W. and Shet, V.},
year = {2019},
howpublished = {url{https://level5.lyft.com/dataset/}}
}

Data Collection

Top-down sensor diagram-smaller
Copy-of-BP9I1484-(1)

All data are collected by a fleet of Ford Fusion vehicles. We have two versions of vehicles. They are indicated in their calibration data as BETA_V0 and BETA_PLUS_PLUS. Each vehicle is equipped with the following sensors depending on their vehicle version:

BETA_V0 LiDARS:

  • One 40-beam roof LiDAR and two 40-beam bumper LiDARs.
  • Each LiDAR has an azimuth resolution of 0.2 degrees.
  • All three LiDARs jointly produce ~216,000 points at 10 Hz.
  • The firing directions of all LiDARs are synchronized to be the same at any given time.

BETA_V0 Cameras:

  • Six wide-field-of-view (WFOV) cameras uniformly cover 360 degrees field of view (FOV). Each camera has a resolution of 1224x1024 and a FOV of 70°x60°.
  • One long-focal-length camera is mounted slightly pointing up primarily for detecting traffic lights. The camera has a resolution of 2048x864 and a FOV of 35°x15°.
  • Every camera is synchronized with the LiDAR such that the LiDAR beam is at the center of the camera's field of view when the camera is capturing an image.

BETA_PLUS_PLUS LiDARS:

  • The only difference in LiDARs between Beta-V0 and Beta++ is the roof LiDAR, which is 64-beam for Beta++.
  • The synchronization of the LiDARs is the same as Beta-V0.

BETA_PLUS_PLUS Cameras:

  • Six wide-field-of-view (WFOV) high dynamic range cameras uniformly cover 360 degrees field of view (FOV). Each camera has a resolution of 1920x1080 and a FOV of 82°x52°.
  • One long-focal-length camera is mounted slightly pointing up primarily for detecting traffic lights. The camera has a resolution of 1920x1080 and a FOV of 27°x17°.
  • Every camera is synchronized with the LiDAR such that the LiDAR beam is at the center of the camera's field of view when the camera is capturing an image.
image1
image2

This dataset includes a high quality semantic map. A semantic map provides context to reason about the presence and motion of the agents in the scenes. The provided map has over 4000 lane segments (2000 road segment lanes and about 2000 junction lanes) , 197 pedestrian crosswalks, 60 stop signs, 54 parking zones, 8 speed bumps, 11 speed humps.

 

All map elements are registered to an underlying geometric map. This is the same frame of reference for all the scenes in the dataset.

Data Format

We chose to use the existing nuScenes format for our dataset to ensure compatibility with existing work that may have been done using the nuScenes dataset.

To explore the dataset, we provide a customized version of the nuScenes devkit and a tutorial on how to use it (see the Download section below).

Download

Please complete the following steps to download the data

    1. Create a login and download the dataset:

      Already have an account? Log in here

    2. Download Lyft’s version of the forked NuScenes SDK.

    3. Follow the README.md to get the SDK set up and start working with the data.

 

Licensing Information

The downloadable Lyft Level 5 AV dataset and included materials are © 2019 Lyft, Inc., and licensed under version 4.0 of the Creative Commons Attribution-NonCommercial-ShareAlike license (CC-BY-NC-SA-4.0).

 

The HD map included with the dataset was developed using data from the OpenStreetMap database which is © OpenStreetMap contributors and available under the ODbL-1.0 license.

 

The nuScenes devkit is published by nuTonomy under the CC-BY-NC-SA-4.0.  Lyft’s forked nuScenes devkit has been modified for use with the Lyft Level 5 AV dataset.  Lyft’s modifications are © 2019 Lyft, Inc., and licensed under the same CC-BY-NC-SA-4.0 license governing the original nuScenes devkit.