CARLA Autonomous Driving Challenge

Challenge 2020 (Finished)

Overview

The CARLA Autonomous Driving Challenge 2020 is organized as part of the Machine Learning for Autonomous Driving Workshop at NeurIPS 2020. This competition is open to any participant from academia and industry. You only need to sign up on the CARLA AD Leaderboard, providing your team name and your institution.

Results and retrospective

Video summary of the CARLA Challenge 2020 and talks from the winning teams:

Congratulations to the winning teams in the SENSORS modality:

Team Submission Driving score Route completion Infraction penalty Collisions pedestrians Collisions vehicles Collisions layout Red light infractions Stop sign infractions Off-road infractions Route deviations Route timeouts Agent blocked Duration
Units % % [0, 1] Infractions/Km Infractions/Km Infractions/Km Infractions/Km Infractions/Km Infractions/Km Infractions/Km Infractions/Km Infractions/Km Minutes
1st MaRLn MaRLn 24.98 46.97 0.52 0.00 2.33 2.47 0.55 0.00 1.82 1.44 0.79 0.94 9035.73
2nd Anonymous Attention Ensemble (NEAT + TransFuser) 13.49 23.2 0.72 0.02 2.06 0.49 1.50 0.0 3.26 0.19 0.0 8.59 452.42
3rd LBC Learning by Cheating (CoRL 2019) 8.94 17.54 0.73 0.0 0.4 1.16 0.71 0.0 1.52 0.03 0.0 4.69 1609.92

Important: The MAP modality is declared deserted this year.

We would like to thank all teams for their participation in the challenge. Remember that the CARLA Autonomous Driving Challenge will reopen in 2021 with new exciting challenges.

Statistics
  • +45 teams participated in the challenge.
  • +6,000 hours of driving simulation.
  • +7,000 Km driven in total.
  • Top-5 submissions registered average infractions per kilometer in the [0.95 - 1.7] range.
  • Road departure (off-road) and collisions with other vehicles were the main infractions for top-5 submissions.

Task

This challenge leverages the existing CARLA AD Leaderboard platform to evaluate submissions. The main goal of the challenge remains the assessment of the driving proficiency of autonomous agents in realistic traffic situations, as defined in the leaderboard mechanics. Teams will have to complete 10 routes in 2 weather conditions through 5 repetitions, leading to a total of 173 Km of driving experiences.

The challenge follows the same structure and rules defined for the CARLA AD Leaderboard. You can participate in any of the two available tracks: SENSORS and MAP, using the canonical sensors available for the challenge.

Important: You are not allowed to make use of any privilege information offered by the CARLA simulator, including world coordinates, planners, or any type of ground truth. Submissions using these features will be rejected and teams will be banned from the platform.

The mechanics of the challenge

The mechanics of the CARLA AD Challenge are the same as the ones defined for the CARLA AD Leaderboard. In a nutshell:

  1. Get familiar with the task by reading the leaderboard introduction.
  2. Learn how to create a new agent by reading the get started guide.
  3. Sign up to the challenge.
  4. Learn how to make a successful submission.
  5. Submit your agent.
  6. Make your submission public before the Challenge closure date (see timeline below).

Important: All submissions made public on the CARLA AD Leaderboard within the Challenge opening and closure dates will be considered for the CARLA AD Challenge.

The top-1 submissions of each track will be invited to present their results at the Machine Learning for Autonomous Driving Workshop. Additionally, all participants are invited to submit a technical report (up to 4 pages) describing their submissions. Based on the novelty and originality of these technical reports, the organization will select up to two teams to present their work at the workshop.

Timeline

Challenge opening October 6th 2020, 00:00 PDT
Challenge closure November 25th 2020, 23:00 PDT
Winners of each track are notified November 30th 2020, 23:00 PDT
Technical report submission November 29th 2020, 23:00 PDT
Accepted technical report notification November 30th 2020, 23:00 PDT
Video presentation submission December 6th 2020, 23:00 PDT
Results presented at ML4AD workshop December 11th, 2020

News and help

If you have any doubts or questions about the CARLA AD Challenge or the leaderboard, please contact us using of the following channels:

Previous events

Find more information about the CARLA AD Challenge 2019, held at CVPR 2019.