Research

DeepUrban

PaperPublisherCodeDataset
Three linked panels: a drone aerial image of an urban street with detected road users, the derived scenario data with agent trajectories along a lane, and a zoomed view of predicted and planned trajectories.

Drone recordings of Munich intersections, seen from directly above, where every road user and every interaction is visible at once.

What it does

From the air there are no occlusions and no ego bias: you see the whole negotiation between car, cyclist and pedestrian. That makes aerial data a clean measuring stick for interaction-aware prediction.

Abstract

The efficacy of autonomous driving systems hinges critically on robust prediction and planning capabilities. However, current benchmarks are impeded by a notable scarcity of scenarios featuring dense traffic, which is essential for understanding and modeling complex interactions among road users. To address this gap, we collaborated with our industrial partner, DeepScenario, to develop DeepUrban-a new drone dataset designed to enhance trajectory prediction and planning benchmarks focusing on dense urban settings. DeepUrban provides a rich collection of 3D traffic objects, extracted from high-resolution images captured over urban intersections at approximately 100 meters altitude. The dataset is further enriched with comprehensive map and scene information to support advanced modeling and simulation tasks. We evaluate state-of-the-art (SOTA) prediction and planning methods, and conducted experiments on generalization capabilities. Our findings demonstrate that adding DeepUrban to nuScenes can boost the accuracy of vehicle predictions and planning, achieving improvements up to 44.1 % / 44.3% on the ADE / FDE metrics. Website: https://iv.ee.hm.edu/deepurban

Result

A trajectory prediction and planning benchmark built from real urban intersections rather than from a vehicle's limited viewpoint.

Citation

@conference{Selzer2024DeepUrban,
title = {DeepUrban: Interaction-aware Trajectory Prediction and Planning for Automated Driving by Aerial Imagery},
author = {Constantin Selzer and Fabian B. Flohr},
url = {iv.ee.hm.edu/deepurban},
year  = {2024},
date = {2024-07-31},
urldate = {2024-07-31},
booktitle = {IEEE International Conference on Intelligent Transportation Systems (accepted for publication)},
volume = {27},
publisher = {IEEE},
abstract = {The efficacy of autonomous driving systems hinges critically on robust prediction and planning capabilities. However, current benchmarks are impeded by a notable scarcity of scenarios featuring dense traffic, which is essential for understanding and modeling complex interactions among road users. To address this gap, we collaborated with our industrial partner, DeepScenario, to develop "DeepUrban"—a new drone dataset designed to enhance trajectory prediction and planning benchmarks focusing on dense urban settings. DeepUrban provides a rich collection of 3D traffic objects, extracted from high-resolution images captured over urban intersections at approximately 100 meters altitude. The dataset is further enriched with comprehensive map and scene information to support advanced modeling and simulation tasks. We evaluate state-of-the-art (SOTA) prediction and planning methods, and conducted experiments on generalization capabilities. Our findings demonstrate that adding DeepUrban to nuScenes can boost the accuracy of vehicle predictions and planning, achieving improvements up to 44.1% / 44.3% on the ADE / FDE metrics.},
keywords = {},
pubstate = {forthcoming},
tppubtype = {conference}
}