nxtAIM

Joint project using generative AI to overcome the limits of today’s automated driving functions. HM’s Intelligent Vehicles Lab works on generative sensor data, scene prediction and foundation models.

About the project

nxtAIM (NXT GEN AI METHODS – Generative Methods for Perception, Prediction and Planning) is a collaborative research project initiated within the VDA lead initiative “Autonomous and Connected Driving” as part of its AI project family and funded by the German Federal Ministry for Economic Affairs (BMWK, now BMWE) in the programme “New Vehicle and System Technologies”. Automated driving functions are still significantly limited in their scope of use; the reasons lie in today’s system architectures and in the discriminative machine-learning methods they use. Building on generative methods, nxtAIM introduces bidirectional information flow as a new paradigm in the operational chain. This promises better scalability through an inexhaustible reservoir of data for offline testing, validation, training and online error detection; better transferability by deconstructing and recombining semantic information and expanding the operational design domain through targeted generation of scenarios and sensor data; and better traceability through online verification of each processing step during operation and an understanding of the latent space. A key outcome for industry will be foundation models for driving data. The consortium of vehicle manufacturers, suppliers, technology providers and research institutions is coordinated by AUMOVIO and Mercedes-Benz; the project budget is EUR 43.5 million, including EUR 27 million in public funding.

Our contribution

HM’s sub-project on holistic scene understanding and self-supervised planning methods is carried out at the Intelligent Vehicles Lab under Prof. Dr. Fabian Flohr, with Prof. Dr. habil. Alfred Schöttl as co-lead. HM researches generative models that produce lidar and camera data for complex urban scenes and extends them with consistent fusion in the latent space. The team also explores sequence models for scene prediction and vehicle trajectory planning, focusing on predicting the trajectories of vulnerable road users and modelling uncertainty, and transfers foundation models for sensor data generation and scene understanding to the automotive domain. Lab publications acknowledging nxtAIM include BEVDriver (IROS 2025), HABIT (WACV 2026) and SatMap (2026).