Autonomous Driving with Generative Pre-trained Transformers
01.01.2025 – 30.06.2029
The research project ADRIVE-GPT is dedicated to the exploration and application of grounded language models (Large Language Models, LLMs) in the context of automated driving. Previous studies have only considered LLMs for simple tasks in stationary robotics. ADRIVE-GPT aims to investigate how multimodal sensor perceptions from cameras and lidar sensors can be combined with natural language to generate action plans for accident-free and human-like autonomous driving.
Additionally, the project explores whether the learning process can be supported by already processed knowledge about static and dynamic objects.
Another area of research is whether traffic rules or detailed map information need to be explicitly provided, or if the model can implicitly learn this information. The latter would allow the model to operate more independently of fixed rules and data, making it more scalable overall. A further focus of ADRIVE-GPT is the research and evaluation of self-prompting methods in the context of automated driving. The model is to be enabled to autonomously generate additional prompts and create action sequences. Through feedback loops and reflection, the model can correct itself, identify alternatives, and monitor and improve its performance in new scenarios.
Over the course of the project, the integration of the innovations described above is to be validated using at least one vehicle demonstrator and evaluated on large datasets provided by industry partners. ADRIVE-GPT contributes to the advancement of highly scalable mobility technologies with several innovations and strengthens Germany’s position in the field of automated driving and artificial intelligence.
