Paderborn University Researchers Develop AI to Predict Pedestrian Behavior for Safer Self-Driving Cars

Autonomous vehicles, poised to become a common sight on our streets, are currently limited in their ability to predict human behavior, particularly that of pedestrians. A research project at Paderborn University aims to bridge this gap by enabling self-driving vehicles to anticipate pedestrian intentions before actions are taken.

The project, led by Dr.-Ing. Sandra Gausemeier and Dr. rer. medic. Tim Lehmann, proposes that autonomous vehicles should be capable of identifying intended actions through a combination of AI methods and motion analysis. This approach could revolutionize the interaction between humans and self-driving systems.

Dr. Gausemeier specializes in driver assistance systems within the ‘Control Engineering and Mechatronics’ research group at the Heinz Nixdorf Institute, while Dr. Lehmann focuses on human motor behavior and neurocognitive processes in the Exercise Science and Neuroscience section of the Department of Exercise and Health. Their collaboration involves conducting experimental studies on human decision-making behavior, which will form the basis for predictive algorithms used in autonomous vehicles.

The researchers’ innovative ideas have earned them Paderborn University’s Research Award, a recognition given to exceptional projects that push the boundaries of science. According to Professor Thomas Tröster, Vice-President for Research and Academic Careers at Paderborn University, “This project, with its combination of artificial intelligence and neurocognitive analysis, is aiming to bring about a paradigm shift in the interactions between people and autonomous systems. It’s not only highly relevant to society but also visionary in the best sense.”

The goal is to develop an AI-based system that can evaluate pedestrians’ intended future actions based on their motor activity, predict their behavior, create risk profiles, and proactively avoid critical situations. To achieve this, the first experimental studies are being conducted on humans’ decision-making behavior in real urban scenarios.

These studies go beyond simulation-based or laboratory-based studies, focusing on the complex, dynamic interactions between humans and machines. Autonomous systems would then be capable of more than simply making collision computations; they could also incorporate pedestrians’ situational awareness and distractions into maneuver planning.

Machine learning methods will be used to understand human motion patterns lasting several seconds with the help of pattern recognition, allowing for reliable prediction of intended actions. The quality of training data is crucial for pattern recognition using AI, so the researchers plan to develop a multi-stage process involving various types of data. This includes eye tracking, mobile electroencephalography, multi-sensor mobile measuring systems, and motion capturing.

After training, autonomous systems should be able to identify intentions using only onboard camera images and use these to infer future motion patterns. Professor Tröster believes that this solution approach could significantly improve the safety of all road users. The team expects initial results by early 2027.