Active Learning with Gaussian Processes for the Investigation of Critical Scenarios in Autonomous Driving

Abstract

The timely introduction and adoption of safe automated vehicles necessitate testing based on scenarios that reflect the diverse aspects they will encounter during autonomous operation.
To deal with a potentially intractably large scenario space, we present a modular framework that leverages a digital twin with machine learning to investigate critical scenarios. The framework describes the scenario-based testing approach under the Bayesian paradigm, where we characterise and update the probability to which a scenario is found to be critical. It builds on three methods for an efficient and coverage-oriented investigation of scenarios.
Firstly, we describe the assessment of scenarios executed in simulation by quantifying the criticality with one of several criticality metrics. In particular, we contribute a novel approach to assess the criticality of a scenario by comparing the behaviour of the automated vehicle with an omniscient safety model, which can be applied across a wide range of possible scenarios.
The main contribution of the presented work is the probabilistic representation of the scenario and its criticality. Based on the collected data from the simulation, we fit a relational, data-driven Gaussian process surrogate model, which allows us to perform inference about the criticality of unknown and untested scenarios. We also explore the application of partitioning
models to manage nonstationarity and heteroskedasticity in the scenario data. The partitioned models subdivide the scenario space into regions, where the criticality behaves similarly.
Finally, an active learning approach ties the framework together and implements an efficient way to build the Gaussian process models. We describe two active learning strategies tailoredto scenario-based testing. The framework either provides high-quality modelling of a specific criticality threshold throughout the scenario space, or it enables holistic coverage and criticality inference of the entire scenario space by sequentially building the surrogate model with
scenarios that offer the biggest information gain.
We show the framework’s application on three case studies, utilising each of the methods presented. It was found that the framework was more efficient and provided better criticality inference in a scenario-based investigation than a static testing design. We envision the use of the framework as a continuous testing tool during development and sign-off or to investigate deficiencies in the implementation of data-driven automated vehicles.

Date of Award	14 May 2022
Original language	English
Awarding Institution	Coventry University
Sponsors	HORIBA MIRA Ltd.
Supervisor	Alireza Daneshkhah (Supervisor), Vasile Palade (Supervisor) & Hamid Taghavifar (Supervisor)