Minimum Viable Demonstrations: Data Requirements for Capable Driving Models and Zero-Shot Track Transfer for Connected Autonomous Vehicles

Abstract

Reinforcement learning has long been the dominant approach for training autonomous driving agents, but it struggles in novel scenarios where running additional training episodes is costly. Imitation learning is a strong alternative to the traditional “trial and error” approach of reinforcement learning, and it allows the agent to simply learn from a human demonstration of a task, thus circumventing the thousands of expensive iterations that reinforcement learn- ing requires. However, imitation learning is not without limitations. Agents trained purely on demonstrations can struggle when encountering scenarios that aren’t presented in their training, such as navigating roads the agent hasn’t seen before. Thus, a combination of reinforcement and imitation learning is needed. By first training an agent on human demonstrations to form a strong behavioral foundation, reinforcement learning can then be applied to extend that behavior, al- lowing the agent to handle edge cases that may not be included in the initial demonstrations. Together, these complementary approaches give a strong framework for effective autonomous driving. This thesis investigates such an approach and presents three questions central to practi- cal deployment: the minimum demonstrations required to train a competent agent, strategies for adapting models to unseen environments, and the overall sensitivity of the model to variation. The results of this research serve as a strong foundation for future training instances where minimal resources are given.