Development of a Reinforcement Learning Environment for Connected Autonomous Vehicle Performance Optimization and Security Analysis

Abstract

The global presence of Connected and Autonomous Vehicles (CAVs) has seen an exponential increase over the last decade. This presence ranges from researchers testing their performance capabilities to studies assessing new security dynamics brought by CAVs. However, due to the costs of purchasing and maintaining CAVs, as well as in-field safety concerns related to safety precautions for test subjects, there is a need to have a flexible cost-effective tool for analyzing CAV impacts. This is the focus of this thesis. A literature search of the current state-ofthe art in autonomous-vehicle-centric simulators is provided in chapter 2. It was found that by better understanding machine learning paradigms – namely, various reinforcement learning (RL) methodologies – CAV performance and security assessment can be enhanced through continuous action space adaptation. Various RL algorithms can be beneficial for environmental exploration and optimization. After conducting an in-depth literature review and brief creation of a prototype reinforcement learning environment, Proximal Policy Optimization (PPO) was selected for its proficiency in both discrete and continuous action spaces. It was found that PPO provided better precision in high-speed driving actions with stochastic considerations than value-based deterministic methods and showcased the approximation of Actor-Critic Neural Network Architectures. PPO driven vehicle agents were trained in small to large scale environments using spatiotemporal performance metrics. The results identified potential road vulnerabilities, provided a better understanding of CAV performance, and constructed a viable algorithm framework that can be used in future analyses.