A Deep Learning Method for GPS/GNSS Independent, Vision-Based Navigation and State Estimation of Class 8 Vehicles

Abstract

This thesis develops a deep learning based method for vision-based navigation and state estimation of class 8 vehicles without relaying on GPS/GNSS. The novel deep learning system is compared to a vehicle-model-based system and a traditional inertial navigation system (INS). For the model-based method, a dead reckoning navigation filter is developed around a tractor-trailer dynamics model. To address the limitations of the model-based system, a deep learning neural network was designed as a data-driven dead reckoning and state estimation system. First, a detailed derivation of a tractor-trailer bicycle model is given. Then, a navigation extended Kalman filter is designed around the vehicle states. Given that a unique challenge for autonomous trucking is high model variability from the trailer unit, accurate parameter knowledge of the vehicle model cannot be assumed. To address this issue, a deep learning neural network is designed to ingest camera and inertial sensor data to produce relative motion and trailer hitch angle predictions. The cameras are mounted on the rear-facing mirrors to monitor the trailer and the vehicle's motion over time. The proposed deep learning network is inspired from classical visual inertial odometry systems, but does not require information about the system or sensor models. The proposed network model is a transformer based model that uses a combination of convolutional neural networks, vision transformers, self-attention, and cross-attention. It is shown through various Monte Carlo simulations that the proposed deep learning system often outperforms the model-based system. Significant benefits are especially evident in cases where the vehicle model is not well known which causes the traditional model-based solutions to degrade. Experiments conducted further validate the benefits of the deep learning approach showing consistent improvements over the model-based method in positioning, orientation, and hitch angle accuracy, even when the model is known accurately. Furthermore, it is shown that the deep learning network is able to maintain accurate positioning for significantly longer periods of time than the model-based system and the INS which is highly valuable for any dead reckoning system.