Self-driving car using udacity's car simulator environment and trained by deep neural networks

Self-driving cars has become a trending subject with a significant improvement in the technologies in the last decade. The project purpose is to train a neural network to drive an autonomous car agent on the tracks of Udacity's Car Simulator environment. Udacity has released the simulator as an open source software and enthusiasts have hosted a competition (challenge) to teach a car how to drive using only camera images and deep learning. Driving a car in an autonomous manner requires learning to control steering angle, throttle and brakes. Behavioral cloning technique is used to mimic human driving behavior in the training mode on the track. That means a dataset is generated in the simulator by user driven car in training mode, and the deep neural network model then drives the car in autonomous mode. Three architectures are compared with respect to their performance. Though the models performed well for the track it was trained with, the real challenge was to generalize this behavior on a second track available on the simulator. The dataset for Track_1, which was simple with favorable road conditions to drive, was used as the training set to drive the car autonomously on Track_2 which consists of sharp turns, barriers, elevations and shadows. To tackle this problem, image processing and different augmentation techniques were used, which allowed extracting as much information and features in the data as possible. Ultimately, the car was able to run on Track_2 generalizing well. The project aims at reaching the same accuracy on real time data in the future.