Jendrik Joerdening and Anthony Navarro on Self-Racing Cars Using Deep Neural Networks

Show Notes

What are you working on today?

• 2:00 When the race happened, I was working at Lockheed Martin on the autonomous systems group, but have since switched over to Udacity to lead the robotics and self-driving car programs.
• 2:30 I set up what the program is going to be, what we’re going to accomplish, and the way we can accomplish the vision.
• 2:55 During the race I worked at Aka Technologies, an engineering service provider, so we did data science work for a lot of customers.
• 3:00 We built autonomous platforms for our customers; but now I work for Arubis as a data science provider.

So you were involved with autonomous systems, but you weren’t using them for neural networks?

• 3:30 Most of the work I had done was probabilistic models, and I hadn’t worked on deep learning models.
• 3:45 For me, I started in January coming from a space background - before that I had only worked on a micro-controller background.

How did you get involved in the autonomous car race?

• 4:15 There’s an autonomous car race in Thunder Hill, California - it happens about once a year; we competed in the second one.
• 4:30 Udacity sponsored a team, and Oliver Cameron (who was at Udacity at the time, and is now the CEO of Voyage) decided to launch the race competition.
• 4:50 We were selected along with 16 other people, and we were handed a problem of the track and asked the question whether you could use deep learning to solve it.
• 5:00 Oliver gave us the constraints of using a single camera and some kind of neural model but otherwise given freedom to solve the problem.

What kind of preparation did you do?

• 5:25 We got dropped into a Slack channel, and the conversations just started.
• 5:45 It was across 5 different countries and 9 different time zones, so the first step was how to organise the work.
• 6:05 So based on the time zones, we organised the work in a way that would work for everyone.

What was the background of the students?

• 6:55 Everyone was a self-driving car student at Udacity.

What had you done at that point?

• 7:05 The self-driving nano-degree at Udacity is a three-term programme, and no-one had been past term 1.
• 7:15 Term one contains computer vision and deep learning techniques.
• 7:20 We had done machine vision, lane detection, and sign recognition.
• 7:30 The behavioural cloning was a way of driving a simulator around a simulated track.
• 7:50 As far as the other similarities between the team - the students all had a computer background, but very diverse.
• 8:30 One person on the team was a race car driver and instructor, which helped to get around the course.

Why did you start by creating a simulator?

• 8:45 We used the Udacity simulator as a basis, and took an overview of the track from Google, and tried to get a height profile as well.
• 9:15 They overlaid the height profile and race track image to give a crude resolution simulated track.
• 9:30 We then used that to test all of our neural networks to go around the track.

Why did you use a convolutional neural network?

• 9:50 One of the reasons was that Udacity was sponsoring us, and requested that we used deep learning.
• 9:55 The second reason why we didn’t use something else, like waypoint following, is that that’s a solved problem.
• 10:00 Everybody at the race who was successful was just using waypoint following, but it’s not hard to do that.
• 10:10 We used a different, interesting approach, which taught us something new.
• 10:35 Most of us hadn’t used convolutional neural networks before the course, so it was an opportunity.
• 10:45 We found that there were a bunch of things that involved real-world problems - to me, that was a more valuable learning experience.

How did you get the data?

• 11:20 We drove around and collected images from the camera, and the steering, gas and brake, and then trained the neural network on those data points.
• 11:40 That’s why it’s called behavioural cloning - because it takes a behaviour and tries to learn it.
• 11:45 The point is to map a set of inputs - the image - and you have a set of outputs - the actuators driving the brake and accelerator.
• 11:55 By feeding a neural network a lot of information, it can modify the weights inside of it to get the correct output given a certain input.

How did the race go?

• 12:15 We made it around every single turn autonomously, but just not in the same lap.
• 12:20 There was even a hairpin turn which we made Jendrik sweat every time.
• 12:30 Being a passenger in an autonomous vehicle was a bit nerve-wracking for him, until he realised that we had the vehicle under control.
• 12:40 In between turns, we would have some kind of random input or output where the steering wasn’t correct and we’d have to take over.
• 12:50 There’s definitely room for improvement - which is why we’re doing it again this year - but we only had six weeks to train.
• 13:00 Given that we were able to get around every single turn of the track, it was pretty amazing what we did accomplish.

What is it like being a passenger in a self-driving car that you’ve had six days to model and two days on the track?

• 13:15 Normally if you’re in a car and you see a wall coming towards you, you scream at the driver to stop.
• 13:20 When you’re training a car, you have to allow it to drive towards a wall so that it can learn what to do.
• 13:30 You have to wait until the last second before you can interrupt the learning process.
• 13:40 We took a journalist around the track for one lap, and sitting in a self-driving vehicle going crazy changes your view on self-driving cars.

Did you use the LIDAR or other sensors on the car?

• 14:00 We didn’t for our race; because we borrowed the car from PolySync, it came equipped with RADAR, LIDAR, and the camera.
• 14:10 We just trying to control the vehicle with the camera and the neural network.
• 14:20 We did have GPS in the vehicle as well, and we did try to augment some of the data with that, but not as a vehicle GPS based follower.

What was the architecture of the platform?

• 15:10 The car itself has all the sensors; for us, it was mainly the camera connected to a computer.
• 15:30 We ran TensorFlow on that, with GPU accelerated predictions to steer the car.

Did you have multiple models operating?

• 15:45 Every member of the race team brought their own laptop, so we had five groups of three people who worked on one laptop with a large GPU.
• 16:05 For each race of the racetrack, we brought all the models and ran all of them, to figure out how far they could go.

How did you troubleshoot the models?

• 16:35 We couldn’t approach it with just a single model.
• 16:40 Because of our timeline, we didn’t know what would work - although we had an idea.
• 16:45 We didn’t want to turn up with a single model, turn up at the race, and have it not work with no backup plan.
• 16:50 Since we had a lot of people, we started going down multiple approaches, and then funnelled that down to a smaller set of models.
• 17:30 Mostly the testing was seat-of-the-pants - if you stay on the track, it’s doing good, and if you aren’t on the track, it’s not.
• 17:40 I ended up keeping Jendrik in the car because we worked well together.
• 17:50 When others would come in the car, it was interesting to see us working, because we were repeatedly testing and occasionally failing.
• 18:00 Jendrik would have a model loaded up, it would fail in some way, he would try again, it would fail in a different way, and then we’d move to the next model.
• 18:15 We would only have about 15 minutes to try everything, after which it was a different car’s turn on the track.
• 18:25 We could then analyse what the models did and could discuss the points.
• 18:30 Some models didn’t work at all, some models got round one turn before failing, but it wasn’t until Sunday morning that we had an idea.
• 19:00 There was a model which we had just finished training on Jendrik’s laptop, that made it around multiple turns - and we had something that we could start improving.

How fast did you go around the track?

• 20:20 We varied it a little during training - we tried to see how you could race it around the track (which is why we needed to replace the brakes by Friday).
• 20:40 By the time we were doing the race, we had settled on 25mph, and use the convolutional network to drive the steering and having the brake and accelerator stay constant.

What takes the output that the model is giving you and steering the car?

• 21:10 We were using the PolySync platform to handle the car.
• 21:25 PolySync core is a middleware platform that can supplement or replace ROS, but used the open-source car controller (OSCC) - now called DriveKit.
• 21:50 That interfaces with the CANbus on the vehicle, so that when the software decides that it wants to turn, it makes the vehicle do that.
• 22:00 The nice thing about OSCC [http://oscc.io] is that it’s open-source - you can go to the code on GitHub and see what it’s doing.

What does it send to OSCC?

• 22:25 It takes messages, which is distributed through the system - you can send steering, brake and throttle messages.

What is ROS?

• 22:55 It’s the Robot Operating System, which allows people to start making robotic systems.
• 23:10 ROS gives you a framework for messaging, so you can publish on topics and have those delivered to other nodes.
• 23:20 You can get your communication across, whether it’s depth information from LIDAR or GPS, they come in the same mechanism.
• 23:40 You can standardise on sending messages, and have nodes that other researchers have given, so you can connect things together.
• 24:10 It can also save a lot of work in developing your own nodes.
• 24:25 You don’t have to develop everything from scratch.

What approach are you taking for the second race?

• 24:55 We’ll use a segmentation network for the perception, which is not directly trying to figure out the steering directly.
• 25:10 Instead, we’ll be using the image to find out where the road is, what obstructions are, and so on.
• 25:25 We then build a road controller to steer based on where we are, instead of having a single model trying to do everything.
• 25:45 It’s easier to debug this at different levels because you can debug each component separately.

How do you train the segnet?

• 26:00 We train on the data we recorded with all the laps from last year.
• 26:00 We have all the images so that we can train the segnet on that data.
• 26:20 We can try and build a controller that’s doing the correct steering for us.

How does it cope with different conditions on the track?

• 26:35 That will come into play when we do the controller in a little while.
• 26:40 It comes into play a bit differently for a convolutional neural network, because it’s more about what’s in the image.
• 26:45 If there’s rain in front of the camera then it’s really an array with numbers.
• 27:00 The rain drops on the camera might mess things up a bit, or the lighting change during the day might cause some variation.
• 27:15 We’re also trying to do the same approach as last year where we’re using the simulator.
• 27:30 The other nice thing we’re doing is that you don’t have to mark you training data if you’re using a simulator - you know where the road is.
• 27:40 We can then try and feed the real data into the model.
• 27:50 Using real data is time consuming, because you have to annotate all the data beforehand.

How did you debug the segnet so that you can adjust it?

• 28:00 Debugging the net mostly involves retraining and using exploration frameworks.
• 28:20 You can also use deep explore, where you can get some neurone coverage.
• 28:40 It’s a bit similar to what adversarial neural networks do.
• 29:00 It’s possible to trace back with TensorFlow but it’s still a hot research topic.

What’s it like when code you wrote is driving a car around a track?

• 29:35 It’s super amazing.
• 29:40 When we got the car, it hadn’t worked with our software.
• 29:55 We needed to spend a while figuring out why our software wasn’t working at first.
• 30:05 Just getting to the stage where we could turn the steering wheel was great.
• 30:15 When we got to the stage of the turns on the course, and the car was doing all the right things up to the last second.
• 30:25 That’s one of the things about being a good safety driver - you have to let the car do things up to the last second in case it’s going to do a correction.
• 30:35 There’s definitely an instinct of when you’re going round those hair-pin turns where the car is going straight, where you think you’re going into the grass or into the wall.
• 30:45 By letting it do its thing, and giving it a few extra seconds, it can rush to take that turn.
• 31:00 One of Jendrik’s favourite parts is when we were debugging to try and get the steering messages to work, and Highway to Hell came on the radio and we were rocking out to that.
• 31:20 We like to call it “adrenaline fuelled development” - a lot of highs, a few lows, and minimal sleep.

How did you deal with failure?

• 31:50 It doesn’t - this is a research platform, not something that’s production ready.
• 32:05 Before you get to that point, you want to make something that’s more redundant, more capable.
• 32:10 Companies like nVidia have their Drive PX2 which is research, but is moving into Pegasus which is a self-driving car platform, which does have that reliability.
• 32:30 Our system wasn’t like that - it was a desktop PC which had the power cable jiggled a bit loose or something happened, the system would have gone down and reboot.
• 32:40 Even if the power cord falls out of the computer, the OSCC would just be forwarding the messages to the car and switches off all the control taken by the computer as soon as you move the steering wheel.
• 33:00 We could still drive even if the computer failed because the OSCC had to cater for that.
• 33:10 Anthony could pull the steering wheel and our code would automatically disconnect.
• 33:20 With most autonomous car research companies in this area, the end-all safety is your safety driver.
• 33:35 You have certain take-over commands that you can do, which switches off all driver inputs by the computer.

How has this experience changed your views on data science?

• 33:50 It has broadened my view - before, I had never used convolutional neural networks on a robot or anything.
• 34:05 When you look on data science topics, if the normal machine learning algorithms fail, try a neural network - especially with the short training cycles we have these days.

What recommendations do you have for picking up machine learning or AI in your job?

• 34:55 Just do it - there are a lot of good resources out there; Ian Goodfellow has “Deep Learning”, MIT has some courses out there for free.
• 35:10 Doing it - what it takes to train a network, the tweaks that you have to do - is invaluable.
• 35:25 If you want to read over projects at GitHub, download it - but understand why it works and edit it.

You can use remote control cars to work on this?

• 35:45 You can download donkey cars, which are RC cars that you can train with, and I think nVidia has a deep learning institute which has a deep learning institute.
• 36:05 The tutorials with the different frameworks; TensorFlow and others have tutorials for how to get started and how to install them.
• 36:25 They usually have good tutorials for creating and training your own neural networks for recognising (for example) the MNIST data set.
• 36:40 A lot of the free content that you can download on the internet can help you get started.