Transcript
Poubel: Without further ado, let me introduce you Dolly, and it's this little robot down here that I prepared just for this presentation today. It's a mobile robot that carries your heavy stuff for you while following you on the sidewalk. It's just meant as a hello-world example of robotics application that has all the main robotics components. Let's take a look at what it does.
You have Dolly here, and this is the human. When the human moves, Dolly just moves behind the human to follow where the human is going. That's all it does. It's not the most complicated robot, but it is running some very powerful software. Actually, the software stack that is running here in Dolly is also present in a lot of self-driving car prototypes, in a lot of warehouse robots all around the world, industrial robots, even quadcopters, humanoid robots. There was a robot in the International Space Station that was running parts of the software that are running in Dolly right now.
You might imagine that I'm talking about ROS, the robot-operating system. It's not exactly an operating system, it's more like a set of tools and libraries that help roboticists put up their robotics applications more quickly than they would if they didn't have all these tools available. It's all free and open-source. But technically speaking, what's running here on Dolly is not what people today would call ROS. It's more the next version of ROS that we're developing. It's been under development for the past 4 years and it's becoming as feature-rich as ROS but it's still under heavy development and it's called ROS 2. So there's no ROS proper running here in the simulation right now. It's all ROS 2 running. I'll be talking about both of them, their differences and their similarities, because there are a lot.
I will also be talking about the third piece of software which is called Gazebo, which is the name of the simulator where this presentation is running right now. Gazebo is a robotic simulator that can be used for a variety of things. It's not a presentation software, but I use it for presentations. Because why not? I'll be talking about that as well.
But first, who am I? My name is Louise Poubel, I'm a software engineer at Open Robotics. I've been there for almost four years and, before that, I was contributing with their software externally. I started as an intern, and then was hired to work full-time on open source, which is awesome. But actually, I'm not trained as a software engineer; I'm a mechanical engineer by training, that's what I did in my undergrad, back in Japan. And then, I did my master's in robotics. This picture is actually from my master's which was actually one of the last times that I dealt with a real physical robot. Because, after this, I went to Open Robotics to work on software and I've been one of the core developers of the simulator ever since. Most of the robots that I touch nowadays are inside the computer. Here it looks like I'm trying to imitate what the robot was doing but actually the robot was imitating what I was doing. That's what my whole thesis was about.
I also wanted to show you a little Easter Egg that I have here in the simulation. I took this world that I'm using as the background for this presentation from a project that we have called CitySim, which people use to train self-driving cars in simulation. I was responsible for putting the world together and I slipped in the height map of Rio de Janeiro, where I'm from, in Brazil. So here you can see an attempt of having the same mountain as this one, which is called Sugarloaf. It's one of the main touristy spots in Rio de Janeiro. So just an Easter Egg. I think that the people who I work with have no idea that this is there. I want to see their reaction to this.
Open Source Robotics
Open Robotics - who are we? We are around 23 people, right now. We're located all over the world. Our headquarters are in Mountain View here, in the U.S. That's where I work from. We recently also opened an office in Singapore where we have four people working, and we also have a few developers working spread around different cities in the U.S. and also in Europe. We are dedicated to developing open-source software for use in robotics. Our two main big projects are ROS and Gazebo, the two that I'm going to be talking about today. But we also have some other smaller projects. But the common theme amongst all of them is being free, open-source, everything is Apache 2 licensed, everything is ready for you to just take, and go home, and use it, and sell your products with it, and you don't need to tell us anything about it.
In today's presentation, I hope that I transmit a little bit of my love for robotics and for open-source to you, and that you get a little bit of the excitement of this really multidisciplinary field that has not only software- I'll be focusing on software, but there's also mechanics, electronics, human-robot interaction. Robotics is so interesting and there are so many unsolved problems, it's a very interesting field to work in right now. Before there was a show of hands and it seems like not a lot of people here are working with robots. I hope that, after this presentation, you get excited to just go home and try this stuff because it's all free and available, and start using your developer expertise for a different field, if you're interested.
Actuation, Sensing and Logic
Let's go back to Dolly and talk a little bit about how a robotics application is structured. I'll explain a little bit about how Dolly's application is structured so we can take a look at some tools and poke around what's happening right here. These are the three main things that are happening in the robotics application at the same time. On a high level, you have actuation, which is basically what separates robotics from many other fields. The robots that I'm talking about here is in the sense of a robot that acts in the physical world and will maybe pick up some object, or it will move in the world and may collide with the world. So it's an object that acts in the world. This is the whole actuation side that you see here, this is the actuation aspect of the robot.
On the other hand, we have sensing, which is how the robot collects information from its environment. The robot is not just moving randomly and hitting things, it's actually acting based on sensor feedback that it's receiving from this environment. Each time the robot acts on the environment, the environment changes so it needs new sensor readings. So sensing is another big block of what's going on.
And finally, there's the whole logic. It looks small here just because the word logic is small, but it's actually where most of the magic happens. The sensing and actuation are mostly device drivers. They are talking to these real components in the physical world, but logic is where your robot decides, "What is a person?" "What is not?" "What should I follow?" "Where should I go?" "What should I pick up?" Usually the big chunk of software that goes in your robot goes in this logic side.
How is this mapped to what Dolly is doing here? Starting with the actuation, Dolly is doing differential drive. If you noticed, Dolly has only two actuated wheels, which are these cylinders in the front. And in the back, it has this caster wheel, which is just passive. It just rolls to keep Dolly balanced. It does have a little joint here in the back, which is the waging tail, but it's not doing anything important. The way that this drive works is that these wheels, they can't be steered, they can only rotate in one axis. According to the difference of the speed that the wheels are turning, it might turn to one side or another side. It's a very basic controller for locomotion. So this is what is going to make Dolly move.
For the sensor side, we have the laser. Dolly only has one single sensor, it's this laser here that we're visualizing in blue. The blue is just a visualization of what the sensor is seeing; if Dolly was a real robot, there would be nothing visible, it's not like it's casting a light or anything, it's just a visualization of the sensor. The sensor captures the depth of points in a plane.
Finally, the logic that Dolly is doing is actually the only part of code that I had to write specifically for this presentation. It's very short. I didn't count the number of lines of code but it's quite short, and it's a follow logic. It receives information from the laser, it decides what's the closest point in front of me, and then sends to the differential driver, "This is the way you should go." That's it. It's very basic and actually Dolly doesn't know what's human is, what's a tree is. So if the human walks too fast, Dolly might just go towards the tree and wait there.
These three different parts of the software, I matched them to what in ROS are called nodes. A ROS Node is a piece of software that does something specific, it tends to be very specialized and only does one specific task. Notice that I'm not saying that they are specific processes or executables; in ROS 1, they tended to be one node per process. Then there were some tricks like Node lets and other concepts that you could put more than one node in a process. But in ROS 2, they are just pieces of software that do something, that communicate with other nodes. When you're writing them, you don't know if they're going to be in separate processes or not. And when you deploy your application, you decide how they map, like which node's going in which process, and so on. So these are nodes. They're only three nodes in Dolly, but a typical robot may have dozens, even hundreds, of nodes running at the same time.
Communication between the Nodes
How is information flowing between the nodes? We have mainly two pipes of information. One is the laser scans that are flowing from the laser to the follow node. And the other one is the commands going towards the differential drive. These are, what we call in ROS, topics. A topic is basically a channel of information and you have a publisher and a subscriber. It's a pub-sub mechanism, it's many-to-many, so you can have many nodes publishing in the same topic and many nodes also subscribing to that same topic. They don't care about where the information's coming from, they just care that it's coming, it's a certain message of a certain type that is coming through a certain channel, which is that topic.
This is the most basic type of communication supported in ROS or ROS 2. But there are other two main methods of communication. One is what we call services, which is more of a request-response kind of thing. And there are also actions, which are more like a request for something that may take a while and will send you periodic updates of how it's progressing. Eventually, it's going to say that it's complete and you can also cancel it. It's a little bit more complex. For our example today, I'll be only using topics because it's the most basic construct in the communication of ROS.
How are these mapped to processes? As I was mentioning before, since we're talking about ROS 2, this line here divides what's on top is in one process, what's in the bottom is in another process. We happen to have two nodes running in the same process here, the laser and the diff drive, and the follow node is actually running in a separate process. And they are talking over ROS 2 across processes here. The reason why the laser and the diff drive are in the same process is because they both need to be tied very closely to the simulation, to Gazebo. In the real world, those would be vice drivers that are real devices that are collecting information from the world or acting in the world. When we're in a simulation, the simulator has to handle generating all the synthetic data, and also, the simulator has to handle moving things inside as if it was moving in the real world. So these parts have to be as close as possible to the simulator. And because it's ROS 2, we have two nodes in the same process. No problem.
Tools
Let's now try out some things for you to see that there's actually something running real here. These are some command-line tools that if you've used ROS 1, these will look very familiar to you. We try to keep the tools as close as possible to what they were in ROS 1 as we migrated to ROS 2, because that's what users are used to. I will talk a little bit later about what the real changes in ROS 2, like the big changes. But the command-line tools, the graphical tools, we are trying to keep them the same as possible so the user can migrate more easily.
Let's try these commands here. This first one is ROS 2 node list. This will list all the nodes that are running right now. As we expected, there are three nodes. There is the laser, the diff drive, and the follow node, as I showed you before. When we try the ROS 2 topic list, that actually has a few more things. These two here, the command vel and the laser scan, are the topics that we saw before. There are actually a bunch of other topics. Some of them are provided by ROS. For example, clock is a central place for all the nodes to agree about time. This happens to be very important for a robot that has sensors collecting data in all different parts and sending commands from many different places. They have to agree about the timing of each sensor. If you're moving very fast, for example, that's very important.
Another topic that is very conveniently provided by ROS is TF, which stands for transforms. It basically keeps track of the physical transforms of the positions between different parts of your application, be it links within a robot- think of a humanoid robots with a lot of different links connected by joints- be it several robots and how they positioned among themselves. There is one central source of where this information can be obtained from, which is the TF topic. There are some others but these are the most interesting ones.
I can do "ros2 topic info," to get information about a topic. I'll do the command vel. We can see here that there is one publisher, one subscriber. We can also echo what is being published in the topic right now. When we echo, we actually create a new subscriber to that topic. Right now, there are two people listening to the topic. My command-line tool and the robot. The robot didn't stop working. We can see that the commands right now are basically 000 velocity because they are stopped. Let's try to make it do something interesting. So if I move the human here, we see that now there is a linear velocity on X, which is the front direction, and there was a little bit of Z. And then, the robot saw that it's pretty close to the human now so it's stopping.
The cool thing about these topics is that if, for example, tomorrow I decide to put some propellers on Dolly and remove the wheels and Dolly's going to become a quadcopter, I don't need to change the follow node, I don't need to change the laser, and I don't need to change these command-velocity messages. I will just change the differential drive for something else that will translate that desired velocity for the whole robot into how fast each propeller needs to move to do that movement. The rest of the program is completely agnostic to the kinematics of the specific robot. So these are some of the tools.
Now, let's go to this other tool, which is a graphical tool, the most powerful tool in ROS, I believe, which is RViz, which stands for ROS visualization. So, "rviz2 rviz2." So open RViz here. I lowered the resolution of my computer so you could read better, but this also means that the UI's get kind of squished. But RViz is this little tool here that has a bunch of topics. It has this 3D view of what is flowing around in your ROS application. Right now, I'm displaying the laser scans, that are here represented by these red lines, and I'm showing the transform of the robot. This is the origin of the robot right here. These red lines here map to where those rays are hitting in the world. So you can see here, it's the two legs. You can see here that there is the sidewalk, and then the sidewalk kind of ends and there is the grass there a little bit farther away.
I think that this example doesn't do justice to all to what RViz is capable of because there's not a lot to be displayed. A lot of people come and ask, "Well, I can already see this here in Gazebo, right? It's basically showing me what these blue lines are showing." But the important thing is that RViz doesn't care about where this data is coming from. If I stop using the simulation and I make Dolly in the real world and I'm publishing that information, RViz is going to show it the same way. RViz will show all the information of your whole application, across multiple robots, all in one place. So I really recommend that you go and search for RViz videos for you to see what people are doing with it. You can put interactive markers inside it, in the 3D world, that when you drag them, it's going to publish topics and send comments to your real robot or to your simulator robot. It's very powerful. This example here doesn't really do it justice, so discard the configuration. So that was RViz.
Summary: ROS and ROS 2
In summary, what is ROS and ROS 2? There is this communication layer that I talked about with topics and services and actions. And the interesting difference in the communication between ROS and ROS 2 is that in ROS, this whole communication was custom. The discovery and the serialization of messages, this was all written from scratch. An interesting fact that I just learned that today is 11 years since the first commit was done to ROS back in SourceForge. So it's an 11-year-old project. Back then, it was all custom code written from scratch.
As we migrated to ROS 2, we decided to build on top of DDS, which is an industry standard for mission-critical applications. It's been tested on things like nuclear power plants and airplanes. So it's very trustworthy, it's real-time safe, it handles quality of service. When ROS was written, it was written originally based on one robot, which was the PR2 robot that had two arms and moved on wheels. Nowadays, it's used for all these different applications. And it doesn't really handle cases, like you have 100 robots, with very little processing power each one of them, and they are communicating over a choppy Wi-Fi. It's not a use case that ROS 1 can handle very well, but that ROS 2 can, and because it can handle quality of service, all because it's building on top of DDS. So this communication layer is the main thing that is changing from ROS 1 to ROS 2.
Another interesting thing about it is that instead of building the whole stack from scratch in each language - ROS has clients for any language that you can imagine because people just write clients for it. But for ROS 2, we created- well, not we, the ROS team, because I wasn't there at the time- there was a thin layer of a wrapper in C that exposes the DDS implementations, which DDS is a protocol. There are many different vendors out there, some are open-source, some are proprietary, and we just created this abstraction. So at compile time, you can decide which DDS implementation you're going to use for your application. If you prefer one, you have the license for another one, you can just use that.
And then, on top of this C layer, you can build thinner clients for each language. The core ROS team is maintaining the C++ and Python layers but there are people in the community already contributing with Java, Rust, C#, you name it. If people are interested in programming robots in those languages, they are creating these client libraries on top of the C layer. There is less repeat repetition of code, less potential for different behaviors, depending on the language that you're using. So that's for the communication layer.
The tools - here is one screenshot of RViz doing some more complex things with the PR2 robot, very dense point clouds, some objects, some depth images. RViz is one of these powerful tools that ROS provides. There are many others, like ROt which allows you to create widgets for creating a dashboard for your robot, for example, or plotting data. Things like this. There are all these tools that are already there for you to use, so as long as you make sure that all your components in your robotic application are speaking ROS and are publishing these ROS messages, you suddenly have access to all these different tools that you can use in your application. So RViz, as you saw, has already been ported to ROS 2, ROt is currently being ported. The target is for it to come out in the next release, in the next distribution. There are many other tools, there are web tools, there are command-line tools. There are a bunch of tools that you just get for free when you start using ROS.
Another important thing that ROS has is all the capabilities. So when you start prototyping your application, there are many different things that you could focus on in a robotics application. ROS, and the whole ecosystem, there are people out there that already solved some of the problems that you're trying to solve. And they're just making these available for free online for anyone to use. So one big example is navigation. If tomorrow I decide that I don't want Dolly to just follow the closest point, but to actually navigate and find itself, and understand the world that it's in, then I can say, "Go to the supermarket," and then Dolly will find its way to the supermarket by itself, there's this thing called the navigation stack that I can plug into Dolly. And because I'm using ROS, I can just tweak some configuration files and it's going to work for that use case. There are many other capabilities. Many of them have not been ported to ROS 2 yet but the navigation stack, for example, is one thing that is being actively worked on right now.
Finally, one of the most important things in ROS is the ecosystem of people from all around the world: thousands of developers in universities and industry and research institutes just contributing code, all free and open. Some of it they can keep private, so you can still be a part of the ecosystem, use the other tools, and build private code on top of the public code that other people are putting out there. So there is this huge ecosystem of people from all around the world contributing with each other, fixing each other's bugs, improving the state of robotics in total as one.
These are the versions of ROS have been released so far. We make releases, we call them distributions, they come with a lot of packages that work together. They are named in the alphabetical order and the common theme is turtles. So the last one to be released for ROS 1 one was Melodic Morenia. The next one's going to be Noetic, targeted for May 2020. And for ROS 2, we're actually going to release Crystal now in December 2018, which is a month away from now. Things that I'm using right here right now--for example, the integration with Gazebo--are going to come in the Crystal release. So right now, if you want to use this, you have to build from a source, like I'm doing. But from December, you can just like get Debian packages and just install it like this.
The idea is that we're not going to stop ROS 1 and tell the whole community to go and use ROS 2. We're going to keep maintaining them side-by-side, improving ROS 2 until it reaches a maturity point, where people organically will start moving their applications and the thousands of robots that are out there running ROS to use ROS 2.
Gazebo
What is Gazebo? Gazebo is the simulator. We've been inside it, you saw a little bit of what it can do. Just to summarize to you what it does, the first important part of Gazebo, or of any simulator, is the physics, which is what people think of when they think of the simulation of a robot. Right? So the physics is the one that is taking care of gravity pulling Dolly down, and making sure that it doesn't pass through the ground because there are collisions between the ground and Dolly. If Dolly drives into a tree, it's going to collide with it. So the physics is keeping track of all of this.
Gazebo actually has an abstraction layer. So there's not one physics engine; we support multiple, we support four right now. Right now, I'm running with The Open Dynamics Engine. Because Gazebo doesn't have its own physics engine, we just use other physics engines that are available out there, and also has support for Bullet, DART, and SimBody. We're improving that, as I will be talking about soon.
For sensors, as you saw with the case of Dolly, in the real world, you have the world generating this data for you and you have these devices that are collecting all that data for you. And, in simulation, you have to fake all that data. You have to create synthetic data for every single sensor. That includes the laser sensor that Dolly is using or, if it had a camera, an RGB camera. It would have to create those pixels for you and pass that image. Or not only things like with rendering, there're physical sensors, contact sensors; if it bumped into something, or a sensor of IMU, how fast it's moving. Gazebo, right now, has support for over 15 sensors, there are things like magnetometers and GPS that you can just plug into your robot inside the simulation.
Finally, there's also the graphical interface, which is a big part of a simulator. Some simulators don't have a graphical interface, which is fine, you can run your simulation, what we call headless. So you just run it, you don't see what's going on, and then you get some results in the end. But Gazebo has a graphical interface, which is what I'm using right now to show all these things to you. It provides a few different things for us.
One of the things is this observation of what's happening in the world with the robot, which is something that is an advantage over the real world, because it's really hard for you to fly over your robot and see what it's doing in the real world when you're trying it out. Another thing related to observing your robot is inspecting aspects of the simulation. So you can make your robot transparent, you can see its joints, and you can see, "Oh, right here, there is a joint that rotates like this, there is also another joint here in the back." So you can inspect your robots inside the simulation.
You can also interact with it. Have you seen those videos of people kicking robots to see if they're going to fall? You can do something like this in simulation too. You can just like apply some force. This little resolution is really hard to see. But, for example, let me push the chassis of Dolly a little bit. And then Dolly is just moving and it tries to find a human again. You can see how your robot is going to react to these external impulses. I can move the human, which is another way of interacting with it. If you had the real robot in front of you, it would also try to move something in front of it to test what it's doing, test its algorithms.
Another thing that you can do with the graphical interface, is actually build the world. You can put these buildings here, you can move them around, you can add things to the world. There is a model database Gazebo has here. For example, I can put, let's see, something like a city, a bus. So I can put a bus here, and there are ways that you can animate it and script it to move in the simulation. There's a car here that has sensors. Yes. This is a Prius, it's equipped with a lot of sensors. It actually has a ray sensor with many more rays than the one that Dolly has, because it also has vertical scanning, those that you see the self-driving cars on the street with that big thing on the top, there is one here on the Prius. You can see right on the graphical interface, you can see some of the messages that are being published here. For example, this is the front camera of the Prius, you can have the back camera here which is turned to the human over there. So these are some of the things you can do with the simulator.
There is also a robot editor, you could create your whole robot inside the simulation. There's a building editor and other things like this. So there are also interfaces. Right now, I'm interfacing with ROS 2. Then the way to interface with Gazebo is to write C++ plug-ins for it. The ROS 2, we, at Open Robotics, provide the plugins for you to interface Gazebo with ROS, Gazebo with ROS 2. But there are people in other projects, in other groups, interfacing Gazebo with other different frameworks for robotics.
Finally, there's also the cloud. We are starting to host a lot of simulation assets in the cloud. The bus and Prius that I put in my simulation right now, they are installed in my computer but we're starting to move that to a cloud server where people can create their own assets, just put them there, and distribute and use each other's models to more quickly build simulations. There's also the whole aspect of running simulations on the cloud. Sometimes you don't have a computer strong enough, powerful enough, to run a very heavy simulation. But you can pay a cloud provider to run it for you, to run a CI. People who are developing robots are running CI on the cloud; every time they make a change, they have simulation running. So these are some of the ways you can use cloud, you can use it for education.
Gazebo Projects
Here are some projects that we've done with Gazebo in the past. Many of them are competitions. So it happens that people love using simulation for robotics competitions, and one of the reasons is that you give the same exact conditions for every single team. Sometimes you go to a real robotics competition, for example, you are in the sea. And one team was just very unlucky and there was a big wave, the other team was just lucky. In simulation, you can control everything and every team gets a fair environment, everybody gets the exact same conditions. So some of the projects here, like this one was a competition about humanoid robots in a rescue mission. The robot had to walk over rough terrain, then drive a little car. This other competition here was a NASA competition of a robot in Mars. Teams had to control the robot from Earth and there was this big latency of each comment that is sent to the robot, so the robot had to have a lot of autonomy there. This is something that, even if we want it, we can't make this competition with a real robot right now.
Another important part of simulation is that sometimes you want teams to compete. But one of these robots can cost millions of dollars. And if you want to be fair to everybody, if you want to give everybody a chance, just make it in simulation, and then everybody can have access to a robot to compete in the competition. There are some other projects there, I can tell you more about it if you're curious.
In the same way that ROS is being refactored into ROS 2 to improve and make use of newer practices and newer architectures that exists out there, Gazebo is also currently being refactored in what we are calling the ignition project. Gazebo right now is a monolith, as I was telling you. It runs in one server, in the back end, and one front-end server. We are trying to make that more distributed so it can support more robots being simulated at the same time. Gazebo is actually an older project than ROS, it's almost 15-years-old.
There's a lot of room for improvement and we're currently in the process of breaking Gazebo, this big monolith, apart into multiple libraries, and improving every single aspect on the way. The physics abstraction library, now it's going to be plug-in based, so you don't need to recompile Gazebo to integrate a new physics library. You can just write it as a plug-in that is loaded at runtime. We're doing the same for rendering, so you can support multiple rendering engines. Right now, Gazebo only supports OGRE but once Gazebo turns into this Ignition Gazebo project, it's going to support OGRE and other ray tracing engines that could be more photorealistic and you could use either one open-source or some expensive proprietary library that you have. It's just up to you to create that plug-in. We're also changing the user interface and also the core of Gazebo, how it works, to support more distributed simulation.
These are some robots that use ROS and/or Gazebo, in some way or another. This is not an extensive list, these are just some examples that I picked up. For example, Baxter here was probably the first commercial product shipped running ROS. The Savioke robots too, they run ROS, and they're currently, right now, running on several robots, all around the road, delivering toothbrushes for people who forgot them. So you can see that it spans all types of robots. You have industrial arms, you have quadcopters. These are not self-driving cars, they're mapping cars, they're just mapping streets. These ones work on supermarkets or warehouses, checking the stock to see if things are still on the shelves where they should be. So it's very versatile. This one is the one that was in the International Space Station for a long time and it was running ROS up there.
Talking about community, Open Robotics hosts a conference, a developers conference every year, called ROSCon. We just had the one in 2018, in Madrid, a month ago. If you're more interested in learning about ROS and Gazebo, watching ROSCon videos is a very good way to get started if you're the kind of person who likes to watch videos, which I am one of.
And that's it. I'll leave these links up here for you, entry points for ROS, ROS 2 and Gazebo. I hope that I was able to spread a little bit of my robot excitement with you, and that you're going to go home and try this all out and make a lot of robotic products and distribute them with free and open-source software. Thank you.
See more presentations with transcripts
