During ROSCon 2019 two interesting tools for visualising and interacting with the Robot Operating System (ROS) were demonstrated: Webviz, and Jupyter Notebooks.
The first tool demonstrated is Webviz. It is an online web-based replacement for RViz. Rviz is a well-used 3D visualization tool for ROS, which can visualize 3D data such as point clouds or poses of a robot sent as a ROS message. Webviz allows other devices to visualise data in a similar manner to RViz, but with greater portability. For example, getting a MacBook to run RViz is a tedious job, which makes it difficult for teams to demonstrate progress to stakeholders who don't have the ROS tools installed.
Webviz gives you the ability to quickly share what your robot or self-driving car sees in an easy way. Webviz can currently visualise images, point clouds, make plots, and print debug messages. Not only can it show this data live, but it's also possible to drag recorded data in a Rosbag into it.
Another interesting approach which gives you more options for interaction is using Jupyter notebooks. Wolf Vollprecht and Chaitanya Deep presented how you can use this interactive browser-based computing environments to visualise and interact with your robot. Generally, programmers use Jupyter Notebooks for exploratory programming, especially in data science and machine learning.
To achieve this, Wolf created ROS support for Jupyter notebooks. He used iPyWidgets, which are Javascript elements which provide a two-way connection between the python kernel and the HTML+Javascript frontend. There are multiple iPyWidgets in the ecosystem. One of them is Bloomberg's plotting library blplot and a library to visualise 3D volumes and 3D scatter plots, iPyVolume.
One challenge Wolf faced was that ROS Python subscribers are multi-threaded. There is no way to stop a thread from a user interface without restarting a kernel. This makes the interactive widgets very useful to pass commands to your robot.
Finally, Wolf showed that you can use Voila on a notebook to turn it into a standalone application. This means you can send co-workers a standalone application to interact with your robot.
