Google is launching two new tools, one proprietary and one open source: AI Hub and Kubeflow pipelines. Both are designed to assist data scientists design, launch and keep track of their machine learning algorithms.
With AI Hub and Kubeflow pipelines, Google is following up on their earlier release of Cloud AutoML in January and continues its strategy to simplify and speed up its customer's ability to adapt to Google's AI techniques and services. Hussein Mehanna, engineering director of the Cloud ML Platform, wrote in a blog post:
Our goal is to put AI in reach of all businesses. But doing that means lowering the barriers to entry. That’s why we build all our AI offerings with three ideas in mind: make them simple, so more enterprises can adopt them, make them useful to the widest range of organizations, and make them fast, so businesses can iterate and succeed more quickly.
Google introduces AI Hub to put AI more in reach of businesses, making it easier for them to discover, share, and reuse existing tools and work. Moreover, the AI Hub is a one-stop destination for ML content, such as pipelines, Jupyter notebooks, and TensorFlow modules. The benefits, according to Mehanna are:
- High-quality ML resources developed by Google Cloud AI, Google Research and other teams across Google are publicly available to all businesses.
- Google provides a private, secure hub where enterprises can upload and share ML resources within their organizations. The hub makes it easy for businesses to reuse pipelines and deploy them to production in GCP—or on hybrid infrastructures using the Kubeflow Pipeline system—in just a few steps.
Next, as a central place where organizations can discover, share and reuse ML resources, they can also build and package their ML resources with Kubeflow Pipelines. Kubeflow Pipelines are an extension of Kubeflow, an open-source framework developed on top of Kubernetes explicitly designed for machine learning. These pipelines are essentially containerized building blocks that users can string together to build and manage machine learning workflows.
Image source: Google Cloud Blog
Essentially, Kubeflow Pipelines are an open-source workbench solution allowing users to compose, deploy and manage machine learning workflows. It offers interoperability and the flexibility to experiment with models before deploying them into production.
Furthermore, Mehanna wrote in the blog post:
Organisations like Cisco and NVIDIA are among the key contributors to this open source project and are collaborating with us closely to adopt Kubeflow pipelines. NVIDIA is already underway integrating RAPIDS, a new suite of open source data science libraries, into Kubeflow. The RAPIDS library leverages GPUs to provide an order of magnitude speed-up for data pre-processing and machine learning, thus perfectly complementing Kubeflow.
Both AI Hub and Kubeflow Pipelines are meant to help data scientists share models across their organization. Also, the early adopters of these tools will be organizations with a sizeable machine learning/data science function and according to Rajen Sheth, director of product management for Cloud AI, they are expected to come from retail, financial services, manufacturing, healthcare and media sectors.