Algorithmia, an AI model management automation platform for data scientists and machine learning (ML) engineers, now integrates with GitHub. The new release integrates machine learning initiatives into organisations's DevOps lifecycles. Users can access centralised code repository capabilities, such as portability, compliance and security. The integration follows DevOps best practices for managing machine learning models in production, with the goal of shortening time-to-value for data science initiatives.
Algorithmia's company history began with an AI algorithm marketplace for data scientists and ML engineers to find and share pre-trained models and utility functions. Launched in 2017, the Algorithmia platform uses a serverless AI layer for scaled deployment of machine and deep learning models. There is also an enterprise offering for those wanting to deploy, iterate, and scale their models on their own stack or Algorithmia's managed cloud.
GitHub brings together a community to discover, share and build software, and is a major repository for software developers and data science teams. By integrating with GitHub, Algorithmia enables data science deployment processes to follow the same software development lifecycle (SDLC) that organisations already use.
With this Algorithmia-GitHub integration, users can store their code on GitHub and deploy it directly to Algorithmia and Algorithmia Enterprise. This means that teams working on ML applications can participate in their organisation's SDLC. For example, multiple users can contribute and collaborate on a centralised ML code base, ensuring code quality with best practices like code reviews through pull requests and issue tracking. Users can also take advantage of GitHub's governance applications around dependency management to reduce the risk of their models utilising package versions with deprecated features.
InfoQ asked Algorithmia CEO Diego Oppenheimer to tell us more about the new integration.
InfoQ: Why are teams increasingly looking to machine learning as part of their applications?
Diego Oppenheimer: As enterprises accumulate vast amounts of data, the ability to manually process it rapidly becomes infeasible. The rate at which machine learning models can produce actionable outputs is exponentially greater than traditional applications. A healthy elastic machine learning lifecycle requires little overhead, which could cut down on expensive headcount.
As companies grow in size, workflow can become siloed (across departments, offices, continents, etc). A centralised ML repository helps with cross-company alignment. As a use case example: Instantaneous customer service or product function is a requisite for any business. If a customer has a negative experience, they are unlikely to continue doing business with you. To that end, ML opens doors to instantaneous, repeatable, scalable service features to ensure positive customer experience, and models can iterate quickly, meaning improvements can happen in far less time than going through a traditional review and update process.
InfoQ: Why would people want to use Algorithmia?
Oppenheimer: Algorithmia seeks to empower every organisation to achieve its full potential through the use of AI and machine learning. Algorithmia focuses on ML model deployment, serving, and management elements of the ML lifecycle. Users can connect any of their data sources, orchestration engines, and step functions to deploy their models from all major frameworks, languages, platforms, and tools. Algorithmia Enterprise offers custom, centralised model repositories for all stages of the MLOps and management lifecycle, enabling data management, ML deployment, model management, and operations teams to work concurrently on the same project without affecting business operations.
Data scientists are not DevOps experts, and DevOps engineers are not data scientists. Time is wasted when a data scientist is tasked to build and manage complex infrastructure, make crucial decisions on how to scale ML deployments efficiently and securely, and all without incurring excessive costs. Algorithmia Enterprise customers don't have to worry about wasting resources getting to deployment. We make it easy for both teams to work together.
InfoQ: Do you see data scientists and ML engineers regularly embedded into product teams? Or are software engineers in these teams upskilling to these roles?
Oppenheimer: The short answer is yes, data scientists and ML engineers are regularly embedded into product teams, though the configuration of teams is still in early stages, and so it's not yet a definitive migratory pattern. The most successful ML teams are ones directly tied to a product or business unit because of how direct the impact can be. We see more and more centres of excellence in data science and ML in which ML teams are assigned to a product for a period of time to develop capabilities.
InfoQ: Will you also be announcing a GitLab integration?
Oppenheimer: Algorithmia's source code management (SCM) system provides flexibility and options for ML practitioners. We have created a flexible architecture that allows for the integration of other SCMs in the future. Our latest integration with GitHub opens doors for a lot of models sitting in GitHub repositories to go directly into production. Algorithmia will be releasing other integrations in upcoming product releases.
InfoQ: What are the most common ML patterns you see today?
Oppenheimer: Data science teams are rapidly growing across all industries as companies rush to get ahead of the AI/ML curve. Top business use cases for machine learning are models to reduce costs, models to generate customer insights, and models that improve customer experience. Most ML-minded companies are within the first year of model development, and only 9 percent of organisations rate themselves as having sophisticated ML programs.
Half of companies doing ML spend between 8 and 90 days deploying one model. For companies with tens or hundreds of models, this timeline can drastically delay ROI. The biggest challenges reported during ML development are with scaling models up, model versioning and reproducibility, and getting organisational alignment on end goals.
AI and ML budgets are growing across all industries, with financial services, manufacturing, and IT leading the charge of increasing their budgets by 26-50 percent. Determining what constitutes ML success varies by job role. Executives and C-level directors deem a return on investment the key success indicator according to our latest State of Enterprise Machine Learning Report.
InfoQ: Are there any particular industries that are leading adoption in this space?
Oppenheimer: We see ML growth across all industries, with ML market growth from $7.3bn in 2020 to $30.6bn in 2024 [43% CAGR] according to Forbes. However, the financial, manufacturing, and IT sectors do seem to be leading the way with increased budgets, well-defined use cases, and models in deployment.
InfoQ: What is your vision for your company in the next 12 to 24 months?
Oppenheimer: Algorithmia is focused on enabling organisations to take advantage of ML and AI to improve their businesses. AI is the most significant technological advancement in our lifetime and it will quickly become a core part of almost every business in the same way that the internet played a huge transformational role since their inception. Our organisation is focused on solving the last mile to delivering these capabilities to line-of-business applications in a scalable and manageable way.
Integration with current organisational software development lifecycles and IT infrastructure is key so we will continue building and delivering flexibility at the pace that technology is advancing. We aim to partner with data science, DevOps, product, and executive team to reduce the time to market and allow them to create a competitive advantage with their ML investments.
The GitHub integration is now available to algorithmia.com public users and for existing Algorithmia Enterprise customers. The 2020 State of Enterprise Machine Learning Report is available here.