IBM has announced they will make a strategic investment in reactive pioneer and Scala language custodian, Lightbend. The two companies will collaborate on a new platform for cognitive development. Cognitive use cases, (such as customer support bots), are on a collision course with application infrastructure built to handle “Data in Motion,” a term used to describe the analysis of data while in transit, without having to store it. According to IBM:
The resulting joint solutions will be designed to provide a complete toolchain for Java and Scala developers to easily build and deploy AI and cognitive applications in both on-premise and cloud environments.
Developers are turning to Scala for their most complex use cases due its ability to handle streaming data and massive scale. Scala is the language of cognitive development. By 2020, according to Gartner, 50 percent of all new business applications will be designed to be cloud native, using microservices to improve agility and scalability. Modern frameworks such as Spark, Kafka and Akka are written in Scala.
The strategic collaboration between IBM and Lightbend will create new code, tools and documentation that will be designed to help developers build Java- and Scala-based applications on the Lightbend Reactive Platform. It is anticipated to integrate WebSphere and key offerings across IBM cloud, including app management services, the Watson Data Platform and the more than 150 services across cognitive intelligence, data analytics, Internet of Things, security, DevOps, and blockchain available to developers through the IBM Cloud.
Mark Brewer, CEO at Lightbend, spoke exclusively to InfoQ to discuss their upcoming collaboration with IBM to support cognitive use cases for the global JVM developer community - 10 million Java and 1.2 million Scala developers.
InfoQ: Why should this partnership matter to Scala developers?
Mark Brewer: IBM is obviously very invested in helping companies leverage artificial intelligence (AI) and the underpinnings of cognitive. Watson is one of the early shining examples of the power of artificial intelligence - and the Watson data science toolkit is not only written in Scala, but leverages the Lightbend platform.
The vision behind IBM and Lightbend’s partnership is to expose Watson and cognitive- driven applications as microservices that can be integrated into broader systems, including Websphere. The reality is that these types of data pipelines need to be operationalized, as microservices that are available in real-time, all the time. It’s really this opportunity to commercialize how Cognitive gets driven into the application architectures.
And it’s not just Scala developers - the broader significance of the partnership is bringing those same opportunities to 10 million Java developers who are building these data in motion applications, which isn’t possible with the traditional Java EE stack.
InfoQ: What is it about Scala that has made it the preferred language for cognitive use cases and “Data in Motion” in general?
Brewer: At a high level, we see a major shift in development where competitive pressures to get value out of data faster is pushing Data in Motion as the new priority for application infrastructure. Languages and frameworks are increasingly being chosen based on their ability to deliver responsiveness, resiliency and elasticity (aka, the fundamentals of the Reactive Manifesto).
Scala was purpose built for dealing with large datasets, and brings a lot of characteristics for making data a first class concern as it flows through systems. From asynchronous message passing and how it deals with concurrency, to the more advanced functional programming capabilities - it quickly became the de facto language for fast data frameworks. Today - because it is the language that most of the complimentary stacks in fast data were written in technologies such as Akka, Apache Spark and Apache Kafka - there are many advantages to working with the same language with native capabilities and integrations with those frameworks, because you’re rarely just running one of those frameworks, you’re running combinations of them.
InfoQ: What types of companies are creating these types of cognitive applications, and what do you see this partnership doing to help them accelerate their efforts?
Brewer: Enterprises actively working on these business apps today include Samsung, Verizon, Hootsuite, William Hill, PayPal, iHeart Radio and LinkedIn, among others.
For many enterprises, using data-driven insights to deepen customer engagement over web, mobile and IoT applications is a focal point of digital transformation. Consequently, architectures are shifting from batch to streaming. For these always-on applications, being reactive is a must.
