Rasa, an open source framework that provides machine learning tools to build and deploy contextual AI assistants, recently held its developer summit in San Francisco. Key topics discussed at the event were: AI assistants can handle question-answering, and repeatable task execution on behalf of users; in addition to machine learning-powered dialogue management systems, infrastructure and workflow tools are required to build production grade contextual assistants; an ongoing area of research is context and the ability of goal oriented dialogue systems and attention models to maintain relevant context.
The speakers at the summit shared interesting enterprise case studies on using Rasa to build AI assistants. "Our primary goal for the Rasa Developer Summit was to build community," said Alex Weidauer, CEO & co-founder of Rasa. "We had speakers sharing the latest research in conversational AI, many members of the community providing real-world examples of contextual assistants in production, and everyone benefitting from the collaboration that comes in a community event," Weidauer added.
Alan Nichol, co-founder and CTO at Rasa, opened the summit by highlighting the need for standard infrastructure to build level-3 contextual AI assistants. Akin to the five maturity levels of autonomous driving, the founders see five levels of AI assistants.
Here are a few key takeaways from the summit.
Josh Converse, founder of Dynamic Offset, took to the stage to explore the concept of a three-part harmony that combines Rasa, Kubernetes, and other disparate open source software (OSS) to build an AI voice receptionist/assistant. He equated this harmony to how the Beatles, as a whole, is greater than the sum of its parts, the band members. Converse discussed the architecture of the AI assistant and how the machine learning based dialogue management system, powered by Rasa, acts as the assistant’s brain. He showcased how Kubernetes can be leveraged to scale the AI assistant.
Edouard Malet, senior data scientist at N26, talked about the need to empower content creators in order to build scalable AI assistants. Malet stressed the fact that data scientists may lack the domain knowledge that is required to design conversations carried out by an AI assistant. After touching upon the responsibilities of data scientists and content creators, Malet went on to discuss the technology stack and the pipeline––that encompasses Rasa, Jenkins, and Nomad––used at N26 to build scalable and robust AI assistants. He also discussed the challenges and solutions to serve multilingual content in multiple markets.
Nouha Dziri, research intern at Google AI, talked about the state-of-the-art entailment techniques that can be used to evaluate coherence in open ended dialogue systems. The challenge, she said, is to come up with an automated metric that can provide an accurate evaluation of a dialogue system without human input. Dziri outlined how conversational logic can be modeled into four aspects in terms of quantity, quality, relevance, and manner. She closed by stating that researchers are still on a quest to identify a reliable metric to evaluate open ended dialogue systems.
Mady Mantha, director of AI/ML at Sirius Computer Solutions, walked the audience through the evolution of Natural Language Processing (NLP). She talked about named-entity disambiguation (NED), the process of figuring out which object is referred to or meant by a named entity. A named entity is a real world object such as a person, a location, or a quantity. Take for instance this sentence "I am going to the bank." "Bank" here can mean either a river bank or a financial institution, depending on the context in which it was said. Mantha showcased how Google’s BERT can be integrated with Rasa to handle NED in a virtual travel assistant. Given a user input "book me a ticket from London to Paris", the BERT model was able to recognize and extract the source (London) and destination (Paris) entities. The model outperformed other competing models and was also able to handle an input like "book me a ticket to Paris" and extract the destination entity even though the source entity was missing from the user’s input. Mantha concluded her talk by highlighting the need for a continuous improvement pipeline in order to build reliable AI assistants and what such a pipeline would entail.
InfoQ sat down with Mantha for a Q&A at the summit.
InfoQ: 2018 saw a backlash on virtual assistants for failing to meet user expectations. Do you see AI assistants making a comeback of sorts?
Mady Mantha: The backlash was reasonable. When people first started interacting with virtual assistants, they often compared it to humans; rule-based chatbots can’t compete with humans at all, especially when it comes to things like context switching, seamless task execution and in general, the unpredictable nature of human conversation.
Recently, we’ve seen a lot of advancements in dialogue management systems and language models like Rasa’s Transformer Embedding Dialogue policy and Google’s BERT and ALBERT. These advancements combined with tempered user expectations is promising. We can now begin to think about how conversational AI can actually add new revenue streams and lower operational costs.
InfoQ: What are the key elements to building and productizing an AI assistant?
Mantha There are a lot of components that go into making a contextual assistant truly successful. You may want to start by defining the problem space, for instance, is it a personal travel assistant, or it is an internal employee-facing digital assistant? After you build a robust NLP pipeline and identify integration points for task execution, establish a baseline to continuously evaluate and improve your model based on real-user conversations and feedback.
The standards and best practices that apply to building, deploying, and scaling software applications apply to conversational AI for the most part. It’s important to automate the entire workflow using a CI/CD pipeline, use a version control system to manage models, and perform holistic end-end testing. Tools that can assist with data tagging, dialogue editing and management, version control, holistic testing, model re-evaluation and re-training based on actionable feedback, essentially a continuous learning cycle, can be extremely helpful.
InfoQ: How can AI assistants augment existing NLP applications?
Mantha: NLP is at the core of many text applications, like information retrieval, document classification, text summarization etc. For instance, if a shopper is trying to return an order, and searches "how do I return an order" in the text box, a backend system can delegate the task of answering this and similar questions, along with assisting the shopper with order returns to the contextual assistant.
So, while an inverted index can be used to search for products, a contextual assistant can take it to the next level by offering proactive customer care. In another example, you have an application that classifies legal documents. A contextual/legal assistant can classify and summarize documents, and in addition, present draft legal briefs to paralegal teams for their review.
Slides from the Rasa Developer Summit are available via SlideShare.
