Microsoft Releases Dialogue Dataset to Make Chatbots Smarter

Maluuba, a Microsoft company working towards general artificial intelligence, recently released a new open dialogue dataset based on booking a vacation - specifically, finding flights and a hotel.

The number of chat bots has risen recently, especially since Facebook opened their Messenger platform to these bots a year ago. At the moment, most bots only support very simple and sequential interactions. Advanced use cases such as travel planning remain difficult for chatbots. With this dataset Maluuba (recently acquired by Microsoft) helps researchers and developers to make their chatbots smarter.

Maluuba collected this data by letting two people communicate in a chatbox. One person played the user, while the other person acted as if he was a computer. The user tried to find the best deal for his flight, while the person who played chatbot used a database to retrieve the information. The interactions only consists of text (there is no spoken interaction), a conscious choice of the researchers. Most people prefer typing to speaking, and it means that this dataset is free from bad speech-recognition and background noise. The result is a dataset with 1,369 dialogues on travel planning, and can be downloaded for free.

Maluuba also presents a way of representing the dialogues. What makes travel planning more difficult is that users often change the topic of their conversation. Simultaneously you might discuss your plan to go to Waterloo, Montreal, and Toronto. We humans have no trouble with keeping apart different plans people make while talking. Unfortunately, If users explore multiple options before booking, computers tend to run into problems. Most chatbots forget everything you talked about when you suddenly enter a new destination. In the left image below you see the interaction with a "traditional" chatbot. As soon as the user utters a new city the bot forgets the old city. On the right you see a pattern that emerged in the datasets that are published: users compare multiple cities before making a decision.

[Click on the image to enlarge it]

This is why Maluuba introduces so-called "frames". Each time the user modifies a previously set value, their wizard creates a new frame. Each tracked frame keeps in memory all the different sets of constraints mentioned by the user. This allows you to simultaneously talk about booking a trip to Montreal for a maximum price of $200, and a trip to Toronto with a maximum price of $300. This kind of conversational memory is a step forward in building bots that help users explore different flights in online travel use case.

Traditional chatbots, like the one you can create on Pandorabots, try to steer a conversation towards so-called "slot-filling". The robot has a few properties (e.g. name and age) that it tries to find in the answers you give. As soon as the chatbot knows these properties the conversation advances and the bot tries to fill the next slots. Other companies, such as the Facebook-acquired Wit, already took this idea a step further with "stories". This flexible layer on top of their language understanding layer already creates more "natural feeling" conversations. The dataset published by Maluuba takes this a step further, by focussing on having multiple conversation topics at the same time.

Last week Tim Peterson published an article about the weaknesses of current chatbots. This dataset addresses one of the issues Peterson talks about: the lack of data chatbot developers get due to the small number of chatbot users. As many chatbots have problems understanding natural language, developers often add "quick replies" to their bot. With these replies users select their reply from a small set of options that the bot surely understands. Although this feature is useful for users who know what they can say, it also means that replies to less-common user queries are never developed.

[Click on the image to enlarge it]