Google Announces Chatbot Meena

Table of Contents

Google how created a conversational chatbot named Meena. While most chatbots perform specific functions like customer support, Meena is trained with a neural conversational model, according to a post on Google’s AI blog. Google has also created Sensibleness and Specificity Average, or SSA, which is a metric system for humans to evaluate chatbots. SSA shows that perplexity, “the uncertainty of predicting the next token,” correlates positively with SSA. Google computed SSA using crowd-sourced, free form conversations with other chatbots like:

  • Mitsuku
  • Xiaolce
  • Cleverbot
  • DialoGPT.

Related: Google Nest Hub Review 

Once the conversations were done, Google asked the crowd workers if the chatbot they spoke with made sense and was specific. When Google compared their results, finding that Meena had an SSA score of 79%, the highest compared to the other chatbots. To give context, humans had an SSA score of 86%, and the second highest score from Mitsuku was 56%.

Meena Chat
Meena Chat. Photo provided by Google.

Google’s Senior Research Engineer Daniel Adiwardana and Senior Research Scientist Thang Luong wrote in the blog post,

“Besides being a fascinating research problem, such a conversational agent could lead to many interesting applications, such as further humanizing computer interactions, improving foreign language practice, and making relatable interactive movie and video game characters…However, current open-domain chatbots have a critical flaw — they often don’t make sense…[Meena] is closing the gap with human performance.”

Google Assistant Is Smartest Voice Assistant

In an 800 question test for Loup Ventures, Google Assistant answered the most questions correctly when compared with Alexa and Siri. While Google Assistant, Google’s voice assistant, answered correctly 92.9% of the time, Siri only answered correctly 83.1% of the time, while Alexa came in last at 79.8%. When it came to understanding queries, Google understood 100% of all queries, Siri, 99.8% and Alexa, 99.9%. The questions covered everything from local information to shopping, information, and navigation.

FAQs

Is Google Assistant a chatbot?

Google Assistant isn’t a chatbot but a voice assistant, primarily.

Is Siri a chatbot?

Siri is both a voice assistant along with a chatbot.

What is the best chatbot?

According to Google, in terms of their Sensibleness and Specificity Average, Meena is the best chatbot. When Google compared their results, finding that Meena had an SSA score of 79%, the highest compared to the other chatbots. To give context, humans had an SSA score of 86%, and the second highest score from Mitsuku was 56%.

Aliza Vigderman

Aliza Vigderman

Aliza is a journalist living in Brooklyn, New York. Throughout her career, her work has spanned many intersections within the tech industry. At SquareFoot, a New York-based real estate technology company, she wrote about the ways in which technology has changed the real estate industry, as well as the challenges that business owners face when they want to invest in property. At Degreed.com, an education technology website, Aliza created digital content for lifelong learners, exploring the ways in which technology has democratized education. Additionally, she has written articles for The Huffington Post as well as her own content on Medium, the online publishing platform. Aliza’s love of journalism and research stems from the excellent Journalism program at Brandeis University. At Brandeis, Aliza interned as a research assistant at the Schuster Institute for Investigative Journalism, a non-profit “news room without walls”. There, Aliza was paired with an investigative journalist and used academic databases to obtain data on everything from the suicide rates in Bhutan to local Boston court cases. Her last position was as an account executive at Yelp, educating business owners on the power of technology to increase revenue. Throughout, however, her heart remained with tech journalism, and she’s thrilled to be writing for Security Baron. When she’s not keeping afloat of the latest tech trends, Aliza likes to cook, read, and write. A former high school “Class Clown,” Aliza has completed two feature-length screenplays, a pilot, and countless comedic sketches. On her days off you can find her relaxing in Prospect Park, trying the latest flavors at Ample Hills Ice Cream, and spending time with friends and family.