Multilingual households can now get more out of their Google Assistant.  The developers behind the Google product announced a bilingual feature at the 2018 IFA conference in Berlin. The feature will be made available to users immediately upon upgrading their system.

Users can now select any pair of the following languages to be recognized at all times: English, Spanish, French, German, Italian and Japanese. Family members speaking a different language can speak directly to the Google Assistant. It should be noted that languages cannot be mixed together, “Spanglish,” for example, cannot be recognized. Below is a demo video of the Google Home switching between French and English.

The Technology Behind Bilingualism

Google has been steadily adding new capabilities to the Google Assistant in routines and general functionality. Identifying a different language seems fairly straightforward, but accomplishing this was much harder than it sounds. Teaching the Assistant to switch back and forth between languages was a “multi-year effort” and a massive machine learning challenge as described in a post on Google’s AI blog. The program has to first identify the language, then comprehend the command, and ultimately make the right call for the user experienceall of which has to be done within milliseconds.

To achieve this, the Assistant essentially has to do twice as much work, running two separate monolingual language identification programs at the same time. The technology behind this capability is LangID, a spoken language identification program that uses neural networks trained in speech and speaker recognition, voice detection, and other inputs to distinguish between pairs of languages.

In addition to the LangID models, the system takes into account how often the language is spoken and which device it is being used on in order to respond appropriately.

The upgrade is supported on all devices with the Google Assistant platform. This includes all phones and smart speakers with the exception of the new smart displays that only function in English.

Google plans to add more languages in the coming months and is working on a trilingual solution as the next step

Featured video courtesy of Naresh Ganatra