Google’s Prototype AI translator Translates Your Tone As Well As Your Words We all know that communication depends on more than just what you say. How you say it is often just as important. That’s why Google’s latest prototype AI translator doesn’t just translate the words coming out of your mouth, but also the tone and cadence of your voice.
The system is called Translatotron, and Google’s researchers go into detail on how it works in a recent blog post. They don’t say that Translatotron will be coming to commercial products any time soon, but that will probably happen in time. As Google’s head of translation explained to The Verge earlier this year, the company’s goal at the moment is to add more nuance to its translation tools, creating more lifelike speech.
Google’s Prototype AI translator Translates Your Tone As Well As Your Words
As you can hear, it’s not a seamless translation, but it’s impressive nonetheless. You can listen to many more audio samples from Translatotron here.
Although capturing the inflection of a speaker’s voice is what’s most impressive to laypeople, Translatotron’s attraction for AI engineers is that it translates speech directly from audio input to the audio output without translating it into the usual intermediary text.
This sort of AI model is known as an end-to-end system, because there are no stops for subsidiary tasks or actions. Google says making translation end-to-end produces results faster while avoiding the risk of introducing errors during multiple translation steps
Perhaps even more interestingly, the data the model is processing isn’t raw audio. Instead, it uses spectrogram data, or detailed visualizations of sound. In essence, that means we’re translating speech from one language to another using pictures, which is mind-boggling
As ever with Google’s translation efforts, there’s reason to be skeptical about how systems like this will work in the wild
READ MORE ON: THE VERGE
