What are ASR, NLU, and TTS?

There are three core technologies that power Alexa. It’s not just Alexa, either. All voice assistants such as Siri, Google Home, etc. use these important technologies. They are Automatic Speech Recognition (ASR),  Natural Language Understanding (NLU), and Text To Speech (TTS).

Automatic Speech Recognition (ASR)

ASR is a key part of Alexa’s system. ASR converts speech to text. It can understand what a human is trying to say, which can be tough. Some people have loud voices and others have quiet voices. People may have accents. Maybe there are other people chatting in the same room. This can be a tough job, even for humans. ASR can handle it.

Example:
Let’s take an example of one of my skills, Wheel of Fun, which gives you a random prize, such as a compliment, joke, poem, or fart sound.

First, Alexa will record what you said (also known as your utterance) and send it to ASR.
Then, ASR will convert the recording into some readable text.
Thanks to ASR, the rest of the process can continue.

Natural Language Understanding (NLU)

Another thing to note is NLU.
The main thing that NLU does is that it can understand what we actually mean when we say something like, “Alexa, tell me the time.”

NLU can also understand that there are many ways to phrase a command. For example, let’s say we have my skill Wheel of Fun. Some people will say, “open Wheel of Fun.” Others may say, “Alexa, spin the Wheel of Fun.” More can say, “Alexa, can you please spin the Wheel of Fun for me?” There are many possibilities, and it is practically impossible to figure them all out. NLU saves you from having to spend hours coding every utterance and saves users from having to commit a certain command to memory.

Text To Speech (TTS)

TTS does literally what the name implies. It converts ‘text to speech’! Near the end of the process, Alexa Voice Service (AVS) needs to convert the text into speech so that the Echo device can actually speak our skill’s response! In the end, TTS and ASR basically undo each other.

Here is an example of the entire Alexa cycle:

This is a picture of how Alexa works from Paul Cutsinger’s video.

You have the skill Wheel of Fun.

First, Alexa will record your utterance and send it to ASR. Next, ASR will convert the recording into text and send it to NLU. Then, NLU will figure out you want to use the skill Wheel of Fun. NLU will then send a JSON request, containing the intent, to your Lambda function. After that, your Lambda function will respond back to Alexa Voice Service. Following this, TTS will convert the JSON response into speech.

Finally, Alexa will speak, and depending on your device’s model, an image may start to form on the screen as well.

The difference between ASR and NLU comes up a lot. ASR converts the speech into text, sends it to NLU, and NLU figures out what we really mean to say. I hope this post clarified things for you!

HOW I KNOW THIS STUFF:

I learned most of the stuff above from this great video by Paul Cutsinger:

Here’s another good video that was very helpful by Nuance Developers Learning:

And finally, here is an amazing article by Amazon on the topic.

Written by

Aariv Modi

37   Posts

Aariv Modi is a blogger, programmer, Alexa skill developer, and AI enthusiast. He is in 9th grade at Round Rock High School and has created several Alexa skills and mobile apps. Aariv was recognized as the Voice/AI Pioneer of the Year by Project Voice in 2021 for his contributions to the conversational artificial intelligence industry. He is also the youngest AWS Certified Alexa Skill Builder and AWS Certified Machine Learning professional. Aariv has contributed much of his learning with the Alexa and voice community through his blog and social media channels such as LinkedIn, YouTube, and Twitter. During the COVID-19 lockdown, he taught hundreds of kids how to develop Alexa skills through webinars, camps, and posts on his website. He was the youngest presenter at several international voice and AI conferences, such as Project Voice 2020, VOICE Global 2020, and VOICE Summit 2020.
View All Posts

3 thoughts on “What are ASR, NLU, and TTS?

Leave a Reply

Your email address will not be published. Required fields are marked *