Home
News
You are here

Google unveils "Expressive Captions" for Android with AI-powered emotion and sound recognition

By Johanna Romero

Published: Dec 05, 2024, 11:01 AM

0comments

Android Google

An image of a sports broadcast on a foldable device displaying expressive captions

Image credit — Google

Android users can now experience a whole new level of captioning with the release of "Expressive Captions." This new feature goes beyond just showing the words people say — it actually captures how they say them. Imagine being able to see the emotion and intensity in someone's voice, even if you can't hear it.

For years, captions have simply displayed the spoken words, but now, thanks to AI, they can do so much more. Expressive Captions analyze things like tone of voice, loudness, and even background sounds to give you a richer understanding of what's happening. This is especially helpful for live videos and social media posts, where captions are often missing or not very accurate.

An image of a smartphone displaying a football came with expressive captions turned on and off

An example of 'Expressive Captions' turned ON vs OFF | Image credit — Google

One of the most interesting things about the new Expressive Captions is how it uses capital letters to show strong emotions. So, if your friend sends you a birthday message and shouts "HAPPY BIRTHDAY!" you'll see those words in all caps in the captions, just like how we've learned that when using all caps in texts means you're shouting. The feature can also pick up on things like sighs, gasps, and even clapping or cheering in the background, giving you a better sense of the whole scene.

An image showing three examples of how expressive captions work

How Android's new 'Expressive Captions' capture emotion | Image credit — Google

Expressive Captions are built right into the latest Android phones, so they work with pretty much any app where you might watch videos. The feature will be available starting today for any Android that is running Android 14 and above and that has the Live Caption feature turned on. For now, it will only be available in the U.S. in English.

This means you can use them for live streams, videos in your Google Photos, and even video messages from friends. And the best part is, they work instantly and even when you're offline, like on an airplane.

Google says that to create this super helpful feature, the teams at Android and Google DeepMind worked together to figure out how people watch videos without sound. They used special AI technology to not only capture the words but also turn them into captions that show emotion and include all those extra sounds.

This is a big step forward in making sure everyone can enjoy online videos, no matter how well they can hear. It shows how AI can be used to make things better for everyone.

I watch a lot of videos on my phone, so I'm pretty excited about Expressive Captions. Being able to see the emotion and hear the background sounds through the captions would make watching videos in noisy places a much better experience. I can't wait to see if this comes to other languages and regions soon.

View Full Bio

Johanna 'Jojo the Techie' is a skilled mobile technology expert with over 15 years of hands-on experience, specializing in the Google ecosystem and Pixel devices. Known for her user-friendly approach, she leverages her vast tech support background to provide accessible and insightful coverage on latest technology trends. As a recognized thought leader and official member of #TeamPixel, Johanna ensures she stays at the forefront of Google services and products, making her a reliable source for all things Pixel and ChromeOS.

Loading Comments...

Google unveils "Expressive Captions" for Android with AI-powered emotion and sound recognition

Recommended Stories

Popular stories

Latest News