Home
News
You are here

Google unveils "Expressive Captions" for Android with AI-powered emotion and sound recognition

0comments

By Johanna Romero

Published: Dec 05, 2024, 11:01 AM

An image of a sports broadcast on a foldable device displaying expressive captions

Image credit — Google

Android users can now experience a whole new level of captioning with the release of "Expressive Captions." This new feature goes beyond just showing the words people say — it actually captures how they say them. Imagine being able to see the emotion and intensity in someone's voice, even if you can't hear it.

For years, captions have simply displayed the spoken words, but now, thanks to AI, they can do so much more. Expressive Captions analyze things like tone of voice, loudness, and even background sounds to give you a richer understanding of what's happening. This is especially helpful for live videos and social media posts, where captions are often missing or not very accurate.

An image of a smartphone displaying a football came with expressive captions turned on and off

An example of 'Expressive Captions' turned ON vs OFF | Image credit — Google

One of the most interesting things about the new Expressive Captions is how it uses capital letters to show strong emotions. So, if your friend sends you a birthday message and shouts "HAPPY BIRTHDAY!" you'll see those words in all caps in the captions, just like how we've learned that when using all caps in texts means you're shouting. The feature can also pick up on things like sighs, gasps, and even clapping or cheering in the background, giving you a better sense of the whole scene.

An image showing three examples of how expressive captions work

How Android's new 'Expressive Captions' capture emotion | Image credit — Google

Expressive Captions are built right into the latest Android phones, so they work with pretty much any app where you might watch videos. The feature will be available starting today for any Android that is running Android 14 and above and that has the Live Caption feature turned on. For now, it will only be available in the U.S. in English.

This means you can use them for live streams, videos in your Google Photos, and even video messages from friends. And the best part is, they work instantly and even when you're offline, like on an airplane.

Google says that to create this super helpful feature, the teams at Android and Google DeepMind worked together to figure out how people watch videos without sound. They used special AI technology to not only capture the words but also turn them into captions that show emotion and include all those extra sounds.

This is a big step forward in making sure everyone can enjoy online videos, no matter how well they can hear. It shows how AI can be used to make things better for everyone.

I watch a lot of videos on my phone, so I'm pretty excited about Expressive Captions. Being able to see the emotion and hear the background sounds through the captions would make watching videos in noisy places a much better experience. I can't wait to see if this comes to other languages and regions soon.

$349

$799

$450 off (56%)

Mint Mobile now sells the Google Pixel 10 with a massive $450 discount. The promo is available on select color variants with 128GB of storage. You also get a 12-month unlimited data plan for $180 instead of $360.

Buy at Mint Mobile

$524

$999

$475 off (48%)

Grab the pro-grade, compact Pixel 10 Pro at Mint Mobile with a 12-month unlimited plan, and you can save a huge $475. The data plan comes with a discount, too: 50% off, to be exact.

Buy at Mint Mobile

$499

$1199

$700 off (58%)

The high-end Gemini AI-enhanced Pixel 10 Pro XL is now available with a mind-blowing discount. You can now save $700 on the phone, plus 50% off unlimited 12-month plans.

Buy at Mint Mobile

$1399

$1799

$400 off (22%)

The foldable Pixel 10 Pro Fold is another standout holiday offer. Right now, you can get the device for $400 off at Mint Mobile. On top of that, you save $180 on 12-month unlimited data plans.

Buy at Mint Mobile

View Full Bio

Johanna 'Jojo the Techie' is a skilled mobile technology expert with over 15 years of hands-on experience, specializing in the Google ecosystem and Pixel devices. Known for her user-friendly approach, she leverages her vast tech support background to provide accessible and insightful coverage on latest technology trends. As a recognized thought leader and former member of #TeamPixel, Johanna ensures she stays at the forefront of Google services and products, making her a reliable source for all things Pixel and ChromeOS.

Read the latest from Johanna Romero