A deeper look into Samsung's Live Translate (3 more languages added to the sci-fi feature)

Samsung's Live Translate feature is now supporting three more languages, for a total of 16 languages! Here are the three new additions:

1. Arabic
2. Indonesian
3. Russian

They join the already supported Chinese (simplified), English, French, German, Hindi, Italian, Japanese, Korean, Polish, Portuguese, Spanish, Thai, and Vietnamese.

Let's take a deeper look into Samsung's Babel Fish. Sorry, sorry – into Samsung's Live Translate feature!

The Babel Fish is, of course, part of Douglas Adams' The Hitchhiker's Guide to the Galaxy. A remarkable, yellow, leech-like creature that feeds on brainwave energy and has the unique ability to translate any language instantaneously. When inserted into the ear of a person who speaks a different language, the Babel fish allows them to understand and be understood by others, effectively breaking down the language barrier between different species and cultures.

The Babel Fish comes to mind when talking about the Galaxy S24 series – Samsung's 2024 flagships came with the Live Translate feature, part of the Galaxy AI pack.

In a nutshell, Live Translate provides a real time translation during a live phone call. Live Translate listens to what the other side is saying and after a short time, translates that into a language you can understand.

Thanks to the One UI 6.1 update, Samsung expanded the Galaxy AI beyond the Galaxy S24 series. Live Translate (and other useful features, like Circle to Search) are now supported by:

• Galaxy S23 series
• Galaxy Z Flip 5
• Galaxy Z Fold 5
• Galaxy Tab S9 series

Once July comes and with it – the summer Galaxy Unpacked event – I expect Live Translate to be available on these phones as well:

• Galaxy Z Fold 6
• Galaxy Z Flip 6

What's up at Samsung Research's Global AI Center

Yoonjung Choi and Yonghyun Ryu from Samsung Research’s Global AI Center explain more about the fascinating AI features like Live Translate.

On-device AI is the key differentiator to Galaxy AI’s Interpreter (another great AI tool for translating, though not used for phone calls) and Live Translate features. Utilizing the advanced computing resources within smartphones, on-device AI ensures privacy by operating without reliance on external servers or the cloud.

"We collected colloquial (casual) data for real-time translation during calls in Live Translate and travel-related data for Interpreter," explained Yoonjung Choi, who led the project. "To provide the most accurate translation, we studied and incorporated casual language used in chatrooms and HTML tags used in web browsers".

The Samsung Research team’s AI translation model is based on deep learning technology that learns from its own data. Yonghyun Ryu, who is in charge of AI research and development, likened this process to raising a child. "Similar to how a child needs excellent educational resources and caregivers to grow and thrive, good language data and talented researchers are required when developing a high-performance AI translation model", he described.

Challenges and wins

As you can imagine, training an AI to translate adequately 13 (now 16) languages is no joke.

Each language carries unique cultural characteristics, encompassing honorifics, tonal variations, and specific punctuation rules. The AI translation model carefully considers these linguistic nuances to ensure precision in translations. For instance, in languages like Korean and Japanese, honorifics are preserved to maintain a respectful tone.

Samsung collaborated with regional R&D centers to gain a comprehensive understanding of languages within their cultural context. "By working closely with researchers and linguists in other countries, we were able to offer a more accurate and complete translation" stated Choi.

However, navigating diverse languages often involves trial and error. Choi noted, "Vietnamese, for instance, is a tonal language. However, we realized during the research process that Vietnamese users often omit tones in casual conversations when chatting," she explained. "We needed additional data to help the features translate sentences without tones".

Similarly, for Thai, Samsung Research devised a unique sentence separator since the language lacks standard punctuation usage.