Home
News
You are here

iOS 15: Apple introduces Live Text, which can locate text inside your photos

By Doroteya Borisova

Published: Jun 08, 2021, 12:54 PM

6comments

iOS Apple

We may earn a commission if you make a purchase from the links on this page.

iOS 15: Apple introduces Live Text, which can locate text inside your photos

Apple is stepping up the game in plenty of areas this year, according to its livestreamed preview of iOS 15 yesterday. The newest annual update to Apple's operating system will enable your iPhone, iPad, or Mac to recognize words or text anywhere inside images on your device, as long as they are visible on-screen.

Live Text on the Camera App

The feature will not only work for past (and future) photos in your camera roll, but is also directly integrated into the camera app, allowing you to look at, copy, and paste text taken directly from a scene in front of you—without necessarily saving a picture you may not need. Talk about a streamlined experience!

This new little Live Text widget will appear on the screen as soon as it detects text anywhere in your environment, in the form of a small yellow circle in the bottom right of the camera app (near the zoom button).

When you click it, the camera will freeze the image, allowing you to select the part of the text that you need with your finger. The options above are immediately available to "copy," "select all," "look up," "translate," and "share" that text.

The "translate" option in particular is part of a huge upgrade to the versatility of Apple's translation capabilities, as iOS 15 will give Apple users the option to translate text from nearly anywhere, including entire web pages and text inside photos, from nearly any language to another.

From Apple's livestreamed visual demonstration, it seems that when you use Live Text to extract text from photos, even details in formatting—such as bullet points—will make their way into your clipboard, so that the text will look as polished as possible wherever you choose to paste it.

If you're trying to save a copy of the professor's notes from the whiteboard in class, for example, Live Text will allow you to instantly copy and save them into your iCloud Notes (or anywhere else, for that matter), which will be immediately synced and ready for viewing from your MacBook.

If only we'd had this back in the days when we had to copy the scribbles on the blackboard, word by word, into a physical notebook. (But then again, they say it's easier to commit something to memory by handwriting it, so there's a balance to everything...)

Live Text on the Photos App

As mentioned earlier, on the iOS 15 launch day, the Live Text function will be applied to all photos you already have in your camera roll, retrospectively. Apple will analyze them, discover any visible text in the background or foreground, and prepare it for your easy selection.

Unlike with the camera app, in the Photos app, you don't always have to press the dedicated Live Text button to activate text recognition.

You can simply treat the text in a photo as you would any regular text: simply hold your finger down on it for a moment, then move your finger to the right or to the left to select what you need. It apparently doesn't matter if the text is sloping or vertical—your device will locate and select it all the same.

The "look up" option lets you instantly search a text or name in Safari, the App Store, or Maps. For example, if you come upon a friend's photo taken in front of an awesome-looking ice cream shop you suddenly feel like visiting, you can instantly select the store's name in the picture, choose "look up," and find out all you need about the place—menu, hours, even directions—at the drop of a dime.

Live Text Will Find Phone Numbers in Photos

If you thought that was it, you're wrong. Apple is also integrating phone-number recognition into the upcoming Live Text feature, letting you call or text a number from a photo with just a couple of taps.

First, toggle the Live Text button in the bottom right. This will force your device to bring out any outstanding features—if it sees a phone number, that will become underlined. Once you tap the underlined number, you'll see a drop-down list of various options for whether you want call, message, FaceTime/FaceTime audio, add to contacts, or copy that number.

Live Text Works Across the System

Apple promises Live Text will work on any text, anywhere, as long as it is visible on the screen. This includes screenshots, QuickLook, and even images you come across while surfing the web.

Live Text Supports Seven Languages (Currently) and Three Devices

At the moment, Live Text is able to select on-screen text in any of the following seven languages (but surely more in the future):

English
Simplified and Traditional Chinese
French
Italian
German
Spanish
Portuguese

It will also be supported on iPhones, iPads and Macs.

Beyond Live Text: Visual Look Up

At the WWDC event, Apple also mentioned the addition of a new Visual Look Up feature, which will be able to recognize a variety of objects and allow you to look up information on them in the blink of an eye. According to Apple, it will easily help you classify things like dog breeds, types of a flower, geographical locations, etc.

However, this particular feature was glossed over as a near side thought during the livestream, leading us to believe that it is not yet in as an impressive or versatile stage of development as Google Lens, which came out with a similar feature back in 2018.