Apple is going all in on its premium MR (Mixed Reality) headset. WWDC 2024 kicked off with an introduction to visionOS 2, the biggest update we’ve seen for Apple Vision Pro. Apple showed off many new exciting features coming to its headset, and here’s a list of everything you should look out for if you own a Vision Pro.
Photos app getting spatial capabilities and SharePlay
The Photos app on visionOS. | Image credit — PhoneArena
Photos on Vision Pro are getting a big update. Spatial photos and videos — media that has depth to it — have been a big marketing point for Apple. Now, using “advanced machine learning”, Apple has made it so that normal images can be transformed into spatial images in the Photos app. Whether videos can get this as well was not mentioned. My guess would be no, because that would require a lot more processing power and time.
To go along with this, Apple is bringing SharePlay to the Photos app as well. You can share media, as well as spatial media, with friends and family. Other people can also appear in your view as Personas: Apple’s digital recreations of people in MR.
New gestures
New gestures in action on visionOS 2. | Image credit — PhoneArena
Apple introduced two new and improved gestures for getting around visionOS. Turning your hand with the palm facing up and then making a pinching gesture will open the home screen. Doing so with the palm facing down will open up the time and allow you to access notifications.
Improvements to Mac Virtual Display
Virtual Mac displays can now be enlarged quite a lot. | Image credit — PhoneArena
Mac Virtual Display — the option to use your Vision Pro as an external monitor for your Mac — is getting improvements too. With visionOS 2 you will be able to enlarge your external display to insane sizes, going even so far as to curve the display around your head.
To facilitate the processing requirements of such a feat, foveated rendering will be carried out on your Mac. Your Vision Pro will use its eye tracking to let the Mac know what to do. Foveated rendering means areas outside of your direct line of sight will be rendered at a lower resolution. This will help the headset run a gigantic external monitor without sacrificing performance.
Travel Mode for trains
Catching up on all your spatial news before getting to work. | Image credit — PhoneArena
Travel Mode lets AR headsets like the Vision Pro work inside moving vehicles without the tracking going haywire. However, Travel Mode on Vision Pro only supported air travel. With visionOS 2 this compatibility extends to trains as well, which should be useful for large metropolitan areas with extensive public transit.
Third-party mouse support
Up till now the Vision Pro has only supported the Apple Magic Trackpad. Other computer mice, and even Apple’s own Magic Mouse, were not compatible. VisionOS 2 finally brings the ability to connect and use whichever mouse you want to.
This is especially useful for people who use the Vision Pro for work because the Magic Trackpad is just a trackpad. And if mouse support were limited to the Magic Mouse, it would’ve made for a very unergonomic experience.
AirPlay casting to Vision Pro
With visionOS 2 you can now cast content from your other Apple devices to your Vision Pro headset. Apple AirPlay will be used for this and allow you to seamlessly send over content to your Vision Pro for a more immersive viewing experience.
Live Captions
The concept is quite futuristic, let’s hope the execution is too. | Image credit — Apple
Live captions are coming to the Apple Vision Pro. With visionOS 2, your headset will be able to transcribe in real-time the dialogue happening around you. Live captions will also work with audio and video playing on the headset. I think Apple might go the way of Google’s Project Astra and Meta’s Project Nazare with this in the future.
Other new features in visionOS 2
Those were some of the big new features coming to Vision Pro with visionOS 2, but there are lots of other things as well. These are:
A new dedicated Vimeo app
Customizable app menu like on iOS and iPadOS
Cinema Mode will now support web videos
Videos in Cinema Mode can now be tilted upwards
Visible external keyboard in immersive backgrounds
Ability to watch multiple sports games together on Apple TV
Support for panoramic viewing of panoramic photos on the web
Quick Look will now let you alter existing models instead of having to load new ones for minor changes
Apple Communication Safety is coming to Vision Pro
Look-to-Dictate (looking at the microphone icon to bring up speech-to-text) will be coming to Messages
Personas will be getting more accurate skin tones and more vibrant clothing
Personas in windowed mode will now be able to choose their background image
Guest User will remember a guest’s calibration for up to a month
In our review of Apple Vision Pro we said it was an absolutely amazing machine. VisionOS 2 is poised to make the headset even better when Apple rolls out the new update in Fall.
Follow us on social media to catch the latest trending stories, watch exclusive videos, and join the conversation with our vibrant community!
Abdullah is an ardent believer in the future of VR, and can't wait to see what more it brings to the table as technology improves. He enjoys covering new software releases in the XR industry, as well as major news about hardware and companies focused on VR.
A discussion is a place, where people can voice their opinion, no matter if it
is positive, neutral or negative. However, when posting, one must stay true to the topic, and not just share some
random thoughts, which are not directly related to the matter.
Things that are NOT allowed:
Off-topic talk - you must stick to the subject of discussion
Offensive, hate speech - if you want to say something, say it politely
Spam/Advertisements - these posts are deleted
Multiple accounts - one person can have only one account
Impersonations and offensive nicknames - these accounts get banned
Moderation is done by humans. We try to be as objective as possible and moderate with zero bias. If you think a
post should be moderated - please, report it.
Have a question about the rules or why you have been moderated/limited/banned? Please,
contact us.
FCC OKs Cingular\'s purchase of AT&T Wireless
Things that are NOT allowed: