It’s here at last! Well .. not until next year and in the US first .. but it’s fair to say that Apple’s Vision Pro mixed reality headset is set to energise the AR/VR category.
Apple’s long awaited mixed reality headset was the last announcement at today’s WWDC keynote – also known as ‘one more thing’. Apple Vision Pro starts at US$3,499. This is where the Pro comes into the name. It’s well beyond the price of HTC and Meta models but it’s also more capable judging from the presentation we saw this morning, Australian time. Apple Vision Pro will be available early next year on apple.com and at Apple Store locations in the U.S – with more countries coming later next year. Customers will be able to try it out and personalise their fit for Vision Pro at Apple Store locations.
“Today marks the beginning of a new era for computing,” said Tim Cook, Apple’s CEO.
“Just as the Mac introduced us to personal computing, and iPhone introduced us to mobile computing, Apple Vision Pro introduces us to spatial computing.
“Built upon decades of Apple innovation, Vision Pro is years ahead and unlike anything created before — with a revolutionary new input system and thousands of groundbreaking innovations. It unlocks incredible experiences for our users and exciting new opportunities for our developers.”
Apple calls the Vision Pro ‘a revolutionary spatial computer’ and from Apple’s point of view and non VR users .. it truly is. Apple’s new category looks remarkably polished. I’ve been a big critic of VR/AR in recent years. I’ve been frustrated with the lack of progress in the field which I’ve been covering for many years now.
PROCESSING POWER
The Vision Pro headset may look like expensive ski goggles but they are packed with new Apple technology. Featuring visionOS, the world’s first spatial operating system, Vision Pro lets users interact with digital content without removing the person from reality. Vision Pro features an ultra-high-resolution display system that packs 23 million pixels across two displays – that’s enough for the equivalent of two 4K TV screens per eye. This is critical for viewer comfort but not as important as processing power.
There’s a bunch of sensors, M2 processor and brand new R1 chip, made to handle the input sensory data the headset collects. The powerful M2 chip simultaneously runs visionOS, executes advanced computer vision algorithms, and delivers stunning graphics, all with incredible efficiency.
And the brand-new R1 chip is specifically dedicated to process input from the cameras, sensors, and microphones, streaming images to the displays within 12 milliseconds — for a virtually lag-free, real-time view of the world. A pair of high-resolution cameras transmit over one billion pixels per second to the displays so you can see the world around you clearly. There’s a digital crown so Apple Watch users will be familiar with its use.
NO HAND CONTROLLERS
The system also helps deliver precise head and hand tracking and real‑time 3D mapping, all while understanding your hand gestures from a wide range of positions. It’s untethered from a computational point of view but it does have an external battery. It’s good for two hours of use and doesn’t look like it’s a problem to use.
A high-performance eye tracking system of LEDs and infrared cameras projects invisible light patterns onto each eye. This system provides ultraprecise input without your needing to hold any controllers, so you can accurately select elements just by looking at them.
To enable user navigation and interaction with spatial content, Apple Vision Pro introduces a new input system controlled by a person’s eyes, hands, and voice. Users can browse through apps by simply looking at them, tapping their fingers to select, flicking their wrist to scroll, or using voice to dictate.
ENHANCED SPATIAL AUDIO
Dual-driver audio pods positioned next to each ear deliver personalized sound while letting you hear what’s around you. Ambient Spatial Audio makes sounds feel like they’re coming from your surroundings. And with audio raytracing, Vision Pro analyses your room’s acoustic properties — including the physical materials — to adapt and match sound to your space.
AUGMENTED REALITY
Apple Vision Pro also features EyeSight. This helps users stay connected with those around them. When a person approaches someone wearing Vision Pro, the device feels transparent — letting the user see them while also displaying the user’s eyes. When a user is immersed in an environment or using an app, EyeSight gives visual cues to others about what the user is focused on.
RECORD IMMERSIVE VIDEOS
Featuring Apple’s first three-dimensional camera, Apple Vision Pro lets users capture, relive, and immerse themselves in favourite memories with Spatial Audio. Every spatial photo and video transports users back to a moment in time, like a celebration with friends or a special family gathering. Users can access their entire photo library on iCloud, and view their photos and videos at a life-size scale with brilliant colour and spectacular detail. Every Panorama shot on iPhone expands and wraps around the user, creating the sensation they are standing right where it was taken.
BIG SCREEN ENTERTAINMENT AND WORK
With two ultra-high-resolution displays, Apple Vision Pro can transform any space into a personal movie theatre with a screen that feels 30 metres wide and an advanced Spatial Audio system. Users can watch movies and TV shows, or enjoy stunning three-dimensional movies. Apple Immersive Video offers 180-degree high-resolution recordings with Spatial Audio, and users can access an exciting lineup of immersive videos that transport them to entirely new places.
Spatial computing makes new types of games possible with titles that can span a spectrum of immersion and bring gamers into all-new worlds. Users can also play over 100 Apple Arcade games on a screen as large as they want, with incredible immersive audio and support for popular game controllers.
Environments allow a user’s world can grow beyond the dimensions of a physical room with dynamic, beautiful landscapes that can help them focus or reduce clutter in busy spaces. A twist of the Digital Crown lets a user control how present or immersed they are in an environment.
USE FACETIME WITH APPLE GENERATED AVATAR
FaceTime calls take advantage of the room around the user, with everyone on the call reflected in life-size tiles, as well as Spatial Audio, so it sounds as if participants are speaking right from where they are positioned. Users wearing Vision Pro during a FaceTime call are reflected as a Persona — a digital representation of themselves created using Apple’s most advanced machine learning techniques — which reflects face and hand movements in real time. Users can do things together like watch a movie, browse photos, or collaborate on a presentation.
DESIGN
Apple Vision Pro looks like a mix between Quest and HTC Vive headsets. A singular piece of three-dimensionally formed and laminated glass is polished to create an optical surface that acts as a lens for the wide array of cameras and sensors needed to blend the physical world with digital content. The glass flows into the custom aluminium alloy frame that gently curves around the user’s face, while the modular system allows for a tailored fit to accommodate a wide range of people. The Light Seal is made of a soft textile, and comes in a range of shapes and sizes, flexing to conform to a user’s face for a precise fit. Flexible straps ensure audio remains close to the user’s ears, while a Head Band — available in multiple sizes — is three-dimensionally knitted as a single piece to provide cushioning, breathability, and stretch.1 The band is secured with a simple mechanism, making it easy to change to another size or style of band.
WHAT IF I WEAR GLASSES?
Users with vision correction needs will use ZEISS Optical Inserts to ensure visual fidelity and eye tracking accuracy.2
PRIVACY
Optic ID is a new secure authentication system that analyses a user’s iris under various invisible LED light exposures, and then compares it to the enrolled Optic ID data that is protected by the Secure Enclave to instantly unlock Apple Vision Pro. A user’s Optic ID data is fully encrypted, is not accessible to apps, and never leaves their device, meaning it is not stored on Apple servers.
Where a user looks stays private while navigating Apple Vision Pro, and eye tracking information is not shared with Apple, third-party apps, or websites. Additionally, data from the camera and other sensors is processed at the system level, so individual apps do not need to see a user’s surroundings to enable spatial experiences. EyeSight also includes a visual indicator that makes it clear to others when a user is capturing a spatial photo or video.