Buried inside Apple’s $3,499 Vision ProVR headset is a feature that continually wows me, but you’ve probably never heard of it. The feature, calledPersonasinvolves two or more users, all wearing Vision Pros, chatting with one another in real time but as virtual replicas.
Now out of beta, Personas are part of Apple’s avatar system for the Vision Pro, creating replicas of yourself via a 3D photo scan.
Taking a scan of myself isn’t a new thing. Some five years ago,I tried telepresence with 3D-scanned avatars on Nreal AR glasses with a company called Spatial. I’ve gotten peeks at Meta’s realistic codec avatars. I explored cartoon avatar telepresence with Microsoft in HoloLens. And I’ve even scanned myself into all sorts of bizarre AI deepfakes using OpenAI’s Sora phone app.
Still, no one is doing anything in VR or AR headsets or glasses as advanced as Apple’s Vision Personas. And we haven’t seen the beginning of how good things could get.
To learn more, I donned an M5 Vision Pro headset and jumped into a FaceTime for an exclusive chat with Apple’s senior director of the Vision Products Group, Jeff Norris, and the senior director of product marketing, Steve Sinclair. The two showed up as Personas in my home office. We wandered in like ghosts when the meeting started, face to face, so to speak. After a few minutes, it felt like we were actually spending time together in person.
Apple doesn’t discuss the future. But Norris and Sinclair did explain some of the very cool 3D tech that makes Personas seem so realistic. As we chatted, I imagined that similar scans could be done on places other than Vision Pro, like maybe your iPhone, which would be accessible to more people.
It’s hard to find another person who has a Vision Pro, but when I have, the eerie sense of someone ghost-walking into my home is like wizardry. Apple’s VisionOS has evolved to allow collaboration between Personas, flexing virtual spaces out for up to five people to see and share virtual objects and apps together. Multiple people in the same room wearing Vision headsets can collaborate with Personas that can beam in remotely, as well.
I’ve dreamed of that Tony Stark-like, Star Wars-hologram telepresence idea for years now. It’s basically here. It’s just walled into very expensive hardware.
Smart glasses haven’t been able to handle the load of avatars like this yet, although AR glasses from Snap and others may be trying soon. My question for Apple is: What technology is making Personas happen, and could it ever appear anywhere else?
Splatting scan technology uses machine learning
In our meeting, Norris explains that Persona technology uses Gaussian splatting to create those surprisingly convincing 3D facial scans. Gaussian splatting is the key tech to many 3D applications right now, often applied to scanning objects or large-scale environments. Meta’s Hyperscape Capture app on Quest can scan whole rooms into 3D-walkable spaces in VR, for example. It knits a 3D image or landscape from a series of 2D images using AI.
What makes Personas unique is the focus on scanning yourself instead of your environment. Using VisionOS 26, Norris showed me the key changes from the earlier Persona versions. The renders can now show greater detail at multiple angles and capture details like jewelry and eyelashes. Bodies and faces are scanned together, which makes the render feel more seamless.
“There’s machine learning involved, but not many people really realize that it’s a concert of networks that come together,” says Norris. “We counted them up, it’s over a dozen, but we actually reduced the number when we moved to this new version of Personas.”
I mentioned the possibility of scanning rooms into Vision Pro down the road (apps like Scaniverse and Polycam already show off 3D scans in headsets). Norris says Apple is already applying Gaussian splatting to the spatial 3D conversions of photos, which now look weirdly immersive in headsets. So, what’s next?
Me in my VisionOS Persona during my first demo of the new version at WWDC earlier this year.
What could this mean for our future sense of virtual identity?
The single Persona I scan and bond to my Apple ID on Vision Pro feels like it’s designed to act as a one-to-one mapping for my virtual self. It’s the closest thing Apple has to a substitute for using a camera to broadcast my actual face, which can’t be done since I’m wearing a headset.
AI companies are already scanning and generating virtual versions of people in increasing numbers of deepfakes, both intentional and unintentional. OpenAI’s Sora app is the most prominent example now, and uses a similar type of face-scanning tech on iPhones to generate a “Cameo” of myself I can lend out to others.
I ask Norris where the line can be drawn going forward. He makes it clear that Apple wants to clearly and securely represent a person in real time, not as a reproduction.
“We have focused Personas on that authentic representation goal,” he says. “We’re trying to grant what I think is a fundamental human wish, which is: ‘I wish you were here.’ That begins by trying to be faithful to how we appear, and how we’re moving, and how we’re emoting as we speak.”
Can I have more than one Persona, or more customization?
Right now, Apple limits you to using one Persona scan at a time, which surprises me. I’d love a variety of Scott Stein avatars in different moods or simply with different clothes. While Apple doesn’t explore identity transformation via scans, I do appreciate the options for realistic glasses, and I’d love to be able to add more accessories.
“People can reenroll or just put on a different shirt and enroll again,” says Norris. “I totally understand why that would be something we’d want. But we’re focusing on just the one at a time right now.”