Meta’s Orion holographic avatars will (finally) be in VR too

The big reveal at Meta’s Connect event was its long-promised AR glasses, the Orion. As expected, the prototype, each reportedly costing around $10,000, won’t be ready for the public anytime soon.

Meanwhile, Meta gave a glimpse of its new holographic avatars, which will allow people to talk to lifelike holograms in augmented reality. The holograms are Meta’s Codec Avatars, a technology it has been working on for several years. Mark Zuckerberg teased the genre last year when he participated in a podcast interview “in the metaverse.”

That technology may now be closer than we think. After the keynote at Connect, I sat down with Mark Rabkin, VP at Meta leading Horizon OS and Quest, who shared more about Meta’s codec avatars and how they will one day come to headsets. -VR in the company.

“Basically, everything you can do in Orion you can do in Quest,” Rabkin said. Codec Avatars in particular are also very easy to create. While they once required advanced camera scans, many indoor avatars are now created with phone scans, Rabkin explains.

“It’s a very similar process in many ways to producing stylized avatars [for VR]but with a different training set and a different amount of computation required,” Rabkin explained. “For stylized avatars, the model has to be trained on many stylized avatars and how they look and how they move. [It has to] she got a lot of training information about what people see as their image, and what they see as good.”

“For Codec avatars … it’s the same process. You collect a large amount of data. You collect data from the highest quality cameras, the best cameras. You collect data from a phone scanner, because that’s how people will be creating, and you just build the model until it’s better. And one of the challenges with both of these problems is to make it fast enough and cheap enough for millions and millions to be able to use it.”

Rabkin said he eventually expects these avatars to be able to play virtually on the company’s headsets. Currently, the Quest 3 and 3S lack the necessary sensors, including eye tracking, needed for photorealistic avatars. But that could change in next-generation VR headsets, he said: “I think, if we do really well, it should happen in the next generation. [of headset].”