At Meta’s Connect event, the primary highlight was the introduction of Orion, the company’s long-anticipated AR glasses. These prototypes, each reportedly costing around $10,000, are not expected to be available to the public in the near future.
Additionally, Meta showcased its new range of holographic avatars. These lifelike holograms are designed to facilitate conversations in augmented reality. The holograms leverage Codec Avatars, a technology Meta has been developing for several years. This technology was previously showcased by Mark Zuckerberg during a podcast interview conducted “in the metaverse” last year.
Post-keynote, Mark Rabkin, a vice president at Meta overseeing Horizon OS and Quest, provided further insights into Meta’s codec avatars and discussed their potential integration with the company’s VR headsets.
Rabkin noted, “Generally, pretty much everything you can do on Orion you can do on Quest.” He highlighted that the creation process for Codec Avatars has become significantly simpler. Previously dependent on complex camera scans, these avatars can now be produced with phone scans.
He explained the process: “It’s an almost identical process in many ways in generating the stylized avatars [for VR], but with a different training set and a different amount of computation required. For the stylized avatars, the model has to be trained on a lot of stylized avatars and how they look and how they move. [It has to] get a lot of training data on what people perceive to look like their picture, and what they perceive to move nicely.”
Regarding Codec avatars, Rabkin stated that the data collection process is extensive, involving high-quality camera scans and phone scans. The goal is to develop a model that eventually becomes more efficient. The major challenge lies in making this technology fast and computationally affordable enough for widespread use.
Rabkin anticipated that these avatars would eventually be compatible with the company’s VR headsets. Currently, the Quest 3 and 3S lack the necessary sensors, such as eye tracking, required for photorealistic avatars. However, he suggested that this could change with the next-generation VR headset: “I think probably, if we do really well, it should be possible in the next generation [of headset].”
