Facebook working on custom silicon, Face and eye-tracking for Quest 3 and 4 - Android

Facebook looks to ditch Android, Qualcomm, Apple, and Google in the future.

What you need to know

• Facebook is looking to become less dependent on Android and Qualcomm as it develops its own AR/VR OS and chipsets.
• Oculus Quest 3 and 4 could very likely feature some sort of eye-tracking or face tracking to enhance social apps by creating more realistic avatars.
• Future Quest products should have significantly higher-resolution displays with HDR tech to better match how our eyes perceive reality.

An Interview with Facebook CEO Mark Zuckerberg revealed several details about the future of the Oculus Quest platform, including Facebook's quest to become less dependent on the other FAANG companies in Silicon Valley. The Oculus Quest 2 was the first step in utilizing a more VR-centric SoC from Qualcomm, whereas the original Oculus Quest utilized a similar chipset that was used in mobile phones. Facebook's goal, moving forward, is to focus on creating custom silicon and operating systems to help tightly optimize the experience for form factors that would be "socially acceptable," as Zuckerberg stated.

Those socially acceptable form factors include smaller and lighter hardware that would also run a lot cooler than the current Quest or Quest 2. While those headsets don't get noticeably hot for the user, Zuckerberg says that Facebook's desire to create significantly more powerful chipsets for use in AR and VR hardware means the company has to come up with new ways of cooling things down. In other words, they "don't want to burn people's faces off" as Zuckerberg so eloquently put it. Included with this custom silicon route is a custom operating system (OS) route that sees Facebook at the helm of control of everything, including ways to better optimize the experience than they currently can with Android.

In addition to that, Facebook is continuing to work in better and brighter displays that would more realistically recreate the world outside the headset. Current displays look mostly fine, but the goal is to create a "retinal display" that would make individual pixels imperceptible — something that's certainly not the case with current VR display tech. Even with a more powerful processor, though, Facebook will need to rely on what's known as foveated rendering in order to create sharper imagery without negatively impacting performance. They also continue to work on varifocal lenses, which is a tech that allows users to focus on different objects more naturally than they are able to with current static VR lenses.

As one would assume, Facebook's next user-facing focus is on the social experience with the Quest platform. This is being achieved by utilizing face and eye-tracking which, while that sounds incredibly creepy, is used to create more realistic avatars that have real expressions and can make actual eye-contact with someone else in VR. The idea is to create a realistic avatar that makes you feel like you're really in the room with another person rather than just having some goofy cartoon avatar. Zuckerberg gave a nod to Epic's recently released MetaHuman avatar generator in Unreal Engine, which you can see in the trailer above.

Zuckerberg also talked about spatial audio and its importance in heightening realism, as well as the concept of developing neural interfaces that are non-invasive, like an armband or something similar that you would wear but would be easily removable. The idea is to create more natural interfaces — not unlike hand tracking that's currently done on Quest — by using the body's ability to create additional neural paths, similar to how physical therapy works. They want to avoid something like Elon Musk's Neuralink, which really isn't something meant for consumers since, as Zuckerberg put it, would require consumers to "get their head drilled open."