Zuckerberg’s new challenge: the viewers that challenge the Turing test
Three prototypes have been announced that condense the latest research carried out by Reality Labs on VR visualization systems. By the end of the year the presentation of Project Cambria, the first device for the metaverse
Jun 21st, 2022
Veronica Balocco
By the end of the year Meta, already Facebook, will officially unveil Project Cambria, its first viewer for the metaverse. But well before that, Mark Zuckerberg wanted to take stock with the international press on the work done by the Reality Labs division on the merits of new technologies that will make it possible to access 3D worlds. The opportunity to talk about it was the round table ‘Inside the Lab: passing the visual Turing test’ which saw the participation of Zuckerberg and the Chief Scientist of Reality Labs, Michael Abrash.
Current VR headsets offer incredible 3D visual experiences but, in many ways, the experience is still different from what we see in the real world. To keep the promise of the metaverse made by Mark Zuckerberg last fall, it is necessary to build an unprecedented VR display system : a viewer that is light and so advanced that it can offer visual experiences as vivid and detailed as those of reality. In order to achieve this result, the research team on the visualization systems of Reality Labs Research is building a new technological system that includes: ‘varifocal’ technology that ensures the correct focus and allows a clear and optimal vision, at the right distance, for prolonged periods; resolution that approaches or exceeds 20/20 of human vision; Hdr technology (High dynamic range) that expands the range of colors, brightness and contrast that can be experienced in Vr; distortion correction to help solve optical aberrations, such as deformation and color fringes, introduced by display optics.
Index of topics
• New challenges on the physical perception of things
• Still a long way to go towards visual realism
• The comparison with the real world
• The three prototypes
New challenges on the physical perception of things
“Today, we’re going to talk about what it takes to create next-generation screens for virtual and augmented reality. Having to make 3D screens that show images as vivid and realistic as those of the physical world will lead us to face some of the fundamental challenges that we will examine today. These are quite interesting issues, because they all affect our physical perception of things, the way our eyes process visual signals, and how they are interpreted by our brain to build a model of the world.”
“Screens that can fully reproduce the breadth of human vision will allow us to achieve truly fundamental results. The first is a realistic sense of presence, that is, the feeling of being with someone or in a place as if we were really there. If we think about how important it is for us to help people create bonds, the great significance of this goal is immediately evident. Another reason that makes these realistic screens so important is their ability to pave the way for a generation of unprecedented visual experiences, which will help make the history of technology and culture over time. We have seen how our culture evolves to integrate the levels of completeness and depth that technology is able to offer. This inevitably leads to access to new forms of art and individual expression. It won’t take much longer to create scenes with virtually perfect fidelity.”
The road to visual realism is still a long way
Zuckerberg then recalled how “current VR systems are already able to offer the feeling of being elsewhere and it is difficult to explain how deep this feeling is. To understand it, you have to try it firsthand.” However, “as far as screens and graphics stacks are concerned, we still have a long way to go before we get to visual realism.” Once a whole range of technical aspects have been resolved, he added, “of course, you need to integrate everything into a device that is comfortable to wear. Ultimately, if all these elements are not implemented and combined optimally, it will be impossible to achieve a feeling of immersiveness that is much higher than that offered by current 2D screens. We have already solved some of these challenges, while for others the road ahead is still long. However, today we will focus only on screens, that is, the last link in the chain. Screens are the components that convert the final graphic result into the photons that the eyes detect. This is obviously an important phase and trying to imagine what it took to make it better was an inspiration for the creation of what we internally call ‘visual Turing test’”.
The comparison with the real world
In this regard, Zuckerberg introduced Michael Abrash: “The visual Turing test, as we call it along with many other academic researchers, is a way to assess whether what is displayed in VR is distinguishable from the real world. It is a completely subjective test because the important element is the human perception of what is seen (the human experience), rather than technical measurements. There is currently no VR technology that can overcome it.” Abrash noted that ‘currently, the most obvious limit is resolution, but the challenges to be overcome are much more numerous. VR introduces new problems that simply do not exist with 2D screens, namely the vergence-accommodation conflict, chromatic aberration, ocular parallax, pupil displacement. And before that, we have to face the challenge of inserting Ar/Vr screens into lightweight and compact viewers and ensuring a long battery life: a decidedly difficult task”.
Another fundamental aspect on which we are working with good results in the new prototypes, explained Zuckerberg and Abrash is the depth of focus. Zuckerberg then explained that “the last big frontier of screen technologies is an important frontier because although resolution, varifocal mode and distortion all make a significant contribution to realism, the probably most incisive factor is the wide dynamic range or Hdr. Essentially, this is the overall level of brightness and contrast of a screen. Experience tells us that, when the light is stronger, the colors stand out and the shadows are darker and it is at that moment that the scenes really begin to come to life. The problem is that the brightness of modern screens is incredibly higher than what the eye sees in the physical world.”
The three prototypes
Zuckerberg then showed three prototypes of viewers.
The first, codenamed Butterscotch, is equipped with a resolution sufficient to guarantee in virtual reality a vision equal to ten tenths of the tables normally used for eye examinations.
Then Holocake 2, which the CEO describes as “the thinnest and lightest headset we’ve ever made and compatible with all existing computer games.” The problem with Holocake 2 is that it requires the use of specialized lasers, still too thick and expensive to be integrated into a mass-market accessory.
The third device shown is a close relative of Meta’s long-awaited goggles, at least in forms. It’s called Mirror Lake, it takes a pair of ski goggles and integrates not only the technology of Holocake 2 but also the others that Meta has been working on for the last seven years.