Researchers at Facebook Reality Labs and the University of Arizona have published new work exploring the use of high-speed mechanical display shifting to reduce the so-called screen gate effect (SDE) of displays immersive. SDE is caused by unlit spaces between pixels leading to the appearance reducing the immersion of a “screen door” between the viewer and the virtual world. Researchers are experimenting with moving the entire screen quickly and carefully so that the pixels on the screen fill in the gaps.
SDE has been one of the main visual artifacts in modern VR headsets since the introduction of the Rift DK1 development kit in 2013. While SDE can be defeated by brute force using very high density screens – in which the spaces unlit between the pixels are too small to the naked eye – most consumer VR headsets still feature an EDS today (with the exception of G2 reverb), affecting immersion and visual clarity.
Beyond ultra high pixel density, other methods have been used to reduce SDE. For example, some helmets choose a smaller field of view which reduces the apparent visibility of the SDE. Other headsets use a diffuser film on the screen to help blend the light from the pixels into the unlit spaces between them.
Another proposal is to quickly and carefully shift the display so that nearby pixels fill the unlit spaces. While this may appear to create the appearance of a dizzying jerky screen, it has been shown with other display technologies that moving a point of light (i.e. a pixel) fairly quickly can create the appearance of a stable image.
Researchers Jilian Nguyen, Clinton Smith, Ziv Magoz, and Jasmine Sears from the University of Arizona and Facebook Reality Labs Research explored and experimented with the idea in an article titled Reduced Screen Door Effect Using Mechanical Shift For Virtual Reality Screens.
Rather than building a VR headset with a mechanical display that moves right out of the door, the aim of the paper was to demonstrate and quantify the effectiveness of the method.
Operation and display modes
The researchers designed a static platform with two piezoelectric actuators that together move the display in a circular motion at 120 Hz – in fact, each pixel traces a 10 µm circle 120 times per second. The size of the circle was chosen according to the distance between the pixels of the screen in order to optimally fill the unlit spaces between the pixels. Researchers call this circular path “non-redundancy” mode.
They also cleverly used a 480Hz display which allowed them to experiment with a more complex pixel shifting path that they called “ redundancy ” mode. This approach was intended not only to fill in the gaps between pixels with some additional overlap, but to divide the displayed frame into four sub-frames which are each shifted and displayed uniquely to account for pixel movement. This means that when a pixel moves to a location where it would fill the SDE space, it uses the correct color that would have be used if a pixel was located in that position in the first place.
Although the article is limited to exploring these two pixel paths, the researchers say that others could be used depending on the display characteristics.
“Pixel shift isn’t just a circular shape. Indeed, an elliptical trajectory or even a figure-eight trajectory could be used by controlling the amplitude of the movement of each axis. Paths can be traced in several ways to explore screen door reduction, ”the researchers wrote. “For the micro OLED display, a circular path was well suited to square pixel and subpixel layouts. This path is used to balance the path length with the fill factor, minimizing the speed at which the actuators must operate. “
With the platform built and able to move the display quickly into desired paths, the next step was to objectively quantify the amount of SDE reduction, which proved difficult.
Quantitative measurement of the mechanical reduction of SDE
The authors initially sought to objectively measure the start and end of each subpixel, but found that the resolution of the camera they were using for the task was not fine enough to clearly delineate the start and end. end of each subpixel, not to mention the spaces between them.
Another approach to quantify the reduction in SDE was to measure the contrast ratio of a section of the screen and compare it to when the screen activation was turned on or off. Lower contrast would imply less SDE due to the displacement of pixels filling the unlit spaces and creating a more solid image. Although the authors argue that this measure does not necessarily reflect the reduction in EDS as seen with the naked eye, they believe it to be a significant quantitative measure.
Qualitative evaluations of the mechanical reduction of SDE
Beyond their efforts to quantitatively measure the reduction in EDS, the researchers also wanted to qualitatively look at the change. The clearest demonstration of the benefits came from looking at a natural photo with intricate landscapes.
Here, the “Non-redundancy” mode clearly reduced the SDE while apparently maintaining even sharpness. Impressively, the “ Redundancy ” mode not only reduced the SDE, but even seemed to make the image noticeably sharper (note the enlarged sections showing detail on the rear of the car).
The sharpness of the image of the ‘Non-redundancy’ mode is an interesting additional advantage because increase resolution screen power without increasing the number of pixels.
Based on their experimentation, the researchers also suggest an approach to study users for future surveys that could be used to quantify any SDE reduction method, be it mechanical displacement, diffusers, or different layouts. and subpixel optics.
The researchers conclude:
Using the mechanical pixel displacement for screen gate reduction, the display dead space must be characterized to define the required path shape and offset distance of the mechanical shift system. With proper application of mechanical movement, SDE can be qualitatively reduced. A promising method of quantifying screen door visibility uses natural scenes and human subjects to determine the magnification at which BDS and screen door reduction artifacts become noticeable.
– – – – –
While the brute-force approach of beating SDE with ultra-high pixel density displays is likely to materialize, a mechanical approach to reducing SDE could allow headset manufacturers to “ get more for less ” by increasing the effective resolution of their screen while reducing the SDE. It could also have indirect benefits for display design, as display manufacturers would be less constrained by the need to achieve exceptionally high fill factors.