How to Use Additional Content in a VR Reference Environment During Temporal Measurements

When testing the Motion-to-photon (MTP) latency of a Head-mounted display (HMD), variables like CPU/GPU load and network bandwidth can affect the result. Therefore, it is quite common that users would like to utilize certain visual content with the HMD during the test, to better match the actual conditions where the device is being used. But there is a fundamental challenge with additional content as the image on the screen is the exact source for the virtual world viewer orientation and location information. And to be able to get that information from an image, certain reference content needs to be viewed.  

Let us take OptoFidelity Buddy as an example. To be able to measure the temporal characteristics of an HMD, you need to have a reference “world” displayed in the HMD from which you analyze the virtual location and orientation of the viewer. In the case of the BUDDY that virtual world has thousands of circular markers, set up in a spherical layout. The virtual camera (or the viewer) is located in the center of the sphere. From the markers visible in each frame, the corresponding view direction can be studied. The virtual world pose can then be compared to the actual real-world HMD pose.  

Adding desired content to this kind of environment might seem like an easy thing to do. But what if the markers were located 2km away from the viewer in the virtual world? The additional content would most certainly be located at least partially closer and therefore blocking the visibility to markers.

On the left: VR content for temporal measurement. On the right: Additional content object obstructing visibility to measurement content.

How could one establish a universal method for allowing any additional content to be presented in a way that in the center of each rendered frame, you could always see the markers, which are very far away? And not only that, but you would always see the markers clearly and with enough luminance to allow the frame capture triggering. 

This was the challenge presented to OptoFidelity’s Senior Software Designer Jouni Mäkitalo. Jouni came up with a perfect solution. First, the desired additional content is rendered. After that, a whiteboard is rendered, for each frame in the exactly same position and orientation from the viewer's perspective. Now the clever part is that the depth testing for the whiteboard is turned off and therefore, regardless of whether there is something located between the board and the viewer, the board will always be on top of previously rendered content. 

On the left: The desired additional content. On the right: A whiteboard rendered in the front with depth testing off.

In the last phase, the marker content is rendered in its original form, a spherical constellation with a 2km radius. Again, the depth testing is turned off leaving the markers perfectly visible on top of the whiteboard while still acting like they are 2 km far when moving the HMD.

On the left: Measurement markers rendered to the original location with depth testing off. On the right: Green dots in the center of markers indicate that the markers have been identified by the analysis software.  

With this kind of method, it is now possible to test and measure the MTP latency in conditions closer to real HMD use. While the plain low CPU load constellation content is useful in certain steps of the HMD device development, users like wireless network providers require more end-user type of content.

With more than 18 years of experience, OptoFidelity is the global leader in designing and manufacturing automated test systems for measuring the optical and tracking performance of AR/VR systems.

Contact us to discuss and find the right solution for your needs!

Planning to attend SPIE AR|VR|MR 2023?

Come and meet our experts at booth #102 to talk more about OptoFidelity's testing capabilities.

Unable to attend SPIE AR|VR|MR 2023? OptoFidelity will be also exhibiting at AWE USA 2023 and Display Week 2023. We are looking forward to meeting you there!