Low Latency Displays

We are developing a low latency tracking and display system suitable for ultra-low latency, optical see-through, augmented reality, head mounted displays. Our display runs at a very high frame rate, and performs in-display corrections on supplied imagery to account for the latest tracking information and the latency in a standard rendering system.

Images

Publications

	Lincoln, P.; Blate, A.; Singh, M.; State, A.; Whitton, M.; Whitted, T. & Fuchs, H. Scene-Adaptive High Dynamic Range Display for Low Latency Augmented Reality ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (i3D), 2017
	Abstract: For generalized augmented reality to be feasible, the augmenting elements must be visible in varied environments and under rapidly changing, high dynamic range lighting, from bright sunlight to deep shadows. We present a high dynamic range, optical see-through, augmented reality display that dynamically adjusts the brightness of the virtual imagery to match the current brightness of the real scene. Critical components include the spatial brightness sensor array and the positional brightness image intensity matcher. The color, scene-adaptive HDR display system is based on a high-rate (15 kHz) DMD projector using a high-speed RGB LED illuminator, each color with independent 16 bit intensity control for each binary DMD frame. The critical input to the intensity matching algorithm is the output of an array of high sensitivity light sensors. This paper discusses the implementation of the system and reports performance via still and video demonstrations under a variety of lighting conditions.
	[Paper] [Video Download] [YouTube] [BibTeX] [DOI]

	Lincoln, P.; Blate, A.; Singh, M.; Whitted, T.; State, A.; Lastra, A. & Fuchs, H. From Motion to Photons in 80 Microseconds: Towards Minimal Latency for Virtual and Augmented Reality IEEE Transactions on Visualization and Computer Graphics, 2016, 22, 1367-1376 Best Paper Award
	Abstract: We describe an augmented reality, optical see-through display based on a DMD chip with an extremely fast (16 kHz) binary update rate. We combine the techniques of post-rendering 2-D offsets and just-in-time tracking updates with a novel modulation technique for turning binary pixels into perceived gray scale. These processing elements, implemented in an FPGA, are physically mounted along with the optical display elements in a head tracked rig through which users view synthetic imagery superimposed on their real environment. The combination of mechanical tracking at near-zero latency with reconfigurable display processing has given us a measured average of 80 microseconds of end-to-end latency (from head motion to change in photons from the display) and also a versatile test platform for extremely-low-latency display systems. We have used it to examine the trade-offs between image quality and cost (i.e. power and logical complexity) and have found that quality can be maintained with a fairly simple display modulation scheme.
	[Paper] [Video Download] [YouTube] [BibTeX] [DOI]