We describe an augmented reality, optical see-through display based on a DMD chip with an extremely fast (16 kHz) binary update rate. We combine the techniques of post-rendering 2-D offsets and just-in-time tracking updates with a novel modulation technique for turning binary pixels into perceived gray scale. These processing elements, implemented in an FPGA, are physically mounted along with the optical display elements in a head tracked rig through which users view synthetic imagery superimposed on their real environment. The combination of mechanical tracking at near-zero latency with reconfigurable display processing has given us a measured average of 80 microseconds of end-to-end latency (from head motion to change in photons from the display) and also a versatile test platform for extremely-low-latency display systems. We have used it to examine the trade-offs between image quality and cost (i.e. power and logical complexity) and have found that quality can be maintained with a fairly simple display modulation scheme.
Paper (Internally Stored): http://telepresence.web.unc.edu/wp-content/uploads/sites/11620/2016/04/lincoln-2016ieeevr-tvcg-lowlatencyproj.pdf Paper (Externally Stored): Video (Externally Stored): http://www.cs.unc.edu/~plincoln/papers/lincoln-2016ieeevr-tvcg-lowlatencyproj.avi Video (Streaming): https://www.youtube.com/watch?v=DvMwDfyjk1E BibTeX:@Article{Lincoln2016,
Title = {From Motion to Photons in 80 Microseconds: Towards Minimal Latency for Virtual and Augmented Reality},
Author = {P. Lincoln and A. Blate and M. Singh and T. Whitted and A. State and A. Lastra and H. Fuchs},
Journal = {IEEE Transactions on Visualization and Computer Graphics},
Year = {2016},
Month = {April},
Number = {4},
Pages = {1367-1376},
Volume = {22},
Abstract = {We describe an augmented reality, optical see-through display based on a DMD chip with an extremely fast (16 kHz) binary update rate. We combine the techniques of post-rendering 2-D offsets and just-in-time tracking updates with a novel modulation technique for turning binary pixels into perceived gray scale. These processing elements, implemented in an FPGA, are physically mounted along with the optical display elements in a head tracked rig through which users view synthetic imagery superimposed on their real environment. The combination of mechanical tracking at near-zero latency with reconfigurable display processing has given us a measured average of 80 microseconds of end-to-end latency (from head motion to change in photons from the display) and also a versatile test platform for extremely-low-latency display systems. We have used it to examine the trade-offs between image quality and cost (i.e. power and logical complexity) and have found that quality can be maintained with a fairly simple display modulation scheme.},
Doi = {10.1109/TVCG.2016.2518038},
ISSN = {1077-2626},
Keywords = {augmented reality;computer displays;field programmable gate arrays;rendering (computer graphics);DMD chip;FPGA;augmented reality;binary pixels;binary update rate;display modulation scheme;end-to-end latency;extremely-low-latency display systems;head tracked rig;image quality;just-in-time tracking updates;mechanical tracking;minimal latency;modulation technique;near-zero latency;optical display elements;optical see-through display;perceived gray scale;post-rendering 2-D offsets;reconfigurable display processing;synthetic imagery;virtual reality;Delays;Field programmable gate arrays;Graphics processing units;Modulation;Optical imaging;Rendering (computer graphics);Tracking;Augmented reality;display modulation;latency}
}