The sensor bar on the HoloLens comprises four ?environment understanding cameras,? two on each side; a depth camera; an ambient light sensor; and a 2MP photo/HD video camera. Some of these are off-the-shelf parts, whereas Microsoft custom-built others.
The environmental sensing cameras provide the basis for head tracking, and the (custom) time of flight (ToF) depth camera serves two roles: It helps with hand tracking, and it also performs surface reconstruction, which is key to being able to place holograms on physical objects. (This is not a novel approach--it?s precisely what Intel is doing with its RealSense 400-series camera on Project Alloy.)
These sensors work in concert with the optics module (described above) and the IMU, which is mounted on the holographic lenses, right above the bridge of your nose.
Said the presenter, ?Environment cameras provide you with a fixed location in space and pose,? and the IMU is working fast, ?so as you move your head around...you need to be able to feed your latest pose information into the display as quickly as possible.? He said that HoloLens can do all of this in <10ms, which, again, is key to preventing ?swimming? and also to ensuring that holograms stay locked to their position in the real world space.
Bookmarks