Social network giant Facebook, in their recent statement, announced that they have open-sourced DeepFocus, an Artificial Intelligence-powered framework for improving focus on close objects. This technology takes benefit of an end-to-end convolutional neural network which generates an accurate retinal blur in near real-time. DeepFocus ensures nearby objects are in-focus, while distant objects appear out of focus, much like cinematic experiences.
This AI-powered framework is the underlying technology behind Half Dome, a prototype headset that utilizes eye-tracking cameras, wide field of view optics and independently focused displays that offer realistic Virtual Reality experiences. Developed by a multidisciplinary team of researchers at Facebook Reality Labs, DeepFocus aims to deliver experiences that are impossible to differentiate from reality. But, VR applications usually focus on objects that are in the user’s mid-distance as it meets the focal plane in which the image is in focus. Head of Enterprise AR/VR at Facebook, Maria Fernandez Guajardo described the challenge with this approach that if users endeavor to see something that is not in the focal plane, like an object that is close to them, things become blurry. To overcome this problem, the VR industry has placed objects at a distance of 2 meters. This is limiting and it’s not realistic. Great VR has to work with objects that are close to the users. The team of researchers at Facebook explored traditional ways to optimize computational displays, but the outcomes didn’t align with expectations, because traditional approaches like utilizing an accretion buffer can attain manually accurate defocused blur. But they can’t generate the effect in real time for sophisticated, rich content, because the processing demands are too high for even state-of-the-art chips.
Rather than waiting for chipsets to advance and lessen costs, the Facebook team utilized a Deep Learning and built an end-to-end convolutional neural network. Deep Learning is already utilized by AI systems to perform specific tasks by training on large data sets of germane data but has generally not be deployed to VR systems.