Efficient and High-Quality Compression for Camera-Captured 6 Degrees of Freedom VR.

Today, the only way to have a 6 degrees of freedom (6-DoF) experience with a VR headset, is to be in a virtual 3D world composed of 3D models (e.g., a game). No high-quality, real-time technology is available for camera-captured content providing 6-DoF video. A few techniques have been designed that allow for free viewpoint video. However, either viewpoint interpolation is hard and lacks quality, or suitable real-time compression does not exist. At the Ghent University, in cooperation with TU Berlin, a novel framework is being developed to tackle the compression of light field video. The major advantages are real-time rendering and elegant mathematics, which allow for high compression ratios. However, the encoding process is still extremely slow and the resulting image quality is often lacking detail in high-frequency regions. The proposed research will fix these two issues: boosting image quality to industry requirements, and making the compression significantly faster. I plan to do this iteratively with an increasing level of complexity. Firstly building fundamentals on 2D images, and slowly moving on to the higher dimensions, required for 6-DoF. Along the way, I will maintain the decoder counterpart, such that there is permanent validation: is everything still fast enough for real-time viewing? In summary, I will push the novel framework to industry standard requirements to make 6-DoF experiences come to live.