< Back to previous page
Project
Optimized render quality based on scalable texture coding. (IWT536)
The goal of the project is to develop a 3D rendering system which provides optimal visual quality for the given hardware at a fixed frame rate. This will be achieved by designing highly scalable rendering techniques and calculation optimal rendering parameters. These techniques will aim for maximal visual quality. Scalability of the 2D artwork resources, providing both classical texture mapping as additional geometric detail, will be ensured by using a wavelet-based codec, combined with virtual texturing. These two techniques are both suitable for use in the traditional GPU rendering pipeline, which is the target platform. Both desktop GPUs as handheld GPUs are considered target platforms.
1. The wavelet-based codec will provide the means to stream textures to the GPU. The stream should contain the lowest resolution data first, which can be used instead of a classical lowest resolution mip map of the texture. Additional streaming enables the reconstruction of consequent resolution levels, up to the resolution of the original texture. Compression should be better than the ratios achieved with current implementations of the DXT codecs, visual quality should be comparable. The end result is a fully scalable 2D resource representation, inherent mip mapping capabilities and improved compression performance, tightly integrated with virtual texturing. Performance will be ensured by caching decompressed pages in GPU memory and by using anisotropic decompressing, making use of the knowledge of the filtering method applied during texture sampling. The codec will be designed in such a way that it can be tightly integrated with virtual texturing (VT), a texture mapping approach that can lower texture memory usage by a significant amount. Additionally, the use of virtual texturing enables the possibility to utilize different compression ratios for each page, resulting in region of interest coding (ROI). This effort will result in a highly scalable and compact texture representation on the GPU.
2. The choice of the rendering parameters can be quite complicated, as hardware resources have to be used for rendering the current frame, as well as an investment in a lower cost of future frames. The optimization problem can thus be phrased as: given the hardware constraints and desired frame rate, choose the rendering parameters such that the user's visual experience is the best for the current frame and the set of future predicted frames, considering their respective probabilities. A quality metric which closely resembles the user's subjective visual experience will be adapted in a way that it can map rendering parameters of a single scene onto a quality value and provide input for the optimization algorithm. Implementing an efficient version of this metric will be a major challenge, as it will need to evaluate multiple predicted scenes during one rendering pass. In order to perform this within the given time constraint, heuristics will be designed. The mapping produced will ensure optimal usage of the available hardware resources.
This work will result in a highly scalable and flexible 3D rendering engine, ready to be used on a variety of consumer GPUs. Memory and bandwidth constrained platforms will benefit from the developed system, resulting in higher detail and smoother frame rates during interactive rendering.
The proposed project is in line with the activities of our research group ETRO-IRIS, which focuses on multidimensional signal processing. Knowledge about state of the art image coding is readily available, as is expertise about constraint optimization. On the other hand, extending these techniques to interactive 3D rendering and GPGPU computing will be a welcome addition to the lab's activities, directly extending the applicability of the lab's research.
1. The wavelet-based codec will provide the means to stream textures to the GPU. The stream should contain the lowest resolution data first, which can be used instead of a classical lowest resolution mip map of the texture. Additional streaming enables the reconstruction of consequent resolution levels, up to the resolution of the original texture. Compression should be better than the ratios achieved with current implementations of the DXT codecs, visual quality should be comparable. The end result is a fully scalable 2D resource representation, inherent mip mapping capabilities and improved compression performance, tightly integrated with virtual texturing. Performance will be ensured by caching decompressed pages in GPU memory and by using anisotropic decompressing, making use of the knowledge of the filtering method applied during texture sampling. The codec will be designed in such a way that it can be tightly integrated with virtual texturing (VT), a texture mapping approach that can lower texture memory usage by a significant amount. Additionally, the use of virtual texturing enables the possibility to utilize different compression ratios for each page, resulting in region of interest coding (ROI). This effort will result in a highly scalable and compact texture representation on the GPU.
2. The choice of the rendering parameters can be quite complicated, as hardware resources have to be used for rendering the current frame, as well as an investment in a lower cost of future frames. The optimization problem can thus be phrased as: given the hardware constraints and desired frame rate, choose the rendering parameters such that the user's visual experience is the best for the current frame and the set of future predicted frames, considering their respective probabilities. A quality metric which closely resembles the user's subjective visual experience will be adapted in a way that it can map rendering parameters of a single scene onto a quality value and provide input for the optimization algorithm. Implementing an efficient version of this metric will be a major challenge, as it will need to evaluate multiple predicted scenes during one rendering pass. In order to perform this within the given time constraint, heuristics will be designed. The mapping produced will ensure optimal usage of the available hardware resources.
This work will result in a highly scalable and flexible 3D rendering engine, ready to be used on a variety of consumer GPUs. Memory and bandwidth constrained platforms will benefit from the developed system, resulting in higher detail and smoother frame rates during interactive rendering.
The proposed project is in line with the activities of our research group ETRO-IRIS, which focuses on multidimensional signal processing. Knowledge about state of the art image coding is readily available, as is expertise about constraint optimization. On the other hand, extending these techniques to interactive 3D rendering and GPGPU computing will be a welcome addition to the lab's activities, directly extending the applicability of the lab's research.
Date:1 Jan 2011 → 31 Dec 2014
Keywords:Low Power Cmos, Numerical Linear Algebra, Digital Image Processing, Image Reconstruction, Embedded System Design, Displays, Audio Processing, Light Detectors, Micro-Electronics Technology, Sige Bicmos Design, Satellite Image Analysis, Medical Image Analysis, Inverse Problems, Telemedicine, JPEGx, Video Compression, Neural Networks, Mine Detection, Vision, Digital Signal Processing, Electronic System Design, Machine Vision, Micro Electronics, Chip Interconnects (Inter / Intra), Cmos Design, Humanitarian Demining, Speech Processing, Mpegx, Light Emitters, Pattern Recognition, Mm-Wave Technology, Robot Vision, Impedance Tomography, Image Compression, Light Modulators, Medical Image Visualization, Opto-Electronics, Motion Estimation And Tracking, Computer Aided Electronic Design, Multispectral Image Analysis, Electronics, Computer Vision, Image Processing, Industrial Visual Inspection, Image Analysis
Disciplines:Electrical and electronic engineering, Mathematical sciences and statistics, (Bio)medical engineering