NVIDIA has introduced a new texture compression algorithm called Neural Texture Compression (NTC) to address the growing demand for video memory. With modern games requiring high-resolution textures and associated properties to realistically represent materials, the development of more data-efficient algorithms is necessary.
NTC offers 4x higher resolution and 16x more texels than the standard block compression (BC) algorithm. The NTC algorithm uses matrix multiplication techniques performed by modern GPUs, making it more practical and functional and less limited by disk and memory.
Direct access and local access are important features of NTC, making it possible to access compressed textures with low cost and zero latency. The new approach compresses multiple channels and mipmaps (textures of different sizes) together, resulting in better quality and bitrate than JPEG XL and AVIF. Unlike traditional block compression algorithms that require special blocks, NTC compresses textures as tensors, making no assumptions about them.
Continued advances in photorealism in rendering are accompanied by an increase in texture data, resulting in increased storage and memory demands. To address this issue, NVIDIA has proposed a new neural compression technique specifically designed for material textures.
With low bitrate compression, it unlocks two more levels of detail, i.e., 16 times the number of texels, with better image quality than advanced image compression techniques like AVIF and JPEG XL. At the same time, the NTC algorithm enables on-demand real-time decompression with random access similar to block texture compression on GPU.
The key idea of the NTC technique is to compress multiple material textures and their mipmap chains together and decompress them using small neural networks optimized for each material. Finally, a custom training implementation is used to achieve practical compression speeds that exceed the performance of popular frameworks like PyTorch by an order of magnitude.
According to the paper, neural textures can be rendered in real-time with up to 16x more texels compared to the BC approach, with a 4K rendering speed of 1.15 ms (measured on the RTX4090). This new algorithm is expected to revolutionize texture compression, making it more efficient and practical for modern games and applications. The details of the research will be presented at SIGGRAPH 2023 on August 6th.
