Sony Could Finally Be Making Good On 8K Gaming With New Real-Time Image-Upscaling Patent

NOV 08, 2021

Sony is working on machine learning technology that could bring improved visual fidelity to PS5 and VR games without sacrificing performance.

Sony is developing a maching learning technique for real-time image upscaling for the PS5 and VR, a new patent from Sony Interactive Entertainment suggests (via Resetera).

The maching learning process is somewhat similar to what Nvidia's Deep Learning Super Sampling (aka DLSS) AI tech does, in that it can improve the resolution of games without sacrificing performance. Nvidia's DLSS allows computers to better run games by rendering the original image at a lower resolution, making it easier for the GPU to process, and using AI to upscale the image to give the appearance it is being run at a higher resolution. This means visually intensive games could be run on higher settings on a 4K screen but without taking hits to FPS as the game is actually being run in 1080p, but giving a very similar appearance to actual 4K resolution.

Sony's patent was filed in April and was published just last month and outlines a method in which machine learning can create "masks" of a certain image to hide or fill in gaps left by missing or corrupted data. In essence, this would allow for real-time upscaling of images without sacrifices to performance as this technique would be less taxing on the hardware.

The patent mentions virtual reality (VR) several times and the technique could be essential to delivering the higher quality image that a superior VR headset would require. Powering a VR headset requires intensive processing power and it was notable that the PSVR headset delivered a lower resolution.

The new patent describes the machine learning technique as using "much less computing power" and "without sacrificing image quality where it's needed most to maintain user comfort and immersion". This might mean the next generation of PSVR headset, for the PS5, could run VR games with much better visual quality and performance.

As the 2020s deepen, we could see more 8K screens come into the mainstream too and this technology could conceivably be used to upscale images to 8K also.

The patent's author, Sony Senior Principal Engineer Andy Bigos, is listed on LinkedIn as "working on the application of machine learning and neural rendering to real-time graphics on the PlayStation 5".

"Most recently I've spearheaded work in using Neural Rendering to enhance traditional rendering methods, focusing on using implicit neural representations and how to make them run efficiently. This includes creating custom high performance interference using compute shaders. The target for this work is PlayStation 5," continues Bigos' experience description on LinkedIn.