NVIDIA is a pioneer in the field of hardware and software for artificial intelligence, driving incredible breakthroughs in every industry. NVIDIA DLSS, one of their first major AI algorithms, was introduced five years ago. Its goal was to enhance performance using neural rendering, made possible by the tensor cores in every GeForce RTX graphics card.
Since its initial launch, the AI models behind DLSS have been continuously trained, leading to increasingly better results and innovations that further accelerate performance:
Today, they are once again refining the rendering technology with the introduction of NVIDIA DLSS 3.5, which includes Ray Reconstruction - a new AI model that creates images with higher quality ray tracing for intensive games and applications.
Alan Wake 2, Cyberpunk 2077, Cyberpunk 2077: Phantom Liberty, Portal with RTX, Chaos Vantage, D5 Render, and NVIDIA Omniverse are adding support for NVIDIA DLSS 3.5 this fall.
See for yourself the advantages in this exclusive new look at Cyberpunk 2077: Phantom Liberty, visualized with DLSS 3.5 and Full Ray Tracing:
"Thanks to the intelligent technology of DLSS 3.5, powered by the strength of artificial intelligence for rendering, you can experience Cyberpunk 2077: Phantom Liberty with cleaner images, more accurate lighting, and the highest possible frame rate." - Jakub Knapik, Vice President of the Art Department, Global Art Director, CD PROJEKT RED
To appreciate the benefits of ray reconstruction, let’s take a look at how ray tracing works.
First, the game engine generates the geometry and materials of the scene, which have physical attributes affecting their appearance and how light interacts with them. Then, from the camera's perspective, a sample of rays is "shot" that determines the properties of the light sources in the scene and how it reacts when it encounters materials. For example, if the rays hit a mirror, reflections are produced.
THE PROBLEM with DLSS so far:
Shooting rays to every pixel on the screen is computationally too intensive, even for offline renders that calculate scenes for several minutes or even hours. Therefore, only a small portion is used - a low number of rays are shot at various points in the scene for a representative sample of the lighting, reflectivity, and shadows within it.
The end result is a noisy, speckled image with gaps, which is still good enough to establish how the scene should look when ray tracing is applied.
To fill in the missing pixels that were not traced by the rays, hand-tuned denoisers (image noise reducers) use two different methods, temporarily accumulating pixels over several frames and interpolating them in space to blend them with neighboring pixels. Through this process, the noisy raw final version is transformed into an image with ray trajectory.
These denoisers are manually tuned and processed for each type of ray-traced lighting in the scene, complicating and burdening the development process, reducing frame rates in games with a high degree of ray tracing, where multiple denoisers work simultaneously to achieve maximum image quality.
Each hand-tuned denoiser accumulates pixels from several frames to increase detail, effectively stealing rays from the past, but at the risk of introducing ghosting, removing dynamic effects, and reducing the quality of others. It also interpolates neighboring pixels and blends this information together, but again at the risk of mixing either too much or insufficiently detailed information, creating uneven lighting effects.
Upscaling is the final stage of the ray-traced lighting pipeline and is crucial for reproducing the most detailed and demanding games at high frame rates. But with the removal or reduction of the quality of effects, the limitations of hand-tuned denoisers increase, eliminating fine details (referred to as high-frequency information) that upscalers use to produce a clear and clean image.
THE SOLUTION:
NVIDIA DLSS 3.5. The latest innovation from Nvidia, Ray Reconstruction, is part of an enhanced neural rendering with artificial intelligence that improves ray image quality for all graphics processors GeForce RTX, replacing manually tuned denoisers with a supercomputer-trained NVIDIA AI network that generates higher quality pixels between discretized rays.
Trained with 5 times more data than DLSS 3, DLSS 3.5 recognizes various ray effects to make smarter decisions about using temporal and spatial data and to retain high-frequency information for high-quality scaling.
Ray reconstruction, trained using offline rendered images that require much more computational power than can be provided during real-time gameplay, recognizes lighting patterns from training data, such as global illumination or ambient occlusion, and recreates them in the game while you play. The results are better than those achieved with manually tuned denoisers.
In Portal with RTX with DLSS turned off, the denoiser struggles with spatial interpolation, where it does not blend enough pixels, creating a blotchy effect. Additionally, it does not accumulate enough good pixels from previous frames, resulting in a visual effect resembling haze. With DLSS 3.5, it recognizes certain patterns related to reflections and keeps the images stable by accumulating accurate pixels while blending adjacent ones to generate high-quality reflections.
In the scene from Cyberpunk 2077 below, the inaccurate lighting of the headlights surrounding the car is the result of the manually tuned denoiser, which pulls inaccurate lighting effects from previous frames. DLSS 3.5 generates accurate lighting so you can distinguish the beam of the headlights and see how the light reflects much more realistically on the curb in front of the car.
The streets of Night City in Cyberpunk 2077 are filled with reflections from rotating billboards and neon lights. With DLSS 3.5 enabled, their quality and clarity are significantly improved:
Creative applications have a wide variety of content that is difficult for traditional denoisers, as mentioned above, they require manual tuning for each scene. As a result, when previewing content, suboptimal image quality occurs. With DLSS 3.5, the artificial intelligence neural network can recognize a wide variety of scenes, creating high-quality images during the preview even before hours are spent on final rendering. D5 Render, an industry-leading application for architects and designers, will feature DLSS 3.5 this fall.
"The integration of DLSS Ray Reconstruction with D5 Render marks our third technical collaboration with NVIDIA after DLSS SR and FG, further enhancing the image quality of D5 Render's real-time rendering capabilities. We look forward to exploring more opportunities with NVIDIA in the future." - Jesse Huang, Vice President, Head of Marketing, D5 Render
With DLSS 3.5, Nvidia provides a faster and better experience as a free upgrade that enhances the already available technologies in your favorite games.
Thanks to RTX, you practically have the power of 2 computers in your PC or laptop - the first is NVIDIA's supercomputer that trains the DLSS AI model with billions of data points to enhance performance and image quality. And the second is your GeForce RTX graphics card, with dedicated tensor cores for executing the AI model in real-time, plus specialized RT cores, innovations like Shader Execution Reordering, and the raw power of every RTX graphics processor providing the best-in-class ray tracing.
In Cyberpunk 2077, the beautiful full ray tracing provided by the Overdrive mode could not be achieved seamlessly without the power of AI. DLSS Super Resolution (SR) reconstructs a 4K image from a lower resolution one, providing a dramatic increase in performance and great picture quality.
To further enhance performance in the most demanding, action-packed moments, we activate DLSS Frame Generation (FG) on the GeForce RTX 40 series graphics cards, which analyzes consecutive frames to create new ones to increase smoothness in gaming.
And now NVIDIA DLSS 3.5 further improves image quality for ray effects by replacing multiple manually tuned denoisers with Ray Reconstruction (RR).
By combining Super Resolution, Frame Generation, and Ray Reconstruction, DLSS 3.5 increases frames per second in Cyberpunk 2077 by a total of 5 times compared to rendering at 4K DLSS OFF.
Note that in games with multiple effects tracked by rays, there may be several denoisers that are replaced with a single neural network for ray reconstruction. In these cases, Ray Reconstruction can also improve performance. In titles with less intensive ray tracing and fewer denoisers, Ray Reconstruction will enhance image quality, although there may be a slight drop in performance.
Source: NVIDIA
