NVIDIA DLSS 3.5 announced: Beyond denoising

[DuckDodgers]

 

 

NVIDIA's pending release, DLSS 3.5, builds on the previous DLSS 3 tech, further advancing its Deep Learning Super Sampling (DLSS) technology. While AMD FSR3 and XeSS have shifted to an open-source approach, NVIDIA continues to refine its proprietary DLSS technology.

 

Quotes

Quote

DLSS 3.5 is focusing in on Ray (Tracing) Reconstruction (RR), delivering a visibly enhanced visual experience compared to traditional denoising techniques. NVIDIA claims that Ray Reconstruction has been trained on five times more data than DLSS3. It has been trained to incorporate additional game and software engine data, recognize various ray-traced effects, distinguish between good and bad temporal and spatial pixels, and preserve high-frequency data for upscaling.

 

The utilization of denoisers was found to strip away essential data required for upscaling, leading to a loss of color data during denoising and subsequent upscaling. Furthermore, traditional methods may result in inaccurate lighting effects due to the accumulation of pixels from previous frames, often resulting in ghosting.

 

Denoisers also contribute to subpar global illumination and lower-quality reflections, as there isn’t enough data for interpolation. This will not be the case with Ray Reconstruction, which is the primary feature of DLSS 3.5 update.

 

Finally some ML love for RT rendering. There's so much regular image denoising can do with so few ray samples per pixel, particularly when using upscaling where there's even less than one RT sample per pixel.

 

 

Source: https://videocardz.com/newz/nvidia-announces-dlss-3-5-with-ray-reconstruction-launches-this-fall

[BetteBalterZen]

So many hate on Nvidia and DLSS and sure, I guess I understand why some do, but man, I'm a sucker for DLSS and Ray Tracing. It's amazing. #nvidiaftw

[Eigenvektor]

4 hours ago, DuckDodgers said:

While AMD FSR3 and XeSS have shifted to an open-source approach, NVIDIA continues to refine its proprietary DLSS technology.

I think you're misinterpreting what the source said: There is no news on FSR3 + XeSS is going open source.

 

Let's see if FSR3 is remotely competitive once it finally releases. Looks like Nvidia isn't keeping still until then.

[starsmine]

So is there no way to run this without the frame gen?
#giveus DLSS 2.5

At some point they need to rename this tech stack if they are seperate technologies. Some that require others to be built off of and some that dont.

[Thaldor]

I just love how again the comparison is DLSS OFF vs. the newest DLSS on. Like what are you hiding? Why cannot you give us clear DLSS X vs DLSS X+1? Do you fear no one would give two cents about the next overpriced GPUs if they really saw how little they get with double the price?

[starsmine]

34 minutes ago, Thaldor said:

I just love how again the comparison is DLSS OFF vs. the newest DLSS on. Like what are you hiding? Why cannot you give us clear DLSS X vs DLSS X+1? Do you fear no one would give two cents about the next overpriced GPUs if they really saw how little they get with double the price?

Quote

 

The “RR” tech will work across all RTX GPUs (unlike Frame Generation).

 

There has to be a way to run 3.5 without the... 3 part just because of this line. It also should not be them selling the next overpriced GPU if possible.

Something that also throws me off is that this also is claiming to both improve quality and frame rates (by 8%)
I may be misunderstanding how this is working, the pictures show it drawing more rays (ish, i get it) or rather, approximating more frames, but it still needs the baseline frames drawn, so it would still be doing more overall work, just more efficient. So where is the performance actually coming from unless its drawing less rays?

Nvidia has said it does not improve performance by definition... so why post an example where it does by such a significant amount?

[porina]

36 minutes ago, starsmine said:

So is there no way to run this without the frame gen?

It says it is compatible with all RTX GPUs, implying it will run on 20 and 30 series too which lack FG.

 

I think the only problem here is that we now have 3.0 requiring 40 series (or newer?) but 2.x and 3.5 do not. This could get confusing and maybe nvidia need to rebrand away from a version number scheme and explicitly call it by function.

[Thaldor]

12 minutes ago, starsmine said:

There has to be a way to run 3.5 without the... 3 part just because of this line. It also should not be them selling the next overpriced GPU if possible.

It seems to run on every RTX card but the question will be can you run it without slapping artificial frames into the mix at which point the GPU question will be "Is DLSS 3.5 that MUCH better than DLSS 2.0 that you should invest into RTX40 card?".

[shermantanker]

44 minutes ago, Thaldor said:

I just love how again the comparison is DLSS OFF vs. the newest DLSS on. Like what are you hiding? Why cannot you give us clear DLSS X vs DLSS X+1? Do you fear no one would give two cents about the next overpriced GPUs if they really saw how little they get with double the price?

At the 5:30 mark in Nvidia's video it shows a side by side between DLSS Off, 2, 3, & 3.5 in Cyberpunk. It showed a ~10% performance uplift over 3.0 in that title.

[porina]

6 minutes ago, Thaldor said:

It seems to run on every RTX card but the question will be can you run it without slapping artificial frames into the mix at which point the GPU question will be "Is DLSS 3.5 that MUCH better than DLSS 2.0 that you should invest into RTX40 card?".

https://youtu.be/sGKCrcNsVzo?t=332

 

The example there shows a small <10% improvement vs 3.0 attributed to RR replacing multiple denoisers, however that is followed up with "In general, the speed of games using Ray Reconstruction is going to be about the same as the speed of games without"

 

If that holds, then games you can already run with RT could look better at the same performance. Maybe in some cases there may be a small improvement, or you can trade off the increased image quality to turn down some other settings for more performance.

[starsmine]

So

RR runs simultaneously with SR. 

2 (DLAA and SR)

3 (FG)
3.5 (RR)

RR does NOT need FG, but DOES need 2.

so... like 3, it runs ONTOP of 2. 
3.5 and 3 can run... at the same time, but do not require each other. 

Nvidia really these are not good names because the numbers look like versions. 

I think what Nvidia wants to do going forward is that the numbers are a feature set a game can include, like 3 can include FG, but it can just be version 3 of DLAA/SR
3.5 can include FG or RR or both, but not necessarily. 

Im using this video as a source to understand it better

From everything I see though, I do NOT understand why it needs to run simutanously with 2 other then to lock this down to nvidia cards so they can claim for the first time in forever they have a graphical advantage over the compitition. Not saying they dont have a reason, just I dont understand it. 

[Brooksie359]

1 hour ago, porina said:

It says it is compatible with all RTX GPUs, implying it will run on 20 and 30 series too which lack FG.

 

I think the only problem here is that we now have 3.0 requiring 40 series (or newer?) but 2.x and 3.5 do not. This could get confusing and maybe nvidia need to rebrand away from a version number scheme and explicitly call it by function.

Honestly I am baffled why they didn't name DLSS 3 something else. I mean it's a different technology as far as I can tell vs the previous DLSS versions. Now we are confused because they decided to name a different technology DLSS 3.0. Personally I don't use 3.0 anyways but I feel like it's going to be super confusing when you will have an option to turn on DLSS 3.5 and one to turn on DLSS 3.0 because people will assume that 3.5 is just a better version of 3.0 when in fact they are totally different technologies. They will most likely just turn on 3.5 thinking that it would be stupid to use the older version. 

[Arokhantos]

So its taking out the ripple effect to make the reflections less realistic and pop out more basically.

[Kinda Bottlenecked]

12 minutes ago, Arokhantos said:

So its taking out the ripple effect to make the reflections less realistic and pop out more basically.

 

It isn't doing that, do watch the video 

[Arokhantos]

17 minutes ago, Kinda Bottlenecked said:

 

It isn't doing that, do watch the video 

 

That is all i am seeing in first iamge on the post, ripples and ripples are gone less realistic looking RT more wet marble RT

[RejZoR]

While it's cool if they improved quality and not affect performance or even improve it, I worry it'll be too shiny and too detailed. We had those faked 100% mirror reflections years ago and I sure as hell hope we aren't returning to them again just for the sake of "ray tracing" effect being more visible. That's like HDR bloom nonsense we had years ago which was turned up to 11 just to be more visible. Or insane depth of field that made everything look like it's smeared with vaseline 10 meters beyond our range of vision. Duke Nukem Forever had both, insane bloom and this DoF crap so bad I had to turn it off when I finally played it during COVID time.

[Quackers101]

43 minutes ago, Arokhantos said:

That is all i am seeing in first iamge on the post, ripples and ripples are gone less realistic looking RT more wet marble RT

would assume it can be changed when creating the game/assets.
but agree, more mirror like and removing the "material's look"? or something. could be adjusted I think?

 

also wouldn't be easier to understand if it was a part of 1.5 or 2.5 DLSS, since assumingly it works on all RTX cards?

 

But I do find it a bit comical, with Nvidia "faking graphics" as they said in the video.
When screens going after showing the best results for  "how the creators intended it to look", but going a bit in circles for 3D content to push "graphics". but some stuff do look good though.

 

as with DLSS 3 and everything else, adding "fake frames", until 80% of what you see is generated or reconstructed work than the native image.

Edited August 22, 2023 by Quackers101

[porina]

2 hours ago, starsmine said:

From everything I see though, I do NOT understand why it needs to run simutanously with 2 other then to lock this down to nvidia cards so they can claim for the first time in forever they have a graphical advantage over the compitition. Not saying they dont have a reason, just I dont understand it. 

It is still a "DL" series feature so will require RTX GPUs regardless. AMD/Intel can probably code equivalent functionality if they want to. Still if we compare FSR2 to native XeSS, Intel may have a better go of it since they have the hardware to back it up.

[starsmine]

1 hour ago, Brooksie359 said:

Honestly I am baffled why they didn't name DLSS 3 something else. I mean it's a different technology as far as I can tell vs the previous DLSS versions. Now we are confused because they decided to name a different technology DLSS 3.0. Personally I don't use 3.0 anyways but I feel like it's going to be super confusing when you will have an option to turn on DLSS 3.5 and one to turn on DLSS 3.0 because people will assume that 3.5 is just a better version of 3.0 when in fact they are totally different technologies. They will most likely just turn on 3.5 thinking that it would be stupid to use the older version. 

DLSS 3 Requries DLSS 2 to work
DLSS 3.5 Requires DLSS 2 to work. 

Frame Gen can not run without running DLSR. So that was the logic. 
The confusing part is 3.5 does not require Frame Gen to run. 

[Eigenvektor]

14 minutes ago, starsmine said:

DLSS 3 Requries DLSS 2 to work
DLSS 3.5 Requires DLSS 2 to work. 

Frame Gen can not run without running DLSR. So that was the logic. 
The confusing part is 3.5 does not require Frame Gen to run. 

They don't run on top of another or require one another. It's not that complicated really.

 

DLSS2 can do SR, which runs on all RTX GPUs

DLSS3 can do SR + FG. SR still runs on all RTX GPUs, while FG only runs on RTX 40-series

DLSS3.5 can do SR + FG + RR. SR and RR run on all RTX GPUs, FG is still exclusive to 40-series.

 

Frame Generation generates additional frames in between frames generated by the game engine. Which means it is independent from the engine and can run without affecting anything else.

 

They have a matrix in their video that shows this pretty clearly, imho.

Spoiler

 

[tikker]

This looks pretty cool. They are doing a pretty impressive job with DLSS.

1 hour ago, Brooksie359 said:

Honestly I am baffled why they didn't name DLSS 3 something else. I mean it's a different technology as far as I can tell vs the previous DLSS versions. Now we are confused because they decided to name a different technology DLSS 3.0. Personally I don't use 3.0 anyways but I feel like it's going to be super confusing when you will have an option to turn on DLSS 3.5 and one to turn on DLSS 3.0 because people will assume that 3.5 is just a better version of 3.0 when in fact they are totally different technologies. They will most likely just turn on 3.5 thinking that it would be stupid to use the older version. 

  Because DLSS 3 expands on the features that were introduced with 2, but is still DLSS. It's probably becoming, or already has become, genericized now that it gets more and more features, but it would probably be more confusing for the consumer if it wasn't. You can still say 3.5 is "better" than 3.0, because it introduces ray reconstruction. You don't turn on 3.5, you turn on ray reconstruction.

 

15 minutes ago, starsmine said:

DLSS 3 Requries DLSS 2 to work
DLSS 3.5 Requires DLSS 2 to work. 

Frame Gen can not run without running DLSR. So that was the logic. 
The confusing part is 3.5 does not require Frame Gen to run. 

I think putting it that way is a misinterpretation of versioning, which the video you quoted also tries to address at the start. A version simply describes a set of features introduced with or available in that version. I like the quoted video, because there you can hear that he does not refer to SR as DLSS 2, but simply as the SR technology of DLSS.

 

As he mentions, SR, DLAA, FG and RR  are all just separate technologies of DLSS. DLSS 3 expands the DLSS feature set by introducing frame generation technology.  Using frame generation might be depdendent on using super resolution, but DLSS 3 does not run "on top of 2" nor does it "require 2" because they are both DLSS. Similarly, DLSS 3.5 expands the DLSS feature set with ray reconstruction, but does not "run simultaneously with 2". Instead, using ray reconstruction is just bound to the use of super resolution due to how ray reconstruction works.

 

You can see this in Cyberpunk 2077's menu, for example (going by screenshots from Tom's Hardware). You don't enable "DLSS 3" if you want to use frame generation. You enable the frame generation option of DLSS.

 

[starsmine]

24 minutes ago, Eigenvektor said:

They don't run on top of another or require one another. It's not that complicated really.

 

DLSS2 can do SR, which runs on all RTX GPUs

DLSS3 can do SR + FG. SR still runs on all RTX GPUs, while FG only runs on RTX 40-series

DLSS3.5 can do SR + FG + RR. SR and RR run on all RTX GPUs, FG is still exclusive to 40-series.

 

Frame Generation generates additional frames in between frames generated by the game engine. Which means it is independent from the engine and can run without affecting anything else.

 

They have a matrix in their video that shows this pretty clearly, imho.

  Reveal hidden contents

 

They do run on top of each other.
Nvidia DLSS FG runs ON TOP of SR. (they have to for latency). 
RR runs ON TOP of SR. (I dont know why, but it does)

 

Yes I know about the chart, it is not relevant to the above. I mentioned how they are using the numbering as like a feature set which is a little weird.


 

  

21 minutes ago, tikker said:

I think putting it that way is a misinterpretation of versioning, which the video you quoted also tries to address at the start. A version simply describes a set of features introduced with or available in that version. I like the quoted video, because there you can hear that he does not refer to SR as DLSS 2, but simply as the SR technology of DLSS.

 

As he mentions, SR, DLAA, FG and RR  are all just separate technologies of DLSS. DLSS 3 expands the DLSS feature set by introducing frame generation technology.  Using frame generation might be depdendent on using super resolution, but DLSS 3 does not run "on top of 2" nor does it "require 2" because they are both DLSS. Similarly, DLSS 3.5 expands the DLSS feature set with ray reconstruction, but does not "run simultaneously with 2". Instead, using ray reconstruction is just bound to the use of super resolution due to how ray reconstruction works.

 

You can see this in Cyberpunk 2077's menu, for example (going by screenshots from Tom's Hardware). You don't enable "DLSS 3" if you want to use frame generation. You enable the frame generation option of DLSS.

 

Yes

SR, DLAA, FG, and RR are all separate technologies. 
FG REQUIRES SR to run. SR is also called, DLSS 2. This is Nvidia's fault for naming things like this, not mine. 
Ray reconstruction runs simultaneously with SR, again called DLSS 2. 

Should nvidia want to disentangle their numbering and feature set for their marketing, they SHOULD, and I wish they would. 

[Eigenvektor]

7 minutes ago, starsmine said:

They do run on top of each other.
Nvidia DLSS FG runs ON TOP of SR. (they have to for latency). 
RR runs ON TOP of RR. (I dont know why, but it does)

Yes, these technologies depend on one another or run one after the other. But the way you worded it makes it sound like DLSS3 runs on top of DLSS2 which is not the case. That like saying Windows 10 requires or runs on top of Windows 95. Its code base may have evolved from Windows 95, but that doesn't mean they run simultaneously or concurrently when you're running Windows 10.

 

• DLSS introduced Super Resolution (SR)
• DLSS2 provides an improved version of SR
• DLSS3 still provides SR and added the ability to enable Frame Generation (FG, on supported hardware)
• DLSS3.5 now introduces Ray Reconstruction, in addition to SR and FG

As @tikker said, "DLSS" is simply the name of an AI powered software suite and newer versions of the suite simply introduce additional features.

 

It would be more correct to say that DLSS 3.5 still requires hardware that can do Super Resolution, just as DLSS2 did. It does not require DLSS2, it requires the same hardware to provide the same features and adds additional features on top of that.

[Brooksie359]

17 minutes ago, tikker said:

This looks pretty cool. They are doing a pretty impressive job with DLSS.

  Because DLSS 3 expands on the features that were introduced with 2, but is still DLSS. It's probably becoming, or already has become, genericized now that it gets more and more features, but it would probably be more confusing for the consumer if it wasn't. You can still say 3.5 is "better" than 3.0, because it introduces ray reconstruction. You don't turn on 3.5, you turn on ray reconstruction.

 

I think putting it that way is a misinterpretation of versioning, which the video you quoted also tries to address at the start. A version simply describes a set of features introduced with or available in that version. I like the quoted video, because there you can hear that he does not refer to SR as DLSS 2, but simply as the SR technology of DLSS.

 

As he mentions, SR, DLAA, FG and RR  are all just separate technologies of DLSS. DLSS 3 expands the DLSS feature set by introducing frame generation technology.  Using frame generation might be depdendent on using super resolution, but DLSS 3 does not run "on top of 2" nor does it "require 2" because they are both DLSS. Similarly, DLSS 3.5 expands the DLSS feature set with ray reconstruction, but does not "run simultaneously with 2". Instead, using ray reconstruction is just bound to the use of super resolution due to how ray reconstruction works.

 

You can see this in Cyberpunk 2077's menu, for example (going by screenshots from Tom's Hardware). You don't enable "DLSS 3" if you want to use frame generation. You enable the frame generation option of DLSS.

 

No DLSS 2.0 and DLSS 3.5 are similar technologies in the fact that they take an image and make it look better. DLSS 3.0 generates a totally new frame so not the same at all. Then again I guess that is how I look at it personally and why I don't like that DLSS 3.0 isn't called something different. 

[tikker]

56 minutes ago, starsmine said:

I mentioned how they are using the numbering as like a feature set which is a little weird.

That is exactly what versioning is for. If you add new features to software you bump the version and give a list of those new features.

 

Windows 11 does not run on top of or require Windows 10 even though it offers many of the same features. Firefox 101 does not run on top of or require Firefox 78, even though many of the same features are in there. They are just newer versions of those pieces of software.

56 minutes ago, starsmine said:

SR is also called, DLSS 2.

But DLSS 1 already covered the super resolution aspect. Shouldn't we call DLSS 2 DLAA in that case? This is where the interpretation goes wrong. DLSS 2 is the version of DLSS that only had SR and DLAA (but improved on the first version). When that still was the only thing DLSS could offer, then sure they were equivalent for all intents and purposes, but not the true meaning of the number 2. When version 3 introduced frame generation as another technology that DLSS as a whole could offer, that equivalency was broken. That is the point where you need to remember that "DLSS 2" means "version 2 of DLSS" and not whatever features that version offers.

56 minutes ago, starsmine said:

SR, DLAA, FG, and RR are all separate technologies. 

Exactly, with the key point being that they are separate technologies of the same "mother package" called DLSS.

 

As you say yourself: it is frame generation that depends on super resolution. FG is not "DLSS 3". DLSS 3 introduced FG. In the same way RR is not "DLSS 3.5". They are separate technologies that were introduced with certain versions of DLSS. Those technologies are free to have their own dependencies within DLSS.

56 minutes ago, starsmine said:

This is Nvidia's fault for naming things like this, not mine.

Not really. If anything it is our own fault for equation technologies to version numbers that introduce them.

 

39 minutes ago, Brooksie359 said:

No DLSS 2.0 and DLSS 3.5 are similar technologies in the fact that they take an image and make it look better. DLSS 3.0 generates a totally new frame so not the same at all. Then again I guess that is how I look at it personally and why I don't like that DLSS 3.0 isn't called something different. 

DLSS 3 added frame generation as a technology under DLSS. It's all DLSS. DLSS 3 is not something different from 2 or 3.5. It's just version 3 of DLSS. You can also see this in the way games advertise the settings to you. I googled a random list of games supporting Frame Generation to see what their menus look like and got this:
 

Hogwarts Legacy:

Spoiler

https://overclock3d.net/reviews/software/hogwarts_legacy_pc_performance_review_settings_guide/5

https://gamingintel.com/hogwarts-legacy-best-pc-settings/

You select DLAA or DLSS as your upscaling and turn Frame Generation on or off.

 

Cyberpunk 2077:

Spoiler

https://www.tomshardware.com/how-to/enable-frame-generation-windows-10

You toggle Super Resolution, DLAA or Frame Generation to your liking (within their requirements).

 

Microsoft Flight Simulator:

Spoiler

https://forums.flightsimulator.com/t/4090-dlaa-frame-generation-wow/549823

A bit confusingly worded in my opinion I'll admit, but you can select DLSS for your AA, DLSS Super Resolution is its own setting and Frame Generation is its own setting.

 

They don't make you choose between "DLSS 2" or "DLSS 3" or "DLSS 3.5", because it is all DLSS and the version numbers are not the features.