Nvidia 4050/60/70 100W+ laptop GPUS are held back by an artificial voltage limit not present on 4080/4090 parts

[PotatoSupremacist]

Summary

 

Jarrod'sTech has discovered that Nvidia has seemingly added a new voltage limit previously not seen on 30-series laptop GPUs, a so called "reliability voltage limit" that causes GPUs nominally rated for power draws in the realm of 120-150W range to never hit those levels, resulting in performance that is within 1% of 100W GPUs. Curiously, the 4080 and 4090 laptop GPUs do not run into this problem and scale with power consumption as expected.

 

Quotes

Quote

I don't think I've seen this before RTX 40 series, but it's happening on the RTX 4050, 4060, and 4070

Quote

100 watts and above gave [the 4070] basically no extra performance

My thoughts

Jarrod takes a fairly measured stance, as he probably should and has to. Me, as just your average tech enthusiast? I'm a bit less convinced. Remember how the 30 series caught flak for wildly differing performance with GPUS operating at a massive range of wattages with the removal of the Max-Q moniker? Manufacturers began to list their laptop GPU wattages and it became a bit of a marketing point, and evidently still is for the example page he showed for an Asus product. It's all a bit too convenient that now you have the illusion of additional performance, with no actual gain. Then we see that the high end 4080 and 4090 do not experience a similar issue, and this combined with Nvidia's attempt to force people into the high end on the desktop side with pathetic VRAM counts on the supposedly "mid range" 4060 and 4070 does not do them any favors. And of course, this wasn't an issue for the 30 series of cards.

 

Sources

 

[Kisai]

10 minutes ago, PotatoSupremacist said:

 

Quotes

My thoughts

Jarrod takes a fairly measured stance, as he probably should and has to. Me, as just your average tech enthusiast? I'm a bit less convinced. Remember how the 30 series caught flak for wildly differing performance with GPUS operating at a massive range of wattages with the removal of the Max-Q moniker? Manufacturers began to list their laptop GPU wattages and it became a bit of a marketing point, and evidently still is for the example page he showed for an Asus product. It's all a bit too convenient that now you have the illusion of additional performance, with no actual gain. Then we see that the high end 4080 and 4090 do not experience a similar issue, and this combined with Nvidia's attempt to force people into the high end on the desktop side with pathetic VRAM counts on the supposedly "mid range" 4060 and 4070 does not do them any favors. And of course, this wasn't an issue for the 30 series of cards.

 

My theory is that the limiter exists because they are effectively the same chip, but marketed for smaller units. Like you can't have a 180w gaming laptop pull 240w every couple seconds because it would kill the power supply, but a laptop that came with the 240w power supply, no problem.

 

However I also feel that gaming laptops should not exist because a 4090 in a laptop is not even remotely comparable to a 4090 desktop performance, and nvidia does itself no favors marketing the parts this way. It's a way to get people to overpay for what is effectively the same 4060 desktop performance at all tiers while paying premium prices for it.

(also nobody nitpick that, this is a flippant statement.)

 

Why make 4 different chips when you can make one chip and just impose a limiter on it to meet a design spec.

 

[leadeater]

Could well be the very large difference in CUDA cores and die size when you go down from the RTX 4080 Mobile, and the memory bus gets quite small paired with GDDR6 rather than GDDR6X too.

 

A lot easier to push power through large GPUs before hitting voltage limits

 

[Supreme Calamitas]

So are they running at 140w and seeing no performance differences or are they being limited to a certain voltage e.i 100w and can’t get to 140/150w

 

One we can work around, the other one we can’t.

 

2 hours ago, Kisai said:

However I also feel that gaming laptops should not exist because a 4090 in a laptop is not even remotely comparable to a 4090 desktop performance

4090 is a luxury gpu and luxury products never make sense. It should still exist though, 4080 Ti was a more appropriate name.

[porina]

The voltage limits definitely aren't new. VRel and VOp have been present for many generations. If I fire up Aida64 stress test and set it to GPU only, I see VRel as the limiting factor on my desktop 3070. I could repeat this on my laptop 3070 in a moment. My laptop 3070 goes straight to power limit, although interestingly it is at 115W not the manufacturer listed 130W.

 

Boost is complicated and the GPU will try to boost until it hits one of several limits, like VRel, VOp, Therm, Pwr. Different workloads and specific GPU configurations may lean more to some of these than others.

[Refinedpluto]

52 minutes ago, leadeater said:

Could well be the very large difference in CUDA cores and die size when you go down from the RTX 4080 Mobile, and the memory bus gets quite small paired with GDDR6 rather than GDDR6X too.

 

A lot easier to push power through large GPUs before hitting voltage limits

 

Really? The 4050, 4060, and 4070 all topping out at 90 watts is just ridiculous and is obviously artificial gimping. The math doesn't math.

 

The desktop 4070ti uses 50% more power (100 watts) then the 4070 just from having more cuda cores at the same memory bus and memory amount.

 

The 4 series is not that efficient, just because a 3080 had transient spikes like a nuclear power plant, doesn't mean the 4 series doesn't scale like normal with power. It's obviously superior to the last Gen AMD 6 series all around, however it is just marginally better frames per watt on a smaller node. Just compare a 6800xt to a 4070ti frames per watt. It's only 10-20% better with a smaller memory bus, less memory, and a smaller node.

 

Saying a laptop 4070 can't use more watts then a 4050 is farking crazy. Look at the desktop 4060ti. The amount of Nvidia apologism and inability of people to call Nvidia out on thier B/S is just crazy. How many times are they gonna get caught doing this kind of thing or releasing a bad driver on purpose to thier old cars or something similar.

 

This is normal, this is par for the course. They gimped the chips purposefully, that is all.

 

Like give me a break, a full power laptop 3070 actually beats a 4070 by a few frames in several games. 

[leadeater]

1 hour ago, Refinedpluto said:

Really? The 4050, 4060, and 4070 all topping out at 90 watts is just ridiculous and is obviously artificial gimping. The math doesn't math.

It doesn't top out at 90W, it can use more if enough of the CUDA cores are being utilized properly otherwise the GPU needs to clock higher but gets voltage limited. That's an important difference. When you start pushing something harder causing it to need to use more power then you need to start looking at what specifically and why, removing the voltage safety isn't necessarily going to alleviate that and it's also there for a reason. All the desktop GPUs are in fact limited by the same thing and it's why OC on them doesn't do a lot because Nvidia isn't allowing the voltage to raise high enough to sustain the demanded frequency being asked, this limit is also there for a reason. I myself would argue it could be relaxed a little but I'm certainly not going to say there is no risk there.

 

But then why does the frequency need to raise so much? The GPU cores can use the extra power but what is it about the workload and pipeline that the GPU frequency becomes the most important factor to the performance? I would argue that the CUDA cores are being under utilized and under feed and the GPU clocks need to raise to compensate for that and I would point directly at the memory bandwidth as the cause.

 

Now the desktop RTX 4070 GPU is larger than the laptop by 1280 CUDA cores and all current RTX 40 desktop series use GDDR6X which give about 40% more bandwidth typically but it's likely more since the GDDR6 being used in laptops won't be getting clocked as high as when used in desktop cards. The RTX 4080 Mobile GPU die is also the same one as used by the desktop RTX 4070.

 

I'm simply not seeing much here beyond small GPUs not able to clock high due to legitimate voltage limits and a substantially weaker memory configuration also limiting performance. You don't take away 40% or greater bandwidth and not be impacted by that.

 

1 hour ago, Refinedpluto said:

Like give me a break, a full power laptop 3070 actually beats a 4070 by a few frames in several games. 

The RTX 3070 Laptop has more CUDA cores and more memory bandwidth, it being faster is as expected as the sun rising in the morning. There has been zero uplift in IPC between RTX 30 and RTX 40 so the marginal increase in clocks for the RTX 4070 Laptop with less CUDA cores and less memory bandwidth makes it no better and in fact worse, absolute performance wise.

 

But lets address something here, performance gains aren't the only goal and it's likely not what Nvidia was targeting. Significant gains in power efficiency is actually more useful in a laptop and it's what has been delivered. That is most likely the target for RTX 40 Laptop not raw performance.

 

https://www.youtube.com/watch?v=ITmbT6reDsw

 

[Quackers101]

price increase for generational improvements, where? maybe taking on raytracing will help my fps, since that is what nvidia wants me to use? shiver my timbers, it got a boost of heat explosion!

[erickft]

On 5/11/2023 at 1:03 PM, leadeater said:

It doesn't top out at 90W, it can use more if enough of the CUDA cores are being utilized properly otherwise the GPU needs to clock higher but gets voltage limited. That's an important difference. When you start pushing something harder causing it to need to use more power then you need to start looking at what specifically and why, removing the voltage safety isn't necessarily going to alleviate that and it's also there for a reason. All the desktop GPUs are in fact limited by the same thing and it's why OC on them doesn't do a lot because Nvidia isn't allowing the voltage to raise high enough to sustain the demanded frequency being asked, this limit is also there for a reason. I myself would argue it could be relaxed a little but I'm certainly not going to say there is no risk there.

 

But then why does the frequency need to raise so much? The GPU cores can use the extra power but what is it about the workload and pipeline that the GPU frequency becomes the most important factor to the performance? I would argue that the CUDA cores are being under utilized and under feed and the GPU clocks need to raise to compensate for that and I would point directly at the memory bandwidth as the cause.

 

Now the desktop RTX 4070 GPU is larger than the laptop by 1280 CUDA cores and all current RTX 40 desktop series use GDDR6X which give about 40% more bandwidth typically but it's likely more since the GDDR6 being used in laptops won't be getting clocked as high as when used in desktop cards. The RTX 4080 Mobile GPU die is also the same one as used by the desktop RTX 4070.

 

I'm simply not seeing much here beyond small GPUs not able to clock high due to legitimate voltage limits and a substantially weaker memory configuration also limiting performance. You don't take away 40% or greater bandwidth and not be impacted by that.

 

The RTX 3070 Laptop has more CUDA cores and more memory bandwidth, it being faster is as expected as the sun rising in the morning. There has been zero uplift in IPC between RTX 30 and RTX 40 so the marginal increase in clocks for the RTX 4070 Laptop with less CUDA cores and less memory bandwidth makes it no better and in fact worse, absolute performance wise.

 

But lets address something here, performance gains aren't the only goal and it's likely not what Nvidia was targeting. Significant gains in power efficiency is actually more useful in a laptop and it's what has been delivered. That is most likely the target for RTX 40 Laptop not raw performance.

 

https://www.youtube.com/watch?v=ITmbT6reDsw

 

I see you presenting points that justify Nvidia limiting the voltage and TGP of the RTX 4050, 4060 and 4070, but there is a very important point that does not fit into this: why advertise cards with a higher TGP, if it is not in fact hit? Does this seem correct to you? If the bandwidth and CUDA core count do not allow the advertised TGP to be achieved, why advertise such TGPs? Isn't this a deception?

That’s insane…

[leadeater]

1 hour ago, erickft said:

I see you presenting points that justify Nvidia limiting the voltage and TGP of the RTX 4050, 4060 and 4070, but there is a very important point that does not fit into this: why advertise cards with a higher TGP, if it is not in fact hit? Does this seem correct to you? If the bandwidth and CUDA core count do not allow the advertised TGP to be achieved, why advertise such TGPs? Isn't this a deception?

That’s insane…

This is a very old thread so it's usually best to not comment on them. Anyway cards with higher TGP have more CUDA cores, each "core" or really execution unit requires power to operate and do work so the more you have the more power you need in total. Voltage is about how high frequency each "core" can operate at and the higher the frequency the more voltage you need so the power per core goes up. But there is a limit to how high the frequency can get before stability is a concern, hopefully before silicon degradation is a concern which is the primary reason voltage limits are imposed.

 

TGP/TDP limits are only a limit, if a card/GPU is not achieving that under what would be an expected sustained load that you would assume would then there is actually a reason for that. If the power draw is under the limit then not power limits are being applied. If the cores are at maximum allowed frequency then they will not draw any more power. The maximum voltage limit is set such that it is below the value require to be limited by the TGB/TDP if all cores are fully utilized.

 

That means if your GPU monitoring is reporting maximum boost frequency is the operating frequency and the power draw/usage is below the TGP/TDP then not all the CUDA cores are being fully utilized. I would also expect something like GPUz monitoring to report the GPU is under voltage limit in that situation and also show memory controller load is 100%

 

 

 

PerCap Reason is the first metric to check

 

We actually have the tools to figure out what a GPU is doing and what if any limiters are active.

 

And the most important point to how/why this happens is not every workload is the same, some require more memory bandwidth and other require more computation. That applies to games as to other things as well.

 

The other thing is it's actually fine if a product doesn't run at TGP/TDP at full load, that is not a problem. Sometimes the power limits are sized to protect from problems or to ensure something will not operate above the designed cooling solution. They aren't actually supposed to be used for "this is the power the card/GPU will operate at always" because that's not true and you can look at basically any review to see this.

[Mark Kaine]

Honestly sounds like a youtube video (no idea what they're talking about lol)

 

tldr: i have no reason to believe their testing methods or results...

 

1 hour ago, leadeater said:

The other thing is it's actually fine if a product doesn't run at TGP/TDP at full load, that is not a problem. Sometimes the power limits are sized to protect from problems or to ensure something will not operate above the designed cooling solution. They aren't actually supposed to be used for "this is the power the card/GPU will operate at always" because that's not true and you can look at basically any review to see this

whats also weird, for example my 3070 uses 270w (easily, and that's also its official tdp, despite "techpowerup" saying otherwise) 

yet it doesn't really perform better than its 220/240w counter parts, at least not significantly. 

 

edit:

 

 

 

 

 

 

https://www.techpowerup.com/gpu-specs/gigabyte-rtx-3070-vision-oc.b8201

 

clueless... cant even read their own data

 

 

 

On 5/11/2023 at 5:31 AM, PotatoSupremacist said:

a so called "reliability voltage limit" that causes GPUs nominally rated for power draws in the realm of 120-150W range to never hit those levels, resulting in performance that is within 1% of 100W GPUs

the problem is, how is this going to be measured when it "cant" hit that power target? Thats why i think "JT" just made this all up (probably honestly thinking its true, i don't want to imply malicious intend, just the usual confirmation bias)

[dizmo]

I don't really have a problem with companies segmenting products like this. Gotta have something for all tiers

[leadeater]

5 hours ago, Mark Kaine said:

whats also weird, for example my 3070 uses 270w (easily, and that's also its official tdp, despite "techpowerup" saying otherwise) 

yet it doesn't really perform better than its 220/240w counter parts, at least not significantly. 

Most desktop GPUs will, because they are less Scrooge McDuck designed and have headroom for partner designs and factory OC etc.

 

Also you have a non-reference 3070, the official Nvidia spec for the RTX 3070 is 220W.

 

And it's not exactly strange that the 3070 can hit it's TGP/TDP limit, the same GPU die is used in higher tier product with a much higher TGB/TDP. Whereas for this story the RTX 40 series mobile parts do not share the GPU dies across models (other than 4050/4060) and are all configured at their design maximums.

Edited October 29, 2023 by leadeater

[Mark Kaine]

3 minutes ago, leadeater said:

Most desktop GPUs will, because they are less Scrooge McDuck designed and have headroom for partner designs and factory OC etc.

 

Also you have a non-reference 3070, the official Nvidia spec for the RTX 3070 is 220W.

oh yes, i know, but the specs on techpowerup for the Gigabyte rtx 3070 vision oc say 220w... when its really 270w (as i linked) which i find weird... they also refuse to fix it lol. ¯\_(ツ)_/¯ 

 

 

 

 

 

[leadeater]

1 minute ago, Mark Kaine said:

oh yes, i know, but the specs on techpowerup for the Gigabyte rtx 3070 vision oc say 220w... when its really 270w (as i linked) which i find weird... they also refuse to fix it lol. ¯\_(ツ)_/¯ 

Well I mean it's not like techpowerup is an official source of anything lol. It's a nice place to look things up but they also put in rumor cards and specs so you have to be real careful about accuracy and also if the information has been updated.

[Mark Kaine]

Just now, leadeater said:

Well I mean it's not like techpowerup is an official source of anything lol. It's a nice place to look things up but they also put in rumor cards and specs so you have to be real careful about accuracy and also if the information has been updated.

yeah, that was my point,  people seem to take it as gospel tho, even when there are obviously inaccuracies (personally i barely use it at all since i learned that, because it's simply not reliable information,  even when "most" of it is probably correct) 

 

8 minutes ago, leadeater said:

And it's not exactly strange that the 3070 can hit it's TGP/TDP limit, the same GPU die is used in higher tier product with a much higher TGB/TDP. Whereas for this story the RTX 40 series mobile parts do not share the GPU dies across models (other than 4050/4060) and are all configured at their design maximums.

ah, i didn't know how/why that works, but that makes sense!

 

i mean,  it might perform better than a 220W counterpart, but frankly the card needs an undervolt for full potential,  without it'll still use 270w but downclocks to its stock boost values due to temps very quickly,  with undervolt it clocks pretty much 2010mhz constantly... which i guess is great, but in most games that really just means like 5-10fps more, which is less impressive.