Nvidia 40-Series power connectors, melting, and burning.

[leadeater]

3 hours ago, Quackers101 said:

GN mostly noted that his adapter is different from igor's

so again quality of cables are different and COULD be an issue.

Seems although the cables were a lower rating the placement and soldering of them was more the issue. Seemed to be just a general all round worse construction quality and different design.

[CarlBar]

50 minutes ago, leadeater said:

Seems although the cables were a lower rating the placement and soldering of them was more the issue. Seemed to be just a general all round worse construction quality and different design.

 

Yeah it seems like there's more than one NVIDIA branded adaptor design going around, which makes it fairly difficult to determine what if anything is going on with the failures, are they all on the weaker design, (i'd expect so but this is somthing we really need confirmation on), or is somthing else going on thats messing things up. Very unclear again all of a sudden. ughh.

[IkeaGnome]

2 hours ago, that_dude said:

It was a bad joke ... sorry

Dw. There’s just been a lot of jumping to conclusions and calling stuff click bait that isn’t really the right conclusion or click bait. That was just the comment I dialed in on. Sorry. 
 

 

2 hours ago, that_dude said:

1. Nobody knows at the moment, except maybe NVidia but they aren't ready to share (legal, PR, etc. working on it) if they know.

For sure. And with the different connectors that GN got compared to Igor with FE cards it just made it more difficult and time consuming to find the root cause. Bad batch? Different cable company cutting costs? We might find out eventually or it’ll be a “we found it. We fixed it.” press release. 
 

 

2 hours ago, that_dude said:

2. GN has no clue what the actual problem is and asked for crowd sourced data/samples

Which is why I do like GNs reporting. Steve was very clear he didn’t know the answer and has to do more research. He doesn’t and didn’t act like he knows everything. Combine that with their willingness to take down videos because of bad information and that gets a lot of respect from me. 

[jos]

It is not just power adapter even native connector melts. So issue runs more deeper

https://www.tomshardware.com/news/rtx-4090-native-16-pin-melting

 

[Kisai]

33 minutes ago, jos said:

It is not just power adapter even native connector melts. So issue runs more deeper

https://www.tomshardware.com/news/rtx-4090-native-16-pin-melting

 

Based on the photo I'd say this is the "bending the cable" type of scenario we were worried about.

 

Too bad he didn't post a photo of the PSU end, though it can be inferred that the melting here is on the GPU end with less damage on the PSU end.

[AtomAlchemist]

At the end of the day it sounds like the ATX 3.0 spec is both in question and not being followed fully by the various manufactures cards and cables. I was hoping to upgrade to a 40 series card, but was putting it off until I could also get my hands on an ATX 3.0 PSU. It is CRAZY to try to blame this on end user cable management, the spec would have to be WAY to tight to have these issues from "user error".

 

There is clearly an issue with too much thermal resistance/heat across the connectors causes the outliers to fail with melted connectors. We've more that doubled the power density per pin from the previous spec and the cables themselves don't seem to be melting or overheating in media attempts to recreate the issue, so it looks like we have too much resistance across the connectors and it's extremely poor customer service to default to "user error" for the outliers being uncovered in this rollout. 

[Quackers101]

12 minutes ago, AtomAlchemist said:

At the end of the day it sounds like the ATX 3.0 spec is both in question and not being followed fully by the various manufactures cards and cables.

It is CRAZY to try to blame this on end user cable management, the spec would have to be WAY to tight to have these issues from "user error".

its not fully by bending, you can bend a lot... just depends on the pressure or force pushed by the bend. but from what others show some bend still adds something to it.

but as said, it just seems fragile of an connector that has too many points of concern. like why couldn't it be "one cable" that also was better in every way compared to the older ones. less points of failure or doing things "wrong".

[Shzzit]

Should i be afraid of using my GF3 1650 atx 3.0?  Or is this being blown way over to push cable sells by certain people?> 

 

My 16 pin cable at the ends is solid copper and you cant even bend it for 1.5".  Hoping that means its built very good. 

[Kisai]

7 hours ago, Quackers101 said:

its not fully by bending, you can bend a lot... just depends on the pressure or force pushed by the bend. but from what others show some bend still adds something to it.

but as said, it just seems fragile of an connector that has too many points of concern. like why couldn't it be "one cable" that also was better in every way compared to the older ones. less points of failure or doing things "wrong".

The smarter solution would have been a single pair of 6AWG wires, but the flexibility wouldn't be there, that's why it's multiple.

12AWG can do 240 watts at 12V. So 3 of those would have been sufficient.

14AWG can do 192 watts at 12V. So 4 x 14AWG (2.08mm) is the minimum. Which begs the question of why the actual contacts aren't 12AWG ( 2.053 mm ) in size to accommodate potential loss-of-one-conductor scenarios.

 

According to this diagram from LAST YEAR the actual plastic is 2.54mm, the center pin space is 0.64mm in diameter.

 

Intel's diagram:

2.4mm is the size to the plastic of each pin with a 0.2mm tolerance. If you do the math that tolerance is pretty tight, with only 0.1mm room for error. My guess is that the connector itself is really too shallow to ensure that the pins have enough surface area connecting the metal should there be pull on the wires. 

 

Like if you've looked at the photos from GN and Igor's lab about the solder jobs on their connectors, note how much larger the actual wires are than the connector. If you lose two conductors, you lose 150w of head room. which means more current goes over the remaining wires. If they are wired like the one in Igor's Lab, then the end pins only have one conductor wired to them (in the adapter). 

 

So what about a ATX 3.0 12VHPWR connector on both ends? Surely there's no solder involved on these? Shouldn't the actual pins be crimped to the connector? I don't know, I don't have any of them to examine.

 

But it does make me wonder if the pin really is 0.64mm on the socket side. Because if it is, that makes it 22AWG equivalent. But again... not that simple, we need to talk about surface area not just the diameter.

 

The PCIe and EATXPWR connectors have 4.2mm pitch

The minifit jr has at least 9mm of depth with 6.7mm of fit depth.

The 12VHPWR connector has apparently 4.6mm of fit depth.

 

By all accounts they should not have picked a smaller connector. Now I'm not an EE, so someone who actually knows the math behind this could probably come up with a better answer. But just based on everything we know right now, the problem is squarely on the design of the connector.

 

So the most logical connector would have to be one that can take the over-current like an 12AWG 8-pin connector where the wires are instead clamped down with a 4 screws, but there's also an easier fix.

 

Put a fused 12pin-to-12pin connector extension, where if the pin exceeds 200w it trips the fuses and shuts down the card. That would prevent the card or PSU from being damaged.

 

[Deadpool2onBlu-Ray]

On 11/4/2022 at 1:12 PM, Kisai said:

Based on the photo I'd say this is the "bending the cable" type of scenario we were worried about.

 

Too bad he didn't post a photo of the PSU end, though it can be inferred that the melting here is on the GPU end with less damage on the PSU end.

This whole thing is starting to get really really really bad lol 

[Kid.Lazer]

I have a question... Since we were re-designing the connectors anyway because having 3x 8-pin connectors was getting stupid, why on earth did we decide on "let's still use a large quantity of tiny wires"? Trying to evenly distribute 50+ amps through 6 pairs of wires is beyond stupid. One single failure on any one pair puts too much load on the remaining ones. You don't have to be an electrical engineer to know that's a bad plan. Why not just use a 2-wire cable, made from 8 AWG with a connector that actually has some meat to it, say, an Anderson Powerpole connector? Bam, done. And sure, throw in some sense wires, or whatever other nonsense, I guess.

 

Oh, and if you think "that wire would be too thick to properly cable-manage," then you should complain to NVIDIA to get their heads out of their butts and stop making cards that require their own power plant...

[Kisai]

2 hours ago, Kid.Lazer said:

I have a question... Since we were re-designing the connectors anyway because having 3x 8-pin connectors was getting stupid, why on earth did we decide on "let's still use a large quantity of tiny wires"?

Because the wires need to flex. Have you seen how thick the cables are on a 12V car battery? Has to be able to sustain 600amps over a few seconds. That's why the cables are so thick, and why jumper cables are massive.

 

GPU's have those micro fluctuations as well. So it will pull more than 600w for moments, but average around 450w.  These melt stories seem to suggest that there was a miscommunication somewhere. Almost certainly on Nvidia's side. Perhaps this problem would not have started had they actually designed it as a single rail that goes over 6 wires instead of what appears to be 4 rails spread over 6 wires.

 

 

2 hours ago, Kid.Lazer said:

Trying to evenly distribute 50+ amps through 6 pairs of wires is beyond stupid. One single failure on any one pair puts too much load on the remaining ones. You don't have to be an electrical engineer to know that's a bad plan. Why not just use a 2-wire cable, made from 8 AWG with a connector that actually has some meat to it, say, an Anderson Powerpole connector? Bam, done. And sure, throw in some sense wires, or whatever other nonsense, I guess.

That makes the most logical sense, but once you get beyond a certain point you no longer make it easy to remove. That's why battery cables have so much area making contact. Where are you going to fit that on a GPU? Plus the potential for arc'ing when the contacts are close to each other.

 

2 hours ago, Kid.Lazer said:

Oh, and if you think "that wire would be too thick to properly cable-manage," then you should complain to NVIDIA to get their heads out of their butts and stop making cards that require their own power plant...

No kidding.

 

Logically the solution is to tell Nvidia to tune down the cards. Maybe extend the PCIe cables with two data pins to signal if it's 75w, 150w, or 300w capable. 

 

[Kid.Lazer]

1 hour ago, Kisai said:

Because the wires need to flex. Have you seen how thick the cables are on a 12V car battery? Has to be able to sustain 600amps over a few seconds. That's why the cables are so thick, and why jumper cables are massive.

I mean, compared to car cables, yeah. Most battery cables are somewhere between 4 and 0 AWG (depending on engine/battery size), which would be difficult to use in a computer case with tight spaces. But even so, an 8 AWG cable isn't that stiff. (And with 90°C cables, it would be good for 660 watts continuous load). A high-quality stranded wire with a good, flexible jacket (not PVC) will handle very well. The ultimate irony is that having 12, let me say it louder, 12 cables! wastes exponentially more insulating material, and will undoubtedly handle as though it is stiffer.

 

Just look at our current 24-pin ATX motherboard plugs. While it admittedly needs more wires because it does more than carry simple 12V and ground, they are a bear to properly manage even though the majority of the wires are very small.

 

1 hour ago, Kisai said:

Where are you going to fit that on a GPU? Plus the potential for arc'ing when the contacts are close to each other.

Fitting shouldn't be a problem. When the GPUs in question that need such a connector use 3-4 slots vertically while also being a foot long, having a proper (larger) electrical termination should be a trivial matter at that point. And clearly arcing isn't a concern if the current plan has them melting to the ground!

 

On the highly subjective side, I think having 2 massive wires running to my GPU would look waaaaaay cooler than a nest of 16 wires. Coupled with water cooling tubes, it would be a sweet look. (For the record, I don't have one of these cards, nor water cooling, but I can imagine the potential.)

[jos]

I have a theory, the highest resistant path that is not rated is the connector. If there are constant fluctuations in transient voltage due to some unknown transistor combinations, it can cause the burning of adapter. Just a theory

[CarlBar]

20 hours ago, Kid.Lazer said:

I have a question... Since we were re-designing the connectors anyway because having 3x 8-pin connectors was getting stupid, why on earth did we decide on "let's still use a large quantity of tiny wires"? Trying to evenly distribute 50+ amps through 6 pairs of wires is beyond stupid. One single failure on any one pair puts too much load on the remaining ones. You don't have to be an electrical engineer to know that's a bad plan. Why not just use a 2-wire cable, made from 8 AWG with a connector that actually has some meat to it, say, an Anderson Powerpole connector? Bam, done. And sure, throw in some sense wires, or whatever other nonsense, I guess.

 

Oh, and if you think "that wire would be too thick to properly cable-manage," then you should complain to NVIDIA to get their heads out of their butts and stop making cards that require their own power plant...

 

The real issue is that it apparently has effectively zero room for any kind of excess draw, anything screws with it's resistance or otherwise creates imbalanced load and it's going to fail.

 

I mentioned this before but the apparent sub 5% safety margin on the design really isn't adequate, it doesn't need to be the insane >2x margin of the old 8 pin PCI_E design, but it needs to be more than it is. If any western nation national safety organisation had jurisdiction here i suspect they'd reject the connector or downrate it significantly.

 

the point about power spikes is interesting as well, i wonder what the behaviour looks like under a range of use cases, could some use case be producing sufficient spikes in a short period of time to exacerbate issues? Average power draw alone isn't allways the only factor.

[Kid.Lazer]

2 hours ago, CarlBar said:

If any western nation national safety organisation had jurisdiction here i suspect they'd reject the connector or downrate it significantly.

16 hours ago, jos said:

I have a theory, the highest resistant path that is not rated is the connector. If there are constant fluctuations in transient voltage due to some unknown transistor combinations, it can cause the burning of adapter. Just a theory

Clearly the connector is the issue. Assuming every wire in the cable is the 16AWG spec, it would be good for 100 Amps, or 1200 Watts (assuming 90°C rating). Since the entirety of the cable is only rated at 600 Watts, they obviously know the connector has short-comings. That disparity in itself is quite confusing too. Why waste money on oversized wires if you could never fully saturate them anyway? The whole design is comically bad. (Not that burning your GPU to the ground is ha-ha funny.)

[leadeater]

58 minutes ago, Kid.Lazer said:

Clearly the connector is the issue. Assuming every wire in the cable is the 16AWG spec, it would be good for 100 Amps, or 1200 Watts (assuming 90°C rating). Since the entirety of the cable is only rated at 600 Watts, they obviously know the connector has short-comings. That disparity in itself is quite confusing too. Why waste money on oversized wires if you could never fully saturate them anyway? The whole design is comically bad. (Not that burning your GPU to the ground is ha-ha funny.)

Cable spec is more complicated than that. Current ratings are at lower ambient than typically found in PC cases and high temperatures reduce the safe current capabilities. Also cable choice isn't only based on current ratings, choice is based on strength and durability as well as the size itself for crimping or solder bonding force.

 

Pick a too smaller cable and it'll fail from mechanical stress factors.

 

Also the actual safe current range for 16AWG under continuous loads is much lower than what you've said above. And for those wondering why not just use fewer much larger cables, voltage drop. You get better and more stable voltage across multiple conducting wires than you do across less, so if you need to stay within say an ATX specification for a 12V application then 6 conducting pairs is superior to 2 or 4.

 

That's what I did for the external battery wiring for my UPS, 76V nominal operating voltage however over the 5m distance the voltage drop was above what I found acceptable and effected the UPS run time due to low battery voltage alarm triggering too soon. Running a second cable was much easier and more effective than increasing the size of the single cable to achieve the same result.

[Kid.Lazer]

2 hours ago, leadeater said:

Also the actual safe current range for 16AWG under continuous loads is much lower than what you've said above.

I did mention that it assumes a 90°C rated wire, which, depending on the data source, has an ampacity of 18-20 amps. 18(amps)*6(positive conductors)*12(volts)=1296(Watts).  My 1200 stated earlier was rounding down considerably. But you are right that the temperatures in a PC would lower that, although not by a ton, in my opinion. A typical PC case has, maybe a 40°C ambient air? while the ratings are based on 30°C ambient.

 

2 hours ago, leadeater said:

Pick a too smaller cable and it'll fail from mechanical stress factors

This is true, but in cases where tight bends are needed (you know, like turning 180° to clear your GPU), the smallest conductor you can get away with would put overall less stress on the conductors and connectors. To be clear, I'm not saying that we should make every cable the exact size, and no bigger. There should be some headroom. But being capable of double the overall adapter's rating is a bit overkill. It's also obvious that it's overkill, since the conductors are fine (with some of the adapter cables being much smaller than 16AWG) yet it's still the connectors themselves that are failing.

 

2 hours ago, leadeater said:

You get better and more stable voltage across multiple conducting wires than you do across less

DC lines are very prone to voltage drop, but in the very short runs within a PC case (typically less than 2'), it's a non-issue. As for using multiple conductors, as I understand it, that is only useful in very high voltage applications. For a 12VDC circuit, I don't think it would serve any use, but I will admit to not being as knowledgeable about that.

[CarlBar]

Remember a wires rating is it's absolute maximum, you should never push it that far. Doing a bit of quick digging suggests that 16AWG shouldn't normally be loaded beyond about 15 amps. Also remember voltage drop due to both distance and load does occur, from what i recall from memory under load drops of a couple of tenths of a volt or so can occur, thats going to drive up the current draw for a given power.

 

Wondering now where @Kisai got their values from earlier as they look way too low by comparision.

[Vishera]

21 hours ago, leadeater said:

Cable spec is more complicated than that. Current ratings are at lower ambient than typically found in PC cases and high temperatures reduce the safe current capabilities. Also cable choice isn't only based on current ratings, choice is based on strength and durability as well as the size itself for crimping or solder bonding force.

 

Pick a too smaller cable and it'll fail from mechanical stress factors.

 

Also the actual safe current range for 16AWG under continuous loads is much lower than what you've said above. And for those wondering why not just use fewer much larger cables, voltage drop. You get better and more stable voltage across multiple conducting wires than you do across less, so if you need to stay within say an ATX specification for a 12V application then 6 conducting pairs is superior to 2 or 4.

 

That's what I did for the external battery wiring for my UPS, 76V nominal operating voltage however over the 5m distance the voltage drop was above what I found acceptable and effected the UPS run time due to low battery voltage alarm triggering too soon. Running a second cable was much easier and more effective than increasing the size of the single cable to achieve the same result.

Just take the tried and proven 8-pin connector and add 8 more pins to it.

The 2 pins in the 6+2 are rated for 75W, so adding 8 more pins should give you a 6+2+8 connector capable of at least 450W.

Now the thing about the 8 pin connector is that it's rated for 150W but is capable of delivering 300W, so 600W should work without issues on the 6+2+8 connector i suggested.

 

After all in the past AMD already pushed the 8-pin connector beyond what it's rated for with the R9 295X2,

That monster pulls 500W from 2x 8-pin connectors (Which is higher than the power consumption of the RTX 4090).

[Kisai]

3 hours ago, CarlBar said:

Remember a wires rating is it's absolute maximum, you should never push it that far. Doing a bit of quick digging suggests that 16AWG shouldn't normally be loaded beyond about 15 amps. Also remember voltage drop due to both distance and load does occur, from what i recall from memory under load drops of a couple of tenths of a volt or so can occur, thats going to drive up the current draw for a given power.

 

Wondering now where @Kisai got their values from earlier as they look way too low by comparision.

There are many charts out there that immediately drop the 16AWG wires off. 

 

Like this one https://www.bluesea.com/support/articles/Circuit_Protection/1437/Part_1%3A_Choosing_the_Correct_Wire_Size_for_a_DC_Circuit

or this one

https://www.crutchfield.com/S-D58FQtFSSC5/learn/learningcenter/car/cable_gauge_chart.html

The second link starts at 12AWG

 

This one has everything below 12AWG

https://4diyers.com/resources/electric/wiregaugechart.php

 

https://www.engineeringtoolbox.com/amps-wire-gauge-d_730.html

The problem is a lot of the fixed charts make assumptions like the boat and car ones start at 15ft, or the temperature is 50 degrees.

 

I did not use a specific calculator, which if I did, would probably suggest 14AWG

This calculator doesn't incorperate the temperature,

https://www.wirebarn.com/Wire-Calculator-_ep_41.html

but recommends 8 AWG @ 50A. (600w x 12V = 50A) to take the full load.

If it's cut down to 25A (thus requiring a minimum of 2 pairs wires) then 12 AWG

If it's cut down to 12.5A (requiring 4 pairs, like PCIe) then 14AWG

(The calculator still recommends 16AWG until 10A, which means only 1 wire could be lost.)

 

That leaves no headroom for a wire failure. Which is why seeing 14AWG wire on the 6 pairs is a bit of a head scratcher if it's supposed to be 600w. By all accounts you should be able to lose two wires and not see melting, but clearly not the case, which suggests, again, the design of the connector is insufficient. If you needed 14AWG wire, why isn't the connector at least 14AWG (1.6mm) as well to the MB?

 

The official spec of the (6-pin/8pin) PCIe connector is 150 watts. and is larger than the 12 pin connector. Incidentally, the EPS12V connector, which also is an 8-pin connector is 336w (28A).  The PCIe and the EPS12V connector are essentially identical except for how it's key'd. So if we have this headroom with both PCIe and EPS12V, why do we not have it with 12VHPWR? What is fundamentally different about these connectors? Just the contact surface area inside the connector really.

 

[leadeater]

21 hours ago, Kid.Lazer said:

I did mention that it assumes a 90°C rated wire, which, depending on the data source, has an ampacity of 18-20 amps. 18(amps)*6(positive conductors)*12(volts)=1296(Watts).  My 1200 stated earlier was rounding down considerably.

 

Quote

Based on Table 310.16 of the NEC, 16 AWG copper wires are suitable for up to 18 amps in standard circuits with insulation and terminations rated for 90°C. Continuous loads used with this size should not exceed 8 amps.

 

Basically nobody considered 18A for 16 AWG safe or practically usable for DC loads.

 

21 hours ago, Kid.Lazer said:

DC lines are very prone to voltage drop, but in the very short runs within a PC case (typically less than 2'), it's a non-issue. As for using multiple conductors, as I understand it, that is only useful in very high voltage applications. For a 12VDC circuit, I don't think it would serve any use, but I will admit to not being as knowledgeable about that.

For 0.5m (less than 2 feet) at 9.3A the voltage drop across just the cable itself would be more than 0.15V or greater than 1.25%. The ATX 12V spec is 5%. Smaller cable conductors would likely be too greater risk of total end to end voltage drop.

 

17AWG would be 2%, so personally I would be rather confident in saying the device would end up regularly getting under the 12V ATX specification.

 

Sometimes gut feel can be significantly off from the situation.

[Kid.Lazer]

32 minutes ago, leadeater said:

Basically nobody considered 18A for 16 AWG safe or practically usable for DC loads.

Then you've never seen me wire up my Christmas lights that you could probably see from space.

 

32 minutes ago, leadeater said:

17AWG would be 4%, so personally I would be rather confident in saying the device would end up regularly getting under the 12V ATX specification.

I unfortunately can't remember what company it was, but one of the custom cable makers actually recommended 17AWG for some reason. If I ever find it I'll edit this. The point being though, clearly they think it's ok and 16AWG is unnecessary.

 

[leadeater]

2 hours ago, Kid.Lazer said:

I unfortunately can't remember what company it was, but one of the custom cable makers actually recommended 17AWG for some reason. If I ever find it I'll edit this. The point being though, clearly they think it's ok and 16AWG is unnecessary.

And they could well be wrong. It's quite common for a PSU to only put out 11.7V-11.8V under load, at 11.8V without any other losses you'll be hovering around 11.5V and the minimum allowed is 11.4V. Nobody should be actually trying to run at minimum allowable, that's not what anyone should be aiming for.

 

There's also very minor losses through the connectors and also through the device power stages before step down to the required device rail voltages.

 

16 AWG seems about right to me, I'd want larger however any bigger then mechanical factors become a problem. So that basically leaves 16 AWG as the only viable option without increasing the number of conducting pairs.

 

Anyway 11A-13A is the more common acceptable current for a 16 AWG (1.5mm²) cable, at 30C ambient.

[Deadpool2onBlu-Ray]

Apparently these are melting because people aren't plugging them in all the way. There was an interesting article from Johnny Guru on the subject. You really have to push these 16 pin connectors in HARD