Folding Community Board

[Favebook]

F@H got a shout-out at the end of the video:

 

[NetoriusNick]

slim chance but does anyone have the board diagram for a vega 64? had a cap blow and would like to replace it

 

[IkeaGnome]

5 minutes ago, NetoriusNick said:

slim chance but does anyone have the board diagram for a vega 64? had a cap blow and would like to replace it

 

Want PDF or boardview?

[NetoriusNick]

both would not hurt but boardview would be the first one please

[IkeaGnome]

10 minutes ago, NetoriusNick said:

both would not hurt but boardview would be the first one please

I've got 3 different sets for different AIBs. The guy that I got my files from did a horrible job at naming them though or someone else up the line did. Here's all three. Hopefully one matches what you've got. They're all for the 64 though.

V368_10(00017967)_Decrypt.pdfV368_10.cadV368_11(00017967)_Decrypt.pdfV368_11.cadV368_20(00017967)_Decrypt.pdfV368_20.cad

[NetoriusNick]

11 hours ago, IkeaGnome said:

I've got 3 different sets for different AIBs. The guy that I got my files from did a horrible job at naming them though or someone else up the line did. Here's all three. Hopefully one matches what you've got. They're all for the 64 though.

V368_10(00017967)_Decrypt.pdf 5.86 MB · 2 downloads V368_10.cad 2.46 MB · 0 downloads V368_11(00017967)_Decrypt.pdf 624.5 kB · 0 downloads V368_11.cad 2.26 MB · 0 downloads V368_20(00017967)_Decrypt.pdf 833.99 kB · 0 downloads V368_20.cad 1.85 MB · 0 downloads

ekk was there time limit on the down loads?

 

[IkeaGnome]

8 hours ago, NetoriusNick said:

ekk was there time limit on the down loads?

 

Time limit for how long it stays in this thread? Not that I'm aware of. If there is and you need them again just feel free to ask. I've got a ton of other boards too. Unzipped it totals to 2 3/4 gigs.

[NetoriusNick]

1 minute ago, IkeaGnome said:

Time limit for how long it stays in this thread? Not that I'm aware of. If there is and you need them again just feel free to ask. I've got a ton of other boards too. Unzipped it totals to 2 3/4 gigs.

[IkeaGnome]

4 minutes ago, NetoriusNick said:

Oh that's the forums doing the broken attachment thing again. I can't remember what you have to remove from the broken link. I can also send them over Discord if you'd want.

Andrew#9623 PFP matches my PFP on here.

[NetoriusNick]

1 hour ago, IkeaGnome said:

Oh that's the forums doing the broken attachment thing again. I can't remember what you have to remove from the broken link. I can also send them over Discord if you'd want.

Andrew#9623 PFP matches my PFP on here.

struggled to find you not offlay with discord sent friend request

 

[dpt17]

New here to the forums.

 

I started doing F@H back at the beginning of the pandemic at Linus' behest.  IIRC, he had a video promoting it at the time.  And, it seemed like a reasonable thing for someone like me, a software developer / PC enthusiast, to help fight diseases.  So I did it and credited the LTT team.  I started by just running the MSI 2070 Super that was in my all-purpose Windows "server" PC 24/7.  Now, it's a 3080 Ti continuously running on Ubuntu.  At some point this card won't keep pace and grabbing a second 4090 (just grabbed one for my gaming PC) for F@H feels painful to my wallet.

 

It never occurred to me, until Linus suggested it this week in his garage heater video, to see if there was a community here to chat about F@H.  Anyway ... hello everyone!

 

https://folding.extremeoverclocking.com/user_summary.php?s=&u=1034023

[justpoet]

52 minutes ago, dpt17 said:

New here to the forums.

Welcome to the fold!

 

Many of us use folding as an extra heat source.  That's actually one of the reasons we also have the winter "folding month" event when we do.  Glad you've found us, and have been participating even without the community support!

[Gorgon]

6 hours ago, dpt17 said:

New here to the forums.

 

I started doing F@H back at the beginning of the pandemic at Linus' behest.  IIRC, he had a video promoting it at the time.  And, it seemed like a reasonable thing for someone like me, a software developer / PC enthusiast, to help fight diseases.  So I did it and credited the LTT team.  I started by just running the MSI 2070 Super that was in my all-purpose Windows "server" PC 24/7.  Now, it's a 3080 Ti continuously running on Ubuntu.  At some point this card won't keep pace and grabbing a second 4090 (just grabbed one for my gaming PC) for F@H feels painful to my wallet.

 

It never occurred to me, until Linus suggested it this week in his garage heater video, to see if there was a community here to chat about F@H.  Anyway ... hello everyone!

 

https://folding.extremeoverclocking.com/user_summary.php?s=&u=1034023

A belated welcome aboard.  If you head over to the Badge Request Thread and request it you can get a badge that will display under your name and link to your stats.

 

If you want to help save up for that 4090 you can run:

nvidia-smi -pm 1
nvidia-smi -i 0 -lgc 0,1440

to run your 3080ti at much better efficiency.

[dpt17]

19 hours ago, Gorgon said:

If you want to help save up for that 4090 you can run:

nvidia-smi -pm 1
nvidia-smi -i 0 -lgc 0,1440

to run your 3080ti at much better efficiency.

Interesting ...

nvidia-smi -pm 1

Looks to be https://docs.nvidia.com/deploy/driver-persistence/index.html ... which talks about keeping the card active even with no monitor.  I already solved that problem years ago by buying a dongle to emulate a 4k monitor.  That was the only way that network/stream software (e.g. remote desktop, chrome remote desktop, Steam Link, Nvidia Gamestream, etc) would work going back to the days where this box was Win10.  I just kept the dongle plugged into the card when switching to Linux so I maybe I don't need this one.

nvidia-smi -i 0 -lgc 0,1440

I'm curious as to why setting the clocks for the GPU to that range would help ... if I run "nvidia-smi -q" the clock speed appears to be ~1650 MHz for the 3080 Ti.  Is the "efficiency" you're describing related to power savings?  The power draw for the card always seems to be ~320W out of 350W max.

[justpoet]

52 minutes ago, dpt17 said:

 

nvidia-smi -i 0 -lgc 0,1440

I'm curious as to why setting the clocks for the GPU to that range would help ... if I run "nvidia-smi -q" the clock speed appears to be ~1650 MHz for the 3080 Ti.  Is the "efficiency" you're describing related to power savings?  The power draw for the card always seems to be ~320W out of 350W max.

Yes. The ppd/watt is far superior lower down the power limit.  You don't loose much in the ppd department, but you drop a lot of wattage use.

[Gorgon]

13 hours ago, dpt17 said:

Interesting ...

nvidia-smi -pm 1

Looks to be https://docs.nvidia.com/deploy/driver-persistence/index.html ... which talks about keeping the card active even with no monitor.  I already solved that problem years ago by buying a dongle to emulate a 4k monitor.  That was the only way that network/stream software (e.g. remote desktop, chrome remote desktop, Steam Link, Nvidia Gamestream, etc) would work going back to the days where this box was Win10.  I just kept the dongle plugged into the card when switching to Linux so I maybe I don't need this one.

nvidia-smi -i 0 -lgc 0,1440

I'm curious as to why setting the clocks for the GPU to that range would help ... if I run "nvidia-smi -q" the clock speed appears to be ~1650 MHz for the 3080 Ti.  Is the "efficiency" you're describing related to power savings?  The power draw for the card always seems to be ~320W out of 350W max.

Persistence mode is necessary so any changes made to the GPU are retained after the current WU is completed and the the next is to begin. Otherwise it will just revert to the default driver settings.

 

Modern GPUs, like modern CPUs, are pushed to the edge of stability by default. This means the manufacturer designs them to runs their clocks as high as they can go, on average, at the expense of excessive power consumption. Just like when overclocking a CPU or GPU, minor increases in Frequency are obtained at the expense of a disproportionate increase in Power.

 

The Power vs. Frequency Graph of a Ampere GPU (3080 running p18601) looks like this:

So we see at the lower Clock speeds an almost Linear increase in Power required for an increase in Graphics Clock. Around 1350-1450MHz things change. Increases in Clock speed now require exponential increases in power.

 

If we look at the Efficiency of the GPU at the same range of Graphics Clock we see:

In the center between 1250 and 1450MHz we see a flattening or a "sweet spot" where the Efficiency peaks. Under 1250MHz and above 1450 the Efficiency falls off a cliff for Ampere.

 

It's kind of like taking a long trip in a car. On the AutoBahn you can drive from point A to B at the maximum speed of your vehicle but you will consume much more gas doing so and be far more likely to have an accident. Driving the same route at 55mph will result in much lower gas consumption and less wear to the vehicle and risk to the passengers.

[dpt17]

Got it.  Thanks for the explanation.  Makes sense.

 

I've been able to do some runs of my new 4090, a PNY Verto OC that has a max TDP of 450W.  When running F@H on my Win11 gaming PC the heat output into the room doesn't feel like as much as the EVGA 3090 FTW3 Ultra that I took out.

 

I was tempted to replace the EVGA 3080 Ti XC3, the always-folding card discussed here, with this 3090 but don't think I will.  It has a width of 2.2 slots as compared to the gargantuan 2.75 that the 3090 does; I could fit another GPU into my mobo alongside the 3080 Ti if I wanted to but certainly could not do that with the 3090.  And, that 3080 Ti model only needs 2 8-pin power connectors vs. the 3 needed for the 3090, making it a much more flexible card.  The folding ppd difference between the two isn't significant from what I have seen.

 

Does PCIe Gen3 vs. Gen4 make a difference with folding?  My folding box is a z590 ASUS Hero w/i9-10850k.  I could swap out the cpu for an  i7-11700k or i9-11900k to unlock Gen4 at some point I doubt that would make a big difference with just the 1 3080 Ti in it.  It would more likely make an impact if there were 2 cards installed as Gen4 x8 ~= Gen3 x16.  The only stinky thing would be dropping from 10 cores / 20 hyperthreads to 8 / 16.  I don't think this would be a big deal as the host Ubuntu system would get 7 / 14 and the guest Win10 VM 1 / 2.  The former should be way more than enough for what I am doing.

[Gorgon]

3 hours ago, dpt17 said:

Got it.  Thanks for the explanation.  Makes sense.

 

I've been able to do some runs of my new 4090, a PNY Verto OC that has a max TDP of 450W.  When running F@H on my Win11 gaming PC the heat output into the room doesn't feel like as much as the EVGA 3090 FTW3 Ultra that I took out.

 

I was tempted to replace the EVGA 3080 Ti XC3, the always-folding card discussed here, with this 3090 but don't think I will.  It has a width of 2.2 slots as compared to the gargantuan 2.75 that the 3090 does; I could fit another GPU into my mobo alongside the 3080 Ti if I wanted to but certainly could not do that with the 3090.  And, that 3080 Ti model only needs 2 8-pin power connectors vs. the 3 needed for the 3090, making it a much more flexible card.  The folding ppd difference between the two isn't significant from what I have seen.

 

Does PCIe Gen3 vs. Gen4 make a difference with folding?  My folding box is a z590 ASUS Hero w/i9-10850k.  I could swap out the cpu for an  i7-11700k or i9-11900k to unlock Gen4 at some point I doubt that would make a big difference with just the 1 3080 Ti in it.  It would more likely make an impact if there were 2 cards installed as Gen4 x8 ~= Gen3 x16.  The only stinky thing would be dropping from 10 cores / 20 hyperthreads to 8 / 16.  I don't think this would be a big deal as the host Ubuntu system would get 7 / 14 and the guest Win10 VM 1 / 2.  The former should be way more than enough for what I am doing.

PCIe3x8 is fine for 3080/3090. You'll see little to no decrease in yield over Gen4.

 

Here's a 3080's PCIe Utilization on a Gen3x8:

 

Lots of headroom left. The 2070 Super in the other slot uses about 1/2 the utilization.

[RollinLower]

On 1/23/2023 at 2:04 AM, Gorgon said:

PCIe3x8 is fine for 3080/3090. You'll see little to no decrease in yield over Gen4.

 

Here's a 3080's PCIe Utilization on a Gen3x8:

 

Lots of headroom left. The 2070 Super in the other slot uses about 1/2 the utilization.

how do you even poll this info? could it be possible to make a grafana board with live insights on this or something?

[Gorgon]

8 hours ago, RollinLower said:

how do you even poll this info? could it be possible to make a grafana board with live insights on this or something?

I use the Nvidia System Management Interface (nvidia-smi) Device Monitoring (dmon) to poll the PCI bus Transmit (Tx) and Receive (Rx) throughputs on my Zabbix server:

UserParameter=pci.rx[*],nvidia-smi dmon -i $1 -c 1 -s t | grep -v ^# | sed -e 's/^ *//g'| sed -e 's/  */ /g' | sed -e 's/ /\t/g' | cut -f2
UserParameter=pci.tx[*],nvidia-smi dmon -i $1 -c 1 -s t | grep -v ^# | sed -e 's/^ *//g'| sed -e 's/  */ /g' | sed -e 's/ /\t/g' | cut -f3

It should be possible to extend this to any monitoring framework.

 

It is not close 100% accurate as with any monitoring Nyquist's Sampling Theorems apply and I'm just polling once per minute so it's just going to capture an approximation of what's going on.

[Mxyzptlk]

A Folder on the F@H is folding two 4090's in a system that he only has 3.0 x 4x. Getting full PPD too!  Not sure how though.

[Gorgon]

20 minutes ago, Mxyzptlk said:

A Folder on the F@H is folding two 4090's in a system that he only has 3.0 x 4x. Getting full PPD too!  Not sure how though.

I actually wouldn't expect PCIe3 x4 to bottle-neck a 4090 too much, if any. A lot less data is sent over the PCIe bus for Folding compared to Gaming. Generally just at the start of a frame (1/100 of a WU) to setup the calculations then the result would be sent back at the end of the Frame.

 

Even if the PCIe bus-width is a little too narrow the result would be a slight delay loading data and pulling results which in comparison to the time spent for calculations is only a small part of the frame time (TPF).

 

Generally mining risers are a poor idea as they're only PCIe3x1.

[keeph8n]

I’m currently folding on two 4090s as well. The SR3 is gen3 and one card is seeing x16 and the other x8 until I can get the waterblocks on. 

 

Both cards perform the same and provide the same ppd. 

 

Seems I don’t have the updated pictures with the two AMP cards installed 

 

[Mxyzptlk]

22 hours ago, Gorgon said:

I actually wouldn't expect PCIe3 x4 to bottle-neck a 4090 too much, if any. A lot less data is sent over the PCIe bus for Folding compared to Gaming. Generally just at the start of a frame (1/100 of a WU) to setup the calculations then the result would be sent back at the end of the Frame.

 

Even if the PCIe bus-width is a little too narrow the result would be a slight delay loading data and pulling results which in comparison to the time spent for calculations is only a small part of the frame time (TPF).

 

Generally mining risers are a poor idea as they're only PCIe3x1.

Im actually not sure what he is running now. This morning he says he is actually on 4.0 x 4, which still doesn't make much sense...?

[Mxyzptlk]

@LAR_Systems I am curious with the PPD Estimates for the 4070ti. They seem.... High. I am seeing my 4070ti in the 11 - 12 range, but almost every WU has a crazy high spike that might be effecting the numbers?