Jump to content
Search In
  • More options...
Find results that contain...
Find results in...

Folding Community Board

Go to solution Solved by GOTSpectrum,

So bad news, todays stream went pretty terribly, good new I should be sorting prizes sometime today 

On 7/7/2022 at 3:08 AM, Gorgon said:

Personally I just gave up on AMD under Linux as their OpenCL drivers are much less performant than NVIDIA's CUDA drivers.

After much complaining to AMD (you'd think they'd notice sooner, as an official partner of Folding@Home), they have finally taken notice of the declining compute performance of AMD cards in OpenCL, and their driver update lists this as a known issue. Hopefully they'll fix it on both Windows and Linux.

Quote
  • Lower than expected Folding@home™ compute performance with OpenCL™ API on some AMD Graphics Products such as the Radeon™ RX 6800.

My part of Norway has fairly cool temperatures and almost free electricity, due to fairly full hydropower magazines and ample wind power, so I'm folding in the middle of summer. I didn't fold as much last summer, since electricity was more expensive and the weather was warmer.

Link to comment
Share on other sites

Link to post
Share on other sites

19 hours ago, Gorgon said:

16 Billion!

16Billion.thumb.jpg.8686684f8fab955747e3b3a28b888c73.jpg

 

@Justaphf the baton will be your in a couple of days - take it to 20! Congratulations.

Slowly working my way up page 1.  I actually just recently consolidated a couple of rigs into one using your excellent Bifrost guide, thanks!

16 hours ago, RollinLower said:

42M a day on a non-competition basis. that's pretty wild!

Yea, I typically run flat out 24/7/365....though I just got last month's power bill and I may need to start using a time-of-use strategy until the heat dies down, first time over $400, ouch.  Even with a lower TPD limit set, those 30 series RTX cards are thirsty.

Link to comment
Share on other sites

Link to post
Share on other sites

15 minutes ago, Justaphf said:

Slowly working my way up page 1.  I actually just recently consolidated a couple of rigs into one using your excellent Bifrost guide, thanks!

Nice. I went back to multiple rigs mostly due to having to baby sit the 6-gpu rig constantly when the download bug hits but the GPUs do run cooler in an open frame.

Quote

Yea, I typically run flat out 24/7/365....though I just got last month's power bill and I may need to start using a time-of-use strategy until the heat dies down, first time over $400, ouch.  Even with a lower TPD limit set, those 30 series RTX cards are thirsty.

Yes, the electricity bill can be a shocker.

 

I'm disappointed with Ampere. After having to wait almost 2 years for them to be a reasonable price Ampere is only marginally more efficient than Turing. It seems most of their performance gains have been due to their higher power budget and their operating at the higher end of the voltage/frequency curve.

 

And the projections for Lovelace don't look any better given the ridiculous power budgets being suggested.

 

It appears both AMD and NVidia have given up on improving efficiency in the battle for top performance.

FaH BOINC HfM

Bifrost - 6 GPU Folding Rig  Linux Folding HOWTO Folding Remote Access Folding GPU Profiling

Systems:

desktop: Lian-Li O11 Air Mini; Gigabyte Aorus ax570 Master; Ryzen 9 5900x;Noctua NH-D15; 4 x 16GB DDR4-3200; 500GB PCIe3 NVMe; 4 x 3TB NAS; AMD RadeonPro W4100; EVGA RTX 2070A; Corsair TX650M

nas1: Fractal Define R4; SuperMicro X10sl7-f; Xeon e3-1231v3; 4 x 8GB DDR3-1666 ECC; 6 x 3TB NAS; 6 x 4TB NAS

nas2: Synology DS-123j; 2 x 6TB NAS

dcn01: Fractal Meshify C; Gigabyte Aorus ax570 Master; Ryzen 9 5950x; EVGA 240 CLC; 2 x 32GB DDR4-3200; 512GB NVMe; EVGA RTX 2070 Super XC Hybrid; Corsair TX650M

dcn02: Fractal Define S; Gigabyte ax570 Pro WiFi; Ryzen 9 3950x; Noctua NH-D15; 2 x 16GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; EVGA RTX 2060 XC Ultra Gaming; Corsair RM750x

dcn03: Fractal Meshify S2; Gigabyte Aorus ax570 Pro WiFi; Ryzen 9 5900x; Noctua NH-U12S; 2 x 16GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; EVGA RTX 2060 Super XC Gaming; Corsair RM750

dcn04: Fractal Meshify S2; Gigabyte x370 Gaming K7; Ryzen 9 3900x; Noctua NH-U12S; 4 x 16GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; Asus TUF RTX3070 ti; Corsair TX750M

dcn05: Fractal Define R4; Gigabyte ax370 Gaming K7; i9-9900K; EVGA 280 CLC; 2 x 8GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; EVGA GTX 1660ti XC Ultra Gaming; Corsair TX650M

dcn10: Fractal Define C; Gigabyte b450 Aorus M; Ryzen 9 5900x; Ryzen 7 2700x; Hyper 212Evo e/w Noctua NF-A12; 2 x 8GB DDR4-3200; Samsung 250GB SSD; EVGA XC GTX 1070 ti 8GB; Corsair TX550M

dcn11: Fractal Focus G mini; Gigabyte b450 Aorus M; Aorus M; Ryzen 7 2700; Wraith Spire; 2 x 8GB DDR4-3200; Samsung 840 Pro 128GB SSD; ATI Radeon HD2400 Pro; Corsair CX450M Bronze

dcn12: Fractal Core 1100; Gigabyte b450 Aorus M; Ryzen 7 2700; Wraith Spire; 2 x 8GB DDR4-3200; Samsung EVO 850 128GB SSD; ATI FirePro 2270; Corsair CX500M Bronze

dcn19: SuperMicro SC731; SuperMicro X9scl; Xeon e3-1240v2; 4 x 4GB DDR3-1333 ECC; 120GB SSD; SuperMicro 300W Bronze

Link to comment
Share on other sites

Link to post
Share on other sites

2 hours ago, Gorgon said:

Nice. I went back to multiple rigs mostly due to having to baby sit the 6-gpu rig constantly when the download bug hits but the GPUs do run cooler in an open frame.

Yes, the electricity bill can be a shocker.

 

I'm disappointed with Ampere. After having to wait almost 2 years for them to be a reasonable price Ampere is only marginally more efficient than Turing. It seems most of their performance gains have been due to their higher power budget and their operating at the higher end of the voltage/frequency curve.

 

And the projections for Lovelace don't look any better given the ridiculous power budgets being suggested.

 

It appears both AMD and NVidia have given up on improving efficiency in the battle for top performance.

I haven't actually hit the download bug on my 6-GPU rig yet, knock on wood. It's actually been more stable than what I had before so far (though I've learned a few things in setting up Linux properly for FAH since my early days with the old rigs).

 

As for efficiency, wondering what the Intel Arc's will be like for FAH, they are saying they are designed for better efficiency than teams Green and Red.  My current rigs are half Pascal and half Ampere, kinda skipped Turing, but thinking I need to start retiring and replacing the 1070s.  They are still going strong running 24/7 since 2017 so can't complain about longevity, but efficiency-wise...

Link to comment
Share on other sites

Link to post
Share on other sites

3 hours ago, Justaphf said:

I haven't actually hit the download bug on my 6-GPU rig yet, knock on wood. It's actually been more stable than what I had before so far (though I've learned a few things in setting up Linux properly for FAH since my early days with the old rigs).

 

As for efficiency, wondering what the Intel Arc's will be like for FAH, they are saying they are designed for better efficiency than teams Green and Red.  My current rigs are half Pascal and half Ampere, kinda skipped Turing, but thinking I need to start retiring and replacing the 1070s.  They are still going strong running 24/7 since 2017 so can't complain about longevity, but efficiency-wise...

Pascal can be a bit challenging but in general they are most efficient at their lowest Power-Limit setting. They'd likely be even more efficient below their lower Power-Limit but, unfortunately, unlike Turing and Ampere they're incapable of having their Graphics-Clock locked (limited to a maximum frequency that would allow them to run at a lower power than the lower Power-Limit).

 

I only have one Pascal GPU left, an EVGA 1070Ti SC. Unlike all my other cards it needs a +150MHz o/c to run at a reasonable (1900-2000MHz) stable clock at defaults and also unlike the other GPUs is also 24x7x365 stable Folding with that overclock.

 

I've retired it to BOINC duty so I don't have it as instrumented but I'll run it for a while over the next couple of days to get some hard numbers but if memory serves it maxes out at about 20kPPD/W peak efficiency.

FaH BOINC HfM

Bifrost - 6 GPU Folding Rig  Linux Folding HOWTO Folding Remote Access Folding GPU Profiling

Systems:

desktop: Lian-Li O11 Air Mini; Gigabyte Aorus ax570 Master; Ryzen 9 5900x;Noctua NH-D15; 4 x 16GB DDR4-3200; 500GB PCIe3 NVMe; 4 x 3TB NAS; AMD RadeonPro W4100; EVGA RTX 2070A; Corsair TX650M

nas1: Fractal Define R4; SuperMicro X10sl7-f; Xeon e3-1231v3; 4 x 8GB DDR3-1666 ECC; 6 x 3TB NAS; 6 x 4TB NAS

nas2: Synology DS-123j; 2 x 6TB NAS

dcn01: Fractal Meshify C; Gigabyte Aorus ax570 Master; Ryzen 9 5950x; EVGA 240 CLC; 2 x 32GB DDR4-3200; 512GB NVMe; EVGA RTX 2070 Super XC Hybrid; Corsair TX650M

dcn02: Fractal Define S; Gigabyte ax570 Pro WiFi; Ryzen 9 3950x; Noctua NH-D15; 2 x 16GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; EVGA RTX 2060 XC Ultra Gaming; Corsair RM750x

dcn03: Fractal Meshify S2; Gigabyte Aorus ax570 Pro WiFi; Ryzen 9 5900x; Noctua NH-U12S; 2 x 16GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; EVGA RTX 2060 Super XC Gaming; Corsair RM750

dcn04: Fractal Meshify S2; Gigabyte x370 Gaming K7; Ryzen 9 3900x; Noctua NH-U12S; 4 x 16GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; Asus TUF RTX3070 ti; Corsair TX750M

dcn05: Fractal Define R4; Gigabyte ax370 Gaming K7; i9-9900K; EVGA 280 CLC; 2 x 8GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; EVGA GTX 1660ti XC Ultra Gaming; Corsair TX650M

dcn10: Fractal Define C; Gigabyte b450 Aorus M; Ryzen 9 5900x; Ryzen 7 2700x; Hyper 212Evo e/w Noctua NF-A12; 2 x 8GB DDR4-3200; Samsung 250GB SSD; EVGA XC GTX 1070 ti 8GB; Corsair TX550M

dcn11: Fractal Focus G mini; Gigabyte b450 Aorus M; Aorus M; Ryzen 7 2700; Wraith Spire; 2 x 8GB DDR4-3200; Samsung 840 Pro 128GB SSD; ATI Radeon HD2400 Pro; Corsair CX450M Bronze

dcn12: Fractal Core 1100; Gigabyte b450 Aorus M; Ryzen 7 2700; Wraith Spire; 2 x 8GB DDR4-3200; Samsung EVO 850 128GB SSD; ATI FirePro 2270; Corsair CX500M Bronze

dcn19: SuperMicro SC731; SuperMicro X9scl; Xeon e3-1240v2; 4 x 4GB DDR3-1333 ECC; 120GB SSD; SuperMicro 300W Bronze

Link to comment
Share on other sites

Link to post
Share on other sites

6 hours ago, Justaphf said:

... My current rigs are half Pascal and half Ampere, kinda skipped Turing, but thinking I need to start retiring and replacing the 1070s.  They are still going strong running 24/7 since 2017 so can't complain about longevity, but efficiency-wise...

Here's a Comparison between Pascal, Turing and Ampere:

GPU_Series_Eff_Comp.thumb.png.c5f62466e1a7286563953a8466cc3151.png

  • Red:        1070ti (Pascal) running p18202
  • Magenta: 1660ti (Turing) running p18202
  • Green:     2070s (Turing) running p18707
  • Blue:        3070ti (Ampere) running p18125

So not exactly apples to apples except for the 1070ti and 1660ti.

  • 17:30 to 18:00 the 1070ti was running at Max PL (217W) with a +150MHz o/c
  • 18:00 to 18:30 the 1070ti was running at Def PL (190W) with NO o/c
  • 18:30 to 17:00 the 1070ti was running at Min PL (90W) with a +150MHz o/c

It would be nice to see if I can get all three generations running the same project at the same time. Perhaps picking an obscure cause preference (one with few active projects) might do the trick.

FaH BOINC HfM

Bifrost - 6 GPU Folding Rig  Linux Folding HOWTO Folding Remote Access Folding GPU Profiling

Systems:

desktop: Lian-Li O11 Air Mini; Gigabyte Aorus ax570 Master; Ryzen 9 5900x;Noctua NH-D15; 4 x 16GB DDR4-3200; 500GB PCIe3 NVMe; 4 x 3TB NAS; AMD RadeonPro W4100; EVGA RTX 2070A; Corsair TX650M

nas1: Fractal Define R4; SuperMicro X10sl7-f; Xeon e3-1231v3; 4 x 8GB DDR3-1666 ECC; 6 x 3TB NAS; 6 x 4TB NAS

nas2: Synology DS-123j; 2 x 6TB NAS

dcn01: Fractal Meshify C; Gigabyte Aorus ax570 Master; Ryzen 9 5950x; EVGA 240 CLC; 2 x 32GB DDR4-3200; 512GB NVMe; EVGA RTX 2070 Super XC Hybrid; Corsair TX650M

dcn02: Fractal Define S; Gigabyte ax570 Pro WiFi; Ryzen 9 3950x; Noctua NH-D15; 2 x 16GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; EVGA RTX 2060 XC Ultra Gaming; Corsair RM750x

dcn03: Fractal Meshify S2; Gigabyte Aorus ax570 Pro WiFi; Ryzen 9 5900x; Noctua NH-U12S; 2 x 16GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; EVGA RTX 2060 Super XC Gaming; Corsair RM750

dcn04: Fractal Meshify S2; Gigabyte x370 Gaming K7; Ryzen 9 3900x; Noctua NH-U12S; 4 x 16GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; Asus TUF RTX3070 ti; Corsair TX750M

dcn05: Fractal Define R4; Gigabyte ax370 Gaming K7; i9-9900K; EVGA 280 CLC; 2 x 8GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; EVGA GTX 1660ti XC Ultra Gaming; Corsair TX650M

dcn10: Fractal Define C; Gigabyte b450 Aorus M; Ryzen 9 5900x; Ryzen 7 2700x; Hyper 212Evo e/w Noctua NF-A12; 2 x 8GB DDR4-3200; Samsung 250GB SSD; EVGA XC GTX 1070 ti 8GB; Corsair TX550M

dcn11: Fractal Focus G mini; Gigabyte b450 Aorus M; Aorus M; Ryzen 7 2700; Wraith Spire; 2 x 8GB DDR4-3200; Samsung 840 Pro 128GB SSD; ATI Radeon HD2400 Pro; Corsair CX450M Bronze

dcn12: Fractal Core 1100; Gigabyte b450 Aorus M; Ryzen 7 2700; Wraith Spire; 2 x 8GB DDR4-3200; Samsung EVO 850 128GB SSD; ATI FirePro 2270; Corsair CX500M Bronze

dcn19: SuperMicro SC731; SuperMicro X9scl; Xeon e3-1240v2; 4 x 4GB DDR3-1333 ECC; 120GB SSD; SuperMicro 300W Bronze

Link to comment
Share on other sites

Link to post
Share on other sites

23 hours ago, Justaphf said:

My current rigs are half Pascal and half Ampere, kinda skipped Turing, but thinking I need to start retiring and replacing the 1070s.  They are still going strong running 24/7 since 2017 so can't complain about longevity, but efficiency-wise...

I went through my HFM.net logs and extracted data on p18202 for the 4 GPUs and looked at my Zabbix Servers to get the Power Consumption.

GPU    Slot    TPF   Points  PPD       Power   Eff.   Adj. MSRP   Value
                                        (W)  (kPPD/W)   ($US)    (kPPD/$)
3070ti dcn04s1 01:17 434,172 4,871,747 144.8   33.6    $639.00     7.6
2070s  dcn05s0 01:43 377,230 3,164,339 106.7   29.7    $564.00     5.6
1660ti dcn05s1 03:07 280,047 1,293,905  68.6   18.9    $315.00     4.1
1070ti dcn10s0 02:53 292,204 1,459,331  89.2   16.4    $470.00     3.1

So a 3070ti is about twice as efficient as a 1070ti and more than twice the value. But then again your 1070s are paid for so unless they fail you'll likely get more benefit out of leaving them folding at minimal power rather than attempting to sell them.

 

One area that I don't include is system efficiency. If your running them in a mining frame off a single motherboard and power-supply then you will be saving a ton of overhead power. Assuming 6 GPUs that were in dual GPU rigs your dropping your system overhead from 3x (60-80W) to just 60-80W which will greatly boost your overall efficiency.

Quote

As for efficiency, wondering what the Intel Arc's will be like for FAH, they are saying they are designed for better efficiency than teams Green and Red. 

Like AMD, Intel ARC's performance will be limited by the fact they have to use OpenCL rather than CUDA so they will be at a disadvantage at the start. Further their Compute performance will be at the mercy of how optimized their drivers are. AMD drivers have been historically much worse compared to NVIDIA and I would argue that Intel's have been even worse.

 

Like AMD, Intel will be playing catch-up and will likely go for the most "bang for the buck" and work on optimizing gaming performance before even thinking about Compute.

 

One area they could differentiate themselves would be to NOT artificially "gimp" the FP64 performance of their GPUs which would put them in a unique position but that would have more of an impact on BOINC than F@H.

FaH BOINC HfM

Bifrost - 6 GPU Folding Rig  Linux Folding HOWTO Folding Remote Access Folding GPU Profiling

Systems:

desktop: Lian-Li O11 Air Mini; Gigabyte Aorus ax570 Master; Ryzen 9 5900x;Noctua NH-D15; 4 x 16GB DDR4-3200; 500GB PCIe3 NVMe; 4 x 3TB NAS; AMD RadeonPro W4100; EVGA RTX 2070A; Corsair TX650M

nas1: Fractal Define R4; SuperMicro X10sl7-f; Xeon e3-1231v3; 4 x 8GB DDR3-1666 ECC; 6 x 3TB NAS; 6 x 4TB NAS

nas2: Synology DS-123j; 2 x 6TB NAS

dcn01: Fractal Meshify C; Gigabyte Aorus ax570 Master; Ryzen 9 5950x; EVGA 240 CLC; 2 x 32GB DDR4-3200; 512GB NVMe; EVGA RTX 2070 Super XC Hybrid; Corsair TX650M

dcn02: Fractal Define S; Gigabyte ax570 Pro WiFi; Ryzen 9 3950x; Noctua NH-D15; 2 x 16GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; EVGA RTX 2060 XC Ultra Gaming; Corsair RM750x

dcn03: Fractal Meshify S2; Gigabyte Aorus ax570 Pro WiFi; Ryzen 9 5900x; Noctua NH-U12S; 2 x 16GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; EVGA RTX 2060 Super XC Gaming; Corsair RM750

dcn04: Fractal Meshify S2; Gigabyte x370 Gaming K7; Ryzen 9 3900x; Noctua NH-U12S; 4 x 16GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; Asus TUF RTX3070 ti; Corsair TX750M

dcn05: Fractal Define R4; Gigabyte ax370 Gaming K7; i9-9900K; EVGA 280 CLC; 2 x 8GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; EVGA GTX 1660ti XC Ultra Gaming; Corsair TX650M

dcn10: Fractal Define C; Gigabyte b450 Aorus M; Ryzen 9 5900x; Ryzen 7 2700x; Hyper 212Evo e/w Noctua NF-A12; 2 x 8GB DDR4-3200; Samsung 250GB SSD; EVGA XC GTX 1070 ti 8GB; Corsair TX550M

dcn11: Fractal Focus G mini; Gigabyte b450 Aorus M; Aorus M; Ryzen 7 2700; Wraith Spire; 2 x 8GB DDR4-3200; Samsung 840 Pro 128GB SSD; ATI Radeon HD2400 Pro; Corsair CX450M Bronze

dcn12: Fractal Core 1100; Gigabyte b450 Aorus M; Ryzen 7 2700; Wraith Spire; 2 x 8GB DDR4-3200; Samsung EVO 850 128GB SSD; ATI FirePro 2270; Corsair CX500M Bronze

dcn19: SuperMicro SC731; SuperMicro X9scl; Xeon e3-1240v2; 4 x 4GB DDR3-1333 ECC; 120GB SSD; SuperMicro 300W Bronze

Link to comment
Share on other sites

Link to post
Share on other sites

5 hours ago, Gorgon said:

I went through my HFM.net logs and extracted data on p18202 for the 4 GPUs and looked at my Zabbix Servers to get the Power Consumption.

GPU    Slot    TPF   Points  PPD       Power   Eff.   Adj. MSRP   Value
                                        (W)  (kPPD/W)   ($US)    (kPPD/$)
3070ti dcn04s1 01:17 434,172 4,871,747 144.8   33.6    $639.00     7.6
2070s  dcn05s0 01:43 377,230 3,164,339 106.7   29.7    $564.00     5.6
1660ti dcn05s1 03:07 280,047 1,293,905  68.6   18.9    $315.00     4.1
1070ti dcn10s0 02:53 292,204 1,459,331  89.2   16.4    $470.00     3.1

So a 3070ti is about twice as efficient as a 1070ti and more than twice the value. But then again your 1070s are paid for so unless they fail you'll likely get more benefit out of leaving them folding at minimal power rather than attempting to sell them.

 

One area that I don't include is system efficiency. If your running them in a mining frame off a single motherboard and power-supply then you will be saving a ton of overhead power. Assuming 6 GPUs that were in dual GPU rigs your dropping your system overhead from 3x (60-80W) to just 60-80W which will greatly boost your overall efficiency.

Like AMD, Intel ARC's performance will be limited by the fact they have to use OpenCL rather than CUDA so they will be at a disadvantage at the start. Further their Compute performance will be at the mercy of how optimized their drivers are. AMD drivers have been historically much worse compared to NVIDIA and I would argue that Intel's have been even worse.

 

Like AMD, Intel will be playing catch-up and will likely go for the most "bang for the buck" and work on optimizing gaming performance before even thinking about Compute.

 

One area they could differentiate themselves would be to NOT artificially "gimp" the FP64 performance of their GPUs which would put them in a unique position but that would have more of an impact on BOINC than F@H.

Awesome level of analysis @Gorgon, thanks!  I have been running my 1070s at ~80% TPD (120W) with 200 MHz memory overclock and no core overclock and getting ~1.0-1.2m PPD each.  I never went through and did a full sweep to figure out optimal operating points like you have in your GPU profiling guide (I've been meaning to, just haven't gotten around to it).  Basically just spot checked 4 or 5 settings, found something that worked 2 years ago and haven't really touched those settings since.  For the 30 series cards I just reduced the TPD to 90% on each of them and left overclock alone after even minimal increments seemed to cause issues (they seemed more sensitive compared to the 10 series).

 

For the 30 series cards, have you ever done testing of PCIe gen 3 vs gen 4?  I didn't think it made any difference for FAH compute, but when I was getting the NVMe to PCIe risers from ADT-Link they had 4.0 for only $6 more than the 3.0 so picked those up instead.  They work fine and show up as running the 30 series cards at 4.0x4 as expected.  Though now that I think of it, wonder if gen 4 costs more power than gen 3.

Link to comment
Share on other sites

Link to post
Share on other sites

1 hour ago, Justaphf said:

Awesome level of analysis @Gorgon, thanks!  I have been running my 1070s at ~80% TPD (120W) with 200 MHz memory overclock and no core overclock and getting ~1.0-1.2m PPD each.  I never went through and did a full sweep to figure out optimal operating points like you have in your GPU profiling guide (I've been meaning to, just haven't gotten around to it).  Basically just spot checked 4 or 5 settings, found something that worked 2 years ago and haven't really touched those settings since.  For the 30 series cards I just reduced the TPD to 90% on each of them and left overclock alone after even minimal increments seemed to cause issues (they seemed more sensitive compared to the 10 series).

 

For the 30 series cards, have you ever done testing of PCIe gen 3 vs gen 4?  I didn't think it made any difference for FAH compute, but when I was getting the NVMe to PCIe risers from ADT-Link they had 4.0 for only $6 more than the 3.0 so picked those up instead.  They work fine and show up as running the 30 series cards at 4.0x4 as expected.  Though now that I think of it, wonder if gen 4 costs more power than gen 3.

Thanks,

 

First - Over-clocking - with the exception of the 1070ti where it's base +boost clock seems way too low at ~1850MHz, it is generally not worth it to try anything over 50MHz as it will not be stable long-term and one or two failed WUs after 2-3 hours will quickly negate any boost in performance. The issue seems to be when the GPUs return from a checkpoint or some other idle task with too high of an o/c they boost up to >2050Mhz briefly and crash. Running:

nvidia-smi -i <x> -lgc 0,2040

can help with this but it's just not worth the effort IMHO. So I typically just run a +50 and leave it at that.

 

Pascal and Turing will both perform most efficiently at the lowest Power-Limit and some Turing cards (2060, for one) have a ridiculously high lower power limit so locking the graphics clock will get you too a more efficient place. Somewhere between 1300-1500MHz seems to be the sweet spot for Turing and Ampere.

 

I haven't tried Gen4 as I only have the 3070Ti that supports it but I suspect it would make only marginal differences and only with possibly 3070ti but definitely 3080 and higher might benefit on Windows and likely less so on Linux as I observed on my 6-GPU build. Likely the power difference from Gen4 is negligible.

FaH BOINC HfM

Bifrost - 6 GPU Folding Rig  Linux Folding HOWTO Folding Remote Access Folding GPU Profiling

Systems:

desktop: Lian-Li O11 Air Mini; Gigabyte Aorus ax570 Master; Ryzen 9 5900x;Noctua NH-D15; 4 x 16GB DDR4-3200; 500GB PCIe3 NVMe; 4 x 3TB NAS; AMD RadeonPro W4100; EVGA RTX 2070A; Corsair TX650M

nas1: Fractal Define R4; SuperMicro X10sl7-f; Xeon e3-1231v3; 4 x 8GB DDR3-1666 ECC; 6 x 3TB NAS; 6 x 4TB NAS

nas2: Synology DS-123j; 2 x 6TB NAS

dcn01: Fractal Meshify C; Gigabyte Aorus ax570 Master; Ryzen 9 5950x; EVGA 240 CLC; 2 x 32GB DDR4-3200; 512GB NVMe; EVGA RTX 2070 Super XC Hybrid; Corsair TX650M

dcn02: Fractal Define S; Gigabyte ax570 Pro WiFi; Ryzen 9 3950x; Noctua NH-D15; 2 x 16GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; EVGA RTX 2060 XC Ultra Gaming; Corsair RM750x

dcn03: Fractal Meshify S2; Gigabyte Aorus ax570 Pro WiFi; Ryzen 9 5900x; Noctua NH-U12S; 2 x 16GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; EVGA RTX 2060 Super XC Gaming; Corsair RM750

dcn04: Fractal Meshify S2; Gigabyte x370 Gaming K7; Ryzen 9 3900x; Noctua NH-U12S; 4 x 16GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; Asus TUF RTX3070 ti; Corsair TX750M

dcn05: Fractal Define R4; Gigabyte ax370 Gaming K7; i9-9900K; EVGA 280 CLC; 2 x 8GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; EVGA GTX 1660ti XC Ultra Gaming; Corsair TX650M

dcn10: Fractal Define C; Gigabyte b450 Aorus M; Ryzen 9 5900x; Ryzen 7 2700x; Hyper 212Evo e/w Noctua NF-A12; 2 x 8GB DDR4-3200; Samsung 250GB SSD; EVGA XC GTX 1070 ti 8GB; Corsair TX550M

dcn11: Fractal Focus G mini; Gigabyte b450 Aorus M; Aorus M; Ryzen 7 2700; Wraith Spire; 2 x 8GB DDR4-3200; Samsung 840 Pro 128GB SSD; ATI Radeon HD2400 Pro; Corsair CX450M Bronze

dcn12: Fractal Core 1100; Gigabyte b450 Aorus M; Ryzen 7 2700; Wraith Spire; 2 x 8GB DDR4-3200; Samsung EVO 850 128GB SSD; ATI FirePro 2270; Corsair CX500M Bronze

dcn19: SuperMicro SC731; SuperMicro X9scl; Xeon e3-1240v2; 4 x 4GB DDR3-1333 ECC; 120GB SSD; SuperMicro 300W Bronze

Link to comment
Share on other sites

Link to post
Share on other sites

On 7/26/2022 at 6:50 AM, Gorgon said:

Pascal can be a bit challenging but in general they are most efficient at their lowest Power-Limit setting. They'd likely be even more efficient below their lower Power-Limit but, unfortunately, unlike Turing and Ampere they're incapable of having their Graphics-Clock locked (limited to a maximum frequency that would allow them to run at a lower power than the lower Power-Limit).

 

I only have one Pascal GPU left, an EVGA 1070Ti SC. Unlike all my other cards it needs a +150MHz o/c to run at a reasonable (1900-2000MHz) stable clock at defaults and also unlike the other GPUs is also 24x7x365 stable Folding with that overclock.

You can set locked clock speeds on Pacal cards in Windows. I did this on both my old 1060 3GB and current EVGA 1070 SC. I'm not sure about Linux though.

9 hours ago, Gorgon said:

Thanks,

 

First - Over-clocking - with the exception of the 1070ti where it's base +boost clock seems way too low at ~1850MHz, it is generally not worth it to try anything over 50MHz as it will not be stable long-term and one or two failed WUs after 2-3 hours will quickly negate any boost in performance. The issue seems to be when the GPUs return from a checkpoint or some other idle task with too high of an o/c they boost up to >2050Mhz briefly and crash.

This is a case where it may help to lock clocks using Afterburner. When folding, I just set my 1070 at 1999/2012Mhz and let it stay at that speed until I change the profile. There may be a minor inefficiency at checkpoints or the end of each WU where there's no work to do but the GPU is still running at full speed, but it's a fair trade off in my eyes.

Desktop 1 : Ryzen 5 3600 (O/C to 4Ghz all-core) | Gigabyte B450M-DS3H | 24GB DDR4-2400 Crucial(O/C to 2667) | GALAX RTX 2060 6GB | CoolerMaster MWE 650 Gold

 

Desktop 2 : i5 10400 | 32GB DDR4-3200(@ 2667Mhz) |  EVGA GTX 1070 SC 8 GB | Corsair CV450M

                        

Laptop : ASUS ROG Strix G17 : i7-10750H, 16GB RAM, Geforce 1660Ti 6GB(90W), 512GB SSD

Link to comment
Share on other sites

Link to post
Share on other sites

4 hours ago, rkv_2401 said:

You can set locked clock speeds on Pacal cards in Windows. I did this on both my old 1060 3GB and current EVGA 1070 SC. I'm not sure about Linux though.

This is a case where it may help to lock clocks using Afterburner. When folding, I just set my 1070 at 1999/2012Mhz and let it stay at that speed until I change the profile. There may be a minor inefficiency at checkpoints or the end of each WU where there's no work to do but the GPU is still running at full speed, but it's a fair trade off in my eyes.

nvidia-smi -i 0 -lgc 0,1440
	Setting locked GPU clocks is not supported for GPU 00000000:06:00.0.
	Treating as warning and moving on

Nope - not supported on Linux under Pascal.

 

I assume maybe your talking about creating a custom voltage/frequency curve in Afterburner/xoc etc. which is also not possible under Linux.

FaH BOINC HfM

Bifrost - 6 GPU Folding Rig  Linux Folding HOWTO Folding Remote Access Folding GPU Profiling

Systems:

desktop: Lian-Li O11 Air Mini; Gigabyte Aorus ax570 Master; Ryzen 9 5900x;Noctua NH-D15; 4 x 16GB DDR4-3200; 500GB PCIe3 NVMe; 4 x 3TB NAS; AMD RadeonPro W4100; EVGA RTX 2070A; Corsair TX650M

nas1: Fractal Define R4; SuperMicro X10sl7-f; Xeon e3-1231v3; 4 x 8GB DDR3-1666 ECC; 6 x 3TB NAS; 6 x 4TB NAS

nas2: Synology DS-123j; 2 x 6TB NAS

dcn01: Fractal Meshify C; Gigabyte Aorus ax570 Master; Ryzen 9 5950x; EVGA 240 CLC; 2 x 32GB DDR4-3200; 512GB NVMe; EVGA RTX 2070 Super XC Hybrid; Corsair TX650M

dcn02: Fractal Define S; Gigabyte ax570 Pro WiFi; Ryzen 9 3950x; Noctua NH-D15; 2 x 16GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; EVGA RTX 2060 XC Ultra Gaming; Corsair RM750x

dcn03: Fractal Meshify S2; Gigabyte Aorus ax570 Pro WiFi; Ryzen 9 5900x; Noctua NH-U12S; 2 x 16GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; EVGA RTX 2060 Super XC Gaming; Corsair RM750

dcn04: Fractal Meshify S2; Gigabyte x370 Gaming K7; Ryzen 9 3900x; Noctua NH-U12S; 4 x 16GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; Asus TUF RTX3070 ti; Corsair TX750M

dcn05: Fractal Define R4; Gigabyte ax370 Gaming K7; i9-9900K; EVGA 280 CLC; 2 x 8GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; EVGA GTX 1660ti XC Ultra Gaming; Corsair TX650M

dcn10: Fractal Define C; Gigabyte b450 Aorus M; Ryzen 9 5900x; Ryzen 7 2700x; Hyper 212Evo e/w Noctua NF-A12; 2 x 8GB DDR4-3200; Samsung 250GB SSD; EVGA XC GTX 1070 ti 8GB; Corsair TX550M

dcn11: Fractal Focus G mini; Gigabyte b450 Aorus M; Aorus M; Ryzen 7 2700; Wraith Spire; 2 x 8GB DDR4-3200; Samsung 840 Pro 128GB SSD; ATI Radeon HD2400 Pro; Corsair CX450M Bronze

dcn12: Fractal Core 1100; Gigabyte b450 Aorus M; Ryzen 7 2700; Wraith Spire; 2 x 8GB DDR4-3200; Samsung EVO 850 128GB SSD; ATI FirePro 2270; Corsair CX500M Bronze

dcn19: SuperMicro SC731; SuperMicro X9scl; Xeon e3-1240v2; 4 x 4GB DDR3-1333 ECC; 120GB SSD; SuperMicro 300W Bronze

Link to comment
Share on other sites

Link to post
Share on other sites

Any ideas for what I should expect to get out of a basic GTX 1080? I picked one up for a good price, mostly just to mess around with it for a bit and eventually put into permanent use in a Folding@home system. The one I bought is an HP OEM card, so I don't expect it to have a great cooler, but that's not too big of a deal. 

 

What should I be expecting in terms of PPD at stock speeds? How low can you get the power draw to go before the PPD drops a lot? I'm not very familiar with Pascal cards, and the only other one I own is a GT 1030. 

 

I also picked up a Quadro M4000 (8GB single slot Maxwell 2.0 card), so I'll be trying to see what I can squeeze out of it as well. 

Phobos: AMD Ryzen 7 2700, 16GB 3000MHz DDR4, ASRock B450 Steel Legend, 8GB Nvidia GeForce GTX 1080, 1TB Samsung SSD 980, 450W Corsair CXM, Corsair Carbide 175R, Windows 10 Pro

 

Pluto: Intel Core i7-2600, 32GB 1600MHz DDR3, ASUS P8Z68-V, 4GB XFX AMD Radeon RX 570, 1TB Samsung 860 EVO, 3TB Seagate BarraCuda, 750W EVGA BQ, Fractal Design Focus G, Windows 10 Pro for Workstations

 

York (NAS): Intel Core i5-2400, 16GB 1600MHz DDR3, HP Compaq OEM, 240GB Kingston V300 (boot), 3x2TB Seagate BarraCuda, 320W HP PSU, HP Compaq 6200 Pro, TrueNAS CORE (12.0)

 

Mid 2011 27" iMac (general use Mac): Intel Core i5-2400, 16GB 1333MHz DDR3, Z68 Logic Board, 1GB AMD Radeon HD 6970M, 500GB Crucial MX500, 2TB WD Blue, macOS High Sierra

Link to comment
Share on other sites

Link to post
Share on other sites

10 hours ago, BondiBlue said:

Any ideas for what I should expect to get out of a basic GTX 1080? I picked one up for a good price, mostly just to mess around with it for a bit and eventually put into permanent use in a Folding@home system. The one I bought is an HP OEM card, so I don't expect it to have a great cooler, but that's not too big of a deal. 

 

What should I be expecting in terms of PPD at stock speeds? How low can you get the power draw to go before the PPD drops a lot? I'm not very familiar with Pascal cards, and the only other one I own is a GT 1030. 

 

I also picked up a Quadro M4000 (8GB single slot Maxwell 2.0 card), so I'll be trying to see what I can squeeze out of it as well. 

If your planning on running the 1080 at a low power limit in the efficiency “sweet spot” then the quality of the cooler doesn’t really matter. I usually prefer EVGA SC or XC over FTW models, for example, as they’re usually less wide (2.0 slot width) and the extra $ not spent on a beefier cooler can be used for more GPUs 🙂

 

My 2070super Hybrids, run around 47-49C and the air cooled cards between 55-65C. Not a lot of cooling required at lower power-limits.

 

For Pascal run the GPU with a slight overclock (+50MHz) and at the lowest power limit supported by the card. Any Quick Return Bonus (QRB) losses will be more than offset by the savings in Power.

 

Install HfM.net (see Remote Access in my Sig). Run the card at default for a week. Drop the Power Limit to the minimum for a week and compare. You’ll be pleased with the results.

FaH BOINC HfM

Bifrost - 6 GPU Folding Rig  Linux Folding HOWTO Folding Remote Access Folding GPU Profiling

Systems:

desktop: Lian-Li O11 Air Mini; Gigabyte Aorus ax570 Master; Ryzen 9 5900x;Noctua NH-D15; 4 x 16GB DDR4-3200; 500GB PCIe3 NVMe; 4 x 3TB NAS; AMD RadeonPro W4100; EVGA RTX 2070A; Corsair TX650M

nas1: Fractal Define R4; SuperMicro X10sl7-f; Xeon e3-1231v3; 4 x 8GB DDR3-1666 ECC; 6 x 3TB NAS; 6 x 4TB NAS

nas2: Synology DS-123j; 2 x 6TB NAS

dcn01: Fractal Meshify C; Gigabyte Aorus ax570 Master; Ryzen 9 5950x; EVGA 240 CLC; 2 x 32GB DDR4-3200; 512GB NVMe; EVGA RTX 2070 Super XC Hybrid; Corsair TX650M

dcn02: Fractal Define S; Gigabyte ax570 Pro WiFi; Ryzen 9 3950x; Noctua NH-D15; 2 x 16GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; EVGA RTX 2060 XC Ultra Gaming; Corsair RM750x

dcn03: Fractal Meshify S2; Gigabyte Aorus ax570 Pro WiFi; Ryzen 9 5900x; Noctua NH-U12S; 2 x 16GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; EVGA RTX 2060 Super XC Gaming; Corsair RM750

dcn04: Fractal Meshify S2; Gigabyte x370 Gaming K7; Ryzen 9 3900x; Noctua NH-U12S; 4 x 16GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; Asus TUF RTX3070 ti; Corsair TX750M

dcn05: Fractal Define R4; Gigabyte ax370 Gaming K7; i9-9900K; EVGA 280 CLC; 2 x 8GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; EVGA GTX 1660ti XC Ultra Gaming; Corsair TX650M

dcn10: Fractal Define C; Gigabyte b450 Aorus M; Ryzen 9 5900x; Ryzen 7 2700x; Hyper 212Evo e/w Noctua NF-A12; 2 x 8GB DDR4-3200; Samsung 250GB SSD; EVGA XC GTX 1070 ti 8GB; Corsair TX550M

dcn11: Fractal Focus G mini; Gigabyte b450 Aorus M; Aorus M; Ryzen 7 2700; Wraith Spire; 2 x 8GB DDR4-3200; Samsung 840 Pro 128GB SSD; ATI Radeon HD2400 Pro; Corsair CX450M Bronze

dcn12: Fractal Core 1100; Gigabyte b450 Aorus M; Ryzen 7 2700; Wraith Spire; 2 x 8GB DDR4-3200; Samsung EVO 850 128GB SSD; ATI FirePro 2270; Corsair CX500M Bronze

dcn19: SuperMicro SC731; SuperMicro X9scl; Xeon e3-1240v2; 4 x 4GB DDR3-1333 ECC; 120GB SSD; SuperMicro 300W Bronze

Link to comment
Share on other sites

Link to post
Share on other sites

17 hours ago, Gorgon said:
nvidia-smi -i 0 -lgc 0,1440
	Setting locked GPU clocks is not supported for GPU 00000000:06:00.0.
	Treating as warning and moving on

Nope - not supported on Linux under Pascal.

 

I assume maybe your talking about creating a custom voltage/frequency curve in Afterburner/xoc etc. which is also not possible under Linux.

Yeah, I was sure you'd already be doing this if it was possible on Linux. 🙂 Just wanted to give a heads up to people who had this question and are folding on a Pascal card on Windows

Desktop 1 : Ryzen 5 3600 (O/C to 4Ghz all-core) | Gigabyte B450M-DS3H | 24GB DDR4-2400 Crucial(O/C to 2667) | GALAX RTX 2060 6GB | CoolerMaster MWE 650 Gold

 

Desktop 2 : i5 10400 | 32GB DDR4-3200(@ 2667Mhz) |  EVGA GTX 1070 SC 8 GB | Corsair CV450M

                        

Laptop : ASUS ROG Strix G17 : i7-10750H, 16GB RAM, Geforce 1660Ti 6GB(90W), 512GB SSD

Link to comment
Share on other sites

Link to post
Share on other sites

Posted (edited)
On 7/26/2022 at 12:04 PM, Gorgon said:

I went through my HFM.net logs and extracted data on p18202 for the 4 GPUs and looked at my Zabbix Servers to get the Power Consumption.

GPU    Slot    TPF   Points  PPD       Power   Eff.   Adj. MSRP   Value
                                        (W)  (kPPD/W)   ($US)    (kPPD/$)
3070ti dcn04s1 01:17 434,172 4,871,747 144.8   33.6    $639.00     7.6
2070s  dcn05s0 01:43 377,230 3,164,339 106.7   29.7    $564.00     5.6
1660ti dcn05s1 03:07 280,047 1,293,905  68.6   18.9    $315.00     4.1
1070ti dcn10s0 02:53 292,204 1,459,331  89.2   16.4    $470.00     3.1
Quote

I have a Intel i7 NUC running a free ESXi 6.7 instance and have a Virtual Machine on that running Ubuntu 18.04.3 LTS that is my Zabbix Server which queries Zabbix Agents installed on my Distributed Computing Nodes. This is a bit of a Black Art but I've been doing System Management a work for over 25 years and so is second nature to me.

@GorgonDid you have a recommended guide related to setup of Zabbix, specifically for accumulating FAH/BOINC relevant metrics? I saw in the BOINC post above you mentioned it was a bit of a black art and that you did system management professionally, just wondering how much of a learning curve (for a non-systems pro) there is and what relevant information you are gathering.  Do you think it is worth setting up (as a novice) if I only have 3 boxes to monitor (after the Bifrost-style system consolidations) and that I don't really care about anything being monitored beyond the FAH relevant stats (i.e nvidia-smi query outputs)?

Edited by Justaphf
Clarification of too subjective question.
Link to comment
Share on other sites

Link to post
Share on other sites

so i still not been able to get mining board working.

atm that cut 25% of my folding points.

 

MSI x399 sli plus  | AMD theardripper 2990wx all core 3ghz lock |Thermaltake flo ring 360 | EVGA 2080, Zotac 2080 |Gskill Ripjaws 128GB 3000 MHz | Corsair RM1200i |100tb | Asus tuff gaming mid tower

Link to comment
Share on other sites

Link to post
Share on other sites

On 7/28/2022 at 9:45 PM, Justaphf said:

@GorgonDid you have a recommended guide related to setup of Zabbix, specifically for accumulating FAH/BOINC relevant metrics? I saw in the BOINC post above you mentioned it was a bit of a black art and that you did system management professionally, just wondering how much of a learning curve (for a non-systems pro) there is and what relevant information you are gathering.  Do you think it is worth setting up (as a novice) if I only have 3 boxes to monitor (after the Bifrost-style system consolidations) and that I don't really care about anything being monitored beyond the FAH relevant stats (i.e nvidia-smi query outputs)?

I don't have a guide written for Zabbix as that is more of a master class in Linux System Administration and will vary significantly depending on which OS your running and the hardware on your monitored systems and especially what your interested in monitoring.

 

I would say its not worth the effort for most people but I'm a bit OCD and interested in doing the most amount of work in F@H and BOINC as efficiently as possible.

 

I need to get my Linux Folding Guide updated for Ubuntu 20.04 LTS first as that will be of more use to most people but I will share my Build Notes for Zabbix.

 

FaH BOINC HfM

Bifrost - 6 GPU Folding Rig  Linux Folding HOWTO Folding Remote Access Folding GPU Profiling

Systems:

desktop: Lian-Li O11 Air Mini; Gigabyte Aorus ax570 Master; Ryzen 9 5900x;Noctua NH-D15; 4 x 16GB DDR4-3200; 500GB PCIe3 NVMe; 4 x 3TB NAS; AMD RadeonPro W4100; EVGA RTX 2070A; Corsair TX650M

nas1: Fractal Define R4; SuperMicro X10sl7-f; Xeon e3-1231v3; 4 x 8GB DDR3-1666 ECC; 6 x 3TB NAS; 6 x 4TB NAS

nas2: Synology DS-123j; 2 x 6TB NAS

dcn01: Fractal Meshify C; Gigabyte Aorus ax570 Master; Ryzen 9 5950x; EVGA 240 CLC; 2 x 32GB DDR4-3200; 512GB NVMe; EVGA RTX 2070 Super XC Hybrid; Corsair TX650M

dcn02: Fractal Define S; Gigabyte ax570 Pro WiFi; Ryzen 9 3950x; Noctua NH-D15; 2 x 16GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; EVGA RTX 2060 XC Ultra Gaming; Corsair RM750x

dcn03: Fractal Meshify S2; Gigabyte Aorus ax570 Pro WiFi; Ryzen 9 5900x; Noctua NH-U12S; 2 x 16GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; EVGA RTX 2060 Super XC Gaming; Corsair RM750

dcn04: Fractal Meshify S2; Gigabyte x370 Gaming K7; Ryzen 9 3900x; Noctua NH-U12S; 4 x 16GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; Asus TUF RTX3070 ti; Corsair TX750M

dcn05: Fractal Define R4; Gigabyte ax370 Gaming K7; i9-9900K; EVGA 280 CLC; 2 x 8GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; EVGA GTX 1660ti XC Ultra Gaming; Corsair TX650M

dcn10: Fractal Define C; Gigabyte b450 Aorus M; Ryzen 9 5900x; Ryzen 7 2700x; Hyper 212Evo e/w Noctua NF-A12; 2 x 8GB DDR4-3200; Samsung 250GB SSD; EVGA XC GTX 1070 ti 8GB; Corsair TX550M

dcn11: Fractal Focus G mini; Gigabyte b450 Aorus M; Aorus M; Ryzen 7 2700; Wraith Spire; 2 x 8GB DDR4-3200; Samsung 840 Pro 128GB SSD; ATI Radeon HD2400 Pro; Corsair CX450M Bronze

dcn12: Fractal Core 1100; Gigabyte b450 Aorus M; Ryzen 7 2700; Wraith Spire; 2 x 8GB DDR4-3200; Samsung EVO 850 128GB SSD; ATI FirePro 2270; Corsair CX500M Bronze

dcn19: SuperMicro SC731; SuperMicro X9scl; Xeon e3-1240v2; 4 x 4GB DDR3-1333 ECC; 120GB SSD; SuperMicro 300W Bronze

Link to comment
Share on other sites

Link to post
Share on other sites

@GorgonThanks for leaving the build notes.  I'll look over them when next I get the chance, though sounds like it might be overkill for my needs.

 

On 7/30/2022 at 2:08 PM, Gorgon said:

I need to get my Linux Folding Guide updated for Ubuntu 20.04 LTS first as that will be of more use

I actually just installed from scratch 20.04.4 this evening for my second Bifrost style build.  I think I've got it down to a dozen or so CLI commands for a clean install for the purpose of just a FAH client (not counting burning the ISO to USB with Rufus and BIOS settings changes).  I do wish they would stop changing how you manage network settings every new release though.  I've given up trying to stay on top of managing that through command line, it's one of the things that's actually just easier to deal with in the desktop now.  Managing the nvidia drivers through UI also seems slightly easier now too, though I still like the control of doing most things by CLI.

Link to comment
Share on other sites

Link to post
Share on other sites

21 hours ago, Justaphf said:

@GorgonThanks for leaving the build notes.  I'll look over them when next I get the chance, though sounds like it might be overkill for my needs.

 

I actually just installed from scratch 20.04.4 this evening for my second Bifrost style build.  I think I've got it down to a dozen or so CLI commands for a clean install for the purpose of just a FAH client (not counting burning the ISO to USB with Rufus and BIOS settings changes).  I do wish they would stop changing how you manage network settings every new release though.  I've given up trying to stay on top of managing that through command line, it's one of the things that's actually just easier to deal with in the desktop now.  Managing the nvidia drivers through UI also seems slightly easier now too, though I still like the control of doing most things by CLI.

Getting Zabbix working is a ton of work and getting it configured to capture relevant information is an additional pile of work.

 

I agree that the network settings are easiest done through the GUI. I usually harcode the IP, Mask, Gateway and DNS on the initial build then just manage everything through SSH via Public Keys after that.

nvidia-smi -i 0 -l 1 --format=csv,noheader --query-gpu=temperature.gpu,power.draw,clocks.gr,fan.speed

is one of my favorite commands. I have PuTTY shortcuts for all my slots so I can watch them.

FaH BOINC HfM

Bifrost - 6 GPU Folding Rig  Linux Folding HOWTO Folding Remote Access Folding GPU Profiling

Systems:

desktop: Lian-Li O11 Air Mini; Gigabyte Aorus ax570 Master; Ryzen 9 5900x;Noctua NH-D15; 4 x 16GB DDR4-3200; 500GB PCIe3 NVMe; 4 x 3TB NAS; AMD RadeonPro W4100; EVGA RTX 2070A; Corsair TX650M

nas1: Fractal Define R4; SuperMicro X10sl7-f; Xeon e3-1231v3; 4 x 8GB DDR3-1666 ECC; 6 x 3TB NAS; 6 x 4TB NAS

nas2: Synology DS-123j; 2 x 6TB NAS

dcn01: Fractal Meshify C; Gigabyte Aorus ax570 Master; Ryzen 9 5950x; EVGA 240 CLC; 2 x 32GB DDR4-3200; 512GB NVMe; EVGA RTX 2070 Super XC Hybrid; Corsair TX650M

dcn02: Fractal Define S; Gigabyte ax570 Pro WiFi; Ryzen 9 3950x; Noctua NH-D15; 2 x 16GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; EVGA RTX 2060 XC Ultra Gaming; Corsair RM750x

dcn03: Fractal Meshify S2; Gigabyte Aorus ax570 Pro WiFi; Ryzen 9 5900x; Noctua NH-U12S; 2 x 16GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; EVGA RTX 2060 Super XC Gaming; Corsair RM750

dcn04: Fractal Meshify S2; Gigabyte x370 Gaming K7; Ryzen 9 3900x; Noctua NH-U12S; 4 x 16GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; Asus TUF RTX3070 ti; Corsair TX750M

dcn05: Fractal Define R4; Gigabyte ax370 Gaming K7; i9-9900K; EVGA 280 CLC; 2 x 8GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; EVGA GTX 1660ti XC Ultra Gaming; Corsair TX650M

dcn10: Fractal Define C; Gigabyte b450 Aorus M; Ryzen 9 5900x; Ryzen 7 2700x; Hyper 212Evo e/w Noctua NF-A12; 2 x 8GB DDR4-3200; Samsung 250GB SSD; EVGA XC GTX 1070 ti 8GB; Corsair TX550M

dcn11: Fractal Focus G mini; Gigabyte b450 Aorus M; Aorus M; Ryzen 7 2700; Wraith Spire; 2 x 8GB DDR4-3200; Samsung 840 Pro 128GB SSD; ATI Radeon HD2400 Pro; Corsair CX450M Bronze

dcn12: Fractal Core 1100; Gigabyte b450 Aorus M; Ryzen 7 2700; Wraith Spire; 2 x 8GB DDR4-3200; Samsung EVO 850 128GB SSD; ATI FirePro 2270; Corsair CX500M Bronze

dcn19: SuperMicro SC731; SuperMicro X9scl; Xeon e3-1240v2; 4 x 4GB DDR3-1333 ECC; 120GB SSD; SuperMicro 300W Bronze

Link to comment
Share on other sites

Link to post
Share on other sites

3 hours ago, Gorgon said:

Getting Zabbix working is a ton of work and getting it configured to capture relevant information is an additional pile of work.

 

I agree that the network settings are easiest done through the GUI. I usually harcode the IP, Mask, Gateway and DNS on the initial build then just manage everything through SSH via Public Keys after that.

 

nvidia-smi -i 0 -l 1 --format=csv,noheader --query-gpu=temperature.gpu,power.draw,clocks.gr,fan.speed

is one of my favorite commands. I have PuTTY shortcuts for all my slots so I can watch them.

I decided I wanted to have all of it reported in HFM directly so I've started a fork to display there.  Have a hacked together proof-of-concept with basic nvidia-smi outputs being retrieved and mapped to slots. It's very much a hack job at the moment, it's display only, no recording of measurements for doing any trending/analysis and using hard coded credentials just to verify functionality.  Basically just added a SSH interface for Linux and PowerShell for Windows (though Windows I only have working local, not remote) and query based on the same polling method used to get the FAHclient stats (which I think is every 30 or 60 seconds).

nvidia-smi --query-gpu=pci.bus,fan.speed,temperature.gpu,pcie.link.gen.current,pcie.link.width.current,pcie.link.gen.max,pcie.link.width.max,pstate,power.draw,power.limit,power.default_limit,clocks.current.graphics,clocks.current.memory --format=csv,noheader,nounits

This is what I've got hooked in so far in the UI.  A number of static things there, like pcie gen and speed, that I was including to be able to quickly verify current config at a glance.

image.thumb.png.69f255b8873ade9e50f82567c3c87c90.png

 

The SSH interface seems pretty straight forward so I'm thinking I can probably get full management of things like power limits, clock speeds, fan speeds, etc eventually.  Though my next task is to get it to use proper SSH key authentication instead of hard coded user/pw.

Link to comment
Share on other sites

Link to post
Share on other sites

I've been folding on my new GTX 1080 for about a day now, and at stock settings FAHControl is giving me an estimate of 1.1 million PPD. That's a bit lower than the 1.4 million my 1660 Ti gets, but this 1080 is much quieter, which is nice considering it's in my main PC. I won't be able to run both of these cards in my main machine at once though unless I upgrade my power supply and downgrade my SSD. The second PCIe X16 slot is disabled when the primary M.2 slot is in use. 

 

The Quadro M4000 also arrived today, and I just got it installed in a test system. It's currently working on a 10 hour WU, but FAHControl is estimating that it'll get around 450K PPD. That's comparable to my Radeon RX 570s at stock settings, but with the benefit of only needing a single slot and a 6-pin power connector. This Quadro probably won't be folding 24/7, but it'll be nice to have when I want to get a few more points. 

Phobos: AMD Ryzen 7 2700, 16GB 3000MHz DDR4, ASRock B450 Steel Legend, 8GB Nvidia GeForce GTX 1080, 1TB Samsung SSD 980, 450W Corsair CXM, Corsair Carbide 175R, Windows 10 Pro

 

Pluto: Intel Core i7-2600, 32GB 1600MHz DDR3, ASUS P8Z68-V, 4GB XFX AMD Radeon RX 570, 1TB Samsung 860 EVO, 3TB Seagate BarraCuda, 750W EVGA BQ, Fractal Design Focus G, Windows 10 Pro for Workstations

 

York (NAS): Intel Core i5-2400, 16GB 1600MHz DDR3, HP Compaq OEM, 240GB Kingston V300 (boot), 3x2TB Seagate BarraCuda, 320W HP PSU, HP Compaq 6200 Pro, TrueNAS CORE (12.0)

 

Mid 2011 27" iMac (general use Mac): Intel Core i5-2400, 16GB 1333MHz DDR3, Z68 Logic Board, 1GB AMD Radeon HD 6970M, 500GB Crucial MX500, 2TB WD Blue, macOS High Sierra

Link to comment
Share on other sites

Link to post
Share on other sites

On 8/2/2022 at 5:11 PM, BondiBlue said:

The Quadro M4000 also arrived today, and I just got it installed in a test system. It's currently working on a 10 hour WU, but FAHControl is estimating that it'll get around 450K PPD.

That sounds about right. My NAS has a pair of those with no adjustments to how they run get a little under 1,000,000 a day.

image.png.26fbd918f2842297257920183c8e0dc2.png

I'm not actually trying to be as grumpy as it seems.

Project Hot Box

CPU 12900k, Motherboard Gigabyte Aorus Elite AX, RAM CORSAIR Vengeance 4x16gb 5200 MHZ, GPU AMD Radeon 6800 XT FE, Case Lian Li O11 Dynamic Mini, Storage Sabrent Rocket Q4 2tbCORSAIR Force Series MP510 1920GB NVMe, CORSAIR FORCE Series MP510 960GB NVMe, PSU CORSAIR sf600, Cooling Arctic Liquid Freezer ii 360, Displays Odyssey G9, LG 34UC98-W 34-Inch,Keyboard Mountain Everest Max, Mouse Mountain Makalu 67, Sound AT2035, Massdrop 6xx headphones, Go XLR 

Oppbevaring

CPU i9-9900k, Motherboard, ASUS Rog Maximus Code XI, RAM, 48GB Corsair Vengeance LPX 32GB 3200 mhz (2x16)+(2x8) GPU Asus ROG Strix 2070 8gb, PNY 1080, Nvidia 1080, MSI 1070 Gaming X 8gb, Case Phanteks Enthoo Evolv X, 2x Storage Samsung 860 Evo 500 GB, 5x Seagate IronWolf 8tb NAS(ZFS1), PSU Corsair RM1000x, Cooling Asus Rog Ryuo 240 with Noctua NF-12 fans, OS Unraid

 

Why is the 5800x so hot?

 

 

Link to comment
Share on other sites

Link to post
Share on other sites

Anyone else not getting many points lately? My GTX 1080 is currently estimating only ~130K PPD, and the Quadro M4000 installed in another system is estimating ~130K PPD as well. Both cards are also not running at full load - the 1080 is only pulling 25W right now. Also, the WU that my 1080 is working on has nearly 17 hours left according to FAHControl, and that number doesn't seem to be going down. Any ideas as to what is going on? 

 

I have two other machines folding without issues right now (2x RX 570s and 1x GTX 1660 Ti), and they don't seem to have any of these weird work units. 

 

On 8/4/2022 at 12:28 PM, IkeaGnome said:

That sounds about right. My NAS has a pair of those with no adjustments to how they run get a little under 1,000,000 a day.

image.png.26fbd918f2842297257920183c8e0dc2.png

I used a couple Quadro M4000s for a bit of folding many years ago, and back then I remember getting around 250K-300K PPD on each one. It's nice to see that they've come down in price a fair amount over the past many months. I remember looking for one on eBay somewhere around December of last year and couldn't find a working one for less than $250 or so. I paid $150 for this one, and so far it's been mostly perfect (see above for the issues I'm having). Doesn't make too much heat either, so that's definitely nice. 

Phobos: AMD Ryzen 7 2700, 16GB 3000MHz DDR4, ASRock B450 Steel Legend, 8GB Nvidia GeForce GTX 1080, 1TB Samsung SSD 980, 450W Corsair CXM, Corsair Carbide 175R, Windows 10 Pro

 

Pluto: Intel Core i7-2600, 32GB 1600MHz DDR3, ASUS P8Z68-V, 4GB XFX AMD Radeon RX 570, 1TB Samsung 860 EVO, 3TB Seagate BarraCuda, 750W EVGA BQ, Fractal Design Focus G, Windows 10 Pro for Workstations

 

York (NAS): Intel Core i5-2400, 16GB 1600MHz DDR3, HP Compaq OEM, 240GB Kingston V300 (boot), 3x2TB Seagate BarraCuda, 320W HP PSU, HP Compaq 6200 Pro, TrueNAS CORE (12.0)

 

Mid 2011 27" iMac (general use Mac): Intel Core i5-2400, 16GB 1333MHz DDR3, Z68 Logic Board, 1GB AMD Radeon HD 6970M, 500GB Crucial MX500, 2TB WD Blue, macOS High Sierra

Link to comment
Share on other sites

Link to post
Share on other sites

Has anyone else been getting WU's that are a lot lower ppd than avg for your gpu? I've tried with and without the advanced client-type tag, and I keep getting these work units that  are 1/3-1/5 normal ppd for 3080 and 3090. It's been doing it for a while, and especially with the heat it is really frustrating getting these low ppd work units, that don't seem to be running efficiently. 

Link to comment
Share on other sites

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share



×