Jump to content

Could use some help for F@H multi-GPU server

Hello everyone!

 

I've been folding on and off with different computers over the years and now that I have settled in an appartment, I'd like to make a dedicated folding server.

 

The hardware I plan on using (mainly because I already have it):

CPU: Xeon E5-2660 8c/16t 2.2GHz

Cooler: Noctua C14 or U12S

Motherboard: Asus P7x79 WS

Ram: 4x4Gb 1600MHz for a total of 16Gb

PSU: I found a great deal on a 1000W Corsair PSU on ebay

A random SATA 120Gb SSD for Windows or Linux.

 

What I need help for is the GPUs... Just like many people I'm not the richest and would like to do this on a budget kinda.

Following Linus' video where he uses a mining specific card with no video output, I'd like to see if I could use these to fold. I've found a good deal where I could get some GTX 1060-6Gb models for less then 50€ each.

I still have to RTFM for the motherboard to know how many GPUs it can support at once.

 

As for case, the PSU is bundled with a rack for crypto-mining, so no worries there. If everything turns out well and I have the money to do so, I'd like to see if I could go hardcore watercooling on this (3x360mm rads, 2 pumps and such).

 

Thank you in advance for any advise you may have.

 

Best regards, seecker

Link to comment
Share on other sites

Link to post
Share on other sites

1 hour ago, yaboistar said:

the GTX 1060 will get about 400-500kppd. it's really not the most efficient card out there but it will work

 

if you can afford to, i highly suggest Turing based GPU's, as for one reason or another they pull way ahead of class in folding PPD output in both the old core21 and the new core22 units. 2060's, 1660's and 1660Ti's are the best performance per dollar folding GPU's at the minute

Thanks for the input Yaboistar. But as I've said, I'm trying to be on a budget and I don't know if there are any Turing GPU models that are meant only for mining (like the 1060 model I refered to).

Link to comment
Share on other sites

Link to post
Share on other sites

1 hour ago, yaboistar said:

the fact that you want to go watercooled indicates you're ptobably going to try and overclock the hell out of these cards. my advice is - don't. FAH is ultra sensitive to errors and sometimes will still complete a work unit with erroring only for it to be rejected as bad at the end.

 

For turing, i've found my GPU (2080Ti) has a happy place at about 2020-2040MHz. Memory clocks do not matter much for folding and the PPD difference is so small it's easily covered up by the normal fluctuations in predicted output. As such, it's best to leave the memory clock alone as GPU memory can start erroring at higher clocks in ways games and benchmarks won't detect, but FAH will pick up on. This is true for my GDDR5, 5X and 6 cards.

 

For pascal, my 1070Ti's and my 1080Ti have a happy place at about 2000-2025MHz.

 

These clocks are assuming you're running full power limit and voltage sliders. For the best Performance Per Watt, reduce the power limit to about 70% and keep the voltage at stock. @TVwazhere runs some underclocked pascal cards IIRC so perhaps he could share his settings.

Going watercooling would only be to make everything more silent. I don't really like overclocking GPUs that much.

I had overclocked my 1060's memory quite high in crypto mining applications thanks to its Samsung memory (+950MHz on the memory with no error!)

Link to comment
Share on other sites

Link to post
Share on other sites

1 minute ago, yaboistar said:

you don't have to get mining-only cards, and infact it will help if you don't in terms of future value when you come to sell hardware on. nobody can use a mining only card except for mining, which ASIC's have basically ensured they're not useful for now. but - people still sell 750Ti's for the better part of £100 because they're a useful budget gaming card

True the resell value of mining specific GPUs is very low... But I don't want this project to cost a lot in the first place and/or end up with 3 different types of cards.

Link to comment
Share on other sites

Link to post
Share on other sites

2 minutes ago, yaboistar said:

+225 on Samsung Gddr5X caused fatal errors in FAH work units for me. Like i said, it's really not worth touching because it doesn't add any tangible benefit

I've been folding on my main PC for a week with +400 on the memory and nothing has changed. I mainly fold with my PC at work so I don't pay for electricity ^^ (don't tell my boss). The work PC's GPU is a Quadro 2000 with 5Gb of VRAM and it gives me about 150k-ish per night.

Link to comment
Share on other sites

Link to post
Share on other sites

Depending on your budget and local prices a 1660ti or Super might be a better option. Twice the PPD as a 1060 and better resale value and at the same power consumption so twice as efficient.

 

1060 450kPPD 120W

1660ti 800-900kPPD 120W

2060 1100kPPD 140W

2060S 1300kPPD 160W

2070 1300kPPD 180W

2070S 1500kPPD 200W

FaH BOINC HfM

Bifrost - 6 GPU Folding Rig  Linux Folding HOWTO Folding Remote Access Folding GPU Profiling ToU Scheduling UPS

Systems:

desktop: Lian-Li O11 Air Mini; Asus ProArt x670 WiFi; Ryzen 9 7950x; EVGA 240 CLC; 4 x 32GB DDR5-5600; 2 x Samsung 980 Pro 500GB PCIe3 NVMe; 2 x 8TB NAS; AMD FirePro W4100; MSI 4070 Ti Super Ventus 2; Corsair SF750

nas1: Fractal Node 804; SuperMicro X10sl7-f; Xeon e3-1231v3; 4 x 8GB DDR3-1666 ECC; 2 x 250GB Samsung EVO Pro SSD; 7 x 4TB Seagate NAS; Corsair HX650i

nas2: Synology DS-123j; 2 x 6TB WD Red Plus NAS

nas3: Synology DS-224+; 2 x 12TB Seagate NAS

dcn01: Fractal Meshify S2; Gigabyte Aorus ax570 Master; Ryzen 9 5900x; Noctua NH-D15; 4 x 16GB DDR4-3200; 512GB NVMe; 2 x Zotac AMP 4070ti; Corsair RM750Mx

dcn02: Fractal Meshify S2; Gigabyte ax570 Pro WiFi; Ryzen 9 3950x; Noctua NH-D15; 2 x 16GB DDR4-3200; 128GB NVMe; 2 x Zotac AMP 4070ti; Corsair RM750x

dcn03: Fractal Meshify C; Gigabyte Aorus z370 Gaming 5; i9-9900k; BeQuiet! PureRock 2 Black; 2 x 8GB DDR4-2400; 128GB SATA m.2; MSI 4070 Ti Super Gaming X; MSI 4070 Ti Super Ventus 2; Corsair TX650m

dcn05: Fractal Define S; Gigabyte Aorus b450m; Ryzen 7 2700; AMD Wraith; 2 x 8GB DDR 4-3200; 128GB SATA NVMe; Gigabyte Gaming RTX 4080 Super; Corsair TX750m

dcn06: Fractal Focus G Mini; Gigabyte Aorus b450m; Ryzen 7 2700; AMD Wraith; 2 x 8GB DDR 4-3200; 128GB SSD; Gigabyte Gaming RTX 4080 Super; Corsair CX650m

Link to comment
Share on other sites

Link to post
Share on other sites

7 hours ago, Gorgon said:

Depending on your budget and local prices a 1660ti or Super might be a better option. Twice the PPD as a 1060 and better resale value and at the same power consumption so twice as efficient.

 

1060 450kPPD 120W

1660ti 800-900kPPD 120W

2060 1100kPPD 140W

2060S 1300kPPD 160W

2070 1300kPPD 180W

2070S 1500kPPD 200W

Wow thanks for the info, I didn't have the points per card scale. That said my 1060 6Gb gets more around 510kPPD.

Less then 50€ per GTX1060 6Gb or at least 250€ for a 1660ti, my choice is unfortunately done...

Link to comment
Share on other sites

Link to post
Share on other sites

1 hour ago, seecker said:

Wow thanks for the info, I didn't have the points per card scale. That said my 1060 6Gb gets more around 510kPPD.

Less then 50€ per GTX1060 6Gb or at least 250€ for a 1660ti, my choice is unfortunately done...

Plotting the numbers shown above by @Gorgon, it's becomes really difficult to deny the power of a GTX 1060 (per USD, I just took prices you mentioned for the 1060 and 1660 Ti, but substituted all the other prices for the USD pricing for the cheapest 'normal' card).

Of course the PPD/watt is less good, but that is about to be expected.

image.png.a95789b34ad6d487a883da3ba64019e4.png

"We're all in this together, might as well be friends" Tom, Toonami.

 

mini eLiXiVy: my open source 65% mechanical PCB, a build log, PCB anatomy and discussing open source licenses: https://linustechtips.com/topic/1366493-elixivy-a-65-mechanical-keyboard-build-log-pcb-anatomy-and-how-i-open-sourced-this-project/

 

mini_cardboard: a 4% keyboard build log and how keyboards workhttps://linustechtips.com/topic/1328547-mini_cardboard-a-4-keyboard-build-log-and-how-keyboards-work/

Link to comment
Share on other sites

Link to post
Share on other sites

24 minutes ago, minibois said:

Plotting the numbers shown above by @Gorgon, it's becomes really difficult to deny the power of a GTX 1060 (per USD, I just took prices you mentioned for the 1060 and 1660 Ti, but substituted all the other prices for the USD pricing for the cheapest 'normal' card).

Of course the PPD/watt is less good, but that is about to be expected.

image.png.a95789b34ad6d487a883da3ba64019e4.png

Thanks for the input Minibois.

That said the cards I'm aiming for lose all resell value because they are mining specific with no video output and locked drivers.

 

That said the 2060 seems like the 'normal' best deal for PPD/EUR

 

BTW, has anyone even tried folding on these GPUs?

Plus, I'm worried the GPUs in question won't be very silent depending on the airflow I put on them... (the server would be under my desk at home)

Link to comment
Share on other sites

Link to post
Share on other sites

Theres been a couple of threads where people have gotten these cards working under Folding at home but I’m not aware of anyone running these long term on our team.

 

Try searching this forum for keywords; Folding Mining and the mining-specific Card models

FaH BOINC HfM

Bifrost - 6 GPU Folding Rig  Linux Folding HOWTO Folding Remote Access Folding GPU Profiling ToU Scheduling UPS

Systems:

desktop: Lian-Li O11 Air Mini; Asus ProArt x670 WiFi; Ryzen 9 7950x; EVGA 240 CLC; 4 x 32GB DDR5-5600; 2 x Samsung 980 Pro 500GB PCIe3 NVMe; 2 x 8TB NAS; AMD FirePro W4100; MSI 4070 Ti Super Ventus 2; Corsair SF750

nas1: Fractal Node 804; SuperMicro X10sl7-f; Xeon e3-1231v3; 4 x 8GB DDR3-1666 ECC; 2 x 250GB Samsung EVO Pro SSD; 7 x 4TB Seagate NAS; Corsair HX650i

nas2: Synology DS-123j; 2 x 6TB WD Red Plus NAS

nas3: Synology DS-224+; 2 x 12TB Seagate NAS

dcn01: Fractal Meshify S2; Gigabyte Aorus ax570 Master; Ryzen 9 5900x; Noctua NH-D15; 4 x 16GB DDR4-3200; 512GB NVMe; 2 x Zotac AMP 4070ti; Corsair RM750Mx

dcn02: Fractal Meshify S2; Gigabyte ax570 Pro WiFi; Ryzen 9 3950x; Noctua NH-D15; 2 x 16GB DDR4-3200; 128GB NVMe; 2 x Zotac AMP 4070ti; Corsair RM750x

dcn03: Fractal Meshify C; Gigabyte Aorus z370 Gaming 5; i9-9900k; BeQuiet! PureRock 2 Black; 2 x 8GB DDR4-2400; 128GB SATA m.2; MSI 4070 Ti Super Gaming X; MSI 4070 Ti Super Ventus 2; Corsair TX650m

dcn05: Fractal Define S; Gigabyte Aorus b450m; Ryzen 7 2700; AMD Wraith; 2 x 8GB DDR 4-3200; 128GB SATA NVMe; Gigabyte Gaming RTX 4080 Super; Corsair TX750m

dcn06: Fractal Focus G Mini; Gigabyte Aorus b450m; Ryzen 7 2700; AMD Wraith; 2 x 8GB DDR 4-3200; 128GB SSD; Gigabyte Gaming RTX 4080 Super; Corsair CX650m

Link to comment
Share on other sites

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×