folding@home What are the properties of a good dedicated folding machine?

[JoeDaCabbie]

So, say I wanted to build a dedicated folding rig with multiple graphics cards. Call the starting price $1,500 -$2,000 with future expansion. I'm trying to figure out what the hardware should be to optimize folding but minimize cost. Any thoughts would be appreciated as I am new to this and don't really understand how the client uses the hardware yet.

 

Thoughts thus far (and feel free to correct misconceptions.)

 

Case: full tower lying on its side, panel replaced with mesh and a custom exhaust system on top. Rationale: not letting multiple graphics cards cook each other in a vertical configuration, but still with lots of room. Maybe even forego this for a simple DIY box.

 

Mobo: multiple pcie 3 x16 slots, preferably four.

 

CPU: overclockable with onboard graphics for the monitor, but maybe only 6-8 cores (one for up to four video cards, the rest to handle the scutwork of the OS.) CPU folding doesn't seem to be worth an investiture in lots of cores to fold with. Or would something like a 32 core Threadripper be worth it? Can you gang cores for workunits? (I know that blows the budget, but if so better to budget something different for 2021 than live with a mistake now.)

 

Memory: does folding use much RAM? Maybe 8gb? Does speed matter?

 

Storage: one 128gb SSD m.2 for the OS, another for the client. Or would RAMdrive work with F@H?

 

Video cards: how many to cram into one case? Four, eventually? Here's where the big money will go. Are there still dedicated types like those miner cards Linus did a video about still being made? If so, useful for folding?

 

Power: go large to start so I don't have to replace it later, maybe 1000-1200w?

[xl3b4n0nx]

You have 2 options the way I see it. You can go mainstream or HEDT. To get your 4 GPU slots HEDT is your best bet. Going with that assumption, I would go with a Threadripper 1900x. Its an 8 core HEDT part from AMD. You can pair it with an X399 Motherboard with 4 x16 slots. As for GPUs, my best guess would be anything good for mining would work well here, but someone can correct me. I haven't done my research on the GPUs. Toss some RAM and an SSD in and you should be good to go.

[Gorgon]

I once saw a post in the Hardware section on Foldingform.org where Folding Legend Ed Olkkala described his systems.

 

He was using 4U server rack-mount chassis with HEDT boards (and Asrock Formula x299) with 3 cards in each and still had to mod the cases adding 120mm fans to the sides to get the heat out so a mesh top might help.

 

A case like the CooMaster HAF-X EVO might also work but I doubt you'd get more than 3 cards in it and even then it would be a tight fit. Your going to have to use either water-cooling or PCIe risers to space the cards out. How about foregoing the case and using an open frame mining rig?

 

Do the math and compare building 1 HEDT system with 4 cards compared to 2 mainstream systems with 2 cards each. You might be surprised. If you willing to play with sketchy Chinese HEDT motherboards and used Xeons from eBay they might be an option but you are right in that getting rid of waste heat is the crux of the issue and with air-cooled cards it is challenging with at best 1-slot of space and with the newer 2.75 slot-width cards these days.

 

I've recently invested in Hybrid cards with an AIO as they were to be found at not too much of a premium over the dual fan variants and placing one in each dual-card system has also significantly lowered the temps on the second dual-fan card.

 

Mainboard. In the mainstream x(3|4|5)70 or z370/390 you will want to look for systems advertised as "SLI-capable" as these will have the PCIe switches to split the 16 CPU PCIe lanes to two x8s. For x299/x399 you have more options due to the many more lanes in HEDT but event with EATX and larger form-factors slot placement and spacing has to be checked carefully.

 

CPU: Yes, the points you can get from CPU folding are insignificant compared to GPUs but there are still lots of projects that cant utilize CPUs and though I initially was using Pentium Gold or AMD Ryzen 3 1200 CPUs I've since switched to 2700(x) and use the spare CPU threads for BOINC.

 

Memory. FaH will run just fine on a single stick of DDR4-2400. BOINC, however, you'll want 8GB but again DDR4-2400 would be fine.

 

PCIe slots. Conventional wisdom is a minimum of 4-lanes of PCIe3 per higher end CPU on Linux and 8 for Windows-based systems due to Windows driver architecture and PCI Bus contention issues. Having said that I am running a GTX 1060 6GB in the PCIe2 x4 ChipSet slot of a Aorus b450M under Linux for the competition and see no discernible loss of performance but YMMV.

[JoeDaCabbie]

Sorry I couldn't reply and say thanks before now, but I was hospitalized for few days; my rigs are sadly more reliable than my heart. 

 

Gorgon: I thought about what you said about the rackmounts and looked at some specs, but I didn't see anything that matched what I want to do. I'm thinking about approaching RIT and offering a prize for a case design to maximize cooling for a minimalist hardware approach that can be DIY.

 

Also, after watching Linus' ATX video I sent an email to the IEEE's computer society suggesting they look at that standard to see if it's really the most efficient design for thermals after almost 25 years. They'll probably ignore it, but who knows? Maybe they need another excuse to throw a conference.

[Gorgon]

On 10/10/2019 at 1:47 PM, JoeDaCabbie said:

Sorry I couldn't reply and say thanks before now, but I was hospitalized for few days; my rigs are sadly more reliable than my heart. 

 

Gorgon: I thought about what you said about the rackmounts and looked at some specs, but I didn't see anything that matched what I want to do. I'm thinking about approaching RIT and offering a prize for a case design to maximize cooling for a minimalist hardware approach that can be DIY.

 

Also, after watching Linus' ATX video I sent an email to the IEEE's computer society suggesting they look at that standard to see if it's really the most efficient design for thermals after almost 25 years. They'll probably ignore it, but who knows? Maybe they need another excuse to throw a conference.

Glad to hear your back at home.

 

A lot of people go for 4U Rosewill or Norco rack-mount chassis. Many 4U Chassis have mid-planes for SAS/SATA disks and you definitely don't need those for folding. Most of the 4U cases support 120 or 140mm fans as front-intake but rear exhaust is usually limited to multiple 80 or 92mm fans so getting the heat out is challenging.

I just don't think traditional chassis are good at thermal management with the heat load GPUs produce. In the enterprise space the Quadro and Volta systems use specialized chassis with small high-speed fans to direct air-low over the GPUs and these high-end GPUs usually don't have fans themselves but rely on the chassis to provide sufficient airflow. These are REALLY loud.

So the approach I'm currently taking is to use a voluminous chassis with lots of fan mounts and an airflow path that is un-restricted by drive bays etc. The Fractal Meshify series and other mesh-fronted chassis work well in this application but are limited to having 2 GPUs, 3 if you get creative.

If you want to have many GPUs running in one system and don't have a data center then going open air is likely the way to go. People used to do creative things with wood but a positive aspect of the Crypto Mining craze is availability of open-air chassis from companies such as Rosewill and Veedha Granted it is not as clean as an enclosed system but it does make managing the heat easier.