Jump to content

GPU Rig 2020.

I've purchased a x16 to X4X4X4X4 adapter to work with my MSI X570 A-Pro. However, I need ideas on mounting 6 GPU's into this case. It's a Thermaltake Level 20 GT. 

 

My current setup. RTX 2080 Super in the rear, RTX 2070 in the front, and a GTX 1060 3GB sat on its rear.

102772206_10220681435258287_130773717766313258_o.thumb.jpg.accd37569dd30688eeb4a796fed634af.jpg

 

The adapter:

https://peine-braun.net/shop/index.php?route=product/product&path=65_63&product_id=95

 

My current PPD for folding at home is just over 4M. I'd like to get that over 6M when the new Ampere GPU's come out. Will be upgrading my CPU to a Ryzen 9, also which when the new ones come out.

Current Build

 

Link to comment
Share on other sites

Link to post
Share on other sites

Some more of an update.

 

After looking at the bios, I found that the X16-1 has two modes. Set either as an x16, or bifurcated to x4x4x4x4. This had given me the idea, that it's possible to run 4 GPU's on the X16 slot, each at X4. This is further made possible as the CPU does not have an iGPU.

 

My current setup as shows,

106913381_10220924079284236_549535928076498248_o.thumb.jpg.81ca12120070ca3b75833d24a5bb7766.jpg

 

M.2: GTX 1060 3GB (Floating on cable ties) running at X4.

X16: RTX 2080 Super running at X4.

X16: RTX 2070 running at X4.

X1: GTX 960 running at X1.

 

As seen in the picture, that blue X1 riser had given me the idea of mounting the GPU's directly onto the case. This lead to further research into riser cards for the X16 slot. I soon came across something from MaxCloudOn, specifically, the X16 to X4X4X4X4. https://riser.maxcloudon.com/en/bifurcated-risers/22-bifurcated-riser-x16-to-4x4-set.html. I have this on order. It's a bit hefty in price, but would be able achieve what I wanted to get.

 

I have two Hyper Evo M.2's, but I was unable to get those working. At least, I couldn't get video output from my 1060.

 

Current Folding at Home PPD:

Screenshot_214.png.d06a72196a6bce89269fce430aeee29b.png

 

More updates to come!

 

Current Build

 

Link to comment
Share on other sites

Link to post
Share on other sites

4 hours ago, RollinLower said:

this is janky af. i love it!

does it still recognise in F@H control tho?

 

 

The 1060 is not recognized in the bios, but windows does detect it and it's fully operational. Not sure if this is normal, as it's the first time plugging in a GPU into the M.2 port. The adapter I'm using is one from China, at £3. It's electronically wired for x4. I'm not sure why they're quite hard to find.

Current Build

 

Link to comment
Share on other sites

Link to post
Share on other sites

@Mnpctech (Bill Owen) sells GPU mounts for 120mm fan mounts ,and while they might be a bit much for this build, you could perhaps use that idea/concept for ideas, especially since you dont need to worry about the I/O :) 

cheap-vertical-gpu-mount-pci-bracket-mnpctech-three_1024x1024.jpg?v=1575138091

GTX-vertical-video-card-bracket-mount_1024x1024.jpg?v=1575138091

"Put as much effort into your question as you'd expect someone to give in an answer"- @Princess Luna

Make sure to Quote posts or tag the person with @[username] so they know you responded to them!

 RGB Build Post 2019 --- Rainbow 🦆 2020 --- Velka 5 V2.0 Build 2021

Purple Build Post ---  Blue Build Post --- Blue Build Post 2018 --- Project ITNOS

CPU i7-4790k    Motherboard Gigabyte Z97N-WIFI    RAM G.Skill Sniper DDR3 1866mhz    GPU EVGA GTX1080Ti FTW3    Case Corsair 380T   

Storage Samsung EVO 250GB, Samsung EVO 1TB, WD Black 3TB, WD Black 5TB    PSU Corsair CX750M    Cooling Cryorig H7 with NF-A12x25

Link to comment
Share on other sites

Link to post
Share on other sites

17 hours ago, TVwazhere said:

@Mnpctech (Bill Owen) sells GPU mounts for 120mm fan mounts ,and while they might be a bit much for this build, you could perhaps use that idea/concept for ideas, especially since you dont need to worry about the I/O :) 

cheap-vertical-gpu-mount-pci-bracket-mnpctech-three_1024x1024.jpg?v=1575138091

GTX-vertical-video-card-bracket-mount_1024x1024.jpg?v=1575138091

I may acutally have use for this, as the 1060 is cable tied (And dangling) to the top of the case, due to the location of the M.2 port. I have a 3D printer, so will have to see what I can find. The GPU mount above would be suitable, but I don't know if he does the 140mm version. I may have to design the part.

Current Build

 

Link to comment
Share on other sites

Link to post
Share on other sites

5 hours ago, LazyDev said:

I may acutally have use for this, as the 1060 is cable tied (And dangling) to the top of the case, due to the location of the M.2 port. I have a 3D printer, so will have to see what I can find. The GPU mount above would be suitable, but I don't know if he does the 140mm version. I may have to design the part.

I dont think he has a 140mm option, but I know you could easily get dimensions for a 140mm fan frame and two PCI-E slots. 

 

If you're going to 3D print it though, make sure it's thick and rigid enough, as mounting it from the top hanging down means it will need to support the entire weight of the card. You could do it from the bottom instead, that way it doesn't need to be "as strong" just enough to hold it upright. 

"Put as much effort into your question as you'd expect someone to give in an answer"- @Princess Luna

Make sure to Quote posts or tag the person with @[username] so they know you responded to them!

 RGB Build Post 2019 --- Rainbow 🦆 2020 --- Velka 5 V2.0 Build 2021

Purple Build Post ---  Blue Build Post --- Blue Build Post 2018 --- Project ITNOS

CPU i7-4790k    Motherboard Gigabyte Z97N-WIFI    RAM G.Skill Sniper DDR3 1866mhz    GPU EVGA GTX1080Ti FTW3    Case Corsair 380T   

Storage Samsung EVO 250GB, Samsung EVO 1TB, WD Black 3TB, WD Black 5TB    PSU Corsair CX750M    Cooling Cryorig H7 with NF-A12x25

Link to comment
Share on other sites

Link to post
Share on other sites

Thanks for the interest guys! 

 

The Stage 2 vertical gpu mount we make with LEGS only requires drilling four mounting holes, so you can mount GPU wherever space will allow. I also have small compact version for smaller cases.

 

https://www.mnpctech.com/collections/gpu-supports-vertical-gpu-brackets

 

 

17 years of helping you void your warranty

Link to comment
Share on other sites

Link to post
Share on other sites

Another update to boot.

 

Today, the bifurcation "Kit" arrived. It took just over a week to get from Bulgaria to the UK.

20200716_143927.thumb.jpg.909ee08d5a363d1b81456b4d82db132d.jpg

 

It took a while to integrate, as I had to re arrange all the GPu's within the system. Whole build had taken 7 hours, as it also included a burnt PCB smell (More a little later). I did my best, but managed to get 8 GPU's in total, detecting and functioning.

20200716_203353.thumb.jpg.91d44960cb545c927de71f33b4fc48a8.jpg

 

Now onto the burnt smell. The space was small, so the 1060 within had to pushed into the M/B to make room for the P106-090, and the 2080 Super. This, however, had an accidental affect. 1. It pushed the P106 into the 1060, forcing the fan to jam, and secondly, it caused the VRM pins to melt into the PCIe cable. This killed the 1060 and the new cable. A £140 mistake, at least.

20200716_213446.thumb.jpg.e9e213ddb7c7331ac7bcf1d6c16390be.jpg

 

Now as you may have heard, how did I get the primary video output onto the second PCIe slot? During my research, as I couldn't find the option within the bios, it's a common problem with Ryzen builds. The solution was to disable the CSM (Compatability Support Module) and enable the UEFI option. This is not a problem with Windows 10. The Reddit post, 

 

In the end, I have 7 working GPU's (Not sure why the x1 slot the 1060 was in, isn't detecting yet. Could be a faulty cable, or slot.)

Screenshot_225.png.87a2ef903ddaaf62d9ae515158d2f7ca.png

The GPU speeds at this time:

RTX 2080 Super: X4

RTX 2070: X4

GTX 1070: X4

GTX 1070: X4

P106 090 (X2): X4

P106 090: X1.

 

As it stands, the P106-090's are just stand in's for the moment, to test the stability of the system. They will eventually be replaced (Waiting for Ampere).

Screenshot_226.thumb.png.4176eb3c25808db65858a6739454c98e.png

 

More system pics:

20200716_203401.thumb.jpg.6ea54cccf4d1c8e7fcc0d31721b3bc8c.jpg

 

20200716_203406.thumb.jpg.1459a374dcc7f1451d668302706491cf.jpg

 

20200716_203413.thumb.jpg.028f7f9bbb42daaa74a2695544032784.jpg

Current Build

 

Link to comment
Share on other sites

Link to post
Share on other sites

On 7/16/2020 at 4:45 PM, LazyDev said:

20200716_203413.thumb.jpg.028f7f9bbb42daaa74a2695544032784.jpg

"Whatcha doin?"

"Not much, just hanging around"

 

Also I see you are a subscriber of which condiment you are allowed to put on a hotdog with those PCI-E cables ;^)

"Put as much effort into your question as you'd expect someone to give in an answer"- @Princess Luna

Make sure to Quote posts or tag the person with @[username] so they know you responded to them!

 RGB Build Post 2019 --- Rainbow 🦆 2020 --- Velka 5 V2.0 Build 2021

Purple Build Post ---  Blue Build Post --- Blue Build Post 2018 --- Project ITNOS

CPU i7-4790k    Motherboard Gigabyte Z97N-WIFI    RAM G.Skill Sniper DDR3 1866mhz    GPU EVGA GTX1080Ti FTW3    Case Corsair 380T   

Storage Samsung EVO 250GB, Samsung EVO 1TB, WD Black 3TB, WD Black 5TB    PSU Corsair CX750M    Cooling Cryorig H7 with NF-A12x25

Link to comment
Share on other sites

Link to post
Share on other sites

32 minutes ago, TVwazhere said:

 

 

Also I see you are a subscriber of which condiment you are allowed to put on a hotdog with those PCI-E cables ;^)

 

String cheese.

 

Also thanks for the tip on maxcloud breakout, saves a few bucks on bifurcation vs going the nvme riser card route (though there's one for sale near me atm but buying the x4 adapters would be a hassle).

Folding@Home | BOINC

CPU: AMD Ryzen 9 3900X + Celsius S36 GPU: ASUS TUF RTX3080 

MB: ASRock x470 Taichi RAM: Corsair Vengeance RGB DDR4-2666 32GB 

CASE: Fractal Design Define 7 Panda STORAGE: WD Black SN770 2TB + WD Red Pro 6TB

Link to comment
Share on other sites

Link to post
Share on other sites

14 hours ago, RollinLower said:

what a beaut!

internal space is overrated anyway, why waste space whe you could use it for more GPU's?

I have designed and printed a part that will allow me to mount 2 GPU's onto the ceiling of the case. I may put the STL file up for sharing, but I need to test the fit, first. I'm awaiting on a couple of PCIe cables from China, so I can mount the GPU's to said bracket.

 

Screenshot_239.thumb.png.bc9f540f3d632629b13b4303879d4bf2.png

 

In terms of space, you nailed the idea. I've always wanted a machine that would max out the capabilities of the space and hardware offered. I do see how another two more GPU's would fit into this case, but I've maxed out the Ryzen 7.

 

Current peak on PPD:

Screenshot_257.png.2d33ba6fa66666f1d9e7208dded22ef7.png

 

I have purchased another GTX 1070. This will replace one of the P106's. 

 

Progress on this system will be limited for the next few months, as I've ran out of funds (Hence the purchasing of the 1070's). My target is to get the system over 12M PPD. This will happen next year!

 

16 hours ago, chaozbandit said:

Also thanks for the tip on maxcloud breakout, saves a few bucks on bifurcation vs going the nvme riser card route (though there's one for sale near me atm but buying the x4 adapters would be a hassle).

I had the same issue with those ADT link pcie cables. They would get the job done, but they were very expensive. On top of that, the GPU's would have had to have some form of custom mounting. The MaxCloudOn uses, what looks to be mini SAS connectors, for the 4 lanes, and a breakout board. In terms of reliability, I've not had one slot fail yet. 

 

Current Build

 

Link to comment
Share on other sites

Link to post
Share on other sites

Your life would be a whole lot easier and quieter with a mining frame instead of a conventional chassis.

 

Nice find on those bifurcation kits.

FaH BOINC HfM

Bifrost - 6 GPU Folding Rig  Linux Folding HOWTO Folding Remote Access Folding GPU Profiling ToU Scheduling UPS

Systems:

desktop: Lian-Li O11 Air Mini; Asus ProArt x670 WiFi; Ryzen 9 7950x; EVGA 240 CLC; 4 x 32GB DDR5-5600; 2 x Samsung 980 Pro 500GB PCIe3 NVMe; 2 x 8TB NAS; AMD FirePro W4100; MSI 4070 Ti Super Ventus 2; Corsair SF750

nas1: Fractal Node 804; SuperMicro X10sl7-f; Xeon e3-1231v3; 4 x 8GB DDR3-1666 ECC; 2 x 250GB Samsung EVO Pro SSD; 7 x 4TB Seagate NAS; Corsair HX650i

nas2: Synology DS-123j; 2 x 6TB WD Red Plus NAS

nas3: Synology DS-224+; 2 x 12TB Seagate NAS

dcn01: Fractal Meshify S2; Gigabyte Aorus ax570 Master; Ryzen 9 5900x; Noctua NH-D15; 4 x 16GB DDR4-3200; 512GB NVMe; 2 x Zotac AMP 4070ti; Corsair RM750Mx

dcn02: Fractal Meshify S2; Gigabyte ax570 Pro WiFi; Ryzen 9 3950x; Noctua NH-D15; 2 x 16GB DDR4-3200; 128GB NVMe; 2 x Zotac AMP 4070ti; Corsair RM750x

dcn03: Fractal Meshify C; Gigabyte Aorus z370 Gaming 5; i9-9900k; BeQuiet! PureRock 2 Black; 2 x 8GB DDR4-2400; 128GB SATA m.2; MSI 4070 Ti Super Gaming X; MSI 4070 Ti Super Ventus 2; Corsair TX650m

dcn05: Fractal Define S; Gigabyte Aorus b450m; Ryzen 7 2700; AMD Wraith; 2 x 8GB DDR 4-3200; 128GB SATA NVMe; Gigabyte Gaming RTX 4080 Super; Corsair TX750m

dcn06: Fractal Focus G Mini; Gigabyte Aorus b450m; Ryzen 7 2700; AMD Wraith; 2 x 8GB DDR 4-3200; 128GB SSD; Gigabyte Gaming RTX 4080 Super; Corsair CX650m

Link to comment
Share on other sites

Link to post
Share on other sites

2 hours ago, Gorgon said:

Your life would be a whole lot easier and quieter with a mining frame instead of a conventional chassis.

 

Nice find on those bifurcation kits.

I chose a conventional chassis, as this is my daily rig. I do everything on this, including gaming. I need something that would look good (Albeit, a challenge) and quiet, as it literally sits next to me. The noise is around 52 DB, with no whines. It's just like a gentle breeze. Also, I wanted RGB.

 

Building a build like this in a conventional case, provides a challenge. I do have a 3D printer, and already have a part to install into the case. I feel like it means more to put more effort into a project like this, than buying a something that is built for the task. It's more custom, and in some cases, you end up with better ascetics, and finding new, unique ways of mountain GPU's. I guess, that's the fun part.

 

I have two of those Hyper Evo M.2's, inspired by your Bifrost rig. I could never get my GPU that's plugged into those to output video. At that time aswell, I didn't know about disabling CSM to force the chipset to use the GPU on the second X16 slot, instead of the first X16 slot.

Current Build

 

Link to comment
Share on other sites

Link to post
Share on other sites

14 hours ago, Gorgon said:

Your life would be a whole lot easier and quieter with a mining frame instead of a conventional chassis.

 

Nice find on those bifurcation kits.

i literally built 4 rigs on that mining frame and m2 splitter cards.  They output video great and setting up the bifurcation was super easy.

 

if your going to fold its the easiest way.

Link to comment
Share on other sites

Link to post
Share on other sites

Another update.

 

I've installed a 3rd GTX 1070, which replaced one of the P106's.

 

Screenshot_266.png.c540fedc343a273bdce401e50099db56.png

 

I did initially plug in, 2X 1070's, 2070 and the 2080, into the HX1000I, but under load, this was overloading the PSU. With just 3 GPU's plugged in, I've been monitoring the PSU. The 12 volt rail is heavily loaded, and this voltage is starting to become a little bit concerning. I currently don't know the lowest acceptable voltage for the 12 volt rail.

 

Screenshot_265.png.775f4aa230f24a3393186e3ed46e4604.png

Current Build

 

Link to comment
Share on other sites

Link to post
Share on other sites

Don't "test" your PSU limits.

I'm a noob Folder, average PC builder, but I'm an experienced and qualified electrician.

The PSU will mostly determine if a fire occurs in your PC, and if you're not there if the fire happens...its guaranteed game over.

 

Love your project/hobby though!

Link to comment
Share on other sites

Link to post
Share on other sites

28 minutes ago, SparkiTom said:

Don't "test" your PSU limits.

I'm a noob Folder, average PC builder, but I'm an experienced and qualified electrician.

The PSU will mostly determine if a fire occurs in your PC, and if you're not there if the fire happens...its game over.

 

Love your project/hobby though!

The PSU is rated at a 1KW load, and is advertised with a 10 year warranty at this load. However, I do see your point. It wouldn't be advisable to run the unit at capacity 24/7, as something is bound to fail. This is the peak load with a, GTX 1070, RTX 2070, RTX 2080 Super and a Ryzen 7 3700X plucking at the 12 rail.

Screenshot_269.png.e03f15f34a91a6a702d6702394cd1a6f.png

 

792 Watts is quite high, not to mention the 50.C temps. I may have to flip the PSU, so it won't be drawing from inside the system. This would have to be considered, as it'll be a pain to flip.

 

I may replace all the 1070's with the 3060's (Depends on the wattage), as it was originally intended for each GPU to draw a maximum of around 120 Watts. This is to ensure that 8 GPU's can run in this system without issues, or accidentally overloading the PSU's.

Current Build

 

Link to comment
Share on other sites

Link to post
Share on other sites

  • 2 weeks later...

Update - 03/08/2020

 

I've designed and 3D printed a bracket to mount two GPU's onto the top of the case. No more cable ties for the RTX 2070!

 

20200801_230811.thumb.jpg.81d200e592767d3058577cc0f095c0a1.jpg

 

20200801_230820.thumb.jpg.956161ea2fbd7be078bc3acdd1dc470d.jpg

 

As you can see, it is a tight fit. I just hope my CPU can hold up, and that I don't have to upgrade it!

20200801_230140.thumb.jpg.f810cd54d0b1c9f30b5f996328b09a2c.jpg

 

20200801_232934.thumb.jpg.e265fc35dcdb4d41d388a71335e3844b.jpg

 

Screenshot_282.thumb.png.c7b317fb8fab37e1b24391b2caf897fe.png

 

I can say now, that my rig if officially built. Over time, the GPU's will be replaced with more power efficient, higher performing units. As of this time, there are 8 GPU's running. In order to keep temperatures down, I've had to remove the glass panel to get more airflow into the case (No pun intended). The motherboard is getting bearly any airflow now, but the system does seem to be stable.

 

I'll have the STL available for the GPU mount at a later date.

Current Build

 

Link to comment
Share on other sites

Link to post
Share on other sites

  • 3 weeks later...

Another update, 23rd August, 2020.

 

I've since pulled out all of the P106's, and installed two RTX 2060's on the x4 lane slots on the front of the case. A little more of a bump in PPD was caused when I changed from Windows to Linux Mint 20 on August 7th. I gained at least 1.5M PPD in the process.

 

2020-08-23_07-06.png.e7cf2ca14874d3ab1c888c70ae59ceda.png

 

Some GPU info,

2020-08-23_07-14.png.0d5e22531f1aa51f6d33fba0b7b9c8c0.png

 

2020-08-23_07-15.png.94c23a5248cc0dc03664ba98ac4c3f16.png

 

As of this time, my folding system is complete.

 

Current Build

 

Link to comment
Share on other sites

Link to post
Share on other sites

6 hours ago, LazyDev said:

...

As of this time, my folding system is complete.

 

Sure ? Thx god i have Thermaltake Level 20 XT Case giving me more than double the space ;) Realy, im not sure to build up 2nd motherboard in this case or just a little refrigator for sixpack of beer for not let it look this empty....

Link to comment
Share on other sites

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×