Building a 4P system

[Whaler_99]

Make sure, or else I'll have to show this.  :P

 

 

 

*Got to love the smileys from Overclockers.com* 

 

Awesome! :D

[Whaler_99]

So here is a bit of a teaser for you guys... few parts starting to trickle in..

 

First up - the coolers from CM arrived as have the 4 x Opterons...

 

 

Here are some close ups of the CPU's - and no, not taking them out of the anti static until the board gets here. :)

 

 

 

 

[qwerty90]

Very nice, looking forward for more pics.

[Whaler_99]

To Update - Canada Post says the new mobo will be here monday. FINALLY got RAM ordered. Was looking for some very specific GSkill Ripsaw ram that is confirmed to work in the board and support overclocking. Was having a heck of a time though getting Newegg to sort out stock, process ordering, etc. Finally got 4 kits of 2x2GB coming and 2 kits of 4x2GB. So.. coming from Jersey and Cali, so maybe a week to get here. Picked up an AX850 as well that was on sale...

 

So... what did come today? My smaller 4P folder. :D  The board, CPU, heat sinks and memory all arrived. The down side - very busy weekend planned. Hope to get this sucker power up though.

 

 

 

 

 

Love the all cooper heat sinks. They ARE heavy.  

 

Now, just need to follow up with Spotswood on delivery times for the tech stations and be all set to go...

[warmon6]

very soon, you'll take off.  :lol:

 

*I'm sure that I'm hearing this stuff around whalers house.*

 

https://www.youtube.com/watch?v=OC_zEsIFCHM

 

 

[Whaler_99]

WWWWEEeeeeeeeeeeeeeeee

[Spotswood]

Your trays have been shipped.  Here are a couple of pics of the upper tier (the posts for the lower tier have connecting rods poking out of them)

 

 

 

Enjoy!   :P

 

[Whaler_99]

WOW - those look amazing! Thanks for the update Spotswood. Really looking forward to getting them all put together and stacked...

[Whaler_99]

I think the mail man is getting nervous when delivering packages now.  :)

 

What could be in here?

 

 

OOHHHH! :D

 

 

The all coveted "Whaler's Thumbs up of approval"

 

 

 

Those are all the screws for the musky's mod I will be doing to get the Hyper 212's mounted to this sucker. Funny story. Lady at Home Depot see's all the screws and single track of channel. Asks if I am forgetting a second. No I say. Also, she points out, those are all machine screws. I probably need wood screws for shelving. I kindly inform her none of it is for shelving, but for a computer mod.

 

That's when I get "the look". Ya, I am crazy. :)

 

Now, just waiting for RAM from Newegg, PSU from NCIX and my case frames from Spotswood...

 

Don't worry - will be doing up proper build logs for both 4P builds when I get down to it...

 

[Levent]

that motherboard is like 4 mini atx board size .Are you planning to use your remaining graphics cards on this system? if you please do a 3dmark benchmark i wonder whats the results look like

[warmon6]

that motherboard is like 4 mini atx board size .Are you planning to use your remaining graphics cards on this system? if you please do a 3dmark benchmark i wonder whats the results look like

 

Well there some problems with trying to do that.  :ph34r:  

 

1. Most guys that I know of that run these type's of system for folding dont typically put gpu's in for folding. Mainly because of power requirements or the negative performance from trying to run Bigadv and GPU at the same time.

 

 

2. The main problem is because this is a 4P board, Whaler would need an Expensive server grade Windows OS to run that benchmark. (Windows 8 pro and WIndows 7 professional & above only support system with a max 2 sockets.)  So unless he wished to buy a $250+ USD OS, he'll be force to use a Linux base OS. Which 3D mark doesn't support.  :(

[warmon6]

Funny story. Lady at Home Depot see's all the screws and single track of channel. Asks if I am forgetting a second. No I say. Also, she points out, those are all machine screws. I probably need wood screws for shelving. I kindly inform her none of it is for shelving, but for a computer mod.

 

That's when I get "the look". Ya, I am crazy. :)

 

People these days. They never expect the unexpected.  :P

[Whaler_99]

Well there some problems with trying to do that.  :ph34r:  

 

1. Most guys that I know of that run these type's of system for folding dont typically put gpu's in for folding. Mainly because of power requirements or the negative performance from trying to run Bigadv and GPU at the same time.

 

 

2. The main problem is because this is a 4P board, Whaler would need an Expensive server grade Windows OS to run that benchmark. (Windows 8 pro and WIndows 7 professional & above only support system with a max 2 sockets.)  So unless he wished to buy a $250+ USD OS, he'll be force to use a Linux base OS. Which 3D mark doesn't support.  :(

 

Actually, I have legal copies of all Microsoft Server OS'es. Certification pays off. :)

 

But, the main reason you don't GPU fold on these units, if you run Windows OS you loose access to BIGADV projects. So, for all the extra power needed for the GPU's you would still earn less PPD. 

 

The most efficient method of folding is still linux based with bigadv projects...

[Whaler_99]

that motherboard is like 4 mini atx board size .Are you planning to use your remaining graphics cards on this system? if you please do a 3dmark benchmark i wonder whats the results look like

 

BTW - would probably be pretty junky compared to a high quality gaming board. These boards are not made for GPU performance...

 

But, if anyone wants to send me 4 Titan's to test, I will try it. :D

[killerx525]

I think the mail man is getting nervous when delivering packages now.  :)

 

What could be in here?

 

IMG-20130506-02961.jpg

 

OOHHHH! :D

 

IMG-20130506-02962.jpg

 

The all coveted "Whaler's Thumbs up of approval"

 

IMG-20130506-02963.jpg

 

IMG-20130506-02964.jpg

 

Those are all the screws for the musky's mod I will be doing to get the Hyper 212's mounted to this sucker. Funny story. Lady at Home Depot see's all the screws and single track of channel. Asks if I am forgetting a second. No I say. Also, she points out, those are all machine screws. I probably need wood screws for shelving. I kindly inform her none of it is for shelving, but for a computer mod.

 

That's when I get "the look". Ya, I am crazy. :)

 

Now, just waiting for RAM from Newegg, PSU from NCIX and my case frames from Spotswood...

 

Don't worry - will be doing up proper build logs for both 4P builds when I get down to it...

I like your thumb of approval Whaler :D

[warmon6]

BTW - would probably be pretty junky compared to a high quality gaming board. These boards are not made for GPU performance...

 

But, if anyone wants to send me 4 Titan's to test, I will try it. :D

 

Only problem is, Unless it's custom Water cooled heatsink attached to the card, you'll only be able to get 3 on that board.  :P

 

http://www.evga.com/Products/Product.aspx?pn=06G-P4-2794-KR

 

(Sadly, even EVGA WC card takes up 2 spots on the case.)

[killerx525]

Only problem is, Unless it's custom Water cooled heatsink attached to the card, you'll only be able to get 3 on that board.  :P

 

http://www.evga.com/Products/Product.aspx?pn=06G-P4-2794-KR

 

(Sadly, even EVGA WC card takes up 2 spots on the case.)

That EVGA Hydro Copper is mean looking!

[Levent]

yeah i saw one benchmark with 2 opterons (and with HD5870) the score is even lower than i3 but thats to be expected 

[Whaler_99]

So, with the new Beta core - some people looking at say quad 7970 systems that could put out 400K PPD - $1,400...

 

OK, first off, pricing - my 32 core 4P system, all in, will run me about $1,600 - most of the parts you can find used on Ebay or other folding sites...

 

Second. I have run 4 and 5 way GPU folders. There is a neat little bug that happens... you never get full PPD potential out of all you cards. It is all random. Typically one to two cards perform where you expect, the rest perform a bit less... Sometimes one tanks... My 660 right now in a quad system can only pull 13k with the Beta core - it used to pull 24K with a normal one...

 

Here is the deciding factor, for me at least... Power and Heat. Ok, two. :)  And maybe noise...

 

My 32 Core system, at full load is epected to draw around 700 Watts. A system with 4x7970 will draw almost 800 Watts, just for the GPU's - and that is a gaming load. I would expect Folding load to be a bit more. Factor in the rest of the components and you are looking at a min 1200+ Watt PSu. Those are not cheap. Over say one year, what does that work out to for electricity costs?

 

Second, heat. Most 4P units have large heat sinks and decent fans, yes, they generate a good amount of heat. But, nothing unmanageable. A single 7970 at load runs almost 80C. Times that by four and you have a LOT of heat and a lot of fan noise. :)

 

In the end, with the new Beta core, GPU folders are good. I just think for long term solutions, running a unit for a year + 7/24, a well priced 4P solution will be more efficient in the long run.

 

My thoughts only. :)  May turn out I am totally wrong...

[killerx525]

So, with the new Beta core - some people looking at say quad 7970 systems that could put out 400K PPD - $1,400...

 

OK, first off, pricing - my 32 core 4P system, all in, will run me about $1,600 - most of the parts you can find used on Ebay or other folding sites...

 

Second. I have run 4 and 5 way GPU folders. There is a neat little bug that happens... you never get full PPD potential out of all you cards. It is all random. Typically one to two cards perform where you expect, the rest perform a bit less... Sometimes one tanks... My 660 right now in a quad system can only pull 13k with the Beta core - it used to pull 24K with a normal one...

 

Here is the deciding factor, for me at least... Power and Heat. Ok, two. :)  And maybe noise...

 

My 32 Core system, at full load is epected to draw around 700 Watts. A system with 4x7970 will draw almost 800 Watts, just for the GPU's - and that is a gaming load. I would expect Folding load to be a bit more. Factor in the rest of the components and you are looking at a min 1200+ Watt PSu. Those are not cheap. Over say one year, what does that work out to for electricity costs?

 

Second, heat. Most 4P units have large heat sinks and decent fans, yes, they generate a good amount of heat. But, nothing unmanageable. A single 7970 at load runs almost 80C. Times that by four and you have a LOT of heat and a lot of fan noise. :)

 

In the end, with the new Beta core, GPU folders are good. I just think for long term solutions, running a unit for a year + 7/24, a well priced 4P solution will be more efficient in the long run.

 

My thoughts only. :)  May turn out I am totally wrong...

Isn't power cheap for you?

[qwertywarrior]

So, with the new Beta core - some people looking at say quad 7970 systems that could put out 400K PPD - $1,400...

OK, first off, pricing - my 32 core 4P system, all in, will run me about $1,600 - most of the parts you can find used on Ebay or other folding sites...

Second. I have run 4 and 5 way GPU folders. There is a neat little bug that happens... you never get full PPD potential out of all you cards. It is all random. Typically one to two cards perform where you expect, the rest perform a bit less... Sometimes one tanks... My 660 right now in a quad system can only pull 13k with the Beta core - it used to pull 24K with a normal one...

Here is the deciding factor, for me at least... Power and Heat. Ok, two. :) And maybe noise...

My 32 Core system, at full load is epected to draw around 700 Watts. A system with 4x7970 will draw almost 800 Watts, just for the GPU's - and that is a gaming load. I would expect Folding load to be a bit more. Factor in the rest of the components and you are looking at a min 1200+ Watt PSu. Those are not cheap. Over say one year, what does that work out to for electricity costs?

Second, heat. Most 4P units have large heat sinks and decent fans, yes, they generate a good amount of heat. But, nothing unmanageable. A single 7970 at load runs almost 80C. Times that by four and you have a LOT of heat and a lot of fan noise. :)

In the end, with the new Beta core, GPU folders are good. I just think for long term solutions, running a unit for a year + 7/24, a well priced 4P solution will be more efficient in the long run.

My thoughts only. :) May turn out I am totally wrong...

A few questions

the bug only happens on 4 + gpu rigs ?
does that only happen on the beta core or did it also happen on older cores ?
are they fixing this problem ?
how much PPD does ur 32core system get because a 4p system with 6168 (48 cores) get about 400k+ ppd at stock

issues with your post for people who live overseas

the motherboards used for 4p systems arnt really offered world wide since i live over seas this isnt ideal because if that $500+ board dies shipping is expensive (RMA)
coolers that work on opteron motherboards are very few (noctua and coolermaster) which is a problem for people living overseas because of shipping (weight)

where i live its mostly dominated by asian distributers (gigabyte msi asus etc)
and electricty is dirt cheap
so for some people its a better alternative to go with a 4 card folding system
but for americans it seems 4p is the way to go

[Whaler_99]

Isn't power cheap for you?

 

Cheap is comparative - yes, cheap compared to other places. But when my bill goes from $200 a month to $300 a month, ya, it hurts. :)

[Whaler_99]

A few questions

the bug only happens on 4 + gpu rigs ?

does that only happen on the beta core or did it also happen on older cores ?

are they fixing this problem ?

how much PPD does ur 32core system get because a 4p system with 6168 (48 cores) get about 400k+ ppd at stock

issues with your post for people who live overseas

the motherboards used for 4p systems arnt really offered world wide since i live over seas this isnt ideal because if that $500+ board dies shipping is expensive (RMA)

coolers that work on opteron motherboards are very few (noctua and coolermaster) which is a problem for people living overseas because of shipping (weight)

where i live its mostly dominated by asian distributers (gigabyte msi asus etc)

and electricty is dirt cheap

so for some people its a better alternative to go with a 4 card folding system

but for americans it seems 4p is the way to go

 

I have seen this GPU issue with as little a two... but typically with three or more. Happens all the time - not specific to the Core or Project or even FAH itself to an extent. I think it is more how Windows works, allocates work and CPU resources, etc. I don't know that there is anything the FAH can do to fix. Playing with the number of cores folding can also affect this. By leaving more free you can help boost the GPU performance, up to an extent. So, on a 8 core unit with 4 GPU's, the sweet spot might be 6 cores folding, or 5... or... you need to adjust, let it run for a few days to get a good metric and then review. 

 

I am still working on getting my 4P units up and running, but the 32 core one, because it can also overclock pretty high, can pull between (estimated from the guy I got the from) 280k on a very slow project to upwards of 400k on the "good" stuff. :)

 

As for overseas - you make valid points. If the material itself isn't available or is simply to expensive, then yes, a GPU folder may be the way to go, especially if you can get GPU's cheaper.

 

What works in one region may not always work in another, important point to keep in mind and thanks for bringing it up.

[qwertywarrior]

I have seen this GPU issue with as little a two... but typically with three or more. Happens all the time - not specific to the Core or Project or even FAH itself to an extent. I think it is more how Windows works, allocates work and CPU resources, etc. I don't know that there is anything the FAH can do to fix. Playing with the number of cores folding can also affect this. By leaving more free you can help boost the GPU performance, up to an extent. So, on a 8 core unit with 4 GPU's, the sweet spot might be 6 cores folding, or 5... or... you need to adjust, let it run for a few days to get a good metric and then review. 

 

I am still working on getting my 4P units up and running, but the 32 core one, because it can also overclock pretty high, can pull between (estimated from the guy I got the from) 280k on a very slow project to upwards of 400k on the "good" stuff. :)

 

As for overseas - you make valid points. If the material itself isn't available or is simply to expensive, then yes, a GPU folder may be the way to go, especially if you can get GPU's cheaper.

 

What works in one region may not always work in another, important point to keep in mind and thanks for bringing it up.

 

if its how windows work

looking at the latest FAH vid

it talks about nvidia working on linux (but didnt mention AMD cards)

maybe that will fix it once and for all

we'll see

[Whaler_99]

Ya, they are apparently doing some work on GPU folding on Linux... that be interesting...