APOLLO (2 CPU LGA1366 Server | InWin PP689 | 24 Disks Capacity) - by alpenwasser [COMPL. 2014-MAY-10]

[alpenwasser]

Table of Contents

01. 2013-NOV-14: First Hardware Tests & The Noctua NH-U9DX 1366

02. 2013-NOV-16: Temporary Ghetto Setup, OS Installed

03. 2014-APR-01: PSU Mounting & LSI Controller Testing

04. 2014-APR-02: The Disk Racks

05. 2014-APR-08: Chipset Cooling & Adventures in Instability

06. 2014-APR-09: Disk Ventilation

07. 2014-APR-11: Fan Unit for Main Compartment Ventilation

08. 2014-APR-12: Storage Topology & Cabling

09. 2014-APR-26: Storage and Networking Performance

10. 2014-MAY-10: Sound Dampening & Final Pics

PDF Version of this Build Log

http://alpenwasser.net/repository/files/apollo.pdf

Hardware - Final Config

CASE:               InWin PP689

PSU:                Enermax Platimax 600 W

MB:                 Supermicro X8DT3-LN4F

CPU:                2 × Intel Xeon L5630 (quadcore, hyperthreaded)

HS:                 Noctua NH-U9DX - Socket LGA1366

RAM:                24 GB Hynix DDR3 1333 MHz ECC

HBA CARD 0:         LSI 9211-8i, flashed to IT mode (Tutorial)

HBA CARD 1:         LSI 9211-8i, flashed to IT mode

HBA CARD 2:         LSI 9211-8i, flashed to IT mode

SSD:                Intel 520, 120 GB

HDD 0:              WD VelociRaptor 150 GB (2.5")

HDD 1-3:            Samsung HD103UJ 1 TB F1 × 3

HDD 4-7:            WD RE4 2 TB × 4

HDD 8-13:           WD Red 3 TB × 6

Total Raw Capacity: 29 TB

 

Pics of Final Form - More in Final Post

 

(click image for full res)

(click image for full res)

 

Wait, What, and Why?

So,   yeah,    another   build. Another   server,    to   be

precise. Why? Well, as  nice of  a system  ZEUS is,  it does

have two major shortcomings for its use as a server.

When I  originally conceived ZEUS,  I did not plan  on using

ZFS (since it was not  yet production-ready on Linux at that

point). The  plan was  to use  ZEUS' HDDs  as single  disks,

backing up the  important stuff. In case of  a disk failure,

the loss of  non-backed up data would  have been acceptable,

since it's mostly  media files. As long as  there's an index

of  what  was  on  the  disk,  that  data  could  easily  be

reaquired.

But right  before ZEUS was  done, I  found out that  ZFS was

production-ready on Linux, having kept a bit of an eye on it

since fall  2012 when I dabbled  in FreeBSD and ZFS  for the

first time. Using  FreeBSD on the  server was not  an option

though since I was nowhere near proficient enough with it to

use it for  something that important, so it had  to be Linux

(that's why I didn't originally plan on ZFS).

So,  I deployed  ZFS on  ZEUS,  and it's  been working  very

nicely  so  far. However, that  brought  with  it two  major

drawbacks: Firstly, I was now missing 5 TB of space, since I

had been  tempted by ZFS  to use those for  redundancy, even

for our media files. Secondly, and more importantly, ZEUS is

not an ECC-memory-capable system. The reason this might be a

problem is that  when ZFS verifies the data on  the disks, a

corrupted bit in your RAM  could cause a discrepancy between

the  data in  memory and  the data  on disk,  in which  case

ZFS  would  "correct"  the  data  on  your  disk,  therefore

corrupting it. This  is not exactly optimal  IMO. How severe

the consequences of this would  be in practice is an ongoing

debate in various ZFS  threads I've read. Optimists estimate

that it would merely corrupt  the file(s) with the concerned

corrupt bit(s), pessimists are  afraid it might corrupt your

entire pool.

The main focus of this machine will be:

• room to install more disks over time
• ECC-RAM capable
• not ridiculously expensive
• low-maintenance, high reliability and

  availability (within reason, it's still

  a home and small business server)

Modding

Instead of some  uber-expensive W/C setup, the  main part of

actually building  this rig will  be in modifying  the PP689

for fitting as many HDDs  as halfway reasonable as neatly as

possible. I have not  yet decided if there  will be painting

and/or sleeving  and/or a window. A window  is unlikely, the

rest depends mostly  on how much time I'll have  in the next

few weeks (this  is not a long-term project, aim  is to have

it done way before HELIOS).

Also, since  costs for this  build should not spiral  out of

control, I will  be trying to reuse as many  scrap and spare

parts I have laying around as possible.

Teaser

More  pics  will  follow  as  parts  arrive  and  the  build

progresses, for now a shot of the case:

(click image for full res)

That's all for now, thanks for stopping by, and so long.

Edited December 28, 2014 by alpenwasser

[Bluejay0]

FIRST AND SUBBED! Can't wait!

[MrSuperb]

yes, another alpenwaser build thread

[nsyedhasan]

He's back again!

[alpenwasser]

Holy crap you guys are fast.

FIRST AND SUBBED! Can't wait!

 

*gives cookie*

 

yes, another alpenwaser build thread

 

Yup, thanks!

 

He's back again!

Was I ever gone?

[Bluejay0]

Holy crap you guys are fast.

 

*gives cookie*

 

 

 

 

om nom nom nom :3

[Lil Chillbil]

can't wait for this, apollo 2 is such a good name 

[legopc]

i luve dual cpu builds

[Kuzma]

any recommendations for where to get a dual socket LGA1366 mobo ? thinking of picking up 2 more e5520s for the hell of it

[megadarkwood]

OMG GIMMIE PLZ <3

[AndyCapets]

Subbed, can't wait :-)

[alpenwasser]

can't wait for this, apollo 2 is such a good name

 

Well, I have a weakness for ancient Greek mythology.

 

i luve dual cpu builds

 

IKR?!

 

any recommendations for where to get a dual socket LGA1366 mobo ? thinking of picking up 2 more e5520s for the hell of it

I bought mine here, there's still 9 left: link

Shipping was insanely fast, I ordered it last Tuesday, it shipped on Wednesday and was here

yesterday. One small disclaimer: I haven't been able to test it yet because the CPUs aren't

here yet. So let's hope it has actually survived its transatlantic journey (it was very well

packaged though, antistatic wrapping and plenty of padding). But if it's still good, then I'll

be very happy indeed.

EDIT:

OMG GIMMIE PLZ <3

 

 

Subbed, can't wait :-)

Thanks guys!

[MG2R]

And subbed. What food is on the menu in this blog?

[jclappers]

 

And subbed. What food is on the menu in this blog?

Subway and i also subbed

[alpenwasser]

And subbed. What food is on the menu in this blog?

 

Thanks! Haven't decided yet, we'll see.

 

Subway and i also subbed

Hm, I will take that into consideration, and thanks!

[Emkryan]

I'll be a close Alpen Watcher of the thread.

[flibberdipper]

Holy crap you guys are fast.

 

*gives cookie*

 

 

Yup, thanks!

 

Was I ever gone?

To start off, FINALLY another build log. Second, how did you get more than 3 lines of stuff in your signature? I MUST KNOW YOUR SECRET, ALMIGHTY OVERLORD!!!

[alpenwasser]

I'll be a close Alpen Watcher of the thread.

 

Hehe, I see what you did there.

And thanks!

 

To start off, FINALLY another build log. Second, how did you get more than 3 lines of stuff in your signature? I MUST KNOW YOUR SECRET, ALMIGHTY OVERLORD!!!

It's just two lines actually. If you make a very long line it will wrap around and

continue on another one.

[alpenwasser]

First Steps

Hardware Tested

M/B, CPUs and memory have all arrived. The CPUs and M/B seem to be working OK.

One of the memory modules seems to be having a bit of trouble being recognized,

the other five work fine. I'll see if it's really defective or if it's just the

IT gods screwing with me a bit.

The Noctua NH9DX 1366

The Noctua NH-U9DX 1366 is a cooler from Noctua's series specifically made for

Xeon sockets. For those who don't know, LGA1366 sockets have an integrated

backplate, just like LGA2011, which makes them much more convenient than their

desktop counterparts. It's quite a nice and sturdy backplate, too, in fact it's

among the most solid backplates I've come across yet. This does, however,

require a slightly different mounting system. You just have four screws which

you bolt directly into the plate.

Aside from that, the cooler is identical to its desktop counterpart as far as I

know. Why the 92 mm version? For one thing, it was in stock, unlike the 120 mm

version of this cooler. Also, the CPUs only produce 40 W TDP each, so there

really is no need for high-end cooling. And as a bonus, I got supplied some

awesome San Ace fans with my case, which also happen to be 92 mm.

The Noctua fans which come with the cooler are just 3 pin fans (the newer models

of this cooler for LGA2011 come with a PWM fan I think), but the San Ace fans I

got with my case are actually PWM controlled! Since the M/B has a full set of

PWM headers (8, to be exact, how awesome is that!? ) I will try the San Ace

fans and see how they play on lower rpm's (they run at 4,800 rpm on full speed

). This does not need to be a super-silent machine since it will be in its

own room, and since I really like the San Ace fans with regards to build quality

(and I'm a total sucker for build quality) I'd love to use them for this. The

Noctuas would admitteldy be better suited, but I'll see how things go with the

SA's first.

The Box

Unlike its shiny desktop counterparts, the NH-U9DX comes in a nice and subtle

(but sturdy) cardbord box with a simple sticker on it. I must admit I like this

box more than the shiny ones.

(click image for full res)

Contents

How it looks packaged...

(click image for full res)

... and out in the open.

(click image for full res)

Noctua Pr0n

A few glory shots of the cooler itself...

(click image for full res)

(click image for full res)

The San Ace 9G0912P1G09

There is no info about this fan on the web, I'm presuming it's something San Ace

makes specifically for InWin in an OEM deal.

I've hooked it up to a fan controller and got a max reading of 4,800 rpm, and

the Supermicro board turns them down to ~2,200 rpm on idle. They seem to be very

good fans, you can only really hear the sound of the air moving, no bearing or

motor noises so far. Also, they are heavy (~200 g per piece), which is always

nice for a build quality fetishist such as myself.

Note: Hooking such a fan up to a desktop board as its power source would not be

advisable, they are rated for 1.1 A and might burn out the circuits on a desktop

board. Server boards usually have better fan power circuitry since they are

desinged with high-performance fans in mind. Just as a side note.

(click image for full res)

Compared to the Noctua fan which comes with the coolers. I might still go with

the Noctuas, but it's not the plan at the moment.

(click image for full res)

The Noctua NH-U9DX 1366 San Ace Edition

I had to improvise a bit with mounting the San Ace's to the tower. The clips

which you'd use with the Noctua fans rely on the fan having open corners, which

the San Ace's do not. Ah well, nothing a bit of cotton cord can't fix.

(click image for full res)

And the current config in its full glory:

(click image for full res)

Side note: The coolers were actually more expensive than the CPUs. :lol:

That's it for now, thanks for stopping by.

[flibberdipper]

I like it. I have a 1.2A desktop fan, that uses molex to start up (or gain speed, or maintain speed at high RPMs since it has a fan control thing too) and uses a standard 3-pin to keep it moving at low speeds and for speed monitoring.

[NohbdyKnows]

Side note: The coolers were actually more expensive than the CPUs. :lol:

 

Wat

[legopc]

that paint finish on the fan looks cinda wierd 

[alpenwasser]

I like it. I have a 1.2A desktop fan, that uses molex to start up (or gain speed, or maintain speed at high RPMs since it has a fan control thing too) and uses a standard 3-pin to keep it moving at low speeds and for speed monitoring.

 

Cool.

 

Wat

 

Haha, yeah I got the CPUs for 60 USD a piece on eBay plus 20 USD shipping plus about 30 USD

VAT & processing fees. Even taking VAT and processing fees into account the coolers were

about 10 USD more expensive I think. They were certainly more expensive when leaving out

VAT/fees.

 

that paint finish on the fan looks cinda wierd

You mean the San Ace? They're not painted, that's just the plastic they're made of. It's

a fiber reinforced industrial plastic. Sturdy as hell.

It might also look a bit weirder on the pic than IRL, I played around with the image in

post-processing quite a bit, contrast is very high in it, which brings out things you

don't usually see under normal conditions. In real life they look mostly dark grey/black.

[flibberdipper]

 

You mean the San Ace? They're not painted, that's just the plastic they're made of. It's

a fiber reinforced industrial plastic. Sturdy as hell.

It might also look a bit weirder on the pic than IRL, I played around with the image in

post-processing quite a bit, contrast is very high in it, which brings out things you

don't usually see under normal conditions. In real life they look mostly dark grey/black.

You beat me to it.

[Darkfeign]

Im surprised that Noctua include a fan that they target specifically at airflow with a CPU heatsink.