The Pure Solid State Server Build Log

[leadeater]

Cable Management: 6/10

Rouge mismatched fan used, minus 1 point.

Having more SSDs than I do, minus 1000 points.

 

Nit pick completed 

[Ben17]

7 hours ago, Windows7ge said:

 

More updates may occur should I come across a good supply of Intel DC/D3 4XXX series 960GB SSDs.

looking good so far  Hope it all goes well for you.

Happy Christmas

[Windows7ge]

7 hours ago, leadeater said:

Cable Management: 6/10

Rouge mismatched fan used, minus 1 point.

Having more SSDs than I do, minus 1000 points.

 

Nit pick completed 

OH? Lets see you do a better job in this box. There is no "behind the motherboard tray", there is no empty 5.25" bay, no space above/below/beside and I've effectively ran 48 SATA cables & 11 SATA power cables. The only thing I could do to make it prettier would be to put some velcro straps on em. LTTStore.com

 

Filling all 10 front bays and then trying to use this server for BOINC the thing was utterly choked for air. I tried using all NOCTUA's but the CPU's were still hotter than I'd like. Swapped em for Deltas. That damned X540 was getting too hot so I swapped the NOCTUA I had there for an old SP Corsair 120 that I knew pushed air like a Delta at 12V. It helped a lot. Removed the "top" & "bottom" fan in substitution of a side panel (made a Status Update about it) which REALLY helped and now even with everything maxed out I'm sitting pretty with everything in the mid 60°C low 70°C in the summer.

 

I only have...16?...18?...lost count. You could 1 up me for cheaper with used 2.5" 1.2TB PCI_e SSDs off eBay if you like. 

 

2 hours ago, Ben17 said:

looking good so far  Hope it all goes well for you.

Happy Christmas

Oh believe me things have gone wrong and utterly derailed several times over the course of this project but right now everything's looking up, thank you.

Merry Christmas to you as well. 

[Bitter]

Delta fans are pretty great, Nidec makes some great ones too and I've found over the years that I prefer the tone of the Nidec to the Delta when it comes to high flow high RPM stuff. However the Nidec's I like are the 38mm thick ones, more of a woosh than a whine if you know what I mean. 1.5 amps each fan though so you HAVE to use a hub if you want to run them via board controls.

[Ben17]

 

34 minutes ago, Windows7ge said:

OH? Lets see you do a better job in this box. There is no "behind the motherboard tray", there is no empty 5.25" bay, no space above/below/beside and I've effectively ran 48 SATA cables & 11 SATA power cables. The only thing I could do to make it prettier would be to put some velcro straps on em. LTTStore.com

 

Filling all 10 front bays and then trying to use this server for BOINC the thing was utterly choked for air. I tried using all NOCTUA's but the CPU's were still hotter than I'd like. Swapped em for Deltas. That damned X540 was getting too hot so I swapped the NOCTUA I had there for an old SP Corsair 120 that I knew pushed air like a Delta at 12V. It helped a lot. Removed the "top" & "bottom" fan in substitution of a side panel (made a Status Update about it) which REALLY helped and now even with everything maxed out I'm sitting pretty with everything in the mid 60°C low 70°C in the summer.

 

I only have...16?...18?...lost count. You could 1 up me for cheaper with used 2.5" 1.2TB PCI_e SSDs off eBay if you like. 

 

Oh believe me things have gone wrong and utterly derailed several times over the course of this project but right now everything's looking up, thank you.

Merry Christmas to you as well. 

I can imagine.... trying to level a ender 3 right now...what a pain

[Windows7ge]

35 minutes ago, Bitter said:

Delta fans are pretty great, Nidec makes some great ones too and I've found over the years that I prefer the tone of the Nidec to the Delta when it comes to high flow high RPM stuff. However the Nidec's I like are the 38mm thick ones, more of a woosh than a whine if you know what I mean. 1.5 amps each fan though so you HAVE to use a hub if you want to run them via board controls.

These Deltas, for being Deltas are surprisingly not that loud. They have gotten the job done though over the Noctuas. If Noctua had much higher RPM 92mm fans I would have bought those but they only have the Industrial 120's as far as I've seen.

 

10 minutes ago, Ben17 said:

I can imagine.... trying to level a ender 3 right now...what a pain

I've got a CR-10. Leveling it is a little tedious but not horrible. What is a pain is trying to get it to print ABS. I've got a couple latch ditch efforts to make it happen and if it doesn't I'm gonna try HTPLA.

[Ben17]

5 minutes ago, Windows7ge said:

I've got a CR-10. Leveling it is a little tedious but not horrible. What is a pain is trying to get it to print ABS. I've got a couple latch ditch efforts to make it happen and if it doesn't I'm gonna try HTPLA.

Do you think a BLTouch would be worth it?

[Windows7ge]

29 minutes ago, Ben17 said:

Do you think a BLTouch would be worth it?

Just watched a video on it. Not gonna lie it looks like a bigger pain in the ass than just leveling the bed manually with a piece of paper. Also I'm not a fan of how much of it is made out of plastic.

[Bitter]

48 minutes ago, Windows7ge said:

These Deltas, for being Deltas are surprisingly not that loud. They have gotten the job done though over the Noctuas. If Noctua had much higher RPM 92mm fans I would have bought those but they only have the Industrial 120's as far as I've seen.

That's good then, the 80MM Delta's on a water cooler from days of yore (early 2000's) are quite loud at full tilt, the other Delta's I've ran across have also been of an offensive tone to my ears, but I'm picky about fan noise. There's the signature Delta 'whine' that grates on my nerves that the Nidec lack, across all sizes this has been the case. I'm glad your Delta aren't torturous.

[Ben17]

13 minutes ago, Windows7ge said:

Just watched a video on it. Not gonna lie it looks like a bigger pain in the ass than just leveling the bed manually with a piece of paper. Also I'm not a fan of how much of it is made out of plastic.

probably will get one at some point and see if its any easier lol. I'll let you know how it goes...

[Windows7ge]

9 minutes ago, Bitter said:

That's good then, the 80MM Delta's on a water cooler from days of yore (early 2000's) are quite loud at full tilt, the other Delta's I've ran across have also been of an offensive tone to my ears, but I'm picky about fan noise. There's the signature Delta 'whine' that grates on my nerves that the Nidec lack, across all sizes this has been the case. I'm glad your Delta aren't torturous.

Well the server also sits in a room nobody occupies along with another server and a DAS. All together it's not quiet by any means but it's also not anywhere near wanting hearing protection either.

[Ben17]

8 minutes ago, Windows7ge said:

Well the server also sits in a room nobody occupies along with another server and a DAS. All together it's not quiet by any means but it's also not anywhere near wanting hearing protection either.

i envy your server tbh lol

[Windows7ge]

8 minutes ago, Ben17 said:

i envy your server tbh lol

I have found some justification for the hardware like how @JacobFW had asked with a DBZ Abridged quote "OK but why though?". I've donated most of the servers free time to BOINC. WCG specifically. The 512GB of RAM will play a role in massive virtualization. When I inevitably fill all 40 bays with SSDs I plan on switching the OS from Windows Server to Linux. I'll use ZFS and create a massive RAID60. It'll be awesome.

[Ben17]

1 minute ago, Windows7ge said:

 When I inevitably fill all 40 bays with SSDs I plan on switching the OS from Windows Server to Linux. I'll use ZFS and create a massive RAID60. It'll be awesome.

if you ever that post a picture of something lol

[Windows7ge]

5 minutes ago, Ben17 said:

if you ever that post a picture of something lol

Will do

Gotta say you have no idea how stupidly long the command is going to be to create a 40 drive RAID60.

[leadeater]

11 hours ago, Windows7ge said:

OH? Lets see you do a better job in this box.

Right now I have zero cables, as I don't have everything to build the complete system. Do you win by default? 

 

That's one of the things I don't like about server PSUs, one that don't use proprietary backlplanes and motherboards, is they aren't modular so it's impossible to do anything nice with all those cables. Well short of cutting them off but who would do that.

 

11 hours ago, Windows7ge said:

The only thing I could do to make it prettier would be to put some velcro straps on em. LTTStore.com

I see you saw through my post and noticed it was actually an LTTStore plug 

 

11 hours ago, Windows7ge said:

I swapped the NOCTUA I had there for an old SP Corsair 120

Wait what? The Corsair pushes more air than a NOC? 

 

11 hours ago, Windows7ge said:

I only have...16?...18?...lost count. You could 1 up me for cheaper with used 2.5" 1.2TB PCI_e SSDs off eBay if you like. 

The price of the P3520's is actually really tempting but I'd actually prefer SATA or SAS server SSD than NVMe, I don't have any systems nor will I get any soon than are optimized for that many NVMes, not unless I win Lotto or something.

[leadeater]

4 hours ago, Bitter said:

Delta fans are pretty great, Nidec makes some great ones too and I've found over the years that I prefer the tone of the Nidec to the Delta when it comes to high flow high RPM stuff. However the Nidec's I like are the 38mm thick ones, more of a woosh than a whine if you know what I mean. 1.5 amps each fan though so you HAVE to use a hub if you want to run them via board controls.

I have a 1.6A Nidec fan and a 3.2A Nidec fan, finger remover 6000. Really great fans and not actually that loud for the RPM and airflow they push but still not comfortable to be around.

[leadeater]

2 hours ago, Windows7ge said:

When I inevitably fill all 40 bays with SSDs

Post a photo of all 40 SSDs in a pile, will look so great.

[Bitter]

8 minutes ago, leadeater said:

I have a 1.6A Nidec fan and a 3.2A Nidec fan, finger remover 6000. Really great fans and not actually that loud for the RPM and airflow they push but still not comfortable to be around.

Mine is only a 35502 at the moment and it makes me uneasy near fingers, having been bitten by some metal bladed fans in the past I think plastic is worse because it's sharper generally. I figure I'll use this one as case air intake on a different curve for extra air to the GPU as needed /derail

[Bitter]

7 minutes ago, leadeater said:

Post a photo of all 40 SSDs in a pile, will look so great.

Like cash, just don't make it rain.

[Windows7ge]

2 hours ago, leadeater said:

Right now I have zero cables, as I don't have everything to build the complete system. Do you win by default? 

 

That's one of the things I don't like about server PSUs, one that don't use proprietary backlplanes and motherboards, is they aren't mobular so it's impossible to do anything nice with all those cables. Well short of cutting them off but who would do that.

I think you win by default if there are no cables to manage. I didn't say the computer had to turn on.

 

If I have to space to cram the excess cable I don't mind the non-modular form-factor.

 

2 hours ago, leadeater said:

I see you saw through my post and noticed it was actually an LTTStore plug 

 

You know it.

 

2 hours ago, leadeater said:

Wait what? The Corsair pushes more air than a NOC?

I don't remember all the numbers but it was like a 1200RPM 140mm Noctua vs a ~2000RPM 120mm SP Corsair. It also didn't help the 140mm was being choked by the original lid but yes. Between the two specific fans in question the Corsair was moving more. It made the X540 go down a few degrees.

 

2 hours ago, leadeater said:

The price of the P3520's is actually really tempting but I'd actually prefer SATA or SAS server SSD than NVMe, I don't have any systems nor will I get any soon than are optimized for that many NVMes, not unless I win Lotto or something.

Buy yourself 2 or 3 9405W-16i's and a pile of those second hand P3520's. It'd be cheaper than what I spent on the RAM.

 

Speaking of if anybody need a stupid amount of RAM for cheap NEMIX is selling the stuff at a steep discount right now: NEMIX RAM

 

2 hours ago, leadeater said:

Post a photo of all 40 SSDs in a pile, will look so great.

With or without drive sleds?

[leadeater]

3 hours ago, Windows7ge said:

With or without drive sleds?

Either, just make the pile look big 

[maxtch]

On 12/24/2019 at 2:34 PM, Windows7ge said:

@maxtch ...7 months later and endless BS dealing with NEMIX support...

 

I have acquired Quad-Channel x2. All matching sticks. >:)

 

May your nitpick be at ease.

512GB in 8 sticks of 64GB huh, with slots for another 512. Curious what you would do on it though with that much CPU and memory. (My instincts screams VM's.) Also what is NEMIX?

 

Well my (well now ex-) server started its life with 8 matching sticks of memory since I know from the get-go that I will compute on it, either as a VM host when it was a server, or as a hybrid CS/EE engineering workstation now. Since I was on DDR3, the memory situation was 128GB in 8 sticks of 16GB RDIMM with room for another 128.

[Windows7ge]

4 hours ago, maxtch said:

512GB in 8 sticks of 64GB huh, with slots for another 512. Curious what you would do on it though with that much CPU and memory. (My instincts screams VM's.) Also what is NEMIX?

 

Well my (well now ex-) server started its life with 8 matching sticks of memory since I know from the get-go that I will compute on it, either as a VM host when it was a server, or as a hybrid CS/EE engineering workstation now. Since I was on DDR3, the memory situation was 128GB in 8 sticks of 16GB RDIMM with room for another 128.

Your instincts are correct. I plan to do a lot of virtualization on here. I want to emulate an office network. NEMIX is the manufacturer of the RAM. I will probably avoid buying from them again. Their support was abysmal.

 

This motherboard can technically hold up to 2TB of DDR4 RAM but I don't plan on going that far.

[maxtch]

8 hours ago, Windows7ge said:

Your instincts are correct. I plan to do a lot of virtualization on here. I want to emulate an office network. NEMIX is the manufacturer of the RAM. I will probably avoid buying from them again. Their support was abysmal.

 

This motherboard can technically hold up to 2TB of DDR4 RAM but I don't plan on going that far.

Since you have that much compute power, you can replace other PC's in the room with Raspberry Pi's acting as thin clients.

 

Meanwhile if you have the funds, you may want to scoop up some of those 4TB Intel DC P4000 NVMe SSD's Facebook was dumping, and a maybe one of those nVidia Tesla cards that can be modded into a GRID. The former are just some cheap and fast SSD, and the latter will allow you assign virtual GPU to the VM's.