Raid Card

[LordMastodon]

Just now, leadeater said:

Looks good, if you want to save a little bit more the E5-2620v4 will be more than enough. I'll have a quick look at some supermicro motherboards to see if there is a better or equal option, I just prefer them since they are primarily in the server market.

1

I didn't think we needed a 10 core either, but that's literally the cheapest one on the Dell customizer so I figured if I wanted to demonstrate to the higher-ups that you could actually get equal or higher performance for half the price by going with a custom-built. I mean seriously, if we go prebuilt we're paying Dell $2,500 for a warranty, and OS (which we're going to format off of the drives if and when it arrives because it's not the one we want and there's no option for not having an OS) and for them to actually assemble it (and a case and motherboard, but those are included in our list too, so it's a shared cost). 

 

I don't actually work for that school, but I work for a tech repair shop that is essentially contracted to help them with things (including this file server), and our hourly rate for that school is around $100 (won't go into too much further detail). I'd say we could spend a good 2-3 hours assembling it, and maybe another 1-2 hours configuring it. That's $300 for building (and $200 for configuration, but it's not like Dell's doing any of that anyway), whereas it would seem like Dell wants to charge us around $1,000 for it. 

[scottyseng]

15 minutes ago, LordMastodon said:

Haha. I've been up till 2 AM on this forum sometimes. It's a really great forum. Also, it's not ugly *cough* *cough* forums.overclockers.co.uk *cough* *cough*.

Haha, I live the vampire life and it's 7:30 AM here in the USA (central time)...time to sleep.

 

Yeah, I have a LSI 9260-8i in my server and it's been chugging along for the past 3 years. I'm still bitter that I bought it and the next month the 93xx series came out. Sigh.

 

I also second that SuperMicro board. I have the non IPMI version and it works well.

 

I have a 14 core Xeon E5-2695V3 and let me tell you, it does absolutely nothing most of a the time as a storage server (I make it render stuff for me so it's actually doing something...with just file storage loads it just laughs at everything and runs at 1-5% load).

 

The main reason why would you go for a OEM like Dell is so that you get a warranty (you'd rather have them pay for stuff than yourself). If you go custom built, it will be cheaper, but you're on your own for support.

[LordMastodon]

7 minutes ago, leadeater said:

Have a look at this motherboard and see what you think, the main benefits are IPMI, dual nics and a server chipset C612 not X99.

 

http://www.newegg.com/Product/Product.aspx?Item=N82E16813182959&cm_re=X10SRA-_-13-182-959-_-Product

 

3

I've never heard of IPMI, but from what I understand on Google it's some sort of monitoring and management system (not entirely sure what that entails) for your CPU, BIOS etc. Could you please explain?

 

Also, we intended to have a Dual 10 GB/s NIC (we have the ports, don't worry), which I'm kinda struggling to find, do you have a clue?

Edited August 3, 2016 by LordMastodon

[LordMastodon]

5 minutes ago, scottyseng said:

Haha, I live the vampire life and it's 7:30 AM here in the USA (central time)...time to sleep.

 

Yeah, I have a LSI 9260-8i in my server and it's been chugging along for the past 3 years. I'm still bitter that I bought it and the next month the 93xx series came out. Sigh.

 

I also second that SuperMicro board. I have the non IPMI version and it works well.

 

I have a 14 core Xeon E5-2695V3 and let me tell you, it does absolutely nothing most of a the time as a storage server (I make it render stuff for me so it's actually doing something...with just file storage loads it just laughs at everything and runs at 1-5% load).

 

The main reason why would you go for a OEM like Dell is so that you get a warranty (you'd rather have them pay for stuff than yourself). If you go custom built, it will be cheaper, but you're on your own for support.

 

Yeah, support is important to them, but I wonder if they'd rather have $2,500-2,000 left over than support.

[leadeater]

6 minutes ago, LordMastodon said:

I've never heard of IPMI, but from what I understand on Google it's some sort of monitoring and management system (not entirely sure what that entails) for your CPU, BIOS etc. Could you please explain? 

As long as the motherboard has power you can connect to a special web interface and check the health of the server. You can also power it on/off, reboot etc if the OS becomes unresponsive. It's a nice way to service a server as if you were physically at it but remotely, you can also mount ISO's as virtual DVD drives to install OS etc.

 

Linus as a video on it a little while back.

 

At work we make heavy use of IPMI or iLO in HP speak. We have 3 different data centers and we manage them at a 4th remote site, we'd be lost without it or traveling a lot.

 

Edit: Forgot to mention it also gives you a remote console connection to it so you can see what you are doing and interact with the server, KVM over network.

[LordMastodon]

Just now, leadeater said:

As long as the motherboard has power you can connect to a special web interface and check the health of the server. You can also power it on/off, reboot etc if the OS becomes unresponsive. It's a nice way to service a server as if you were physically at it but remotely, you can also mount ISO's as virtual DVD drives to install OS etc.

 

Linus as a video on it a little while back.

 

At work we make heavy use of IPMI or iLO in HP speak. We have 3 different data centers and we manage them at a 4th remote site, we'd be lost without it or traveling a lot.

 

Yeah, that sounds like something extremely useful for us considering we're about an hour's drive away from that school, and a lot of what we've been doing lately has been trying to do is automate things over the Internet, like setting up remote deployment, etc. I'm sure my boss would be ecstatic if I told him we could have IPMI if we could go custom-built. Damn, this is sounding better by the minute.

[scottyseng]

1 minute ago, LordMastodon said:

Yeah, support is important to them, but I wonder if they'd rather have $2,500-2,000 left over than support.

Yeah, that's a call I can't make...Since servers run 24/7, if something does break, they usually have to pay express shipping to get the replacement part out to you as quick as possible.

 

Otherwise with custom built, you'd have to stock some drives as emergency spares incase one dies. You wouldn't want to wait for one to die then order a new drive (and wait for the 3-5 day shipping) while the array is degraded...not a good idea.

 

That is a lot of money potentially saved though.

[leadeater]

8 minutes ago, LordMastodon said:

Also, we intended to have a Dual 10 GB/s NIC (we have the ports, don't worry), which I'm kinda struggling to find, do you have a clue?

I use Intel X540 10Gb NICs but they are a little $$$, QLogic and Mellanox have cheaper options. First what type of 10Gb connection are you going to use? Direct attached Copper (DAC)/Twinax, 10GBase-T or Fibre?

[LordMastodon]

Just now, scottyseng said:

Yeah, that's a call I can't make...Since servers run 24/7, if something does break, they usually have to pay express shipping to get the replacement part out to you as quick as possible.

 

Otherwise with custom built, you'd have to stock some drives as emergency spares incase one dies. You wouldn't want to wait for one to die then order a new drive (and wait for the 3-5 day shipping) while the array is degraded...not a good idea.

 

That is a lot of money potentially saved though.

3

Yeah, I think that amount of money saved we can afford to have one extra of those power supplies and one extra drive.

[leadeater]

3 minutes ago, LordMastodon said:

Yeah, that sounds like something extremely useful for us considering we're about an hour's drive away from that school, and a lot of what we've been doing lately has been trying to do is automate things over the Internet, like setting up remote deployment, etc. I'm sure my boss would be ecstatic if I told him we could have IPMI if we could go custom-built. Damn, this is sounding better by the minute.

Just so we are fair about it Dell also has this ability and they call it iDRAC, usually the remote console KVM is an extra cost license too.

[LordMastodon]

8 minutes ago, leadeater said:

Just so we are fair about it Dell also has this ability and they call it iDRAC, usually the remote console KVM is an extra cost license too.

 

 

Yeah, it's nice that iDRAC is free, but from what I can see SuperMicro's IPMI is free as well, not to mention they have a freaking iPhone app. Wow. I mean that's genuinely one of the most useful things ever. Also, with this motherboard, I don't need that 950, because it has onboard video! Huzzah!

[scottyseng]

4 minutes ago, LordMastodon said:

Yeah, I think that amount of money saved we can afford to have one extra of those power supplies and one extra drive.

Ah, curious, do you have a UPS battery back up for this file server as well? Otherwise I'd consider getting one. Just relying on the Cachevault for power loss protection isn't a good idea...some power conditions can cause the Cachevault to discharge accidently. I had a brownout in our house when a bad storm hit, and the server power supplies would keep trying to turn on using the low power available. The RAID card though the power was stable, but it turned completely off in about 2 seconds and I lost my RAID card's RAM cache that night. I didn't have any corruption luckily because the RAID card had just finished the consistency check and patrol read the night before nothing was used on the server when the power went out (I was sleeping and the server was idling). I now have a UPS unit because of that storm.

[LordMastodon]

1 minute ago, scottyseng said:

Ah, curious, do you have a UPS battery back up for this file server as well? Otherwise I'd consider getting one. Just relying on the Cachevault for power loss protection isn't a good idea...some power conditions can cause the Cachevault to discharge accidently. I had a brownout in our house when a bad storm hit, and the server power supplies would keep trying to turn on using the low power available. The RAID card though the power was stable, but it turned completely off in about 2 seconds and I lost my RAID card's RAM cache that night. I didn't have any corruption luckily because the RAID card had just finished the consistency check and patrol read the night before nothing was used on the server when the power went out (I was sleeping and the server was idling). I now have a UPS unit because of that storm.

 

 

Yeah, we have a UPS.

[scottyseng]

Just now, LordMastodon said:

Yeah, we have a UPS.

Yeah, sadly I fell into that meme because I was kind of holding off on getting a UPS until I could find one on sale...Though mine is for home use.

 

Yeah, I think you should be set for your server build then.

[LordMastodon]

13 minutes ago, leadeater said:

Just so we are fair about it Dell also has this ability and they call it iDRAC, usually the remote console KVM is an extra cost license too.

 

I found this 10 GiB NIC, which seems pretty nice, except for the PCIe 2.0. Will that really matter, or should I find something else?

[LordMastodon]

Just now, scottyseng said:

Yeah, sadly I fell into that meme because I was kind of holding off on getting a UPS until I could find one on sale...Though mine is for home use.

 

Yeah, I think you should be set for your server build then.

 

You should have gotten the UPS that Linus has! You know, the one he featured on Holy Sh!t! That one is obviously necessary for everyone, right?

[scottyseng]

Just now, LordMastodon said:

You should have gotten the UPS that Linus has! You know, the one he featured on Holy Sh!t! That one is obviously necessary for everyone, right?

Haha, let me just wire that into my house...

 

I got something still fairly expensive, the 2000VA 1800W Eaton 9130 UPS rackmount unit. It can handle both my server and PC (400 watts) and has a 40 minute runtime. Sadly it cost me 1.2k...

[leadeater]

4 minutes ago, LordMastodon said:

Yeah, we have a UPS.

I can't remember what our UPSs are in the main data center but they are 160kva each.

 

I've also used two of these installed at 2x 60kva in A + B power feed for a server room with 2 36U cabinets of batteries between them for both, so 4 cabinets in total.

http://powerquality.eaton.com/Products-services/Backup-Power-UPS/9390.aspx?cx=3

[leadeater]

8 minutes ago, LordMastodon said:

You should have gotten the UPS that Linus has! You know, the one he featured on Holy Sh!t! That one is obviously necessary for everyone, right?

Pff cheap compared to what I just posted 

 

Edit:

@LordMastodon @scottyseng

Haha we managed to make this thread get automatically marked as a hot topic, guess we have made it to page 3 in an hour.

Edited August 3, 2016 by leadeater

[leadeater]

6 minutes ago, LordMastodon said:

I found this 10 GiB NIC, which seems pretty nice, except for the PCIe 2.0. Will that really matter, or should I find something else?

Yep very good, I do prefer Intel NICs where ever possible. They have the best OS support and really good drivers.

[LordMastodon]

Just now, leadeater said:

I can't remember what our UPSs are in the main data center but they are 160kva each.

 

I've also used two of these installed at 2x 60kva in A + B power feed for a server room with 2 36U cabinets of batteries between them for both, so 4 cabinets in total.

http://powerquality.eaton.com/Products-services/Backup-Power-UPS/9390.aspx?cx=3

 

Just now, scottyseng said:

Haha, let me just wire that into my house...

 

I got something still fairly expensive, the 2000VA 1800W Eaton 9130 UPS rackmount unit. It can handle both my server and PC (400 watts) and has a 40 minute runtime. Sadly it cost me 1.2k...

 

 

The UPS is real folks. 

 

For context, at school, we had an AC fan lying around and I wanted to tear apart a laptop charger cable and use it to power the fan. We were working on recreating a record player and the normal DC fans wouldn't spin the right way, so I almost did that. My teachers said that if I did do that, I'd most likely A) start a fire, B) kill myself, and C) get thrown out of the school.

 

I also once took apart a Macbook, and while it was still apart, the IT guy plugged it into power and turned it on to see if it still worked. The thing was fine until the display cable touched the inverter, short-circuited the whole computer and sent a few sparks flying. Needless to say, both the PCB and cable were badly burnt, but looked awesome so no one really cared.

 

Those are the two most dangerous things I've (almost, in the first one) done with electricity. I pale in comparison.

 

[LordMastodon]

7 minutes ago, leadeater said:

Pff cheap compared to what I just posted 

 

Edit:

@LordMastodon @scottyseng

Haha we managed to make this thread get automatically marked as a hot topic, guess we have made it to page 3 in an hour.

 

I'm not sure whether to be proud of that, or...

[LordMastodon]

8 minutes ago, leadeater said:

Yep very good, I do prefer Intel NICs where ever possible. They have the best OS support and really good drivers.

 

The PCIe 2.0 shouldn't be a problem, right?

 

EDIT:

I found this one, which is PCIe 3.0, so I'll go with it.

[leadeater]

5 minutes ago, LordMastodon said:

The PCIe 2.0 shouldn't be a problem, right?

Yes will be fine. A PCIe 2.0 x8 slot has 4GB/s bandwidth which is spot on for both 10Gb ports fully used full duplex.

Edited August 3, 2016 by leadeater

[scottyseng]

1 minute ago, LordMastodon said:

Those are the two most dangerous things I've (almost, in the first one) done with electricity. I pale in comparison.

Worst for me was being shocked by 480v...I was lucky that the circuit tripped. The worst accident we've seen related to electricity is where one of our friends who does electricity got shocked by the same 480v above and fell down 18 feet onto concrete in a department store.

 

Yeah, the PCIe 2.0 will be fine. We're still not even bottlenecking PCIe 2.0 with today's tech.