45Drives Alternative

[BenChurchill]

I work for a production company in NYC and I'm looking to build/buy a centralized media server to replace the our current system (50+ external drives siting on a bookshelf labeled with sharpie) and looking at the 30 drive model of 45Drives Storinator. This is the perfect fit with 10TB drives to fit our existing ~100TB of data and provide lots of growing room for the next few years. My problem is spending ~$7000 on the server alone is a little out of our price range and was looking to build a comparable or less powerful system for much cheaper, I plan on running UnRaid and would like to have it rack mounted. Just looking for suggestions on hardware, mostly an inexpensive chassis that can handle 24+ drives, MB with enough PCI lanes and raid controllers to connect all the drives. Desired budget is anything less than around $2,500 (excluding drives). I have experience building systems but nothing with more than 6 HDs. 

[Chaz042]

@BenChurchill If you're looking for new, your best bet is most likely 45 Drives, anything else that's going to support 24+ drives will be $10,000+ $4,000 for an enterprise system. Other solutions like Direct Attached Storage/Drive Shelves start round $2,500, and the entry level solutions also start with 12 HDD bays, so you'll need 2, and you still need a server to process everything.

 

 

Now, if you want to talk about used enterprise equipment, you could get away with 2 or so Dell R510 with 12 3.5" bays. Using these servers you can create a storage cluster with GlusterFS. I found this overview on the subject from 45 Drives actually, http://45drives.blogspot.com/2016/11/an-introduction-to-clustering-how-to.html.

 

You can get a Dell R510 on eBay starting around $250~$350. I personly have used www.savemyserver.com  in the past and I couldn't be happier, and all their servers have a 2-year warranty. A custom Dell R510 12 bay starts at $190, however, at the moment, there configuration page for it is bugged.

 

If you were to go the Save My Server route, I would say get 32GB (4x 8GB) of RAM, 64 (4x 16GB) if you really want, 2x X5670 (2.93GHz 6Cores/12Threads), no RAID card, IDRAC 6 Express, DVD if free, no PSUs, no drive trays, no rails, and bezel. The PSUs, drive trays, rails, and bezel can be found on eBay for much cheaper. 

 

Next, the Dell R510 has 1 x8 PCIe 2.0 slot, and 2 x4 PCIe 2.0 slots and you still need an SAS HBA card that works with the hot swap backplane, can't really help you with this part, but I'm sure someone here has a bit of experience with HBAs.

 

Lastly, you may want to think about upgrading your network to support 10Gbps uplinks to the servers, SFP+ NICs can be found for $15~$100 on eBay and 48 port gigabit switches with 10Gbps uplinks can be found starting at $90, but 10Gb this is a different topic altogether. 

 

 

Edit: Pricing info was off, thanks @GDRRiley

Edited July 28, 2017 by Chaz042
Wrong Price Info

[BenChurchill]

This is exactly what I was looking for appreciate the help! 

[Mikensan]

The Dell 2100 also a good one to look for.

 

Brand new you'll spend ~$1,000-$2,000 on hardware without disks... $500 for the case, $200 for the mobo $200 for the processor, $200 for memory, $100 for the P/S...

HDDs your mileage may vary.

 

As for HBAs you can find them on ebay for $50-$100, you'll need an expander card if you're going to fill all 24 slots or 3x HBA cards. The Dell H310 and LSI 9211s can be put into IT mode. Unless you're doing SSDs, an expander ($20) will work just fine.

[Chaz042]

7 minutes ago, Mikensan said:

The Dell 2100 also a good one to look for.

 

Brand new you'll spend ~$1,000-$2,000 on hardware without disks... $500 for the case, $200 for the mobo $200 for the processor, $200 for memory, $100 for the P/S...

HDDs your mileage may vary.

 

As for HBAs you can find them on ebay for $50-$100, you'll need an expander card if you're going to fill all 24 slots or 3x HBA cards. The Dell H310 and LSI 9211s can be put into IT mode. Unless you're doing SSDs, an expander ($20) will work just fine.

2

Are you talking about the Dell C2100 by chance? The C2100 is a good server but with a few issues. The "C" stands for Cloud because they were sold in batches of 100s for web/cloud hosting use. The cloud line is designed with no to little reliability in mind because they are meant to be in huge clusters where the reliability comes from the cluster and not the hardware itself. Another factor is long term driver and BIOS update support, C2100 support was weak, however, the Dell R510 has released almost 8 years ago and it's still getting updates for parts like the iDRAC which was last updated on 7/25/2017, it has way more support. 

 

Also, take my thoughts with a grain of salt, I maybe a little biased because I got a Dell C1100 / CS24-TY Cloud server a few years back and it had non-standard rails, no support for ECC RAM, no BIOS updates because the BIOS was custom to the hosting company, and firmware on the software based RAID controller was also custom.  It made ESXi 5.5 very hard to install, and anything newer than ESXi 6.0 won't work. Also to my knowledge, the server was less than 3 years old when I bought it and Dell offered 0% support for it.

[GDRRiley]

https://www.supermicro.com/products/system/4U/6049/SSG-6049P-E1CR36H.cfm this you can find starting at about 5.6k for dual 6 cores 96gb of ram and not storage, os. http://www.thinkmate.com/system/superstorage-server-6049p-e1cr36h

[GDRRiley]

PCPartPicker part list / Price breakdown by merchant

CPU: Intel - Core i3-6100 3.7GHz Dual-Core Processor  ($107.99 @ SuperBiiz) 
Motherboard: Supermicro - MBD-X11SSL-F-O Micro ATX LGA1151 Motherboard  ($167.98 @ Newegg) 
Memory: Crucial - 32GB (4 x 8GB) DDR4-2400 Memory  ($341.94 @ Adorama) 
Storage: Samsung - 850 EVO-Series 120GB 2.5" Solid State Drive  ($87.27 @ Amazon) 
Storage: Samsung - 850 EVO-Series 120GB 2.5" Solid State Drive  ($87.27 @ Amazon) 
Power Supply: Corsair - HX Platinum 1000W 80+ Platinum Certified Fully-Modular ATX Power Supply  ($199.99 @ Amazon) 
Other: Norco rpc-4224 ($430.00)
Other: intel multi port sff8087 ($110.00)
Other: sff-8087 cables 6x ($60.00)
Total: $1592.44
Prices include shipping, taxes, and discounts when available
Generated by PCPartPicker 2017-07-27 16:38 EDT-0400

as an alt to the early post (you may want more some more cpu power for this) it should do well

[Jarsky]

3 hours ago, GDRRiley said:

Other: intel multi port sff8087 ($110.00)
Other: sff-8087 cables 6x ($60.00)

 

The RES2SV240 is a port multiplier, you still require a controller - and at least 1 port needs to be sacraficed for controller connection, so you only get 5 ports - or 20 devices.

You'd be best to swap those out and get one of these: https://www.amazon.com/Controller-05-25699-00-9305-24i-24-Port-PCI-Express/dp/B01BDZWLV6

 

Or if you want to go the cheap route, get a card like the 9211-8i and that RES2SV240 - but you would need to direct attach 4 disks to 1 of the ports on the HBA and use the other port to uplink to the expander giving you 24 in total. 

[GDRRiley]

4 minutes ago, Jarsky said:

 

The RES2SV240 is a port multiplier, you still require a controller - and at least 1 port needs to be sacraficed for controller connection, so you only get 5 ports - or 20 devices.

You'd be best to swap those out and get one of these: https://www.amazon.com/Controller-05-25699-00-9305-24i-24-Port-PCI-Express/dp/B01BDZWLV6

 

Or if you want to go the cheap route, get a card like the 9211-8i and that RES2SV240 - but you would need to direct attach 4 disks to 1 of the ports on the HBA and use the other port to uplink to the expander giving you 24 in total. 

ok cot those confused, thanks

[Chaz042]

8 hours ago, GDRRiley said:

https://www.supermicro.com/products/system/4U/6049/SSG-6049P-E1CR36H.cfm this you can find starting at about 5.6k for dual 6 cores 96gb of ram and not storage, os. http://www.thinkmate.com/system/superstorage-server-6049p-e1cr36h

I was always under the idea that Supermicro was prices similarly to HP and Dell, this is actually really competitive.

[GDRRiley]

Just now, Chaz042 said:

I was always under the idea that Supermicro was prices similarly to HP and Dell, this is actually really competitive.

yeah 

[Jarsky]

10 minutes ago, Chaz042 said:

I was always under the idea that Supermicro was prices similarly to HP and Dell, this is actually really competitive.

 

They do once you get into support contracts and such - but Supermicro sell barebone stuff for pretty reasonable prices.

[Mikensan]

17 hours ago, Chaz042 said:

Are you talking about the Dell C2100 by chance? The C2100 is a good server but with a few issues. The "C" stands for Cloud because they were sold in batches of 100s for web/cloud hosting use. The cloud line is designed with no to little reliability in mind because they are meant to be in huge clusters where the reliability comes from the cluster and not the hardware itself. Another factor is long term driver and BIOS update support, C2100 support was weak, however, the Dell R510 has released almost 8 years ago and it's still getting updates for parts like the iDRAC which was last updated on 7/25/2017, it has way more support. 

 

Also, take my thoughts with a grain of salt, I maybe a little biased because I got a Dell C1100 / CS24-TY Cloud server a few years back and it had non-standard rails, no support for ECC RAM, no BIOS updates because the BIOS was custom to the hosting company, and firmware on the software based RAID controller was also custom.  It made ESXi 5.5 very hard to install, and anything newer than ESXi 6.0 won't work. Also to my knowledge, the server was less than 3 years old when I bought it and Dell offered 0% support for it.

Yup, C2100 and oddly enough I also have a C1100 and had somewhat of a similar experience. Mine supports ECC RAM, has 32GB in it right now (More than likely it's very picky about what ECC RAM you can put in it) however my issue was they did not have anything later than ESXI 4.0. Also same issue with the BIOS, even spoke with Dell about these things being CTO.

[leadeater]

Building a smaller server with only OS/cache disks and dual SAS HBA is also an option. Buy some used disk shelves and fill with new disks.

 

You'll also be able to add more shelves easily by chaining them using the SAS link ports on them.

[Chaz042]

On 7/28/2017 at 9:07 AM, leadeater said:

Building a smaller server with only OS/cache disks and dual SAS HBA is also an option. Buy some used disk shelves and fill with new disks.

 

You'll also be able to add more shelves easily by chaining them using the SAS link ports on them.

The NetApp DS4243 is a good choice for this, you can get one for $200+- with 4x PSUs and 2x IOM3 controllers. There are a few trade offs that @BenChurchill would need to accept before using this method however.

1. Higher power usage due to older drive shelves not handling larger drives, like the DS4243 can only take up to 3TB drives. Also more power = more heat you'll need to get rid of.
2. More rack space, again, due to the drive size constraints, you'll need more drives, and more drives = more physical space. In NYC, space costs $$$$$, so unless @BenChurchill has a 24U~45U server rack already setup or space reserved/available for the rack, 5U+ of space may be unreasonable. 
3. The cost per TB is typically lower the higher the capacity the drive is, meaning, you may incur higher cost when buying the storage.

 @leadeater @Jarsky I would like to hear your opinions on the NetApp DS4243, do you know of anything else that's used that doesn't have a 3TB limit while being reasonably priced or something new that's sub $1000?

[leadeater]

5 hours ago, Chaz042 said:

 @leadeater @Jarsky I would like to hear your opinions on the NetApp DS4243, do you know of anything else that's used that doesn't have a 3TB limit while being reasonably priced or something new that's sub $1000?

IBM EXP3512, anywhere from $25-$180 USD on ebay.