So I want to build a RAID server... help?

[ThioJoe]

I've built several PCs but never a RAID server. In fact (quite embarassingly I've never used RAID before).  Basically I am not sure where to start, since the requirements for a server are so different from a PC.  This server would not be used as a final backup solution, but rather for video footage storage while working, and short term storage. Final backup would be onto LTO drives.

 

I know I want enterprise grade drives with UREs at least 10^15, but beyond that not much.  So what I don't know about is:

 

• CPU: Probably a Xeon
• Motherboard: Never shopped for a server mobo before, not sure.
• Memory: I'm assuming something with ECC, but not sure how much
• RAID Controller: No clue about good brands / types here either
• OS: Not sure, maybe FreeNAS? My main consideration is driver compatibility for any interface cards I'd be installing now and in the future, such as Thunderbolt III or whatever.
• Array: Probably going to use RAID 6, but was thinking about ZFS because I hear good things about it.

 

Anyone want to point me in the right direction?

[TheGeeker]

FReeNas is great. Hit me up if you need help.

[manikyath]

dont bother with raid for home use, its not worth it.

[.:MARK:.]

The Xeon you choose will depend on what software you plan on running on the sever. Same with RAM too.

Supermicro make really good boards for server use, I have one myself and they are really good.

The RAID controller is a key component, I personally prefer and use an LSI card.

You could run Windows Server 2012 R2, and run hardware RAID using the RAID controller and not have to have extortionate amounts of RAM.

 

I'm currently building a 96TB server so if there is anything you need, let me know.

[EmeraldFlame]

I've built several PCs but never a RAID server. In fact (quite embarassingly I've never used RAID before).  Basically I am not sure where to start, since the requirements for a server are so different from a PC.  This server would not be used as a final backup solution, but rather for video footage storage while working, and short term storage. Final backup would be onto LTO drives.

 

I know I want enterprise grade drives with UREs at least 10^15, but beyond that not much.  So what I don't know about is:

 

• CPU: Probably a Xeon
• Motherboard: Never shopped for a server mobo before, not sure.
• Memory: I'm assuming something with ECC, but not sure how much
• RAID Controller: No clue about good brands / types here either
• OS: Not sure, maybe FreeNAS? My main consideration is driver compatibility for any interface cards I'd be installing now and in the future, such as Thunderbolt III or whatever.
• Array: Probably going to use RAID 6, but was thinking about ZFS because I hear good things about it.

 

Anyone want to point me in the right direction?

 

What is your budget and intended use case? Is this something that only you will be using or are you going to have multiple people accessing the footage on this server (even if those people are accessing at seperate times)? How many people do you expect to be using it simultaneously? If you're planning on having multiple people use this in like a small business/startup situation I would recommend going with Windows Server simply for the active directory/domain support and security options that come along with it, along with the wide support. If it's just you, I'd go with FreeNAS.

As far as raid controllers, LSI is kind of the big dog in the market, and they have great cards, but good raid controllers aren't cheap. You can normally find used cards on ebay for cheaper, but they still typically aren't cheap. However if you are planning to use FreeNAS, it might not be necessary.

As far as your RAID setup goes RAID6 or ZFS would be fine choices. If you end up with Windows, I'd go 6, FreeNAS, I'd go ZFS.

 

dont bother with raid for home use, its not worth it.

 

You've got to be kidding right?

[Altecice]

If you literally only want a storage server you could put an I3/Pentium inside of it and it will still be more than up to the task of storing data. The main issue will be your drives/sustained read/writes of those drives. RAID controller.

[manikyath]

You've got to be kidding right?

i'm not, theres much better options for at home.

 

wendell has a video that explains some of the issues you could face with raid, and why theres better options for home use.

[You_are_a_cunt]

Why on earth would you want to use LTO as final backup?

Magnetic storage is pretentious as hell and with LTO you're ASKING for trouble.

 

LTO is expensive and requires proper storage of the cartridges to avoid data loss and corruption

[ThioJoe]

The array would just be for me, and would need good performance, able to handle at least 500MB/s read for 4K raw footage (even though I probably won't be recording much raw).  

 

I have another fundamental question that might sound stupid, but has to do with raid controllers vs the server itself and want to make sure I understand it to make sure I need an actual server.  From how I understand it, if I built a whole RAID server, it would be a standalone computer obviously, running an OS with its own components. If I got a RAID controller though, would that replace the need for a whole server?  By that I mean could I use a RAID controller card and just hook it up to a dummy enclosure filled with drives.  I just want to get the terminology right.

 

 

Why on earth would you want to use LTO as final backup?

Magnetic storage is pretentious as hell and with LTO you're ASKING for trouble.

 

LTO is expensive and requires proper storage of the cartridges to avoid data loss and corruption

 

 

The footage isn't something that I absolutely have to back up, I just want to, and LTO is the cheapest way to store huge amounts of data.  The drives are expensive but you can't beat the tapes for price.  I guess I incorrectly used the term "backup".  Basically I would use the LTO drives as an alternative to deleting the footage.

[You_are_a_cunt]

The footage isn't something that I absolutely have to back up, I just want to, and LTO is the cheapest way to store huge amounts of data.  The drives are expensive but you can't beat the tapes for price.  I guess I incorrectly used the term "backup".  Basically I would use the LTO drives as an alternative to deleting the footage.

 

That sort of makes sense. Just keep'em safe.

 

On that note: how much do LTO drives/cartridges go for these days?

[EmeraldFlame]

The array would just be for me, and would need good performance, able to handle at least 500MB/s read for 4K raw footage (even though I probably won't be recording much raw).  

 

I have another fundamental question that might sound stupid, but has to do with raid controllers vs the server itself and want to make sure I understand it to make sure I need an actual server.  From how I understand it, if I built a whole RAID server, it would be a standalone computer obviously, running an OS with its own components. If I got a RAID controller though, would that replace the need for a whole server?  By that I mean could I use a RAID controller card and just hook it up to a dummy enclosure filled with drives.  I just want to get the terminology right.

 

The footage isn't something that I absolutely have to back up, I just want to, and LTO is the cheapest way to store huge amounts of data.  The drives are expensive but you can't beat the tapes for price.  I guess I incorrectly used the term "backup".  Basically I would use the LTO drives as an alternative to deleting the footage.

 

Well technically 'RAID Server' isn't really a common terminology. You can have a Server with a RAID, but as far as terminology goes they are two independent things. As far as the physical attributes go, the work together, but don't necessarily need each other. You can have a server without it having a raid, and you can have a raid on a normal computer not just a server. 

That said, regardless, a RAID controller is always going to need some sort of host. You can't throw a RAID controller into a case with some HDD's and nothing else. It needs some sort of host computer it runs from. That being said, if it is just you using this raid, if you wanted to save money you could add the RAID controller to your existing workstation and run the RAID completely locally. This would save you a good chunk of change because you wouldn't need to build an entire separate computer. It does have some drawbacks though, RAID controllers are notoriously picky for the hardware it works with, you may have a hard time finding one that works well with your existing setup. You would also miss out any of the features that dedicated server OS's have. At the same time, your workflow currently sounds like you may not need all that much in the way of features. It would also be harder, but still doable, to share this data over the network down the line if you bring someone else into the network.

This is just going to be a storage server correct? You don't plan to use it for transcoding or anything? Do you happen to have a budget in mind or is this a 'the cheaper the better' scenario? As that could change my recommendation dramatically in your specific use case.

Also on the LTO storage thing, unless you have a room with extremely strictly controlled temp and humidity I wouldn't recommend it. LTO is notoriously temperamental. It is used widely in enterprise for mass backup, because it is cheap, but precautions on it's storage really need to be taken for it to have an effective lifespan. Long term may not be something you care about either in which case it doesn't matter.

[ThioJoe]

Well technically 'RAID Server' isn't really a common terminology. You can have a Server with a RAID, but as far as terminology goes they are two independent things. As far as the physical attributes go, the work together, but don't necessarily need each other. You can have a server without it having a raid, and you can have a raid on a normal computer not just a server. 

That said, regardless, a RAID controller is always going to need some sort of host. You can't throw a RAID controller into a case with some HDD's and nothing else. It needs some sort of host computer it runs from. That being said, if it is just you using this raid, if you wanted to save money you could add the RAID controller to your existing workstation and run the RAID completely locally. This would save you a good chunk of change because you wouldn't need to build an entire separate computer. It does have some drawbacks though, RAID controllers are notoriously picky for the hardware it works with, you may have a hard time finding one that works well with your existing setup. You would also miss out any of the features that dedicated server OS's have. At the same time, your workflow currently sounds like you may not need all that much in the way of features. It would also be harder, but still doable, to share this data over the network down the line if you bring someone else into the network.

This is just going to be a storage server correct? You don't plan to use it for transcoding or anything? Do you happen to have a budget in mind or is this a 'the cheaper the better' scenario? As that could change my recommendation dramatically in your specific use case.

Also on the LTO storage thing, unless you have a room with extremely strictly controlled temp and humidity I wouldn't recommend it. LTO is notoriously temperamental. It is used widely in enterprise for mass backup, because it is cheap, but precautions on it's storage really need to be taken for it to have an effective lifespan. Long term may not be something you care about either in which case it doesn't matter.

 

Yea it would basically be a storage server but I'd also be live editing from it.  Budget could go to an absolute max of about $3000 I guess if everything was perfect, since that seems to be what a 24TB G-Tech drive array would be.  The only reason I'd build my own is to be able to customize the interfaces, since the big G-Tech drives only use thunderbolt, which I don't have and would have to buy an expansion card for anyway.  I'd like to be able to add Thunderbolt III or any future USB standards pretty easily.  I don't need it to be accessible by the network, and if I ever got to the point of needing that I'd probably just upgrade again anyway.  

 

I'm leaning towards ZFS as opposed to RAID6 if I ended up building my own.  From what I understand, if I went with ZFS, I wouldn't need a controller card since you want direct access to each drive.  So I believe it would basically be a separate machine with all the drives inside attached directly to its mobo.

[scottyseng]

-snip-

 

You can get by with a i3 (haswell) or a Pentium for the CPU. You don't need a quad core Xeon.

 

Motherboard, get the Intel Enterprise chipset (Cxxx series) for proper ECC support. I can vouch for ASUS or SuperMicro.

 

RAM: I would recommend 16GB ECC for FreeNAS, 8GB min if Windows Server (16 GB would be better).

 

If you still are going with a RAID controller, LSI is the market king. I think for you, a LSI MegaRAID 9361-8i is what you need. Keep in mind you will also need to buy the appropriate battery back up unit for the RAID card itself.

 

Your two choices for OS are Windows Server or FreeNAS. If you go freenas, you will not need the RAID card. However, depending on the chassis, if it has a SAS expander on the backplane of the chassis (I'm assuming you're buying a hot swap server of some kind), you will need a SAS HBA to connect to it (The SATA ports on the motherboards will not connect to SAS).

 

Array: RAID6 is good, but if you go FreeNAS, but what it has (I think Z2 would be good...I don't know much about FreeNAS other than what hardware it requires. I run Windows Server myself with a LSI MegaRAID card).

 

Network: I see no one has mentioned the network. If you just use the 1Gb/s LAN, you will be bottlenecked to 80-100Mb/s regardless of how fast your server drive is. You will need 10Gb/s Ethernet cards connected to your PC (Which also need 10Gb/s itself) to get up to 1GB/s to 1.25GB/s connection speed. Right now my server has that as the bottleneck.

 

As for speed, I have 24TB (six 4TB WD Reds) in RAID 10, and they achieve 550MB/s sequential Read and Write. I think the WD RE SAS (WD's enterprise lineup...you don't really need SAS though...it's more if you use a RAID controller) is 140MB/s. I have two RE SAS 4TB in RAID0 and it gets around 270MB/s sequential read / write.

 

Noise: The server chassis is usually quite loud, so be prepared for that...or swap out the fans like me.

 

Battery backup: Budget some kind of UPS unit for the server in case the power goes out for either Windows Server or FreeNAS (APC is my preferred brand).

[Jetfighter808]

Are you the real ThioJoe? Cos if you are you could just download some ECC memory from www.downloadmoreram.com. That video made me LMFAO.

[EmeraldFlame]

-snip-

Came back to check on the thread and I gotta say I pretty much agree with @scottyseng. What I will add is that from your description of what you need, I think you would likely be served best by FreeNAS. I'm not overly familiar with it, I've set some up for people but I use Windows Server 2012R2 personally for the domain features. But from what it sound with Windows Server you would just be paying for functionality that you don't want/need. FreeNAS should suit you fine. 

[Blake]

Why on earth would you want to use LTO as final backup?

Magnetic storage is pretentious as hell and with LTO you're ASKING for trouble.

 

LTO is expensive and requires proper storage of the cartridges to avoid data loss and corruption

LTO is cheap, but it should only really be used for an off-site backup where internet costs are very high. should be backing up to disk locally and replicating to off-site.

 

Just need to keep the tapes in an air-conditioned environment and you are sorted.

[Blake]

That sort of makes sense. Just keep'em safe.

 

On that note: how much do LTO drives/cartridges go for these days?

~$32AUD for an LTO5, very cheap...