Large Scale Home Storage Server

[JRowe95]

Hi guys/girls, first time posting on the forum.

 

I currently have a Netgear 4 Bay Ready NAS filled with 2TB WD Green drives, but its not up to the task to hold all my files anymore so i'm in need of something a little (alot) more beefier.

 

Can anyone suggest a solution that is expandable and redundant. I have looked at some rack mountable ATX cases from Norco and was thinking along the lines of building my own server to run something like FreeNAS or Windows Home Server. I have never really done anything like this before, can anyone recommend components for a build like that, eg raid cards, cases etc... 

 

Hard drives aside I was thinking something in the region of $2000-$2500 budget.

 

Any input would be much appreciated, Thanks. 

[Hazy125]

I think @looney is the guy to talk to if he's still online

FreeNAS is a good way to go about it

[whatthe_fuzz]

looney will hook you up with some great system ideas. 

 

there is a thead located HERE that is home to 10TB+ Club. those guys are pretty smart if you want to PM on of them

 

also Whaler_99 is another great storage expert. 

[WoodenMarker]

How much/many storage/capacity/drives are you shooting for?

[JRowe95]

How much/many storage/capacity/drives are you shooting for?

At first, something like 10-15TB

[JRowe95]

Iv had a look around on the 10TB+ thread and have gotten a few ideas and I was thinking something along these lines:

 

Norco Tek RPC-4224 | Norco Tek RL-26 Rails | Silverstone Strider 1500watt (Not sure if this is over kill) | Western Digital 3TB Red x 24 | LSI MegaRAID SAS 9280-24i4e | Intel Pro/1000 PT Quad NIC

 

Not sure about motherboards, CPU or RAM, im gona say some kind of low power xeon and like 8GB DDR3

[looney]

Iv had a look around on the 10TB+ thread and have gotten a few ideas and I was thinking something along these lines:

 

Norco Tek RPC-4224 | Norco Tek RL-26 Rails | Silverstone Strider 1500watt (Not sure if this is over kill) | Western Digital 3TB Red x 24 | LSI MegaRAID SAS 9280-24i4e | Intel Pro/1000 PT Quad NIC

 

Not sure about motherboards, CPU or RAM, im gona say some kind of low power xeon and like 8GB DDR3

Hi,

 

The 1500 watt PSU is pretty overkill :p, in most systems you will need the high wattage for graphics cards but you don't have those here.

I went for a 850Watt PSU, and that is still a bit overkill but it will give some nice headroom and that watt range has a very compelling price point.

 

A very important thing to know is what you will store on your server, what kind of files, are they just movie and series or will it also store more I/O demanding stuff.

 

I would also like to know if you prefer a certain OS. (windows or linux)

[JRowe95]

Hi,

 

The 1500 watt PSU is pretty overkill :P, in most systems you will need the high wattage for graphics cards but you don't have those here.

I went for a 850Watt PSU, and that is still a bit overkill but it will give some nice headroom and that watt range has a very compelling price point.

 

A very important thing to know is what you will store on your server, what kind of files, are they just movie and series or will it also store more I/O demanding stuff.

 

I would also like to know if you prefer a certain OS. (windows or linux)

 

Mainly media files and small files like program source code. I was thinking FreeNAS or something of the kind.

[looney]

Mainly media files and small files like program source code. I was thinking FreeNAS or something of the kind.

Oke,

 

If your going for FreeNAS (or any other software RAID) then don's use that RAID card.

That card is for hardware RAID, all you need is a HBA (Host Bus Adapter) that will simply connect the drives to the system without RAID.

RAID will be handled by the OS/ Software.

 

I would go for something like a IBM m1015 (flashed to IT mode, cheap on ebay) and a HP SAS expander, just like @RandomNOOB system (http://linustechtips.com/main/topic/21948-ltt-10tb-storage-show-off-topic/?p=273858

)

[JRowe95]

Oke,

 

If your going for FreeNAS (or any other software RAID) then don's use that RAID card.

That card is for hardware RAID, all you need is a HBA (Host Bus Adapter) that will simply connect the drives to the system without RAID.

RAID will be handled by the OS/ Software.

 

I would go for something like a IBM m1015 (flashed to IT mode, cheap on ebay) and a HP SAS expander, just like @RandomNOOB system (http://linustechtips.com/main/topic/21948-ltt-10tb-storage-show-off-topic/?p=273858

)

How about a hardware raid solution, what kind of operating system would you recommend?

[looney]

How about a hardware raid solution, what kind of operating system would you recommend?

For simple storage I would not go for hardware because its so much more expensive.

 

OS, really comes down to what you favor, both windows and linux work well with both software and hardware RAID.

[MG2R]

How about a hardware raid solution, what kind of operating system would you recommend?

As Looney said, hardware RAID isn't ideal. Not only because of the expensive controllers, also because it can be tough to recover the RAID array in case of a component (RAID card) failure or a system upgrade. With software RAID, you have a way greater flexibility and system independence.

 

To back this statement up: my software RAID array (using mdadm on Debian) has survived two complete OS reinstalls and three hardware upgrade cycles.

[JRowe95]

Sounds like software raid is the go in this case. Thanks for the great help

[Whaler_99]

SuperMicro also make some great HBA's - I use one and some RocketRAID controllers, which were a bit more funcky.

 

If you use Windows Home Server or Storage Spaces in Windows 8 / Server 201 / Essentials - then DO NOT do hardware RAID. Their software based raid is based on physical disk ID's and a hardware RAID can completely mess this up. Using something like FLEXraid in Windows is also based on you just having a JBOD.

 

Something like FreeNAS and unRaid also do their software based raid very well.

 

If you look at most of the builds that the linux systems are using, you don't need anything to powerful.

 

PSU - you don't need MUCHO power, but for ease, make sire whatever you pick, has a single 12V rail. Makes things a lot simpler.

[snowComet]

I'm interested in building a home server as well, but I know nth about linux. Do u guys think ubuntu server is a good choice for home server??

[MG2R]

I'm interested in building a home server as well, but I know nth about linux. Do u guys think ubuntu server is a good choice for home server??

Yes, very.

 

If you're not sure how to use/configure Linux, then I'd suggest you take a look at the Debian Home Server Howto. Ubuntu is derived from Debian, so everything in my tutorial should for you as well. If you go for Ubuntu, do take the Server edition. The Desktop edition is waaaaay too cluttered, that's why I've left Ubuntu and went to Debian as my OS of choice.

[snowComet]

Yes, very.

 

If you're not sure how to use/configure Linux, then I'd suggest you take a look at the Debian Home Server Howto. Ubuntu is derived from Debian, so everything in my tutorial should for you as well. If you go for Ubuntu, do take the Server edition. The Desktop edition is waaaaay too cluttered, that's why I've left Ubuntu and went to Debian as my OS of choice.

Thx for replying my post. I would definitely check that. Btw, can u clarify a bit about the system independence of software raid from yr comment above? How could the system recover data of the software raid after re-installation or system crashes?? By the remaining data in those disks?? Thx~~

[MG2R]

Thx for replying my post. I would definitely check that. Btw, can u clarify a bit about the system independence of software raid from yr comment above? How could the system recover data of the software raid after re-installation or system crashes?? By the remaining data in those disks?? Thx~~

The following explanation is true for mdadm, which I use on my machine. Although I'm farily sure other software RAID solutions oprate in a very similar way, I can not garantee that this explanation is correct for every software RAID solution. It is unlikely to be 100% correct, but this is what I understand about it:

 

mdadm operates by creating so called superblocks on the drives you include in an array. Such a superblock is a specific piece of data which is tored in the beginning of the drive. It contains information about the drive, the array and so on. Say you've made a RAID1 array with two drives, then those two drive will have a superblock saying that they belong to a certain RAID1 array (with a unique identifier) and what position they are in respectively. Now say you want to build a new, more powerful machine running a differen Linux distribution in which you still want this RAID array. You buy new hardware and install a new OS. Now, when you install mdadm on this new machine and add the two drives you had from the old machine, mdadm will recognise the superblocks and will be able to read that those two drives belong to a RAID1 array. By reading the superblocks, it will be able to open the RAID1 array you created on another machine and another operating system.

 

If you want to compare this to hardware RAID. Say you have a certain RAID controller, to which you've connected two drives. You set up this RAID controller to put those drives in a RAID1 array and all is well. Now say that you want a new, more powerful controller. Chances are that this new controller won't know that the drives you had were part of a RAID1 array, so the only way to migrate to that new controller safely is to buy new drives and have both controller set up and running while you move the data. If your controller dies and you're forced to buy a new one, there is a chance you're screwed and lose your array.

[snowComet]

The following explanation is true for mdadm, which I use on my machine. Although I'm farily sure other software RAID solutions oprate in a very similar way, I can not garantee that this explanation is correct for every software RAID solution. It is unlikely to be 100% correct, but this is what I understand about it:

 

mdadm operates by creating so called superblocks on the drives you include in an array. Such a superblock is a specific piece of data which is tored in the beginning of the drive. It contains information about the drive, the array and so on. Say you've made a RAID1 array with two drives, then those two drive will have a superblock saying that they belong to a certain RAID1 array (with a unique identifier) and what position they are in respectively. Now say you want to build a new, more powerful machine running a differen Linux distribution in which you still want this RAID array. You buy new hardware and install a new OS. Now, when you install mdadm on this new machine and add the two drives you had from the old machine, mdadm will recognise the superblocks and will be able to read that those two drives belong to a RAID1 array. By reading the superblocks, it will be able to open the RAID1 array you created on another machine and another operating system.

 

If you want to compare this to hardware RAID. Say you have a certain RAID controller, to which you've connected two drives. You set up this RAID controller to put those drives in a RAID1 array and all is well. Now say that you want a new, more powerful controller. Chances are that this new controller won't know that the drives you had were part of a RAID1 array, so the only way to migrate to that new controller safely is to buy new drives and have both controller set up and running while you move the data. If your controller dies and you're forced to buy a new one, there is a chance you're screwed and lose your array.

oh ic.... thx for clarifying it. I think I will build my home server based on ubuntu and zfs... but I still need to wait for Lian Li to release my dream case for the home server - the PC-D7000....

[MG2R]

oh ic.... thx for clarifying it. I think I will build my home server based on ubuntu and zfs... but I still need to wait for Lian Li to release my dream case for the home server - the PC-D7000....

Hope you get it to work. A while ago (couple of years), I tried installing ZFS on Ubuntu and couldn't get it to work. Of course, that was then and this  is now

[snowComet]

Hope you get it to work. A while ago (couple of years), I tried installing ZFS on Ubuntu and couldn't get it to work. Of course, that was then and this  is now

You are scaring me now  :unsure: ....

[MG2R]

You are scaring me now  :unsure: ....

You'll be fine ;)

[Whaler_99]

Just to throwh this out there on RAID and solutions - I have been working with RAID and various OS'es for over 18 years...

 

Most of the more common "free" solutions that use a software RAID solution are all very different. Most of the Linux based ones, say like FreeNAS and unRAID use some sort of LiveCD to install then are completly managed by a web interface. Very easy to install, setup and use. I know that on both the support forums for these, and similar products, they have excellent community support, so if you do need advanced help using  the old CLI, most of them will walk you through step by step.

 

Now, onto RAID - software based first. This one is a bit more scary as various solutions can do it various ways. I know from experience that something like unRAID will always write a whole file to a drive, so in the event of a complete system failure, another install, or even Windows, can be setup to read the data contents. As well, using Windows Storage Spaces is similar, should Windows die, any Server 2012 or Win8 system can read the contents of a disk. FreeNAS and ZFS? Not sure. Here is where you need to do your research on what specific DR options you have based on your config. Windows Home Server used another disk extension solution that I believe that data could also be read on any other Windows install. Windows server based "software" RAID in versions prior to 2012 used a Logical Disk feature that was actually quite dangerous. This logical disks were actually bound to the physical ID of the drive, so if you also had hardware RAID running and did software RAID and a drive failed, you typically lost everything as drive ID's would never match. I saw this happen more then once. With Windows Server prior to 2012, either use software RAID or hardware RAID -but never both.

 

Also, remember, RAID does NOT equal BACKUPS. It is simply a way of protecting data, typically in the home environment, from a single disk failure. There could be many other things that could happen which could fry every drive and all your data - for example, a massive power surge. Always make sure you have a backup of all the data on your storage solution. Many even offer solution for doing it. I user schedule rsync on my unRAID to backup data to a secondary unit.

 

Now, Hardware RAID. Again, do your research. Most server class solutions now adays store all the RAID config on each disk. So, in theory you could pull all the disks from one server, to another server with the same or similar RAID card and everything will run fine. More advanced cards will even correct of you swap the drives around. Fun.

Some though, typically cheaper ones, do store the config on chip, so if the RAID card dies, then yes, you lose your config. And often you data. Hence the popularity of software RAID more recently. Slower rebuild times and not always the advanced feature, but if you are not willing to shell out a grand plus, at least on a system failure, it can be much easier to get your data back from a software RAID solution than a hardware RAID solution.

 

As has been mentioned - if going software based and you are looking at HBA's - do not pay extra for anything that does RAID and run it disabled. There are many excellent options of multi port HBA cards that just do JBOD, essentially desgined for software RAID solutions. Always use the mobo ports first, before using HBA ports. More reliable and a bit faster, less bottle necks. Also, PCI slots at most, can handle two drives on the channel. PCIe slots can typically handle 8. Oh way, are your slots "sharing" channels? Research also needs to be done on this. You can go get a the fastest disks and a great HBA and stuff and watch your system grinf to halt, because all your disks are running over a single PCIe lane because your mobo manufacturer is cheap.

 

Also, if using HBA's and software RAID, ALWAYS make sure your parity drive, if you have one, runs off the onboard SATA port. Lot more reliable. Esp if the HBA card fails and you need to replace. You will still have all your parity info.

 

So, final point, do your research on whatever solution and parts you use and know your DR options. Depending on what you do, this can vary widely. And.... do backups...