Home storage server

[kamikazi]

I thought this would be the best place to put this. I have never made a home storage server but have wanted to make one for some time now. My 2TB drive just isn't cutting it anymore. So I wanted to make a system but I really don't know were to start for this build. Mobo, CPU, PSU, Raid card, ETC. Would like to stay on the less expensive side and make a system with a raid 6 config. Was looking at this as a starting point, http://us.ncix.com/products/?sku=58605&vpn=RPC-2212&manufacture=Norco%20Technologies%20Inc%2E

I don't need anything to extreme something just as a storage for movies, music, and a place to store my edited game footage. Also to stream off of to my other devices via streaming software such as "air video server"

Would also like to be capable of having clan mates connect to my server and upload their game footage for me to edit and post. (not editing from server, I have dedicated machine just upload)

If a machine like this would be on the more expensive side I could just be patient and do little at a time. Also I have an old P4 and mobo out of cookie cutter computer. Would something like that be sufficient to run a server for my needs?

[wpirobotbuilder]

If you want to do lots of 1080p streaming to other devices, you don't need the best, but you might want to get an i3 or a quad-core AMD. This might also handle people connecting to your server.

Hardware RAID tends to be faster, but it adds another point of failure to your machine and the cards can run hot. Check out Windows Storage Spaces and FreeNAS for decent software RAID options.

No idea what power supplies fit in that case, but get one that's 80+ certified if possible, since the machine will be running all time.

As far as streaming software goes, Plex Media Server is fantastic, and works with Android devices and the Roku http://www.roku.com/

[Rain]

A P4 is not going to cut it for something like this. You need at least an i3. Using software RAID is a great way to keep the cost down; I highly recommend ZFS, FreeNAS supports this. I have personal experience with ZFS and can assure you that it is great and performs really well. The ZFS equivalent of RAID6 is "RAIDZ2" (Dual-parity, up to two drives can fail).

The chassis you have selected is a great starting point; it will be a little tricky to find a PSU that will fit, but you should be able to find one. Do you have a specific price (before drives) that you are trying to stay under? I can provide very specific suggestions if given a rough budget.

[azeotrope]

P4 is not going to cut it for something like this. You need at least an i3

Not true. I run a mdraid RAID6 setup with an AMD a4 3400, which is far below an i3 and I can sustain 400MB/s writes and 700MB/s reads. RAIDZ/RAIDZ2 is roughly equivalent to mdraid in performance if you don't use data deplication or compression, which is useless for media storage anyway.

If you're going with hardware raid you DEFINATELY don't need a better cpu, SMB, NFS, FTP, SSHFS etc takes up hardly any resources

[Rain]

Not true. I run a mdraid RAID6 setup with an AMD a4 3400' date=' which is far below an i3 and I can sustain 400MB/s writes and 700MB/s reads. RAIDZ/RAIDZ2 is roughly equivalent to mdraid in performance if you don't use data deplication or compression, which is useless for media storage anyway. If you're going with hardware raid you DEFINATELY don't need a better cpu, SMB, NFS, FTP, SSHFS etc takes up hardly any resources[/quote']But an i3 supports ECC memory. If you are setting up a new server today, as cheap as ECC memory is now (16GB for ~$100), it is pointless to get a CPU and board that don't support ECC memory; especially if you are using software RAID. You get far, far, far more data security with ECC memory. When your doing a ZFS scrub and your non-ecc memory flips a bit (however rare; it could happen) it will assume that corruption has occurred and possibly "fix" the problem with incorrectly calculated garbage and actually corrupt the data on-disk. Worse, if a disk dies and you are currently doing a resilver with a new disk and a memory error occurs it will definitely corrupt the data. Same applies to mdraid consistency checks and rebuilds.

If you are using a hardware RAID controller, I suppose your chances of seeing data corruption are greatly reduced, but then you have a single point of failure: The RAID controller. Down the road if the controller craps out and you can't get the same exact RAID controller (or one by the same manufacturer that supports older RAID volumes on older controllers), your array is forever lost. Also, data corruption can still occur with a hardware controller; for example: When the system is reading data from the network card prior to sending it to disk; it is temporarily storing it in memory.

If OP really wants to keep things as cheap as possible, yes an i3 is an unnecessary cost (something better than a P4 is still needed in my option). But it could lead to long-term problems down the road. Depends on how much you value the data your storing on the array (family photos vs. pirated movies). That said, RAID is not a backup; you should always have another backup solution in place. BUT: If data corruption occurs prior to your backup or while you are backing up (whether it be copying to an external drive, sending the data off-site, what ever) you will "backup" corrupted data; yet another reason you should use ECC memory wherever possible in long-term storage-type applications.

[azeotrope]

If you are setting up a new server today' date=' as cheap as ECC memory is now (16GB for ~$100), it is pointless to get a CPU and board that don't support ECC memory[/quote']

That's pretty subjective. Yes, its more secure but chances of data corruption from this are incredibly small. I'd hardly say it's "far, far more data security". no all in one NAS that I know of uses ECC memory, and you'd say that due to this, they are insecure? ECC is used by enterprise for high availability, not really for security.

something better than a P4 is still needed in my option

For what the OP said they want to do, it can definately be done with a decent P4. If configured correctly, you should still bottleneck on gigabit ethernet, that's if you're even using gigabit with that setup.

[Rain]

That's pretty subjective. Yes' date=' its more secure but chances of data corruption from this are incredibly small. I'd hardly say it's "far, far more data security". no all in one NAS that I know of uses ECC memory, and you'd say that due to this, they are insecure? ECC is used by enterprise for high availability, not really for security.[/quote']I disagree. Although unlikely, Non-ECC memory can still glitch and flip a bit at any time, especially if the RAM stick is on the edge of actually failing. If you want to live on the edge and hope your data will survive, be my guest. Personally, I think the added cost on proper server hardware is worth the data security in the long run, especially if you are building a server from scratch. The hardware is a one-time cost and you are likely planning on running for many years; why not get hardware that protects your data? I wouldn't trust any of my important data (Family pictures, videos, documents, ect) on any all-in-one NAS. Not only do they not have ECC memory (usually), the even bigger factor is what if the entire enclosure craps out and a replacement that is still compatible is not available, you lose everything.

For what the OP said they want to do' date=' it can definately be done with a decent P4. If configured correctly, you should still bottleneck on gigabit Ethernet that's if you're even using gigabit with that setup.[/quote']But saturating the network isn't everything. Scrubs, consistency checks, and rebuilds will be considerably faster if the CPU can actually keep up with the disks. The chances of a disk failing during a rebuild are higher than during simple normal operation. If you CPU cannot keep up with your disks during a rebuild, your only extending the critical time during which a second (or third, in the case of RAID6) drive failure could render the array useless and lose all of your data.

You can get a Supermicro MBD-X9SCM-O (dual Intel NICs, a lot of PCIe for future HBA expension), an i3-3220, and 16GB of Kingston ECC memory for a bit over $400. Add a case and a PSU and your still sub-$600 (in the range of most all-in-one NAS units w/o drives) but with much more power, data security, and many more future expansion options.

[azeotrope]

Although unlikely' date=' Non-ECC memory can still glitch and flip a bit at any time[/quote']

Yes ECC memory gives you more protection, but bit flipping is about as rare as bit rot on rotational media, bad ram stick or not. There is no hard data that I'm aware of to prove that ECC memory is generally beneficial to consumers, you're the first person I've ever come across that seems to think that it is. Wouldn't more hardware companies be pushing ECC on to consumers if that were the case? In the event of a bad stick causing data corruption, the chances of this corruption occuring more than once on a single stripe is even lower, and mitigated by using more than one parity. Potentially, yes this could cause a drive to be kicked out of the array however data loss is unlikely since the raid algorithm can correct the bad block based on what is on the remaining drives.

consistency checks' date=' and rebuilds will be considerably faster if the CPU can actually keep up with the disks.[/quote']

consistency checking on ZFS should be done in the background and take little resources overall. It is also prioritized as to not affect performance. This has to occur constantly anyway to protect against silent corruption. conventional raid does none of this, and only checks consistency during reads.

The chances of a disk failing during a rebuild are higher than during simple normal operation

Only under the assumption that you don't have IO going 24/7. Even then it's difficult to prove that since rebuild read/write rates are generally low and also done in the background on software raid as to not affect general use. The whole point of using raid in the first place is high availability.

[Rain]

There have been studies proving frequency the of bit-flip occurrences in DRAM; even Google warned at it at one point (I remember reading about it a while ago). There has also been research on how ZFS recovers from bit-flips in its various data-structures in memory. Do some research; there have been a few academic publications in the last decade. There are many people, even home users and/or academics, that agree.

I'm not saying buy the most expensive memory on the planet and an 8-core Intel processor, I'm just saying spend an extra $200 or so and get proper server-grade hardware that will last you for many years without problems and will thwart memory corruption.

Wouldn't more hardware companies be pushing ECC on to consumers if that were the case?

No, they shouldn't. The general consumer doesn't store TBs of data. The general consumer browses Facebook and Reddit, plays games, watches videos, and thinks that bigger numbers are always better in computing. A bit-flip shouldn't cause much damage to them so purchasing ECC memory and a CPU that can handle it is not worth it. ECC memory (and proper server-grade hardware) should be purchased for mass storage-specific applications when more than a few TBs of important data are in question.

[wpirobotbuilder]

ECC memory (and proper server-grade hardware) should be purchased for mass storage-specific applications when more than a few TBs of important data are in question.

I can back this up. I work for Dell Equallogic and the storage arrays we sell use ECC memory for RAID controllers and processors. If one of these arrays were to fail due to hardware (and not firmware) problems, it would definitely not be a good thing.

[kamikazi]

Would it be "over kill" to use a 8120 in the server? I was thinking of upgrading to a i5 3570 for my main gaming rig and then using the server as a "server" and "editing" machine.

[Guest]

Would it be "over kill" to use a 8120 in the server? I was thinking of upgrading to a i5 3570 for my main gaming rig and then using the server as a "server" and "editing" machine.

Yes, I personally wouldn't. If this and I assume it will be, on 24 hours a day you'll want a fairly low power CPU and system in general. That CPU alone is rated for somewhere between 100-130w from memory.

[Rain]

Would it be "over kill" to use a 8120 in the server? I was thinking of upgrading to a i5 3570 for my main gaming rig and then using the server as a "server" and "editing" machine.

Yes' date=' I personally wouldn't. If this and I assume it will be, on 24 hours a day you'll want a fairly low power CPU and system in general. That CPU alone is rated for somewhere between 100-130w from memory. [/quote']Those numbers are maximum (under 100% load) wattages. You have to look up idle power-draw to get an accurate measure of how much power the CPU will be drawing from the wall during 90+% of the time the system is on.

The other hardware (drives, memory, board, everything else...) will probably out-draw the CPU at idle. That said, I know nothing about AMD CPUs; they may be power-hungry beasts even at idle. Like I said, do some research!

[kamikazi]

So all I really need to worry about is my Idle consumption? The server would be used for most of the day just not fully loaded people streaming music off it with occasional movies being streamed as well. So through out a normal day mostly streaming music and then people uploading pictures video.... etc. So shouldn't be to taxing on the Cpu with those tasks. Right? Once in a blue moon someone streaming a movie off there which would tax cpu.

[looney]

I have been sick lately so sorry if im just repeating things that have already been said :p

Now, first off i would recommend you to look at this topic for some inspiration:

http://linustechtips.com/main/forum/storage-solutions/7147-storage-show-off-topic

As far as the actual system goes i would recommend something like Rain said in a previous post, which is a nice i3 with ECC ram.

ECC is always recommended on a file server and at the moment i3 is a nice bang for the buck and will also have a low power usage.

Which brings me on to the next thing, the power supply, and no you don't want to just look at the idle consumption as the spinning up of the drives takes more power.

That said my personal server (first post in the topic above) has a 850W PSU which is just utter overkill but i put it in in-case i wanted to upgrade to a dual socket motherboard.

Now on to the actual storage side of things, you can go for a hardware raid which is likely to break your budget or you can go for software raid.

If you decide to go for hardware just let us know and we will find a good card for you. But personally i would go for software.

And with software you again have some choices, a linux distro (like freenas), or a windows package (like FlexRAID ($60)).

Personally I've been using FlexRAID for a while now because i needed to have windows on my Fileserver.

[kamikazi]

Yea I have been pulling parts together and making a list and when I got to raid cards I nearly had a heart attack. Recently I have decided to go with this case instead of the one stated above. http://www.newegg.com/Product/Product.aspx?Item=N82E16811219037

Just for shits and giggles what would be the amount of a lower end raid card? More then likely I am going to go software since I don't want to spend 600+ dollars. So with software would that be the same as going hardware? What is the difference between the two besides the obvious of physical and virtual. So do I just plug in all the drives to the motherboard and make an array within the software of FlexRaid? Is there redundancy? Like a drive fails and I do or don't loose all my data. I saw your picture in your post and it seemed like you have a bunch or various drives. Is there a way to pool that into one massive drive?

[Guest]

Would it be "over kill" to use a 8120 in the server? I was thinking of upgrading to a i5 3570 for my main gaming rig and then using the server as a "server" and "editing" machine.

Yes' date=' I personally wouldn't. If this and I assume it will be, on 24 hours a day you'll want a fairly low power CPU and system in general. That CPU alone is rated for somewhere between 100-130w from memory. [/quote']Those numbers are maximum (under 100% load) wattages. You have to look up idle power-draw to get an accurate measure of how much power the CPU will be drawing from the wall during 90+% of the time the system is on. The other hardware (drives, memory, board, everything else...) will probably out-draw the CPU at idle. That said, I know nothing about AMD CPUs; they may be power-hungry beasts even at idle. Like I said, do some research!

I know but even so, they will more than likely idle higher than a lower power processor.

[kamikazi]

The drives will eat power regardless of which CPU I use though. My biggest worry is CPU. The only reason I was thinking of using a 8120 is because I am thinking like I stated above going to an i5 or i7 for my main gaming/editing rig from my 8120.

[Guest]

Would it be "over kill" to use a 8120 in the server? I was thinking of upgrading to a i5 3570 for my main gaming rig and then using the server as a "server" and "editing" machine.

Yes' date=' I personally wouldn't. If this and I assume it will be, on 24 hours a day you'll want a fairly low power CPU and system in general. That CPU alone is rated for somewhere between 100-130w from memory. [/quote']Those numbers are maximum (under 100% load) wattages. You have to look up idle power-draw to get an accurate measure of how much power the CPU will be drawing from the wall during 90+% of the time the system is on. The other hardware (drives, memory, board, everything else...) will probably out-draw the CPU at idle. That said, I know nothing about AMD CPUs; they may be power-hungry beasts even at idle. Like I said, do some research!

I swear I wrote back to this...

Anyway, even so - they're more likely to idle higher than a lower max power CPU - that's just what I've found personally on a few CPU's.

[looney]

Just for shits and giggles what would be the amount of a lower end raid card?

If you going for software raid and all you need it extra connectors then a IBM M1015 would do the job (around $100 for 8 ports)

More then likely I am going to go software since I don't want to spend 600+ dollars. So with software would that be the same as going hardware? What is the difference between the two besides the obvious of physical and virtual. So do I just plug in all the drives to the motherboard and make an array within the software of FlexRaid? Is there redundancy? Like a drive fails and I do or don't loose all my data. I saw your picture in your post and it seemed like you have a bunch or various drives. Is there a way to pool that into one massive drive?

First of that picture is outdated, here are some new ones.

[ATTACH=CONFIG]n3089[/ATTACH] [ATTACH=CONFIG]n3090[/ATTACH]

As you can see its now listed as one volume. the second picture shows the data drives (blue bars) and the parity drives (green bars) so now im running the equivalent to raid 6. but i can easily add another parity or data drive. keep in mind that this is a snapshot raid so it will only calculate the parity when you want it to do so (there is a schedule option)

[looney]

Would it be "over kill" to use a 8120 in the server? I was thinking of upgrading to a i5 3570 for my main gaming rig and then using the server as a "server" and "editing" machine.

Yes' date=' I personally wouldn't. If this and I assume it will be, on 24 hours a day you'll want a fairly low power CPU and system in general. That CPU alone is rated for somewhere between 100-130w from memory. [/quote']Those numbers are maximum (under 100% load) wattages. You have to look up idle power-draw to get an accurate measure of how much power the CPU will be drawing from the wall during 90+% of the time the system is on. The other hardware (drives, memory, board, everything else...) will probably out-draw the CPU at idle. That said, I know nothing about AMD CPUs; they may be power-hungry beasts even at idle. Like I said, do some research!

I swear I wrote back to this...

Anyway, even so - they're more likely to idle higher than a lower max power CPU - that's just what I've found personally on a few CPU's.

yes you did write back but the page was full and i doesn't show the second page :p