A wakeup call + Home server post

[Huntsman]

No, this is not a novel although the title sounds like one. This is just something that has happened to me that I wanted to share with everyone.

Yesterday I had my WD Blue 1TB(not thrashing on the brand, just to be clear) die on me. It will not mount, it runs very hot even though nothing is done to it, the hdd activity LED will stay lighted up even though resource managet tells me that nothing is going on, startups and shutdowns will take forever with the drive connected(sometimes wont get into desktop) and with it my entire movies, series and song collection. oh and explorer.exe would crash if i try to open up file explorer. basicly the thing is causing me a pain in the butt.

now after 15 hours of chkdsk the problem is now sorted out with approx 500MB of bad sector and a movie being corrupted. Not too bad i know but it is an experience which make me feel like kicking myself in the butt because I always insisted that my clients to use a NAS box due to it having a RAID storage solution but did not get one myself. So now im planinng to build a dedicated home server(current one serves as brother's gaming rig plus server) and implement a proper RAID system.

current rig:

Intel Core 2 Duo E7400 oc to 3.8GHz

Asus P5Q SE PLUS

Seagate 500GB os drive

WD Blue and WD Green 500GB storage drive

WD Blue 1TB storage drive (which failed)

Sapphire HD5750

Cooler Master 850W

ThermalTake Commander MS-1 casing

Im thinking of building a dedicated home server to run Samba file server, torrent client, BOINC, TeamSpeak 3 server, FTP server and backup. It can be WHS or linux though im familliar with ubuntu so think im going for that. gigabit LAN connection is already in place with a dlink gigabit switch. this is what i have in mind so far:

OS: Ubuntu

Processor: thinking about i3

Mainboard: something low end LGA1155

RAM: 8GB is probably overkill since RAM is so cheap these days

Storage: 4TB

PSU: 550W?

Casing: Xigmatek Gigas

Please give me some suggestion as I have never really built a dedicated home server before as my current one is a ghetto-just-enough-to-work rig. Thanks in advance.

[azeotrope]

Do you have a budget? It looks like you're leaning towards software raid. If you stick to Ubuntu you have the options of FlexRAID or mdraid.

Do you have any drives that you plan to include in the server already? If not, it's better to decide on which kind of raid you want to use first. for mdraid you may want to start off with slightly larger drives, like 3 or 4 TB in a raid5 array so that you can grow the array in the future. If you're set on 4TB, then 3x2TB raid5 is probably the way to go.

550W is definitely overkill in this case, you could easily get away with something around 350W even with more hard drives.

You could also go for a AMD FM2 board and low end apu to save some cash, which will handle a basic mdraid setup just fine and easily saturate a gigabit line, even on raid5 writes.

[MG2R]

Next week, I'll get my components back from RMA and I'll be rebuilding my home server from the ground up. I'll be writing a howto on the software setup on the forums as I go, so keep your eyes open for that ;) I'll be using Debian (so the guide will be pretty Ubuntu compatible) to create a file server implementing samba, apache, owncloud, transmission, and so on and so forth...

As for advice I can give you right now: pretty much what azeotrope said in the comment above. Also, you said it yourself, 8GB of RAM is completely overkill unless you're planning on running a minecraft server ;)

[Huntsman]

Do you have a budget? It looks like you're leaning towards software raid. If you stick to Ubuntu you have the options of FlexRAID or mdraid.

Do you have any drives that you plan to include in the server already? If not, it's better to decide on which kind of raid you want to use first. for mdraid you may want to start off with slightly larger drives, like 3 or 4 TB in a raid5 array so that you can grow the array in the future. If you're set on 4TB, then 3x2TB raid5 is probably the way to go.

550W is definitely overkill in this case, you could easily get away with something around 350W even with more hard drives.

You could also go for a AMD FM2 board and low end apu to save some cash, which will handle a basic mdraid setup just fine and easily saturate a gigabit line, even on raid5 writes.

No, not really. I would like to keep it cheap but the last time i did that i got into trouble. So, think i'll go for a price within reason. It's all just planning at this stage, am saving some cash to get this thing built in the meantime. And im not leaning to either software or hardware raid since I wont be booting from any of the drives in the array. Also, Im more worried about RAID 5 without a dedicated RAID card. Heard that the performance will suck.

I was thinking about using RAIDs which doesnt need a dedicated card like RAID 1 or 10 but I would need alot of drives for that. My main concern about RAID 5 is like I've said, the performance issue without a dedicated RAID card and those card are expensive and rare at where I live.

As for the PSU, there's not much choice at where I live. The lowest non-crap PSU i can get is a Cooler Master 450W. To be honest I don't trust Cooler Master PSUs since I have 2 of them died on me before.

An APU would be a waste in my application since I won't be using the display after I've got it set up. Would prefer speding the processing power on raw number crunching instead.

[Huntsman]

Do you have a budget? It looks like you're leaning towards software raid. If you stick to Ubuntu you have the options of FlexRAID or mdraid. Do you have any drives that you plan to include in the server already? If not' date=' it's better to decide on which kind of raid you want to use first. for mdraid you may want to start off with slightly larger drives, like 3 or 4 TB in a raid5 array so that you can grow the array in the future. If you're set on 4TB, then 3x2TB raid5 is probably the way to go. 550W is definitely overkill in this case, you could easily get away with something around 350W even with more hard drives. You could also go for a AMD FM2 board and low end apu to save some cash, which will handle a basic mdraid setup just fine and easily saturate a gigabit line, even on raid5 writes.[/quote']

No, not really. I would like to keep it cheap but the last time i did that i got into trouble. So, think i'll go for a price within reason. It's all just planning at this stage, am saving some cash to get this thing built in the meantime. And im not leaning to either software or hardware raid since I wont be booting from any of the drives in the array. Also, Im more worried about RAID 5 without a dedicated RAID card. Heard that the performance will suck.

I was thinking about using RAIDs which doesnt need a dedicated card like RAID 1 or 10 but I would need alot of drives for that. My main concern about RAID 5 is like I've said, the performance issue without a dedicated RAID card and those card are expensive and rare at where I live.

As for the PSU, there's not much choice at where I live. The lowest non-crap PSU i can get is a Cooler Master 450W. To be honest I don't trust Cooler Master PSUs since I have 2 of them died on me before.

An APU would be a waste in my application since I won't be using the display after I've got it set up. Would prefer speding the processing power on raw number crunching instead.

[azeotrope]

Also' date=' Im more worried about RAID 5 without a dedicated RAID card. Heard that the performance will suck[/quote']

As I've said, software raid5 with a low end apu/i3 will easily saturate a gigabit network with both sequential reads and writes.

I was thinking about using RAIDs which doesnt need a dedicated card like RAID 1 or 10

You can do software raid5, without a raid card

An APU would be a waste in my application since I won't be using the display after I've got it set up.

Well, a current gen i3 has an integrated gpu and will do the same thing. Most motherboards will allow you to disable it anyway.

[MG2R]

Do you have a budget? It looks like you're leaning towards software raid. If you stick to Ubuntu you have the options of FlexRAID or mdraid. Do you have any drives that you plan to include in the server already? If not' date=' it's better to decide on which kind of raid you want to use first. for mdraid you may want to start off with slightly larger drives, like 3 or 4 TB in a raid5 array so that you can grow the array in the future. If you're set on 4TB, then 3x2TB raid5 is probably the way to go. 550W is definitely overkill in this case, you could easily get away with something around 350W even with more hard drives. You could also go for a AMD FM2 board and low end apu to save some cash, which will handle a basic mdraid setup just fine and easily saturate a gigabit line, even on raid5 writes.[/quote']

No, not really. I would like to keep it cheap but the last time i did that i got into trouble. So, think i'll go for a price within reason. It's all just planning at this stage, am saving some cash to get this thing built in the meantime. And im not leaning to either software or hardware raid since I wont be booting from any of the drives in the array. Also, Im more worried about RAID 5 without a dedicated RAID card. Heard that the performance will suck.

I was thinking about using RAIDs which doesnt need a dedicated card like RAID 1 or 10 but I would need alot of drives for that. My main concern about RAID 5 is like I've said, the performance issue without a dedicated RAID card and those card are expensive and rare at where I live.

As for the PSU, there's not much choice at where I live. The lowest non-crap PSU i can get is a Cooler Master 450W. To be honest I don't trust Cooler Master PSUs since I have 2 of them died on me before.

An APU would be a waste in my application since I won't be using the display after I've got it set up. Would prefer speding the processing power on raw number crunching instead.

If you can get your hands on a Corsair CX 430 unit, pick up one of those. I've been using Cosrair PSUs for 6 builds in the last year (not all for me) and I'm quite surprised. Clean units that work with very little noise, what more do you want?

[Huntsman]

Do you have a budget? It looks like you're leaning towards software raid. If you stick to Ubuntu you have the options of FlexRAID or mdraid. Do you have any drives that you plan to include in the server already? If not' date=' it's better to decide on which kind of raid you want to use first. for mdraid you may want to start off with slightly larger drives, like 3 or 4 TB in a raid5 array so that you can grow the array in the future. If you're set on 4TB, then 3x2TB raid5 is probably the way to go. 550W is definitely overkill in this case, you could easily get away with something around 350W even with more hard drives. You could also go for a AMD FM2 board and low end apu to save some cash, which will handle a basic mdraid setup just fine and easily saturate a gigabit line, even on raid5 writes.[/quote']

No, not really. I would like to keep it cheap but the last time i did that i got into trouble. So, think i'll go for a price within reason. It's all just planning at this stage, am saving some cash to get this thing built in the meantime. And im not leaning to either software or hardware raid since I wont be booting from any of the drives in the array. Also, Im more worried about RAID 5 without a dedicated RAID card. Heard that the performance will suck.

I was thinking about using RAIDs which doesnt need a dedicated card like RAID 1 or 10 but I would need alot of drives for that. My main concern about RAID 5 is like I've said, the performance issue without a dedicated RAID card and those card are expensive and rare at where I live.

As for the PSU, there's not much choice at where I live. The lowest non-crap PSU i can get is a Cooler Master 450W. To be honest I don't trust Cooler Master PSUs since I have 2 of them died on me before.

An APU would be a waste in my application since I won't be using the display after I've got it set up. Would prefer speding the processing power on raw number crunching instead.

that is kinda of a problem, over at my area good PSUs are hard to come by. if not very expensive.

[Huntsman]

Also' date=' Im more worried about RAID 5 without a dedicated RAID card. Heard that the performance will suck[/quote']

As I've said, software raid5 with a low end apu/i3 will easily saturate a gigabit network with both sequential reads and writes.

I was thinking about using RAIDs which doesnt need a dedicated card like RAID 1 or 10

You can do software raid5, without a raid card

An APU would be a waste in my application since I won't be using the display after I've got it set up.

Well, a current gen i3 has an integrated gpu and will do the same thing. Most motherboards will allow you to disable it anyway.

I'm hoping to hit 80MBps read/writes that my current setup can do. And yes, I'm leaning towards an i3 too since it is efficient and has enough processing power for anything I'm about to run. Still wouldn't put an APU in there though, since that chip is built around having a better integrated graphics and sacrificing pure processing power in the process.

Also, I'm thinking about going with ubuntu and ZFS filesystem, using RAID-Z. Any comment on that?

[Huntsman]

I'm hoping to hit 80MBps read/writes that my current setup can do. And yes, I'm leaning towards an i3 too since it is efficient and has enough processing power for anything I'm about to run. Still wouldn't put an APU in there though, since that chip is built around having a better integrated graphics and sacrificing pure processing power in the process.

Also, I'm thinking about going with ubuntu and ZFS filesystem, using RAID-Z. Any comment on that?

[azeotrope]

Also' date=' Im more worried about RAID 5 without a dedicated RAID card. Heard that the performance will suck[/quote']

As I've said, software raid5 with a low end apu/i3 will easily saturate a gigabit network with both sequential reads and writes.

I was thinking about using RAIDs which doesnt need a dedicated card like RAID 1 or 10

You can do software raid5, without a raid card

An APU would be a waste in my application since I won't be using the display after I've got it set up.

Well, a current gen i3 has an integrated gpu and will do the same thing. Most motherboards will allow you to disable it anyway.

I have a server with a low end a4 apu and it hits 400MB/s writes with mdraid. ZFS is a good choice, the main reason I don't run it is because arrays cannot be expanded like on mdraid. If you decide on ZFS, you should stick to BSD, as zfsonlinux is not officially considered stable by any distro.

[Huntsman]

Also' date=' Im more worried about RAID 5 without a dedicated RAID card. Heard that the performance will suck[/quote']

As I've said, software raid5 with a low end apu/i3 will easily saturate a gigabit network with both sequential reads and writes.

I was thinking about using RAIDs which doesnt need a dedicated card like RAID 1 or 10

You can do software raid5, without a raid card

An APU would be a waste in my application since I won't be using the display after I've got it set up.

Well, a current gen i3 has an integrated gpu and will do the same thing. Most motherboards will allow you to disable it anyway.

How does a gigabit connection get you 400MBps? 1000Mbps equals to a theoretical maximum of 125MBps...

I thought ZFS can be expanded easily? And does not need to be taken offline while doing so too..

[azeotrope]

Also' date=' Im more worried about RAID 5 without a dedicated RAID card. Heard that the performance will suck[/quote']

As I've said, software raid5 with a low end apu/i3 will easily saturate a gigabit network with both sequential reads and writes.

I was thinking about using RAIDs which doesnt need a dedicated card like RAID 1 or 10

You can do software raid5, without a raid card

An APU would be a waste in my application since I won't be using the display after I've got it set up.

Well, a current gen i3 has an integrated gpu and will do the same thing. Most motherboards will allow you to disable it anyway.

I don't get that speed over the network, that is just from some benchamrking I did. The point is that I get full gigabit speed quite comfortably with that cpu. And it's not 400Mb/s it's 400MB/s with a B. zfs can be expanded, but the raid itself cannot. I suggest you look here: http://linustechtips.com/main/forum/storage-solutions/89446-zfs-vs-hardware-raid-vs-software-raid-vs-anything-else

post #12 explains how to grow a zfs array or storage pool in some detail.

[Huntsman]

Also' date=' Im more worried about RAID 5 without a dedicated RAID card. Heard that the performance will suck[/quote']

As I've said, software raid5 with a low end apu/i3 will easily saturate a gigabit network with both sequential reads and writes.

I was thinking about using RAIDs which doesnt need a dedicated card like RAID 1 or 10

You can do software raid5, without a raid card

An APU would be a waste in my application since I won't be using the display after I've got it set up.

Well, a current gen i3 has an integrated gpu and will do the same thing. Most motherboards will allow you to disable it anyway.

hmm.. then I'll have to change my approach on things since my emphasis is on expandability and data security. Speed doesn't bother me that much since the worst load the system is going to get is a weekly backup. Most of the time it would just serve media files to my htpc.

The security part is easily dealt with, but what about the expandability? And I will be hosting TeamSpeak 3 server and folding on the server too so the OS has to support this. FTP i know for sure majority of the OS supports.

[azeotrope]

The comments thing is starting to annoy me so I'll post here instead.

what about the expandability

How exactly were you planning to expand your storage?

I will be hosting TeamSpeak 3 server and folding on the server too so the OS has to support this

This shouldn't be much of a problem on either linux or BSD. I know for a fact that linux has folding and a teamspeak server. If there isn't a native BSD port, you can run linux software on BSD with a certain kernel module loaded.

[Huntsman]

Yea LOL, I have posted a thread about it http://linustechtips.com/main/forum/...478#post106478

How exactly were you planning to expand your storage?

I know for certain that capacities are only going to get bigger since media files will always increase. So I'm thinking like maybe a year or two down the road and I might run out of storage spaces and need to expand the array. Probably by adding drives instead of swapping out to larger capacity drives. Any suggestion between the two?

This shouldn't be much of a problem on either linux or BSD. I know for a fact that linux has folding and a teamspeak server. If there isn't a native BSD port' date=' you can run linux software on BSD with a certain kernel module loaded. [/quote'] I was afraid to have to modify the kernels since I expect it to just run silently at a corner without having me to tinker with it every few hours.

edit: Okay, just found out that TeamSpeak 3 has a freeBSD version but BOINC doesn't. There is a port for it but not very stable.

[azeotrope]

Probably by adding drives instead of swapping out to larger capacity drives. Any suggestion between the two?

That will depend on a few things like budget and how quickly you tend to fill up storage. slower increases in data probably favours a full drive swap.

If you plan to grow to more than 7 or 8 drives, I'd start out with raid6 instead. mdraid will allow you to switch between raid versions, however this can take a long time, but so can growing an array. growing from 6 to 7 drives for example will take at least a day if not more. It's difficult to reccommend one over the other, but leaving some headdroom to grow is probably a good idea in either case. Flexraid may be a better option for you if future expansion is important.

[Huntsman]

That will depend on a few things like budget and how quickly you tend to fill up storage.

Budget wise I can't really do a thing right? LOL when capacity needs to be increased you just HAVE to dig out the cash to spend on it.

How quickly do I fill up my storage? Erm, 4TB is going to last quite a while I think since I've only filled up(not completely) 1TB in one and a half years and I'm not running short yet. My current home server has 2TB of space scattered around 2x500GB and a 1TB harddisk. When the final server is built, I would opt for all 1TB drives if possible. NO to WD Green drives lol, heard too much bad stuff about them dropping out of arrays. Correct me if I'm wrong but Seagate drives seemed to perform better in arrays?

[azeotrope]

NO to WD Green drives lol' date=' heard too much bad stuff about them dropping out of arrays.[/quote']They only drop out in hardware raid. Software raid works differently, but I won't try convince you differently as seagates are fine.

4TB is going to last quite a while I think

If 4TB will last a long time you probably don't have to worry about expansion too much, but there's no harm in having a contigency plan in case you want to add more drives.

[Huntsman]

They only drop out in hardware raid. Software raid works differently, but I won't try convince you differently as seagates are fine.

Price wise, wd and seagate are priced the same in my area. So I'm alright with any of them. Also, difference in hardware and software raid, I heard its because for hardware raid cards its because they have a specific response time that the drives ave to meet else the card would drop it out of the array. Am I correct or totally shot off the orbit?

[azeotrope]

That's the main reason. raid cards don't have much of a tolerance for error recovery and drives that don't have a short time limit on error recovery, such as WD green drives will be kicked out of the array due to that fact that they are not responding. This is fine in enterprise server configs which raid cards are mainly aimed at since performance is top priority, but not so much for home storage.

[Huntsman]

Okay, you've talked me out of spending cash on hardware raid. Which I am very hoping to not have to use lol.

Another question would be what if the motherboard on the server gets screwed? Or if the OS is screwed. What would happen to my array? Since it's not a hardware raid which you could recover the data just by moving the raid card to another system and power it back up. So what happens if the software side fails? Could I install the same thing on another system and migrate everything over there or am I tied to that installation of OS?

[azeotrope]

If your mobo suddenly dies, you can simply run the array on another rig, get another mobo etc since its completely software based. If the OS becomes corrupt it would be very unlikely to affect the array, especially if you run the OS on a seperate drive. raid software like mdraid is very stable and commonly used in enterprise so I wouldn't worry about the OS being "screwed". The most common way that software raid arrays go out of sync are power failures. NEVER run software raid with HDD cache on in the firmware AND without a BBU, that's just asking for trouble in the case of a power failure. In the case of a linux raid setup, you should be able to migrate between different OSes if for some reason you need to. the only thing I would suggest is to make sure you don't use kernel version lower than what you created the array on in the frist place.

[Huntsman]

The most common way that software raid arrays go out of sync are power failures. NEVER run software raid with HDD cache on in the firmware AND without a BBU' date=' that's just asking for trouble in the case of a power failure. [/quote'] How do you turn off the HDD cache in the firmware? It IS built in isn't it? Or we could opt to instruct the drive to flush the buffer more frequently?

In the case of a linux raid setup' date=' you should be able to migrate between different OSes if for some reason you need to. [/quote'] So, I just unplug the aray from the defective system and plug it into another system and it'll just detect the array? Isn't there supposed to be some configuration files or something from the original system that has to be transferred to the new system?

[azeotrope]

How do you turn off the HDD cache in the firmware

On linux you can use the hdparm tool to modify some hard drive firmware settings. The -W flag will let you turn cache cache on/off. With windows you can do it through device manager.

So' date=' I just unplug the aray from the defective system and plug it into another system and it'll just detect the array?[/quote'] For, the most part, yes (on linux/BSD anyway). There is a config file where you can specify your array but generally udev will pick it up and build the array for you on startup.