Looking for light servergrade memory and motherboards

[HueyTheGod]

Hey guys! I am looking to build a highly-upgradable NAS, and plan on having tons of storage in it eventually. Right now I am trying to find a server grade motherboard, preferably 1550 socket, or some socket that doesn't have only high end CPU's in it. What I want:

 

• At least 6 - 10 SATA III ports
• 16Gb+ ECC RAM possible
• Server Grade
• Micro-ATX or Mini-ITX would be amazing, but ATX is fine
• Under $200 ($150 preferably, but I understand)

 

I also have a question about RAID Controllers if you want to answer that here. Will I need as many SATA III sockets as I have hard drives, or does my raid controller manage all that depending on how many plugins are in the raid controller?

[Flight1sim]

If you want true server grade motherboards, LGA 2011 is probably the best

[Wats]

how havent you heard of the asrock c2550d4i? itx board, prolly works great

 

for ram.. same as all of my ram advice. as long as it works, its great ram. Go newegg and sort by lowest price

 

 

 

 

EDIT

forgot the C in that model#

[Vitalius]

• Micro-ATX or Mini-ITX would be amazing, but ATX is fine
• Under $200 ($150 preferably, but I understand)

So... you're looking for a motherboard that's under $200, or a motherboard+CPU?

I know it isn't what you are asking for, but it's still amazing, so I'll mention it: ASRock c2550d4i as @Wats mentioned.

It has 12 SATA ports, 4 SATA III and 8 SATA II. 4 dimm slots with support for 64 GB of ECC RAM and a built-in 4 core Atom processor. The bigger brother has 8 cores but is $120 more ($400 total) and the extra 4 cores are kind of the only different AFAIK. It's also M-ITX. 

It might not be that upgradeable, but honestly, you won't need much more than that for like, ever. And I do mean ever. Unless you intend to do some crazy stuff. 

The reason I suggest that is because I sincerely don't believe you'll find something this amazing for $200 or less.

[HueyTheGod]

So... you're looking for a motherboard that's under $200, or a motherboard+CPU?

I know it isn't what you are asking for, but it's still amazing, so I'll mention it: ASRock c2550d4i as @Wats mentioned.

It has 12 SATA ports, 4 SATA III and 8 SATA II. 4 dimm slots with support for 64 GB of ECC RAM and a built-in 4 core Atom processor. The bigger brother has 8 cores but is $120 more ($400 total) and the extra 4 cores are kind of the only different AFAIK. It's also M-ITX. 

It might not be that upgradeable, but honestly, you won't need much more than that for like, ever. And I do mean ever. Unless you intend to do some crazy stuff. 

The reason I suggest that is because I sincerely don't believe you'll find something this amazing for $200 or less.

Yeah, that board is good I guess. Will SATA II bottleneck my regular WD Red 4Tb hard drives?

[Vitalius]

Yeah, that board is good I guess. Will SATA II bottleneck my regular WD Red 4Tb hard drives?

Nope. 

Normal 7200rpm HDDs can only push out about 120 MB/s maximum in the best of circumstances. Most of the time it'll be 40-70 MB/s at best. SATA I's theoretical max speed is 150 MB/s. SATA II is 300 and SATA III is 600 MB/s.

Only SSDs can max out SATA II & III and only >=10k HDDs can max out SATA I. 

[HueyTheGod]

Nope. 

Normal 7200rpm HDDs can only push out about 120 MB/s maximum in the best of circumstances. Most of the time it'll be 40-70 MB/s at best. SATA I's theoretical max speed is 150 MB/s. SATA II is 300 and SATA III is 600 MB/s.

Only SSDs can max out SATA II & III and only >=10k HDDs can max out SATA I. 

Thanks! Do you know any about raid cards, or should I make a separate thread asking for help about that as well?

[Vitalius]

Thanks! Do you know any about raid cards, or should I make a separate thread asking for help about that as well?

I don't know much about RAID cards, but I know @alpenwasser and @wpirobotbuilder do among a few others (don't like to mention everyone at once). 

I only know I recommend against RAID 5 or any other parity RAID (6 or 7) when possible. I don't like it personally, but that may just be me. For WD Reds, it's fine since you actually get redundancy, but I don't like the performance hit that calculating parity causes. Some people would rather have more storage space and sacrifice performance (that's the trade off with parity RAID), but I'm willing to sacrifice space for RAID 1 or RAID 10. 

Besides, RAID should never be considered a backup unless it's exclusively on the backup machine. Like, if you have two copies of your data, one on your machine, and one on this light server, then that's fine, but if the only copy is on this server then it shouldn't be important data (i.e. movies and such). 

That's just sort of what I've learned as time has gone on. RAID 5-7 are only good for machines that don't need the performance. Only backup/redundancy.

So it just depends on your setup. 

[wpirobotbuilder]

I also have a question about RAID Controllers if you want to answer that here. Will I need as many SATA III sockets as I have hard drives, or does my raid controller manage all that depending on how many plugins are in the raid controller?

Yes, you will need at least that many, if not more. Though with RAID cards you get 4 SATA ports for every port on the card (8087, 8088, etc.) Depending on your performance needs, you will want to pick up a RAID card with built-in cache and battery backup unit.

 

There are lots server grade boards out there, but they tend to mostly be ATX or larger (SSI EEB). If performance isn't the first priority, this is a decent option with twelve onboard ports.

[alpenwasser]

As said, that Asrock board (either the 4 or 8 core) should do

pretty well for file storage.

Since it has enough SATA ports already, you won't need an additional

RAID card just to get more ports. You can of course still use one

if you want proper RAID, but personally I'd probably stick to the

M/B ports and use software RAID, for instance via ZFS or mdadm,

or whatever solution you prefer.

Alternatively, you could try looking for used server components on

eBay. Since performance isn't that crucial, it wouldn't matter if

the components aren't the newest anymore. However, it's probably

not that likely that you'll find something in mITX. But I've gone

this route for our server and am very happy with the result.

[HueyTheGod]

Yes, you will need at least that many, if not more. Though with RAID cards you get 4 SATA ports for every port on the card (8087, 8088, etc.) Depending on your performance needs, you will want to pick up a RAID card with built-in cache and battery backup unit.

 

There are lots server grade boards out there, but they tend to mostly be ATX or larger (SSI EEB). If performance isn't the first priority, this is a decent option with twelve onboard ports.

Ok, thanks for clearing this up. So a raid card add's 4 more SATA ports per SAS port? Or...

 

As said, that Asrock board (either the 4 or 8 core) should do

pretty well for file storage.

Since it has enough SATA ports already, you won't need an additional

RAID card just to get more ports. You can of course still use one

if you want proper RAID, but personally I'd probably stick to the

M/B ports and use software RAID, for instance via ZFS or mdadm,

or whatever solution you prefer.

Alternatively, you could try looking for used server components on

eBay. Since performance isn't that crucial, it wouldn't matter if

the components aren't the newest anymore. However, it's probably

not that likely that you'll find something in mITX. But I've gone

this route for our server and am very happy with the result.

Well, this is going to hold tons of data, both random movies and backups for important clients.

 

For the important clients, I will have the data on my PC, this NAS, and an external hard drive though. So backing up isn't much of an issue, this is just another way. I plan on going RAID 10 since I can have the benefits of raid 0 and raid 1.

[wpirobotbuilder]

Ok, thanks for clearing this up. So a raid card add's 4 more SATA ports per SAS port? Or...

The ports on RAID cards break out to 4 SATA or SAS ports if you plug in an SFF 8087 cable, like this one.

[HueyTheGod]

The ports on RAID cards break out to 4 SATA or SAS ports if you plug in an SFF 8087 cable, like this one.

Oh, ok! I will have to do more research on this

[wpirobotbuilder]

Oh, ok! I will have to do more research on this

I assist.

 

This raid card has two 8087 ports that you'd plug the aforementioned cables into. That would give you 8 SATA ports.

[HueyTheGod]

I assist.

 

This raid card has two 8087 ports that you'd plug the aforementioned cables into. That would give you 8 SATA ports.

Ok, I have been away for finals but I would not pay $500 for only 8 ports! I would for like, 15 if it is really worth it but I don't want super-enterprise datacenter grade stuff I suppose. 

[wpirobotbuilder]

I would not pay $500 for only 8 ports! I would for like, 15 if it is really worth it but I don't want super-enterprise datacenter grade stuff I suppose. 

That card isn't very high-end enterprise, far from it. You cannot get cards with 15 ports for $500 and expect them to be from a reputable manufacturer or to have good performance.

 

Good RAID controllers are not cheap. If you want decent performance out of your storage box, that card is the minimum I would recommend. You can get cheaper cards like the LSI 9211-8i, but it has terrible overall performance because there is no onboard cache to speed up random operations. Areca has some cards with lots of ports around the $500-600 mark, but they also lack onboard cache.

 

The only time lower-end cards are recommended is when they are used with a file system like ZFS, where you don't want any onboard features like cache and battery backup because the software will handle those things.

[HueyTheGod]

That card isn't very high-end enterprise, far from it. You cannot get cards with 15 ports for $500 and expect them to be from a reputable manufacturer or to have good performance.

 

Good RAID controllers are not cheap. If you want decent performance out of your storage box, that card is the minimum I would recommend. You can get cheaper cards like the LSI 9211-8i, but it has terrible overall performance because there is no onboard cache to speed up random operations. Areca has some cards with lots of ports around the $500-600 mark, but they also lack onboard cache.

 

The only time lower-end cards are recommended is when they are used with a file system like ZFS, where you don't want any onboard features like cache and battery backup because the software will handle those things.

I am not doing raid for speed as much as I am doing it for backups (raid 1, 5, or 10 is my plan) and large amounts of storage. 

[Vitalius]

I am not doing raid for speed as much as I am doing it for backups (raid 1, 5, or 10 is my plan) and large amounts of storage. 

Then I would rock on with ZFS or mdadm as alpen mentioned. 

[HueyTheGod]

Then I would rock on with ZFS or mdadm as alpen mentioned. 

Would using FreeNAS be fine then? To manage like 30Tb - 60Tb in drives?

[Vitalius]

Would using FreeNAS be fine then? To manage like 30Tb - 60Tb in drives?

If you are willing to buy around 64-92GB of ECC RAM, sure. That's about $700 btw, by itself. 

Note that ZFS will need that much regardless of what OS it's running on. Linux or FreeNAS.

You could buy 64GB of ECC RAM, that ASRock (or the SuperMicro version, which is a little bit cheaper but has less SATA ports) motherboard for $270, then whatever your drives are. 

Assuming twelve (12) 4TB WD Reds (48TB of storage), 64 GB of ECC RAM, and that motherboard, you are looking at about $3000-$3200 (depending on deals and which you go with) without tax or shipping.

[HueyTheGod]

If you are willing to buy around 64-92GB of ECC RAM, sure. That's about $700 btw, by itself. 

Note that ZFS will need that much regardless of what OS it's running on. Linux or FreeNAS.

You could buy 64GB of ECC RAM, that ASRock (or the SuperMicro version, which is a little bit cheaper but has less SATA ports) motherboard for $270, then whatever your drives are. 

Assuming twelve (12) 4TB WD Reds (48TB of storage), 64 GB of ECC RAM, and that motherboard, you are looking at about $3000-$3200 (depending on deals and which you go with) without tax or shipping.

Why would I need so much ram exactly? 

[Vitalius]

Why would I need so much ram exactly? 

Short answer: Because that's how ZFS works. 

Long answer: 

ZFS stores tables of your data in RAM as a sort of read cache. The general rule is that you should have 1GB of RAM for every TB of storage space. 48TB means a bare minimum of 52GB of RAM (4GB for the OS itself). And even then, you'd probably have crap performance because ZFS relies on RAM for performance like a PC relies on a GPU for performance to play a video game. Even if all you use it for is backups, backing that much data up at 1MB/s is going to suck hard, and you are probably going to outpace it in data creation (meaning you never stop backing up).

Then you have to consider if it's a media server, or torrent server, or wtv. Every plugin you add, you want around 1-2GB of RAM for it as well.

The reason I suggested 64GB is because it's evenly divisible by 8 (so you can get 8 or 16GB dimms) and it gives decent headroom for performance as well. 

You could use UFS in FreeNAS and then you'd only need 4GB of RAM (and it doesn't have to be ECC RAM either), but then you lose all of ZFS' epic features. And with 30-60TB of data, you probably want those features. 

One option you have is buying used. I bought a server off of Ebay for $250 and I got 2 Xeon quad core CPUs, a server motherboard (dual socket LGA 771), and 32GB of ECC DDR2 RAM. That's a cheap way to get lots of ECC RAM for FreeNAS' ZFS. Obviously only buy from Ebay trusted seller's.

The speed of DDR2 and the fact that most of those motherboards use FSB (a terrible way to connect RAM to CPU that can be a huge bottleneck) won't matter much as long as all you are doing is a few plugins and backups. Anything more intensive and it might matter. 

[HueyTheGod]

Short answer: Because that's how ZFS works. 

Long answer: 

ZFS stores tables of your data in RAM as a sort of read cache. The general rule is that you should have 1GB of RAM for every TB of storage space. 48TB means a bare minimum of 52GB of RAM (4GB for the OS itself). And even then, you'd probably have crap performance because ZFS relies on RAM for performance like a PC relies on a GPU for performance to play a video game. Even if all you use it for is backups, backing that much data up at 1MB/s is going to suck hard, and you are probably going to outpace it in data creation (meaning you never stop backing up).

Then you have to consider if it's a media server, or torrent server, or wtv. Every plugin you add, you want around 1-2GB of RAM for it as well.

The reason I suggested 64GB is because it's evenly divisible by 8 (so you can get 8 or 16GB dimms) and it gives decent headroom for performance as well. 

You could use UFS in FreeNAS and then you'd only need 4GB of RAM (and it doesn't have to be ECC RAM either), but then you lose all of ZFS' epic features. And with 30-60TB of data, you probably want those features. 

One option you have is buying used. I bought a server off of Ebay for $250 and I got 2 Xeon quad core CPUs, a server motherboard (dual socket LGA 771), and 32GB of ECC DDR2 RAM. That's a cheap way to get lots of ECC RAM for FreeNAS' ZFS. Obviously only buy from Ebay trusted seller's.

The speed of DDR2 and the fact that most of those motherboards use FSB (a terrible way to connect RAM to CPU that can be a huge bottleneck) won't matter much as long as all you are doing is a few plugins and backups. Anything more intensive and it might matter. 

I google'ed ZFS, but can't really find anything that makes it worth the extra $600 in ram. Why is ZFS so awesome? And why would I have 1mb/s... 

[Vitalius]

I google'ed ZFS, but can't really find anything that makes it worth the extra $600 in ram. Why is ZFS so awesome? And why would I have 1mb/s... 

Because your RAM is holding you back. 

ZFS has data integrity features that help it prevent your data from becoming corrupt. The reason you basically have to have ECC RAM with ZFS is because if you don't, and your RAM has an error (it's very rare, but it happens), all your data will be corrupted. Like putting it through a paper shredder effectively. 

ZFS enables you to do a lot of things. Without it, you can't use Plugins on FreeNAS (such as Plex Media Server, OwnCloud, Backula, etc). You can't use snapshots, replication, scrubs, or encryption. It has great potential, but also requires great care in setting it up or it can come back and bite you later. 

If it's not worth the money to you (remember, I mentioned you can buy used on Ebay for much cheaper hardware), I would just use Linux with mdadm. I don't know the requirements or features of that. @alpenwasser does though, I believe. 

UFS is simple and hard to get wrong, but it won't protect your data. If a drive dies, you are hosed (unless you use RAID 3 which is like RAID 5 but only supports 3 HDDs and no more than that). If your RAM has an error and corrupts some of your files, you are hosed. If your data decays over time due to magnetic fields and general environment damage (vibrations), you are hosed. 

Those are the types of things that ZFS protects against. It's preferred for long term active storage of large amounts of data. Active meaning it is accessed, used, and updated regularly. Unlike tapes which aren't.

Just remember, having software RAID means you don't have to pay for a RAID card which saves you quite a bit of money (since good RAID cards are usually expensive).

[HueyTheGod]

Because your RAM is holding you back. 

ZFS has data integrity features that help it prevent your data from becoming corrupt. The reason you basically have to have ECC RAM with ZFS is because if you don't, and your RAM has an error (it's very rare, but it happens), all your data will be corrupted. Like putting it through a paper shredder effectively. 

ZFS enables you to do a lot of things. Without it, you can't use Plugins on FreeNAS (such as Plex Media Server, OwnCloud, Backula, etc). You can't use snapshots, replication, scrubs, or encryption. It has great potential, but also requires great care in setting it up or it can come back and bite you later. 

If it's not worth the money to you (remember, I mentioned you can buy used on Ebay for much cheaper hardware), I would just use Linux with mdadm. I don't know the requirements or features of that. @alpenwasser does though, I believe. 

UFS is simple and hard to get wrong, but it won't protect your data. If a drive dies, you are hosed (unless you use RAID 3 which is like RAID 5 but only supports 3 HDDs and no more than that). If your RAM has an error and corrupts some of your files, you are hosed. If your data decays over time due to magnetic fields and general environment damage (vibrations), you are hosed. 

Those are the types of things that ZFS protects against. It's preferred for long term active storage of large amounts of data. Active meaning it is accessed, used, and updated regularly. Unlike tapes which aren't.

Just remember, having software RAID means you don't have to pay for a RAID card which saves you quite a bit of money (since good RAID cards are usually expensive).

Yeah I am not doing any of that.... I would rather just throw $600 at a raid card.