Freenas Server PC Help

[waraich]

I am thinking of building a freenas server however i am running in to some trouble choosing the right parts,

I currently have the following parts:

 

i5-3540

120Gb SSD (using as cache)

2 x 2tb western digital green drives (I am aware that there no ideal for 24/7 operations)

intel PRO/1000 MT Gigabit Dual Port Server Adapter PCIZ PCI-X NIC Network Card

16 GB DDR3 Ram (not ECC)

 

besides from the stuff I currently have I am considering to get the following:

 

2 x intel xeon quad core E5450 (depends on whether i will be more effective to use this instead of i5-4570)

Dell powerconnect 5324 24-Port Gigabit Ethernet fully managed switch (can obtain one for fairly cheap)

Intel motherboard BB5000XALR ( i will get this if i go for the dual xeon)

intel z79 board (for the i5, if i decide to go down that route)

Power supply ( if anyone could recommendation on a efficient power supply)

Raid card (if needed)

 

So i have a few questions:

1)i was wondering would it be better to go for dual xeon instead of the i5 as i can use the i5 for another build and as xeon processors are designed for server use?

2)If i using an ZFS file system should i out more two drives in raid and if so could you recommend a decently priced raid card?

3)I was watching linus`s home server video and he talked about something called link aggregation so i was wondering someone could shed some light on that matter?

4) I don't have error-correcting memory (ECC), would it make a massive difference to use standard ddr3 ram that i already have.

 

 

 

 

 

[Vitalius]

 

1)i was wondering would it be better to go for dual xeon instead of the i5 as i can use the i5 for another build and as xeon processors are designed for server use?

2)If i using an ZFS file system should i out more two drives in raid and if so could you recommend a decently priced raid card?

3)I was watching linus`s home server video and he talked about something called link aggregation so i was wondering someone could shed some light on that matter?

4) I don't have error-correcting memory (ECC), would it make a massive difference to use standard ddr3 ram that i already have.

1. Between those two, the i5-3540, although I consider that somewhat overkill for just a FreeNAS machine (unless it's doing some other things on top of file storage). See my recommendation at the end of this post. 
2. It entirely depends which RAID you mean. If you mean RAID 10, then sure, buy 2 more HDDs. If you mean RAID 5 or 6 (Z1 and Z2 in FreeNAS respectively), then no. That is a bad idea. Don't get a RAID card. They are pointless if you are using ZFS. ZFS has software RAID that is as good, if not better, than any reasonably priced RAID card.
3. Link Aggregation is basically taking multiple ethernet ports and making them work together. Other terms are "teaming", "bridging", although they can mean slightly different things. Link Aggregation basically would mean that 2 or more ethernet ports would look like a single connection and have the bandwidth of however many ethernet ports you have.
4. Yes. You wouldn't be able to safely use some of ZFS' special features (what makes it special) like scrubbing. In fact, if something does go terribly wrong with that RAM you currently have, it could very well corrupt all of your data. Or at least it would corrupt your data until you realized something was wrong. It is a very bad idea to use ZFS with non-ECC for this reason. You can, but it's not safe and if you care about your data at all, you won't do it. 

About your hardware considerations...

I don't recommend using the CPU and RAM you already own personally. Save that for some other build (pfSense, or a VM host or something maybe). 

This motherboard is very powerful. I recommend it for a FreeNAS build because it is very good for what it is but only for people who will be doing a lot. Then grab 8GB of ECC RAM (1x8GB is fine imo rather than 2x4GB), and just rock on. You don't need a RAID card if you plan to use FreeNAS. If you do get 2 more HDDs, then go for 16GB of ECC RAM. 

If you don't need that much power (and it is a lot), look at ShadowCaptain's post. FreeNAS with ZFS can be ran on a very weak CPU as long as you don't plan on doing a lot. Just remember: RAM capacity and being ECC are most important to ZFS performance and data integrity. More and ECC is always preferred, ECC basically being required.

[ShadowCaptain]

I am thinking of building a freenas server

 

A) what do you want this server to do? What types of files/applications? How big is your network? Gigabit ethernet? etc etc

 

Need to know more before we can recommend components. My home server runs fine on a crappy G3220 Pentium and I stream movies and music with plex, do nightly backups to 5 computers, even have my steam games installed mapped  here to access from any pc in the house.

 

[waraich]

 

1. Between those two, the i5-3540, although I consider that somewhat overkill for just a FreeNAS machine (unless it's doing some other things on top of file storage). See my recommendation at the end of this post. 
2. It entirely depends which RAID you mean. If you mean RAID 10, then sure, buy 2 more HDDs. If you mean RAID 5 or 6 (Z1 and Z2 in FreeNAS respectively), then no. That is a bad idea. Don't get a RAID card. They are pointless if you are using ZFS. ZFS has software RAID that is as good, if not better, than any reasonably priced RAID card.
3. Link Aggregation is basically taking multiple ethernet ports and making them work together. Other terms are "teaming", "bridging", although they can mean slightly different things. Link Aggregation basically would mean that 2 or more ethernet ports would look like a single connection and have the bandwidth of however many ethernet ports you have.
4. Yes. You wouldn't be able to safely use some of ZFS' special features (what makes it special) like scrubbing. In fact, if something does go terribly wrong with that RAM you currently have, it could very well corrupt all of your data. Or at least it would corrupt your data until you realized something was wrong. It is a very bad idea to use ZFS with non-ECC for this reason. You can, but it's not safe and if you care about your data at all, you won't do it. 

About your hardware considerations...

I don't recommend using the CPU and RAM you already own personally. Save that for some other build (pfSense, or a VM host or something maybe). 

This motherboard is very powerful. I recommend it for a FreeNAS build because it is very good for what it is but only for people who will be doing a lot. Then grab 8GB of ECC RAM (1x8GB is fine imo rather than 2x4GB), and just rock on. You don't need a RAID card if you plan to use FreeNAS. If you do get 2 more HDDs, then go for 16GB of ECC RAM. 

If you don't need that much power (and it is a lot), look at ShadowCaptain's post. FreeNAS with ZFS can be ran on a very weak CPU as long as you don't plan on doing a lot. Just remember: RAM capacity and being ECC are most important to ZFS performance and data integrity. More and ECC is always preferred, ECC basically being required.

 

 

Thanks for the detail reply,

The motherboard that you recommended is more expensive then the dual xeon and the dual socket motherboard, so would you still recommend it?

Also i will be running the owncloud plugin and several other plugins, so do you still believe the motherboard is powerful enough ?

 

 

A) what do you want this server to do? What types of files/applications? How big is your network? Gigabit ethernet? etc etc

 

Need to know more before we can recommend components. My home server runs fine on a crappy G3220 Pentium and I stream movies and music with plex, do nightly backups to 5 computers, even have my steam games installed mapped  here to access from any pc in the house.

 

 

Thanks for the reply,

There are 5 computers that will be connected via gigabit Ethernet and i will be running the owncloud plugin as well as several other freenas plugins.

[Vitalius]

Thanks for the detail reply,

The motherboard that you recommended is more expensive then the dual xeon and the dual socket motherboard, so would you still recommend it?

Also i will be running the owncloud plugin and several other plugins, so do you still believe the motherboard is powerful enough ?

Ah, then that does change things a little. 

Actually, I just realized that's what I have. Not even kidding, I bought dual Xeons (although I got the X5472's as they were the same price at the time) with a similar motherboard for my FreeNAS machine primarily because it was the cheapest way to get a lot of ECC RAM for very cheap. I got 32GB of it along with the dual xeons and the motherboard for around the price of that motherboard I listed. 

Buying used has it's qualms (there are multiple types of ECC RAM and I bought the wrong ones originally, be careful of that). 

Do note that these CPUs & their platform are horribly inefficient, and the RAM sticks you need for them use 10 watts each at idle. I'm not kidding. I have 8 of them, so those alone use roughly 80w at idle and 120-150w when under load. It's crazy. That's not even considering the dual CPUs.

Another thing to consider is that those CPUs are on a platform that has a FSB (Front Side Bus). It is a huge bottleneck for performance between the CPUs and the RAM sticks. Having dual CPUs and Quad Channel RAM means nothing because of it. That's why DDR3 platforms don't use them. 

Don't get me wrong, I get good performance with my server, but this is just something to consider with how many plugins you intend to use and how much they will be doing (Plex uses a looot of resources to transcode media on the fly). 

All-in-all, I would do what you intended to to begin with. Get Dual Xeons with some cheap ECC RAM and a motherboard for less than that M-ITX Server motherboard with embedded CPU. Just only buy from Ebay trusted sellers and make sure you get the correct ECC RAM. There is Fully Buffered and Unbuffered. They are different, and it depends on the motherboard I think (not sure, it might be the CPUs). 

[waraich]

Ah, then that does change things a little. 

Actually, I just realized that's what I have. Not even kidding, I bought dual Xeons (although I got the X5472's as they were the same price at the time) with a similar motherboard for my FreeNAS machine primarily because it was the cheapest way to get a lot of ECC RAM for very cheap. I got 32GB of it along with the dual xeons and the motherboard for around the price of that motherboard I listed. 

Buying used has it's qualms (there are multiple types of ECC RAM and I bought the wrong ones originally, be careful of that). 

Do note that these CPUs & their platform are horribly inefficient, and the RAM sticks you need for them use 10 watts each at idle. I'm not kidding. I have 8 of them, so those alone use roughly 80w at idle and 120-150w when under load. It's crazy. That's not even considering the dual CPUs.

Another thing to consider is that those CPUs are on a platform that has a FSB (Front Side Bus). It is a huge bottleneck for performance between the CPUs and the RAM sticks. Having dual CPUs and Quad Channel RAM means nothing because of it. That's why DDR3 platforms don't use them. 

Don't get me wrong, I get good performance with my server, but this is just something to consider with how many plugins you intend to use and how much they will be doing (Plex uses a looot of resources to transcode media on the fly). 

All-in-all, I would do what you intended to to begin with. Get Dual Xeons with some cheap ECC RAM and a motherboard for less than that M-ITX Server motherboard with embedded CPU. Just only buy from Ebay trusted sellers and make sure you get the correct ECC RAM. There is Fully Buffered and Unbuffered. They are different, and it depends on the motherboard I think (not sure, it might be the CPUs).

Thanks again for the very detailed reply,

What would be your ideal freenas hardware?

You say they are inefficient but I are they still powerful enough for running a server also isn't it odd that they are made for server use yet still are very inefficient.

Also, for link aggregation should go with the switch that I have listed and if you don't mind telling me what's the advantages of a switch of this grade?

Again thank you for help so far you have cleared a lot up for me.

[Vitalius]

Thanks again for the very detailed reply,

What would be your ideal freenas hardware?

You say they are inefficient but I are they still powerful enough for running a server also isn't it odd that they are made for server use yet still are very inefficient.

Also, for link aggregation should go with the switch that I have listed and if you don't mind telling me what's the advantages of a switch of this grade?

Again thank you for help so far you have cleared a lot up for me.

You're welcome. I try.

To be honest, if we just ignore price completely, it's that M-ITX ASRock motherboard with embedded CPU. It's very powerful, efficient, feature packed, and small. The only downside is that it is expensive (but it's not really expensive compared to current server hardware). 

Well, that's the thing. The CPU you are looking at was released in 2007. 7 years old. They didn't care about efficiency then. They cared about power. 

I don't know anything about what makes a good switch other than if it's Gigabit, from a reputable brand, and it has useful features. That one is from Dell, has some nice features, and is Gigabit, so I figure it's a pretty good switch. 

Glad to help.

[wpirobotbuilder]

Thanks again for the very detailed reply,

What would be your ideal freenas hardware?

You say they are inefficient but I are they still powerful enough for running a server also isn't it odd that they are made for server use yet still are very inefficient.

The CPU/Mobo all-in-one he recommended earlier is pretty much ideal in terms of the price-to-functionality metrics. You get ECC support and enough CPU power to do most tasks required of a file server for a decent price and with minimal power consumption.

 

More CPU power and more memory is better, but will only really matter in certain circumstances. I have a dual Xeon (12 cores total) FreeNAS setup with 80GB of ECC memory. With this setup it is very unlikely that I will ever have slow performance because of all the caching going on, and I can use intensive compression and deduplication without any performance hit.

 

Such a system would ideally be in an enterprise environment, where dozens of machines access shared storage. For simple sharing of files and transcoding, the one Vitalius suggested is plenty.

[waraich]

You're welcome. I try.

To be honest, if we just ignore price completely, it's that M-ITX ASRock motherboard with embedded CPU. It's very powerful, efficient, feature packed, and small. The only downside is that it is expensive (but it's not really expensive compared to current server hardware). 

Well, that's the thing. The CPU you are looking at was released in 2007. 7 years old. They didn't care about efficiency then. They cared about power. 

I don't know anything about what makes a good switch other than if it's Gigabit, from a reputable brand, and it has useful features. That one is from Dell, has some nice features, and is Gigabit, so I figure it's a pretty good switch. 

Glad to help.

I will probably end up going with the motherboard you recommended.

I was wondering what kind of performance your getting on you server and also how expensive is it to run?

[waraich]

The CPU/Mobo all-in-one he recommended earlier is pretty much ideal in terms of the price-to-functionality metrics. You get ECC support and enough CPU power to do most tasks required of a file server for a decent price and with minimal power consumption.

 

More CPU power and more memory is better, but will only really matter in certain circumstances. I have a dual Xeon (12 cores total) FreeNAS setup with 80GB of ECC memory. With this setup it is very unlikely that I will ever have slow performance because of all the caching going on, and I can use intensive compression and deduplication without any performance hit.

 

Such a system would ideally be in an enterprise environment, where dozens of machines access shared storage. For simple sharing of files and transcoding, the one Vitalius suggested is plenty.

Thank for reply,

I will probably end up going with motherboard recommend but there is something awesome about running a dual xeon system.

Also, I will have plex media plugin as well as the owncloud plugin installed. I will be port forwarding for the owncloud so would still be powerful enough.

[wpirobotbuilder]

I will probably end up going with motherboard recommend but there is something awesome about running a dual xeon system.

It is kind of cool, but server hardware in general can be difficult to work with.

 

Also, your switch is decent. If you plan to use those SFP+ ports I would contact Dell support to see if they only support switch-to-switch connectivity. On some switches you can wire a 10GbE device to those ports as a backbone. I'm looking at doing something similar with a 10G SFP network card.

[waraich]

It is kind of cool, but server hardware in general can be difficult to work with.

 

Also, your switch is decent. If you plan to use those SFP+ ports I would contact Dell support to see if they only support switch-to-switch connectivity. On some switches you can wire a 10GbE device to those ports as a backbone. I'm looking at doing something similar with a 10G SFP network card.

Thanks for the advice,

I don't have any 10GBe device or at least I have no device with a 10GBe port.

if you don't mind could you help me out with what you mean by backbone in this context. You can point to a link where I can read up about it you prefer instead of me making you do all the research.

[wpirobotbuilder]

Thanks for the advice,

I don't have any 10GBe device or at least I have no device with a 10GBe port.

if you don't mind could you help me out with what you mean by backbone in this context. You can point to a link where I can read up about it you prefer instead of me making you do all the research.

I mean that (theoretically) ten gigabit devices could concurrently access that 10GbE enabled device. If the switch supports plugging something other than a switch into that SFP port, then you could have that setup.

 

Basically that one port is much faster than the others.

[waraich]

I mean that (theoretically) ten gigabit devices could concurrently access that 10GbE enabled device. If the switch supports plugging something other than a switch into that SFP port, then you could have that setup.

Basically that one port is much faster than the others.

Thanks, that really clears it up and thank you for the advice

[Vitalius]

I will probably end up going with the motherboard you recommended.

I was wondering what kind of performance your getting on you server and also how expensive is it to run?

Well, when I'm moving files to FreeNAS' cache, I get around 90MB/s. When I start moving them to the actual HDD, once the cache is full, it drops to 5-10MB/s. This is because the HDD I got is a WD Green 4TB drive. Bad choice imo for myself. I think the drive may be bad as well (in terms of performance, not "it's dying"). 

Not sure. I haven't checked it. I'll do so at some point and let you know. Just remember, electricity prices are different everywhere. I live in Texas where electricity is notoriously (imo) cheap, so it's viable for me. It might not be elsewhere (EU for example). 

[madcow]

...

Dell powerconnect 5324 24-Port Gigabit Ethernet fully managed switch (can obtain one for fairly cheap)

...

Is noise of these devices a concern to you at all? That switch could be *very* loud. I got the 5424 but it was so loud that I ate the cost and replaced it with something else. I'm guessing the 5324 is similar.

 

For reference I have two 4U boxes each with 5 fans at 5-6K RPM and the switch could be heard over these fans. Also if you plan to swap out the fans in the switch, you should do some research on compatible fans. The fan controller, at least on my unit was unreasonably picky and would not start the switch if the fans were missing or had been replaced with lower powered units.

[waraich]

Is noise of these devices a concern to you at all? That switch could be *very* loud. I got the 5424 but it was so loud that I ate the cost and replaced it with something else. I'm guessing the 5324 is similar.

 

For reference I have two 4U boxes each with 5 fans at 5-6K RPM and the switch could be heard over these fans. Also if you plan to swap out the fans in the switch, you should do some research on compatible fans. The fan controller, at least on my unit was unreasonably picky and would not start the switch if the fans were missing or had been replaced with lower powered units.

 

Ahh thanks for the heads up, i was not expecting the switch to be loud. So what did you replace it with?

[waraich]

Well, when I'm moving files to FreeNAS' cache, I get around 90MB/s. When I start moving them to the actual HDD, once the cache is full, it drops to 5-10MB/s. This is because the HDD I got is a WD Green 4TB drive. Bad choice imo for myself. I think the drive may be bad as well (in terms of performance, not "it's dying"). 

Not sure. I haven't checked it. I'll do so at some point and let you know. Just remember, electricity prices are different everywhere. I live in Texas where electricity is notoriously (imo) cheap, so it's viable for me. It might not be elsewhere (EU for example). 

 

I mean that (theoretically) ten gigabit devices could concurrently access that 10GbE enabled device. If the switch supports plugging something other than a switch into that SFP port, then you could have that setup.

 

Basically that one port is much faster than the others.

 

Thanks for the help guys,

 

I have decided to go with dual xeon x5560 as I am able to get a pair of them for around $260. I am ware they are a few generations old but i think they are still quite powerful and i can use them for other tasks later on if need be.

I am just struggling to find a good dual socket lga 1366 except for the evga sr-2, which however is out of stock.

I am also going grab some 16gb ecc ram and get a decently efficient power-supply.

 

If think i am making the wrong decision, please let me know.

[Vitalius]

Thanks for the help guys,

 

I have decided to go with dual xeon x5560 as I am able to get a pair of them for around $260. I am ware they are a few generations old but i think they are still quite powerful and i can use them for other tasks later on if need be.

I am just struggling to find a good dual socket lga 1366 except for the evga sr-2, which however is out of stock.

I am also going grab some 16gb ecc ram and get a decently efficient power-supply.

 

If think i am making the wrong decision, please let me know.

Monetarily, maybe. You could get the CPUs, motherboard, and 32GB RAM for $260-$300 off ebay if you went with LGA 771 Xeons. Yes, they're older, and less efficient, but two of them are probably more than you will ever fully utilize. 

Otherwise, not really. It will be powerful and perform relatively well. Being DDR3 and not having an FSB is a major plus though, so it might be worth the extra money. 

Honestly, the prices on used parts makes it a hard choice. I think you'll be fine with whatever you go with as long as it's server hardware (i.e. xeons).

[wpirobotbuilder]

Thanks for the help guys,

 

I have decided to go with dual xeon x5560 as I am able to get a pair of them for around $260. I am ware they are a few generations old but i think they are still quite powerful and i can use them for other tasks later on if need be.

I am just struggling to find a good dual socket lga 1366 except for the evga sr-2, which however is out of stock.

I am also going grab some 16gb ecc ram and get a decently efficient power-supply.

 

If think i am making the wrong decision, please let me know.

You'll be hard-pressed to find anything except for a server 1366 board. Supermicro is probably the one you'll find the most support from the enthusiast community on, and they make good boards. What's more, many of them are still available on Newegg for sale and come with warranties and official support, which is a good thing if you're using lots of used hardware.

 

Make sure your memory is compatible with the board you choose using Kingston's aforementioned memory search.

[madcow]

Ahh thanks for the heads up, i was not expecting the switch to be loud. So what did you replace it with?

I ended up going with the Cisco SG300-10.