Tips for my new NAS

[Grande_Forti]

Hello Everyone,

so, i am planning to build a new NAS for my personal use and install FreeNAS on it.

I've already built some computers, but i have no experience building a NAS or with FreeNAS itself, so i decided to ask here, cause you guys are certainly more experienced that i am and i hope that you can help me make the right decisions.

I'll use this NAS mainly for streaming movies with Plex and  to store files and other stuff. 

Here is the components list, i know it may looks overkill, and maybe it is, i'll explain why i chose each component so that you can tell me is it is appropriate or not.

Thanks in advance!

 

All the parts are from Amazon IT

• CPU: 1x Intel Xeon E3-1245v5 Skylake @ 3.5GHz
• Motherboard: 1x Asus P10S WS
• RAM: 2x Kingston KVR21R15D4/16 16GB ECC @ 2133MHz653$
• OS Drive: 1x Transcend MTS600 M.2 SSD 64GB
• Data Drives: 5x WD Red 3TB WD30EFRX
• PSU: 1x Corsair CS550M 550 Watt
• Case: 1x Fractal Design Define R5

I was also planning to buy a Samsung 850 EVO SSD 120GB to install jails, but i'm not completely sure about it.

I chose that CPU cause i need to stream at least 3 simultaneous 1080p movies, and since transcoding is very CPU intensive, i went with that. Also i want scrubbing to be extremely fast, and that should help a lot.

I needed at least two ethernet ports cause i wanna use different ports for the file sharing, and the Plex Server, so that when i'm streaming movies i am also able to transfer files at full speed since i still have 1Gb of bandwidth (on another computer of course). That motherboard has built-in dual Gigabit Ethernet, so it was a big plus, it is a Workstation Motherboard, so it's certainly more reliable than a 70€ motherborad and it has the 1151 socket to support my Skylake CPU.

FreeNAS and RaidZ likes ram, a lot, that's why i went with 32GB, 16 would be enough, but i couldn't find two 8GB DDR4 ECC sticks, and since i wanted the ram to run in dual-channel, i ended up with that.

Now there's the part where i get really unsure, first of all, about the OS Drive, FreeNAS recommend to install it on a USB, i didn't find it much reliable so i opted for an M.2 SSD, but i've never used them, i don't even know Transcend really much, i don't know even if it is a good choice to run FreeNAS of of that, so i'm really open to suggestions.

On the Data Drives, even here i'm not really sure of what i have done, i want as much space as i can get within that price range, which is about 570€ (652 USD), but i want safe storage as well, and my idea is to use RaidZ2, so that two drives can fail before i lose all my data,  but i've never worked with RAID or the ZFS filesystem, so i don't know, maybe less drive but with more capacity will be better, i just don't know ahahah, so please, help!

Case and PSU were an easy choice, the Define R5 is just awesome, and it has the space to put all my drives, and the PUS, well, maybe it's a little overkill, i'm not sure, but it is 80 Plus Gold, and i wanted a reliable power supply.

 

Aand, thank you for reading that far, it you have any kind of suggestion or anything i'll be really happy to know it.

Greetings.

Filippo

 

[KaasKop]

Do not dedicate an entire 64gb drive to FreeNAS, run it off of an USB3.0 stick and plug it into an USB 3.0 port.

 

FreeNAS will NOT use any space on that drive for anything besides itself, so do what they tell you and run it off of an USB.

I've placed a freenas in a semi-big company (20-50 employees) and I dont feel any lag or hiccups.

 

Also install FreeNAS in an VM before using it to get an feeling of how to use it.

[79wjd]

A simple i3 can easily handle three 1080p streams, although with 4k coming around, it may still be worthwhile to just go with a Xeon (especially when you consider that it's not that much more in the end). Although, I would personally go for a Xeon E3 1230v5 instead since the only difference is that the 1245v5 has integrated graphics (which you don't actually need as a lot of boards have built in onboard graphics). I would personally swap the board for something from Supermicro since you'll likely find that they have cheaper (but equally good) boards. 

 

FreeNAS 9.x.x does NOT support XHCI but only legacy EHCI. With Skylake, Intel removed EHCI support and just left XHCI support. So If you plan on running FreeNAS 9.x.x you can't actually use a USB for the OS (Haswell still can use a USB), and when FreeNAS 10.x.x eventually comes out you will also be able to use a USB for the OS. You don't actually write to the drive much, so USBs are plenty reliable (I currently have my FreeNAS server booting off of two mirror'd 8GB flash drives). So if you want to go Skylake you'll need an HDD/SSD (or a SATADOM) to handle booting, any are all equally good....I would just recommend getting the cheapest SATA SSD you can find (that's not garbage) and using that for the boot drive (if you want to stick with Skylake).

 

The drives are solid. 

 

Keep in mind, that the CS series isn't all that reliable. AND the higher efficiency will only save you like $5/year at most. So a higher end 80+ bronze unit would be a better idea. My PSU recommendation based on current prices: http://it.pcpartpicker.com/part/xfx-power-supply-p1450sx2b9

 

You also don't need a drive just for jails unless you're doing something super crazy (which you're not). You can just put your jail on your normal RAIDZ2 array. 

 

 

I would personally go with Haswell simply because there's a pretty big price difference between the two platforms.

 

PCPartPicker part list / Price breakdown by merchant

CPU: Intel Xeon E3-1230 V3 3.3GHz Quad-Core Processor  (€258.90 @ Amazon Italia) 
Motherboard: ASRock E3C224-V+ ATX LGA1150 Motherboard  (€175.61 @ Amazon Italia) 
Memory: Kingston 32GB (4 x 8GB) DDR3-1600 Memory  (€188.65 @ Amazon Italia) 
Storage: Western Digital Red 3TB 3.5" 5400RPM Internal Hard Drive  (€114.80 @ Amazon Italia) 
Storage: Western Digital Red 3TB 3.5" 5400RPM Internal Hard Drive  (€114.80 @ Amazon Italia) 
Storage: Western Digital Red 3TB 3.5" 5400RPM Internal Hard Drive  (€114.80 @ Amazon Italia) 
Storage: Western Digital Red 3TB 3.5" 5400RPM Internal Hard Drive  (€114.80 @ Amazon Italia) 
Storage: Western Digital Red 3TB 3.5" 5400RPM Internal Hard Drive  (€114.80 @ Amazon Italia) 
Case: Fractal Design Define R4 (Black Pearl) ATX Mid Tower Case  (€87.27 @ Amazon Italia) 
Power Supply: XFX ProSeries 450W 80+ Bronze Certified ATX Power Supply  (€56.18 @ Amazon Italia) 
Total: €1340.61
Prices include shipping, taxes, and discounts when available
Generated by PCPartPicker 2016-05-01 21:45 CEST+0200

[Grande_Forti]

19 hours ago, KaasKop said:

Do not dedicate an entire 64gb drive to FreeNAS, run it off of an USB3.0 stick and plug it into an USB 3.0 port.

 

FreeNAS will NOT use any space on that drive for anything besides itself, so do what they tell you and run it off of an USB.

I've placed a freenas in a semi-big company (20-50 employees) and I dont feel any lag or hiccups.

 

Also install FreeNAS in an VM before using it to get an feeling of how to use it.

Yes, i know FreeNAS uses all the drive space for itself, i was just unsure about it speed and reliability. From what you've said i'd go straight with an USB, but from what djdwosk97 said i simply can't if i wanna use Skylake. And yeah, i've put freenas on a VM and used for a while, that's what made me chose it, i like it a lot and it's great and simple for what i'm doing. Thanks a lot for answering.

19 hours ago, djdwosk97 said:

A simple i3 can easily handle three 1080p streams, although with 4k coming around, it may still be worthwhile to just go with a Xeon (especially when you consider that it's not that much more in the end). Although, I would personally go for a Xeon E3 1230v5 instead since the only difference is that the 1245v5 has integrated graphics (which you don't actually need as a lot of boards have built in onboard graphics). I would personally swap the board for something from Supermicro since you'll likely find that they have cheaper (but equally good) boards. 

 

FreeNAS 9.x.x does NOT support XHCI but only legacy EHCI. With Skylake, Intel removed EHCI support and just left XHCI support. So If you plan on running FreeNAS 9.x.x you can't actually use a USB for the OS (Haswell still can use a USB), and when FreeNAS 10.x.x eventually comes out you will also be able to use a USB for the OS. You don't actually write to the drive much, so USBs are plenty reliable (I currently have my FreeNAS server booting off of two mirror'd 8GB flash drives). So if you want to go Skylake you'll need an HDD/SSD (or a SATADOM) to handle booting, any are all equally good....I would just recommend getting the cheapest SATA SSD you can find (that's not garbage) and using that for the boot drive (if you want to stick with Skylake).

 

The drives are solid. 

 

Keep in mind, that the CS series isn't all that reliable. AND the higher efficiency will only save you like $5/year at most. So a higher end 80+ bronze unit would be a better idea. My PSU recommendation based on current prices: http://it.pcpartpicker.com/part/xfx-power-supply-p1450sx2b9

 

You also don't need a drive just for jails unless you're doing something super crazy (which you're not). You can just put your jail on your normal RAIDZ2 array. 

 

 

I would personally go with Haswell simply because there's a pretty big price difference between the two platforms.

 

PCPartPicker part list / Price breakdown by merchant

CPU: Intel Xeon E3-1230 V3 3.3GHz Quad-Core Processor  (€258.90 @ Amazon Italia) 
Motherboard: ASRock E3C224-V+ ATX LGA1150 Motherboard  (€175.61 @ Amazon Italia) 
Memory: Kingston 32GB (4 x 8GB) DDR3-1600 Memory  (€188.65 @ Amazon Italia) 
Storage: Western Digital Red 3TB 3.5" 5400RPM Internal Hard Drive  (€114.80 @ Amazon Italia) 
Storage: Western Digital Red 3TB 3.5" 5400RPM Internal Hard Drive  (€114.80 @ Amazon Italia) 
Storage: Western Digital Red 3TB 3.5" 5400RPM Internal Hard Drive  (€114.80 @ Amazon Italia) 
Storage: Western Digital Red 3TB 3.5" 5400RPM Internal Hard Drive  (€114.80 @ Amazon Italia) 
Storage: Western Digital Red 3TB 3.5" 5400RPM Internal Hard Drive  (€114.80 @ Amazon Italia) 
Case: Fractal Design Define R4 (Black Pearl) ATX Mid Tower Case  (€87.27 @ Amazon Italia) 
Power Supply: XFX ProSeries 450W 80+ Bronze Certified ATX Power Supply  (€56.18 @ Amazon Italia) 
Total: €1340.61
Prices include shipping, taxes, and discounts when available
Generated by PCPartPicker 2016-05-01 21:45 CEST+0200

Thank you really much for the time you spent to answer this. And you configuration is really great, it's 200€ less than what i did, but it's still badass. I might go with Haswell then, i would save up a lot of money and still get great performance. I stil have one small question, so putting five 3TB drives in a RaidZ2 array is a good thing? Or maybe it would be better to get four 4TB drives? 

[79wjd]

52 minutes ago, Grande_Forti said:

Yes, i know FreeNAS uses all the drive space for itself, i was just unsure about it speed and reliability. From what you've said i'd go straight with an USB, but from what djdwosk97 said i simply can't if i wanna use Skylake. And yeah, i've put freenas on a VM and used for a while, that's what made me chose it, i like it a lot and it's great and simple for what i'm doing. Thanks a lot for answering.

Thank you really much for the time you spent to answer this. And you configuration is really great, it's 200€ less than what i did, but it's still badass. I might go with Haswell then, i would save up a lot of money and still get great performance. I stil have one small question, so putting five 3TB drives in a RaidZ2 array is a good thing? Or maybe it would be better to get four 4TB drives? 

Well...3tb drives (not reds afaik) seem to have a higher failure rate for some odd reason, but if there was a major problem with reds it would be widely publicized by now, so it shouldn't be an issue. I would honestly want RAIDZ2 since I'd be worried about hitting a URE and having the array fail to rebuild on such a large Z1 array. So I would either do five 3tb drives or five 4tb drives. 

 

I recently upgraded my NAS to ivy bridge (over haswell or skylake) because I was able to get it pretty cheap (1230v2, supermicro board, and 32gb of ECC for $350) and honestly, there really isn't much of a compelling reason to pay more for anything past sandy bride. 

[WaterproofBeanie]

On 2/5/2016 at 1:05 AM, Grande_Forti said:

Yes, i know FreeNAS uses all the drive space for itself, i was just unsure about it speed and reliability. From what you've said i'd go straight with an USB, but from what djdwosk97 said i simply can't if i wanna use Skylake. And yeah, i've put freenas on a VM and used for a while, that's what made me chose it, i like it a lot and it's great and simple for what i'm doing. Thanks a lot for answering.

Thank you really much for the time you spent to answer this. And you configuration is really great, it's 200€ less than what i did, but it's still badass. I might go with Haswell then, i would save up a lot of money and still get great performance. I stil have one small question, so putting five 3TB drives in a RaidZ2 array is a good thing? Or maybe it would be better to get four 4TB drives? 

In my opinion if data integrity matters to you all your drives should be in mirrored pair vdevs. Easier to upgrade, easier to maintain, faster and faster to resilver with much less strain than RAIDZ.

I would go for 4 4tb drives in 2 mirrored vdevs, which are then combined in effectively a software RAID10.

[79wjd]

49 minutes ago, ShaolinBear said:

In my opinion if data integrity matters to you all your drives should be in mirrored pair vdevs. Easier to upgrade, easier to maintain, faster and faster to resilver with much less strain than RAIDZ.

I would go for 4 4tb drives in 2 mirrored vdevs, which are then combined in effectively a software RAID10.

I'd still personally take RAIDZ2 over mirror'd vdevs. In that RAID10 config if you lose the wrong two drives you're screwed, whereas in RAIDZ2 you can lose any two drives and be fine. 

[Mikensan]

Just to add on - you mentioned splitting traffic between two NICs - 3 transcoding streams + file moves are going to stress out the array (iops) faster than you capping out 2gbps. I second @djdwosk97 - solid build. 

 

Those reds are 5400rpm - if budget allows (with the above change in parts from djd it should) then maybe look at HGST's NAS drives (0S03660). $US about $14 per drive difference, and well worth the 5400rpm to 7200rpm and rock solid brand. You'll get higher IOPS out of them at least.

 

I'm an advocate for raid 10, sacrificing space for iops. True, in a worse-case scenario if you lose both drives in one of the mirrors then you're screwed. However with backups and all the other advantages (ability to expand the array, higher best-case scenario # of drive failures, increased IOPS, significantly faster resilver and less impact) I think 10 is the way to go. If I could redo my array I'd switch to 10.

 

With Raidz2, minimum of 4 drives and should be dividable by 2 for best performance. My 4x 3TB RaidZ2 WD reds, is 9TB usable in case you were curious.

[79wjd]

29 minutes ago, Mikensan said:

Just to add on - you mentioned splitting traffic between two NICs - 3 transcoding streams + file moves are going to stress out the array (iops) faster than you capping out 2gbps. I second @djdwosk97 - solid build. 

 

Those reds are 5400rpm - if budget allows (with the above change in parts from djd it should) then maybe look at HGST's NAS drives (0S03660). $US about $14 per drive difference, and well worth the 5400rpm to 7200rpm and rock solid brand. You'll get higher IOPS out of them at least.

 

I'm an advocate for raid 10, sacrificing space for iops. True, in a worse-case scenario if you lose both drives in one of the mirrors then you're screwed. However with backups and all the other advantages (ability to expand the array, higher best-case scenario # of drive failures, increased IOPS, significantly faster resilver and less impact) I think 10 is the way to go. If I could redo my array I'd switch to 10.

 

With Raidz2, minimum of 4 drives and should be dividable by 2 for best performance. My 4x 3TB RaidZ2 WD reds, is 9TB usable in case you were curious.

Even a 5400rpm Red should have more than sufficient read/writes to not be an issue. 

 

What benefit is there to hav an even number of drives for Z2? Five drives would be fine. Also, how do you have 9TB usable in a 4x3TB Z2 array...that doesn't make sense. 

[Mikensan]

The WD red gets 45 read IOPS and 112write IOPS, the HGST gets 115/160. So there's a clear difference, as well as in seek time. I think it's worth the slight extra $$. I personally max out the IOPS from multiple transfers way before I hit the top speeds. It sounds like he'll be dealing with essentially multiple transfers... (if you consider the activity from streaming a "transfer" to the player) 

 

There's large articles over at the FreeNAS forum about proper RaidZ configurations that's above my head, but the guides as well as the FreeNAS GUI recommend RaidZ2 be done in even # of drives with a minimum of 4. The only exceptions to # of drives will depend on the block size, which none of the drives mentioned here are 4k. Solaris actually recommends RaidZ2 be 4+2 (6) minimum drives. FreeNAS however says 4 is fine. 5 will work, but is not "optimal." I believe you also lose even more storage if you use an odd number.

https://doc.freenas.org/9.3/freenas_storage.html

 

I'll take a screenshot when I'm home, RaidZ2 isn't exactly the same as Raid6 which you essentially just predict a loss of 2 drives worth of storage. I'm assuming you're thinking it should be 6TB and not 9TB? Sometimes it is more than 2 drive loss sometimes it is less. There isn't even a good RaidZ2 calculator lol - they exist but are often wrong compared to real world.

 

@djdwosk97