freeNas build. (new user)

[suyspeedy]

(hope this is the correct place for this post:)) freeNas build! Need help:)

(!Be advised, this post contains bad English!)

Hello,

I'm going too build myself a freeNas, and  I need advice on what kind of hardware I should run.

Im going too throw some money at this since I am looking for a safe place too store my family doc,pics,vids and movie collection. And some music

I currently have a windows home-server running, but Its getting outdated and starting to become unstable (so I will not be reusing any of the hardware from it), I have also noticed that some of my data are getting corrupt and unusable, and I don't want too loose my data. So I have decided to get a new setup running freeNas. 

Some of my requirements for the new setup will be, at least 12TB  of usable storage-space (I currently have 9,2tb of data and need some room too grow). Also want too use the snapshot/shadow-copy function. So the ZFS filesystem is what I`m going too be using. I have some ideas off what kind of hardware I need, but I don't want too use allot of money, and then figure out I got it wrong and overlooked something.

-----------------------------------------------------------------------------------------------------------------------------------------------

My choices:

*5, WD Red 4TB NAS Harddrive`s

SATA 6Gb/s (SATA 3.0), 64MB, 3.5", 24x7 reliability, IntelliPower, 

*2 Crucial DDR3 BallistiX Tactical 16GB, 16GB kit (8GBx2),1600MHz, 1.5V, CL8-8-8-24 (for a total of 32GB)

*SanDisk Cruzer® Fit 8GBUSB2.0 (For FreeNas-install)

*ASUS H87M-PLUS, Socket-1150m-ATX, H87, DDR3, 2xPCIe-x16, CFX, VGA, DVI, HDMI, Haswell

*Intel Core i3-4330Socket-LGA1150, Dual Core, 3.5GHz, 4MB, 54W, HD4600, Boxed w/fan, Haswell

*Fractal Design ARC Mini R2 SortFans: 1x 120mm Front, 120mm rear, mATX, mITX 

*Corsair CS 550M, 550W PSUATX 12V v2.4, 80 Plus Gold, Modular, 2x 6+2pin PCIe, 5x SATA, 4x Molex, 1x FD

*Samsung SSD 840 EVO 120GB Basic KIT, 540/410MB/s read/write, Samsung MEX controller. (cache disk)

--------------------------------------------------------------------------------------------------------------------------------------------

So is this a reasonable setup? not powerful enough? too much? what do you think guys and/or girls?,- I would really appreciate any feedback I can get. 

[bobhays]

do you really need 16gb for a nas?

[Vitalius]

do you really need 16gb for a nas?

He's using ZFS and his total RAM is 32GB.

And yes, you absolutely need that much RAM with 20 TB of Raw Storage.

If he's using UFS, he loses the features he wants (shadow copies and such), but he would only need about 4GB of RAM.

FreeNAS' ZFS is an enterprise level File System. It has amazingly useful features, but you gotta throw a ton of strong hardware at it for it to perform well.

I'm tagging @Vitalius because I know he's currently working on a freeNas build.

Thanks.

 

-snip-

It looks like you did your research before posting here. That is a very good build for FreeNAS. Like, really. I wouldn't change anything about it at all about it considering your use case.

Every piece of hardware makes sense and is what I would constitute as the 'Sweet spot' for what you want.

[bobhays]

snip

 

sorry I was just wondering because i've never made a nas with that much storage, or using freeNAS. What does it need that much ram for?

[Vitalius]

sorry I was just wondering because i've never made a nas with that much storage, or using freeNAS. What does it need that much ram for?

It's cool bro. People learn.

In the RAM, it logs where all the data is within a huge table. So for every 1TB of space, you need at least 1GB of RAM for the table to tell you what is in that 1TB of storage. 

It's not cache. It just directs the OS to where the data is stored very quickly, then it reads it off the HDDs.

The OS itself needs 4GB to run (for the various things it does), then you add 1GB per 1TB of storage, so for him, he needs at least 24GB of RAM. It being RAIDed is irrelevant, as the amount of RAM you need is based off RAW storage, not actual storage (so not 12TB after it is RAIDed, but the 20TB as if it were not RAIDed).

If you added in Deduplication (to save on storage space), you would need 4GB of RAM for every 1TB of storage. Which is insane. That is exclusive for Enterprise level things, where you would have 128GB of RAM or more. 

[bobhays]

DAMNNNNNNNNNNNNNNNNNNN that's a lot of RAM for storage. ok I see now haha. Does that limit freeNAS's to 28tb of storage max? (32 - 4 for OS)?

[bobhays]

snip

[Vitalius]

DAMNNNNNNNNNNNNNNNNNNN that's a lot of RAM for storage. ok I see now haha. Does that limit freeNAS's to 28tb of storage max? (32 - 4 for OS)?

It limits his build to that much storage. If he upgrades RAM, he can have more.

[bobhays]

It limits his build to that much storage. If he upgrades RAM, he can have more.

well most consumer boards have a max of 4 slots x 8gb right? so 32gb max?

[Vitalius]

well most consumer boards have a max of 4 slots x 8gb right? so 32gb max?

Admittedly, yeah. He'd need to upgrade his motherboard and everything else. 

This is why used Server parts are my preferred NAS building components.

[Atlantisman]

I personally run FreeNAS, i love it. This build will do awesomely. In my experience the SSD would probably be overkill though. I have an SSD in mine(the same one actually) and i can write fast enough to the disk that it saturates the NIC before anything else.

[suyspeedy]

do you really need 16gb for a nas?

Its 32Gb, I know that ZFS can be really memory hungry, so maby yes ? ( I heard that you need 8gb for freenas+1,5Gb for very TB of space) do you know? 

[suyspeedy]

He's using ZFS and his total RAM is 32GB.

And yes, you absolutely need that much RAM with 20 TB of Raw Storage.

If he's using UFS, he loses the features he wants (shadow copies and such), but he would only need about 4GB of RAM.

FreeNAS' ZFS is an enterprise level File System. It has amazingly useful features, but you gotta throw a ton of strong hardware at it for it to perform well.

Thanks.

 

It looks like you did your research before posting here. That is a very good build for FreeNAS. Like, really. I wouldn't change anything about it at all about it considering your use case.

Every piece of hardware makes sense and is what I would constitute as the 'Sweet spot' for what you want.

Tyvm for the insight, I`m really exited about this build, I have never used freeNas, so Im going to test it out properly before I start migrating all of my data over too it:) 

[Atlantisman]

Its 32Gb, I know that ZFS can be really memory hungry, so maby yes ? ( I heard that you need 8gb for freenas+1,5Gb for very TB of space) do you know? 

Yeah, you will need the RAM, especially if you want to do higher compression or deduplication. And with that much storage it may not be a good idea to even do deduplication... FreeNAS LOVES RAM.

[suyspeedy]

I personally run FreeNAS, i love it. This build will do awesomely. In my experience the SSD would probably be overkill though. I have an SSD in mine(the same one actually) and i can write fast enough to the disk that it saturates the NIC before anything else.

So you think I could drop the SSD without any noticeable performance loss?  Maby even replace it with more storage, like one more 4tb disk?

[wpirobotbuilder]

So you think I could drop the SSD without any noticeable performance loss?  Maby even replace it with more storage, like one more 4tb disk?

It'll depend on your usage. Your SSD cache will certainly be of help if you do lots of I/O at once, and it's a reasonably sized SSD for the amount of RAM you have, though I'd reduce it to a 64GB SSD (you can get them for like $50 from Adata).

 

In general, unless you've maxed out your RAM, an SSD as an L2ARC will not be of much benefit. Running an L2ARC also uses memory, which will reduce the total amount available to index your storage's contents and run your OS and such. I think Vitalius ranted about this at some point.

 

I personally would go LGA 2011 for a ZFS machine, because you can install 64GB of RAM if you need it, can get support for ECC memory and have access to a lot of CPU horsepower.

[Atlantisman]

So you think I could drop the SSD without any noticeable performance loss?  Maby even replace it with more storage, like one more 4tb disk?

Yeah, the SSD would be overkill in my experience. Unless you have a 10gb NIC, or plan to team multiple gig NICs and will have potentially mutiple clients writing to the NAS at the same time.

 

I have 3x 3TB WD Reds in a RaidZ-1

and 5x Seagate Enterprise 750gb HDDs in another RaidZ-1 (i got them for free from a friend that used them in his works data center)

 

The SSD is attached to the WD raid array and i can remove it without seeing any performance loss or anything, because i can already completely saturate the gigabit NIC (around 100-110ish MB/s), full gigabit theoretical is about 121MB/s, but with overhead and etc you can expect to see around 100-110ish. I have 16gb of ram and an old Athlon II X2 245 Processor.

[wpirobotbuilder]

Yeah, the SSD would be overkill in my experience. Unless you have a 10gb NIC, or plan to team multiple gig NICs and will have potentially mutiple clients writing to the NAS at the same time.

NIC  

 

Faster than a 4x1GbE NIC, same price, and backwards compatible.

[suyspeedy]

Yeah, the SSD would be overkill in my experience. Unless you have a 10gb NIC, or plan to team multiple gig NICs and will have potentially mutiple clients writing to the NAS at the same time.

 

I have 3x 3TB WD Reds in a RaidZ-1

and 5x Seagate Enterprise 750gb HDDs in another RaidZ-1 (i got them for free from a friend that used them in his works data center)

 

The SSD is attached to the WD raid array and i can remove it without seeing any performance loss or anything, because i can already completely saturate the gigabit NIC (around 100-110ish MB/s), full gigabit theoretical is about 121MB/s, but with overhead and etc you can expect to see around 100-110ish. I have 16gb of ram and an old Athlon II X2 245 Processor.

 

 

It'll depend on your usage. Your SSD cache will certainly be of help if you do lots of I/O at once, and it's a reasonably sized SSD for the amount of RAM you have, though I'd reduce it to a 64GB SSD (you can get them for like $50 from Adata).

 

In general, unless you've maxed out your RAM, an SSD as an L2ARC will not be of much benefit. Running an L2ARC also uses memory, which will reduce the total amount available to index your storage's contents and run your OS and such. I think Vitalius ranted about this at some point.

 

I personally would go LGA 2011 for a ZFS machine, because you can install 64GB of RAM if you need it, can get support for ECC memory and have access to a lot of CPU horsepower.

 

 

He's using ZFS and his total RAM is 32GB.

And yes, you absolutely need that much RAM with 20 TB of Raw Storage.

If he's using UFS, he loses the features he wants (shadow copies and such), but he would only need about 4GB of RAM.

FreeNAS' ZFS is an enterprise level File System. It has amazingly useful features, but you gotta throw a ton of strong hardware at it for it to perform well.

Thanks.

 

It looks like you did your research before posting here. That is a very good build for FreeNAS. Like, really. I wouldn't change anything about it at all about it considering your use case.

Every piece of hardware makes sense and is what I would constitute as the 'Sweet spot' for what you want.

Ty guys, this is very good feedback I'm getting here, I think im gonna stick to my initial idea. (im kind of stretching my budget already   ). Do any of you know if its possible to add the cache disk later, after the system is up and running, or do I need to set it up initially with the rest of the disks? 

[Atlantisman]

Ty guys, this is very good feedback I'm getting here, I think im gonna stick to my initial idea. (im kind of stretching my budget already   ). Do any of you know if its possible to add the cache disk later, after the system is up and running, or do I need to set it up initially with the rest of the disks? 

You can add and remove cache disks later.

[wpirobotbuilder]

Ty guys, this is very good feedback I'm getting here, I think im gonna stick to my initial idea. (im kind of stretching my budget already   ). Do any of you know if its possible to add the cache disk later, after the system is up and running, or do I need to set it up initially with the rest of the disks? 

Pretty sure you can add it and remove it as you wish.

[suyspeedy]

NIC  

 

Faster than a 4x1GbE NIC, same price, and backwards compatible.

hmm, I was thinking something like this could be a nice upgrade later down the road. When and/or if I can afford a 10Gbit switch that is: hehe

[Atlantisman]

NIC  

 

Faster than a 4x1GbE NIC, same price, and backwards compatible.

http://www.ebay.com/itm/HP-PCIe-Express-Quad-Port-Gigabit-Server-Adapter-Long-Bracket-NC364T-436431-001-/121290194698?pt=US_Internal_Network_Cards&hash=item1c3d757f0a

 

Data center surplus parts on ebay are an awesome resource for super high end enterprise stuff that is a bit outdated. Still wicked awesome for home.

 

Edit: you can even find them cheaper a lot of the times. I picked one of those up on ebay a few weeks ago for like 60$ shipped.

[wpirobotbuilder]

hmm, I was thinking something like this could be a nice upgrade later down the road. When and/or if I can afford a 10Gbit switch that is: hehe

You don't need a switch, you can directly connect two computers with one NIC each.

 

If you end up wanting all computes to have 10GbE, Netgear has a phenomenally cheap switch for around $850

 

 

http://www.ebay.com/itm/HP-PCIe-Express-Quad-Port-Gigabit-Server-Adapter-Long-Bracket-NC364T-436431-001-/121290194698?pt=US_Internal_Network_Cards&hash=item1c3d757f0a

 

Data center surplus parts on ebay are an awesome resource for super high end enterprise stuff that is a bit outdated. Still wicked awesome for home.

 

Edit: you can even find them cheaper a lot of the times. I picked one of those up on ebay a few weeks ago for like 60$ shipped.

To be fair, you also have internet speeds that legitimize you having an awesome home network

[Atlantisman]

You don't need a switch, you can directly connect two computers with one NIC each.

 

If you end up wanting all computes to have 10GbE, Netgear has a phenomenally cheap switch for around $850

Why do you have to do this to me, now i am going to be drooling over that until one night i when i am bored i drop a couple grand picking up that and three 10gb NICs for my main computer and two servers. Damn.