M.2 cache accelerated 60TB NAS?

[Ripper7620]

Good afternoon, I was just wondering if adding an M.2 will work to speed up the performance of my planned NAS build? Here's a link to my planned NAS build: 

 

https://pcpartpicker.com/user/Isabella_Shiri/saved/gfrQVn

 

Thank you in advance! 

[Electronics Wizardy]

What OS? What filesystem?

 

What use case?

 

what network speed?

Why do you have a 1060?

 

why only 2 dimms?

 

 

[unkn0wn1]

I would recommend amd from this because consumer chipset support ecc so you don't need to pay extra for the xeon.

 

Additionally what operating system do you plan to use?

 

If you are using freenas i believe you can use an nvme ssd to avoid having to need crazy amounts of ram.

[Ripper7620]

Thank you very much for the feedback. I will be using FreeNas as the OS, and it will be a multi-function apparatus. I'm new to NAS and will have to further explore what is possible. I want to leave as many options open as possible as far as specific uses.

[Electronics Wizardy]

3 minutes ago, Ripper7620 said:

Thank you very much for the feedback. I will be using FreeNas as the OS, and it will be a multi-function apparatus. I'm new to NAS and will have to further explore what is possible. I want to leave as many options open as possible as far as specific uses.

What do you mean by multifunction? VMS? backup?

 

 

You can use the ssd as a l2arc or a slog. 

 

A l2arc is the place where the ram cache goes when its not needed. Its normally not that usefull, but depends on use. I was messing with one on my server and only getting a hit ration of about 2% compared to the arc hit ratio of about 60%.

 

A l2arc has a ram overhead, so ram is needed to index it. Normally get more ramm before you get a l2arc.

 

A slog keep track of what the file system is doing and works as a write cache in simple terms. This only helps for sync writes, so it may be completely useless if you aren't using sync.

 

 

Is this server already built or still in planning? if its still in planning, id change some parts around, its not optimal as a nas.

[Ripper7620]

Sorry, I forgot to add that I will be using using it in conjunction with Mac, Windows, and iOS devices. 

[Electronics Wizardy]

Just now, Ripper7620 said:

Sorry, I forgot to add that I will be using using it in conjunction with Mac, Windows, and iOS devices. 

Then you probably using cifs, so a slog won't help at all.

 

If this is a nas for sharing file over a gig network(it seems like it) then a ssd cache will do almost nothing. I wouldn't get one here.

[Ripper7620]

Thank you very much, I'm very new to NAS and just wanted to have all my options open since I will be using it for more tasks as I discover what is possible. I'll mainly be using it in conjunction with content creation workstations.

[leadeater]

59 minutes ago, Ripper7620 said:

Good afternoon, I was just wondering if adding an M.2 will work to speed up the performance of my planned NAS build? Here's a link to my planned NAS build: 

 

https://pcpartpicker.com/user/Isabella_Shiri/saved/gfrQVn

 

Thank you in advance! 

Swap out the motherboard for a Supermicro one that actually supports ECC Registered memory, it's highly likely the system will not work at all. Also optimal configuration is to also add RAM in set of 4 for that CPU. 

 

Edit:

NVM assumed that X99 board wouldn't support Registered memory, it does. I'd still go with Supermicro though and a C series chipset that is designed for Xeons and server use.

[sykulohi]

@Ripper7620 Checkout RockStor instead, you seem to be looking for performance, in my testing it was about 40% faster with Raid 10. Others have found it to be 5X faster.

 

Getting high baseline speeds + adding a read cache is hard. With FreeNAS you'll have a lot of support for mediocre baselines speeds, but decent caching options. Though not as mediocre as unRaid's baseline, most people get around that by dumping money into dangerous caching solutions or quad 2TB SSD's.

 

I'm considering a unRaid VM to handle caching because setting up bcache on Rockstor is a pita.

[Ripper7620]

Thank you very much for the feedback, I will swap out the motherboard for something appropriate. 

[Ripper7620]

I'm not sure if this is an improvement, but I swapped out some parts, and added a Supermicro motherboard that supports ECC ram, as well as upgrading the CPU to a Broadwell class 8-core, and a Noctua cooler. Here's the updatedparts list: 

 

https://pcpartpicker.com/user/Isabella_Shiri/saved/2f77hM

[Electronics Wizardy]

2 hours ago, Ripper7620 said:

I'm not sure if this is an improvement, but I swapped out some parts, and added a Supermicro motherboard that supports ECC ram, as well as upgrading the CPU to a Broadwell class 8-core, and a Noctua cooler. Here's the updatedparts list: 

 

https://pcpartpicker.com/user/Isabella_Shiri/saved/2f77hM

What are you doing with the cpu, that seems a bit overkill here.

 

Id go xeon d with a board like this 

https://www.newegg.com/Product/Product.aspx?Item=N82E16813182964

 

NO need for that ssd as discussed above, won't make a difference here.

 

Why the 1060?

 

Why the red pro, won't make a performance difference here, esp with a 1gbe link.

 

Id probably go 12tb drives. The segate 12tb ironwolfs are about the same price.

 

 

[Jarsky]

Speedof your filesystem at home doesn't really matter at all with a 1gigabit port - even a 4 x 1gb trunk port smiles happily at most disk based arrays. 

Really you're only going to see benefit if you go to a 10gbE link, and again its only going to be noticable when multitasking really. 

 

As above, i'd recommend going with a workstation or server board that gives you ECC memory support by default and integrated graphics controller - otherwise some Ryzen boards support ECC (like the Asrock B250 Pro4) but you'd want a 2nd gen Ryzen APU with built in graphics, or an Intel board with a Xeon. 

 

As for actually using M.2 as a cache, you can create tiered storage in Storage Spaces (Windows Server), l2arc cache in FreeNAS, bcache in MDADM (Linux Software) or just a ssd cache in unRAID or Rockstor - so can be pretty flexible with what OS you go with. 

 

Looking at your build, you can get rid of the GTX1060 - that motherboard has integrated graphics. 

[leadeater]

6 hours ago, Jarsky said:

As for actually using M.2 as a cache, you can create tiered storage in Storage Spaces (Windows Server)

Min of 2 SSD required and cannot be used for OS. Kind of a downside and upside to Storage Spaces, it doesn't let you create something that puts your protected data at risk in an unprotected cache. You can do single SSD with a simple virtual disk but that's RAID 0 so not a good idea at all.

[Ripper7620]

17 hours ago, Electronics Wizardy said:

What are you doing with the cpu, that seems a bit overkill here.

 

Id go xeon d with a board like this 

https://www.newegg.com/Product/Product.aspx?Item=N82E16813182964

 

NO need for that ssd as discussed above, won't make a difference here.

 

Why the 1060?

 

Why the red pro, won't make a performance difference here, esp with a 1gbe link.

 

Id probably go 12tb drives. The segate 12tb ironwolfs are about the same price.

 

 

Thank you for the feedback. I picked the WD Red Pro's because they're the highest quality and least likely to fail NAS Drives that I'm currently aware of. I'll likely ditch the 1060. I'm not sure about the best choice for a mobo, and if the Xeon 8-core turns out to be completely uneccisarry and overkill, then I'll re-purpose it in another build and go with something else with the NAS.

[Electronics Wizardy]

2 minutes ago, Ripper7620 said:

Thank you for the feedback. I picked the WD Red Pro's because they're the highest quality and least likely to fail NAS Drives that I'm currently aware of. I'll likely ditch the 1060. I'm not sure about the best choice for a mobo, and if the Xeon 8-core turns out to be completely uneccisarry and overkill, then I'll re-purpose it in another build and go with something else with the NAS.

You can't really compare relability like that. Failure is unknown and will be about the same as a normal red or a segate drive.

 

The gpu is just wasting power in the build. it won't be doing anything.

 

 

[Ripper7620]

20 minutes ago, Electronics Wizardy said:

You can't really compare relability like that. Failure is unknown and will be about the same as a normal red or a segate drive.

 

The gpu is just wasting power in the build. it won't be doing anything.

 

 

Thank you, I did some more research ,and indeed almost everyone concurs that the Red Pros are a complete waste of resources.

[Electronics Wizardy]

1 minute ago, Ripper7620 said:

Thank you, I did some more research ,and indeed almost everyone concurs that the Red Pros are a complete waste of resources.

there a bit fasher, but you dont seem to need that here, the 12tb segates seen like a better buy.

[Ripper7620]

18 minutes ago, Electronics Wizardy said:

there a bit fasher, but you dont seem to need that here, the 12tb segates seen like a better buy.

thank you, and the reason I chose 10TB drives is to match the RAM up to the store using the 1gb of Ram to 1 TB of storage ratio that I've read about when using FreNas. So I'll be running 60TB of store to 64gb of Ram. Also, my understanding is that it's advantageous to spread the Ram out over many smaller sized sticks, as opposed to only two 32gb sticks, which I chose solely based on cost.

[Electronics Wizardy]

1 minute ago, Ripper7620 said:

thank you, and the reason I chose 10TB drives is to match the RAM up to the store using the 1gb of Ram to 1 TB of storage ratio that I've read about when using FreNas. So I'll be running 60TB of store to 64gb of Ram. Also, my understanding is that it's advantageous to spread the Ram out over many smaller sized sticks, as opposed to only two 32gb sticks, which I chose solely based on cost.

The 1gb per tb is kinda bs.

 

In zfs and freeNAS ram is used for a few things.  The os and web interface need about a gig or two. ZFS needs a gig or two to work right for things like the zil. Anything extra is used for a ARC or the read cache. More is better, but won't make a big change in performance(depending on use). You can get away with 8gb here if you want.

 

Some functions need more ram, like a large l2arc or dedup, but your not using those.

[AshleyAshes]

3 hours ago, Ripper7620 said:

thank you, and the reason I chose 10TB drives is to match the RAM up to the store using the 1gb of Ram to 1 TB of storage ratio that I've read about when using FreNas.

The 1GB RAM per 1TB of storage requirement for ZFS is basically a myth that's been debunked by actual developers of the ZFS file system.