Storage Design (FreeNAS / VMware ESXi)

[phoenix3dfx225]

Afternoon, 

 

Looking for advice on planning a build. I currently have an Asus X99-s Mobo with a Xeon E2696 22c 44t CPU. I have a VMWare ESXi 6.5 licence and have played around with getting a VM system to work with GPU Passthrough on my bench successfully. (128GB RAM will be installed, currently only 16GB)

 

I want to build out this system some more to combine all my other requirements into one box. The systems I plan to have are:

 

Daily Driver / Gaming (including Plex and DVR(Tv Tuner) OS (Windows 10 8core / 32GB RAM)

CCTV (Windows 10 8core / 16GB RAM)

FreeNAS (8Core 32GB / RAM)

Sandbox / VPN (Window 10 2core 8GB RAM

 

The storage I'll have available would be:

 

2TB NVMe Drive - Thought I'd run the Daily Driver / Gaming OS, CCTV OS, and Sandbox PC OS on this:

FreeNAS boots from a USB

VMWARE boots from a USB

 

I have one these Dual USB Motherboard headers with each port having a mini 32GB USB Storage connected:

 

https://www.newegg.com/black-en-labs-n-a-adapter/p/181-00WC-00003

 

So my question is,  how to plan out the freenas storage side of things. Thinking x2 10TB 7200rpm HDD matched with a 256GB SSD I have spare? (I have x4 256GB SSD spare currently)

 

Do I just create one big data-store for steam library, CCTV footage and Plex Media? Using 2drive mirror and a SSD Cache? 

 

Any advice thanks

[Windows7ge]

If this is to be accessed over SMB and/or SSH/SFTP there's no reason for a cache drive. You won't see any performance benefit because ZFS doesn't work that way for those services.

 

It's generally ill-advised to run FreeNAS within a VM. Since (to my knowledge) ESXi lacks Software RAID support I'd pass an HBA through to FreeNAS and connect all your drives to that.

 

To address your question what most people will do is create one large pool then break it out into datasets. Each dataset can be individually controlled with permissions, network shares, services, etc. So create the pool then create a dataset for CCTV, a dataset for File Sharing for general storage, a dataset for PLEX, etc.

[GDRRiley]

for CCTV I would use its own WD purple or seagate surveillance  drives rated for that. You aren't going to get good recordings or good throughput if your forcing the drives to do 2 things.

[phoenix3dfx225]

4 hours ago, Windows7ge said:

If this is to be accessed over SMB and/or SSH/SFTP there's no reason for a cache drive. You won't see any performance benefit because ZFS doesn't work that way for those services.

 

It's generally ill-advised to run FreeNAS within a VM. Since (to my knowledge) ESXi lacks Software RAID support I'd pass an HBA through to FreeNAS and connect all your drives to that.

 

To address your question what most people will do is create one large pool then break it out into datasets. Each dataset can be individually controlled with permissions, network shares, services, etc. So create the pool then create a dataset for CCTV, a dataset for File Sharing for general storage, a dataset for PLEX, etc.

This is where ideas started getting developed and questions prompted. I played around with the iSCSI on FreeNAS. Would be good to incorporate as the VM OS's would see it as a physical drive so installing and running apps from the storage pool would work well. So OS C:\drive would still be on the high speed NVMe 2TB drive, yet a D:\ (Data drive) could come from a iSCSI rather than just SMB network NAS sort of share.

 

As a side note, I had a problem on my current setup installing Oculus software as it would only install on NTFS and at the time I was running Storage Spaces, which is ReFS. Using FreeNAS and iSCSI it would present to the OS as NTFS yet the back end would have the benefits of ZFS.

[Windows7ge]

23 minutes ago, phoenix3dfx225 said:

This is where ideas started getting developed and questions prompted. I played around with the iSCSI on FreeNAS. Would be good to incorporate as the VM OS's would see it as a physical drive so installing and running apps from the storage pool would work well. So OS C:\drive would still be on the high speed NVMe 2TB drive, yet a D:\ (Data drive) could come from a iSCSI rather than just SMB network NAS sort of share.

 

As a side note, I had a problem on my current setup installing Oculus software as it would only install on NTFS and at the time I was running Storage Spaces, which is ReFS. Using FreeNAS and iSCSI it would present to the OS as NTFS yet the back end would have the benefits of ZFS.

From my own experience using iSCSI on FreeNAS is it worked great for a short time then the performance dropped to utter garbage unexpectedly and I never solved the cause. I wouldn't recommend it.

 

iSCSI does have it's uses and it's appear but I wouldn't use it as a general storage device.