I need some advice from anyone experienced in building a FreeNas server to ensure I don't create myself some bottlenecks by mistake. My goal is to have 100+TB of 7200RPM disks with 20TB of SSD. The server is running an Intel Xeon chip, with 128GB RAM, 1GBe & 10GBe.
The connections made to this box will be via iSCSI.
I'm trying to find out really if FreeNas is going to be best, or if I should use another OS initially, but then also any configuration I should be aware of? Should I create multiple volumes in Raidz-2 - any particular compression level?
Should I use my SSD as a log volume or just as another tier of storage, I know having a cache won't help me as 80% of my operations are going to be writes and not reads.
I'm using some rubbish drives at the moment while I wait for the proper ones to turn up (Enterprise), just so I can work with the configuration and have set-up the following for my tests:
1 volume with 6x500GB
1 volume with 6x500GB, 1x120GB SSD as log
1 volume with 6x500GB, 1x120GB SSD as log, 1x120GB SSD as cache
This project isn't committed to FreeNas so I'm willing to change, even to Windows Server 2016, any input would be appreciated.
Update #1 - 23/05/2019
This will be used as a back-up target for long term retention at a DR site.The application sits on a Windows Server and will be connected via 10GBe directly into this storage device (no switch).
Update #2 - 23/05/2019
In my tests using non-enterprise HDD's (moving 230GB) the top transfer speed using a log didn't increase, but the average speed increased so my transfer time decreased by 10%.
The log & cache volume had no benefits over log alone.