FreeNas configuration advice

[PooBear85]

Hey all,

 

I need some advice from anyone experienced in building a FreeNas server to ensure I don't create myself some bottlenecks by mistake. My goal is to have 100+TB of 7200RPM disks with 20TB of SSD. The server is running an Intel Xeon chip, with 128GB RAM, 1GBe & 10GBe.

 

The connections made to this box will be via iSCSI.

 

I'm trying to find out really if FreeNas is going to be best, or if I should use another OS initially, but then also any configuration I should be aware of? Should I create multiple volumes in Raidz-2 - any particular compression level?

 

Should I use my SSD as a log volume or just as another tier of storage, I know having a cache won't help me as 80% of my operations are going to be writes and not reads.

 

I'm using some rubbish drives at the moment while I wait for the proper ones to turn up (Enterprise), just so I can work with the configuration and have set-up the following for my tests:

 

• 1 volume with 6x500GB
• 1 volume with 6x500GB, 1x120GB SSD as log
• 1 volume with 6x500GB, 1x120GB SSD as log, 1x120GB SSD as cache

This project isn't committed to FreeNas so I'm willing to change, even to Windows Server 2016, any input would be appreciated.

 

Thanks!

 

Update #1 - 23/05/2019

This will be used as a back-up target for long term retention at a DR site.The application sits on a Windows Server and will be connected via 10GBe directly into this storage device (no switch).

 

Update #2 - 23/05/2019

In my tests using non-enterprise HDD's (moving 230GB) the top transfer speed using a log didn't increase, but the average speed increased so my transfer time decreased by 10%.

The log & cache volume had no benefits over log alone.

 

[UrbanFreestyle]

pics or it didn't happen

[Eniqmatic]

You didn't explain your use case, how are you going to be interacting with this storage, your saying ISCSI, is it going to be a datastore? Purely datastore and nothing else?

[PooBear85]

9 minutes ago, Eniqmatic said:

You didn't explain your use case, how are you going to be interacting with this storage, your saying ISCSI, is it going to be a datastore? Purely datastore and nothing else?

Added an update to original post.

[Chris Moore]

On 5/23/2019 at 3:25 AM, PooBear85 said:

with 20TB of SSD.

Why do you want any SSDs in a backup system? 

On 5/23/2019 at 3:25 AM, PooBear85 said:

The connections made to this box will be via iSCSI.

Does it need to be iSCSI or can it be a regular SMB share?  The iSCSI connection is going to always be slower because of the block storage nature of the interface.

On 5/23/2019 at 3:25 AM, PooBear85 said:

Should I create multiple volumes in Raidz-2 - any particular compression level?

You should check this resource on the FreeNAS forum:
https://www.ixsystems.com/community/threads/some-differences-between-raidz-and-mirrors-and-why-we-use-mirrors-for-block-storage.44068/

 

On 5/23/2019 at 3:25 AM, PooBear85 said:

Should I use my SSD as a log volume or just as another tier of storage,

Normal SSDs are not fast enough to be used as SLOG and you would only need a maximum of 30 or 40 GB of SLOG anyhow.  There are some great articles about that here:

The ZFS ZIL and SLOG Demystified
http://www.freenas.org/blog/zfs-zil-and-slog-demystified/

Some insights into SLOG/ZIL with ZFS on FreeNAS
https://www.ixsystems.com/community/threads/some-insights-into-slog-zil-with-zfs-on-freenas.13633/

Testing the benefits of SLOG using a RAM disk!
https://www.ixsystems.com/community/threads/testing-the-benefits-of-slog-using-a-ram-disk.56561/

SLOG benchmarking and finding the best SLOG
https://www.ixsystems.com/community/threads/slog-benchmarking-and-finding-the-best-slog.63521/

[Mikensan]

The tl;dr for SLOG/ZIL is get a write-intensive SSD or grab an optane drive - would be useful since you're going to use iSCSI + 10GBe (Sync. writes).

 

If the OS is aware the target is iSCSI and has a protocol developed for faster operations (like vmware's VAAI) iSCSI will be faster than if you were to just attach it to windows (treats it like a local disk). A VHD on an iscsi datastore mounted to a windows VM is going to perform better than a SMB share.

 

If this is for "backup" then SSDs / SLOG/ZIL are all wasted. If this is for high availabilty and to be used as production storage for a DR site that needs live / real-time updates, then you're on the right track.

 

The overall configuration of this server depends on the type of data that will be on it. Single large files like movies, tiny little files like user data (docs/images), or complex data like SQL. SQL data in itself isn't complex per se, but the configuration for a SQL database can be.