Suggestions for NAS platform

[arcadeperfect]

Hello

 

 

TLDR:

 

Can anyone recommend a NAS platform that prioritizes performance over integrity, but is more than just a standard OS serving software raid?

 

 

Long version:

I already have the following hardware:

 

Supermicro X10SDV-4C-TLN2F - mini ITX Xeon-D SOC with on board 10GBase-T, 32 GB ram

5 x WD Red 2TB sata SSD

1 x 500 GB nvme consumer ssd

I do have a chelsio SFP+ nic that could go in there if it would be better than the on board nic, but would prefer to keep that for something else.

 

Right now, it's running Truenas with the drives in RaidZ 1. I threw in the nvme with its own pool just to see what speeds I could get out of it.

 

However I am starting to think ZFS is overkill. The nvme with no parity or compression can't come close to saturating the nic even on reads. AFAICT it's bottle necking on the CPU.

 

As long as the data is backed up nightly, I think I can accept a total loss of the array, and performance is higher priority. But given that the nvme runs pretty slow I don't think striping the drives with ZFS would be any better (?)

 

I would like at least a few creature comforts though, preferably a web interface and scheduling for backups to another local server and / or backblaze. I don't need jails / vm's etc.

 

Can anyone recommend a NAS platform that fits the bill?

 

Thanks

[LIGISTX]

18 minutes ago, arcadeperfect said:

Hello

 

 

TLDR:

 

Can anyone recommend a NAS platform that prioritizes performance over integrity, but is more than just a standard OS serving software raid?

 

 

Long version:

I already have the following hardware:

 

Supermicro X10SDV-4C-TLN2F - mini ITX Xeon-D SOC with on board 10GBase-T, 32 GB ram

5 x WD Red 2TB sata SSD

1 x 500 GB nvme consumer ssd

I do have a chelsio SFP+ nic that could go in there if it would be better than the on board nic, but would prefer to keep that for something else.

 

Right now, it's running Truenas with the drives in RaidZ 1. I threw in the nvme with its own pool just to see what speeds I could get out of it.

 

However I am starting to think ZFS is overkill. The nvme with no parity or compression can't come close to saturating the nic even on reads. AFAICT it's bottle necking on the CPU.

 

As long as the data is backed up nightly, I think I can accept a total loss of the array, and performance is higher priority. But given that the nvme runs pretty slow I don't think striping the drives with ZFS would be any better (?)

 

I would like at least a few creature comforts though, preferably a web interface and scheduling for backups to another local server and / or backblaze. I don't need jails / vm's etc.

 

Can anyone recommend a NAS platform that fits the bill?

 

Thanks

SMB is a single threaded operation, so that could be your issue… truenas and ZFS are not the limiting factor. While ZFS in its current form and especially the older versions in truenas are not exactly tuned for SSD performance, but it shouldn’t have any issue at all saturating 10gbe. It runs into issues when you start trying to push full flash arrays with 40 or 100 gbe interfaces. 
 

I am not an expert tho… not with those speeds or with SSD’s in ZFS in general. Potentially may be worth while posting on the truenas forums as well. 

[arcadeperfect]

Someone else also told me that SMB is single threaded and could be the issue. I did test with ZFS and Ubuntu on the client, and iSCSI. Neither yielded much improvement. iSCSI was worse actually.

But that's interesting... correct me if I'm wrong but server CPU's tend to go for cores over clock. So if this is the case to serve SMB at high speeds you need something clocked pretty high. The humble Xeon-D in question is 2.4 Ghz.

[LIGISTX]

13 minutes ago, arcadeperfect said:

Someone else also told me that SMB is single threaded and could be the issue. I did test with ZFS and Ubuntu on the client, and iSCSI. Neither yielded much improvement. iSCSI was worse actually.

But that's interesting... correct me if I'm wrong but server CPU's tend to go for cores over clock. So if this is the case to serve SMB at high speeds you need something clocked pretty high. The humble Xeon-D in question is 2.4 Ghz.

Hmm, are you using Scale or Core? I know SCALE has had some performance issues, notably slower iSCSI performance than expected. 

 

Even a decently new Xeon would have “plenty” of Ghz for SMB performance. Looks like the chip you have boosts to 2.7 ghz and is based on some variant of 14nm (looks like broadwell, so not exactly new technology…). That said, I am not sure that is the issue… I would post on the truenas forums and see if they have insight. There are certainly folks here who would know more then I and would be able to provide better information, but that forum is basically filled with people who would have advice and knowledge on this. 

[ShakataGaNai]

I don't think you'll find something "NAS" that focuses on performance over integrity. That would defeat the purpose of most NAS. That being said, I'm sure there is tuning to be done.

 

Like on the network side, are you using jumbo frames? Have you tried using something like iperf to eliminate the disks as an issue? How does a local read speed test of the NVMe compare to an over-the-network test?

 

And in today's market, it seems the options are basically TrueNAS or Unraid. Or Windows... I guess? If you don't care about the data you could always nuke TrueNAS and install something else and try it.