storage NVME NAS setup

[spikkedd]

I'm working on setting up an NVME SuperMicro based NAS, and was looking at Linus for some inspiration. 
Does anyone know what ended up happening with this server setup?

 

 

I have the same SuperMicro server but have been encountering the same issue as his other NAS.

 

[Electronics Wizardy]

7 hours ago, spikkedd said:

I have the same SuperMicro server but have been encountering the same issue as his other NAS.

 

what model do you have?

 

That issue was mostly fixed in the epyc system with changing to pulling.

 

What full specs do you have?

[spikkedd]

14 hours ago, Electronics Wizardy said:

what model do you have?

 

That issue was mostly fixed in the epyc system with changing to pulling.

 

What full specs do you have?

I have the

A+ Server 2113S-WN24RT 

with:

AMD EPYC 7451
128GB RAM
FreeNAS11.3-U4.1
4x 4TB Intel P4510 NVME SSDs (pooled)

 

There are separate SSDs for the OS.

[Electronics Wizardy]

6 hours ago, spikkedd said:

I have the

A+ Server 2113S-WN24RT 

with:

AMD EPYC 7451
128GB RAM
FreeNAS11.3-U4.1
4x 4TB Intel P4510 NVME SSDs (pooled)

 

There are separate SSDs for the OS.

Are you having any issues? What are the issues?

 

How is the zfs pool setup?

[spikkedd]

2 hours ago, Electronics Wizardy said:

Are you having any issues? What are the issues?

 

How is the zfs pool setup?

I am encountering the same issues as his other NAS.

My NAS is running FreeNAS 11.3-U4.1

with a pool of 4 4TB Intel NVMEs in stripe

dedup and compression are set to off, sync set to off.

 

My issue is that the transfer will start and accomplish ~900MB/s write just fine and then will come to a halt for a few seconds before picking it back up again.

I've tried changing the record size and playing around with sync settings but no luck so far.

The SuperMicro comes with a built in 10G NIC, my network is 10G, and my iMac is also 10G so I don't think its on the network.