NVMe Array Write performance problems

[xl3b4n0nx]

I am working on testing out some through put issues on my Dell R740xd with 2 Xeon Gold 6254 CPUs. I am using 8 Intel DC4510 8TB SSDs in a mdadm raid 0 array with xfs on top. Running parallel (8 to 12) dd commands (bs=32K count=8M) I am getting about 24GB/s reads and 6GB/s writes. The writes seem to be kind of slow and I can't figure out where the bottleneck is. Does anyone have any experience troubleshooting something like this?

 

I know LTT posted a video about this, but my problem is a bit different and is on an Intel platform so the same techniques may not apply.

[Electronics Wizardy]

Try using fio instead of dd, dd is not made for benchmarking.

 

WHat is system usagae? DIsk usage? 

[BloodKnight7]

Have you updated your firmwares/bios/idracs lately? You can do it easy through life cycle controller.