Im currently running an Infiniband QSFP+ link between 2 servers which both have 8 drive 2TB nvme raid arrays.
QSFP+ is just 4 link aggregated SFP+ connections, so when I use something like SMB or NFS on a file transfer I only get about 1.2 Gigabytes per second max. Which is the theoretical max of what a single SFP+ could do. I haven't tested SMB Multi-channel yet over Infiniband to see how much that helps but im not expecting much unless its hit from multiple angles.
What im truly curious about is should I upgrade to SFP28 Infiniband adapters?
The overall link is slower at 25Gbit vs 56Gbit but since its a single channel im thinking it might help the SMB transfers. If i had to guess it would be a theoretical x2 on transfer speed. A big plus would be the network adapters, transceivers, and fiber are all cheaper. The only issue I see is that SFP28 is still new so the switches havent come down in price on the used market yet.
If anyone has the capability can someone test to see if SFP28 or QSFP+ [is faster on SMB, NFS, Crystal disk mark speed test] is faster for single machine file transfers?