Dual 10g SFP+ card in PCIE 3.0 x 4 slot - will one port be able to saturate?

arcadeperfect · January 22, 2021

Hi,

I'm putting together a poor man's 10G freenas.

I have an old motherboard with a single PCIe 3.0 x16 slot, and a single x4 slot (with physical size of x16). I plan to use the x16 slot for storage.

It seems that the cheap 10G nics on ebay are often PCIe 2.0 x8 cards (especially if I go for Intel or Chelsio as has been recommended for freenas). Single port cards are also quite rare. It's a shame that Melanox cards don't seem to play nice as they're easy to find in the config I need I suppose they cheap for a reason.

My questions are....

Would a 2 x 10G port card, with PCIe 2.0 x 8 interface, in a PCIe 3.0 x 4 slot, technically allow me to saturate one of the SFP+ ports, if the other is ignored?

I assume this would run at PCIe 2.0 x4 speed which should max out at 16 Gb/s. But don't know if it's as clear cut as that.

Similar question - if I spring for a PCIe 3.0 x 8 card can I assume that one port would be able to saturate since that slot should be able to go well beyond 10Gb/s?

Does it even matter which way around I put the cards since the data all has to go through the x4 slot anyway? ie I may as well use the x4 slot for the storage?

PS I know these are theoretical maximums and I'm unlikely to sustain 10Gb/s in real life, I'm just talking theory here.

Thanks and apologies if anything is dreadfully stupid.

Windows7ge · January 22, 2021

5 minutes ago, arcadeperfect said:

It's a shame that Melanox cards don't seem to play nice as they're easy to find in the config I need I suppose they cheap for a reason.

I don't know about the latest versions of FreeNAS but I can vouch for the Mellanox ConnectX-2 MNPA19-XTR. Plug'n'Play from my experience.

You should be able to saturate one but you will likely struggle to saturate both on that slot. If you don't NEED the ability to push the NIC to the max all the time then it should work just fine for you.