Which of these 10Gbps cards would work best in Linux?

[road hazard]

So I've decided to take the plunge and switch over my Linux boxes (media server and backup server) to 10Gbps networking.

 

 

 

Warning, even though I've been running Linux for a few years on my media server (Mint) and a couple of other PCs (Antergos, now EndeavourOS), I still consider myself a total newbie. For right now, my media server is using Mint but I might be moving it to EndeavourOS down the road so whichever card I go with, it needs to work with the Debian world and Arch.

 

 

 

For the 10Gbps switch, I'm looking at getting a Dell PowerConnect 5524. Looked at the Aruba and some others but it seems that if you don't pay for an account, you can't access the latest firmware updates so I think I'm going with the Dell.

 

Now the cards..... here are the ones I'm considering:

 

 

Buy a generic FS branded one: https://www.fs.com/products/75851.html

Mellanox MCX311A-XCAT CX311A ConnectX-3 EN

Intel X520-DA1

 

It looks like I'll need to compile the driver for all of them, no matter which one I go with, but do one of those 3 stand out above the others as working perfectly with Linux? (Or, will give me less headache?)

 

For the DAC cable, does it matter which brand I go with or will the FS branded one work just fine with any of those cards and the Dell switch? (https://www.fs.com/products/35194.html .... or go with a Dell branded one since my switch will be a Dell?)

Read that I might have some issues with Arch and the Mellanox driver, should I just stay away from that card? https://community.mellanox.com/s/question/0D51T00006Zi6PPSAZ/how-can-i-compile-either-mlnxen-or-mlnxofed-from-source

 

I also keep seeing lots of people warning about getting fake Intel X520's off eBay/Amazon so maybe the FS branded card is the one to use?

[mariushm]

How much do you plan to pay for the network switch?

 

Unixplus has a 24 port SFP+ + 4x1gbps Quanta LB6M for 300$ : https://www.unixplus.com/collections/network-switches/products/quanta-lb6m-24-port-10gbe-sfp-4x-1gbe-l2-l3-switch?variant=5126589710363 (or with an intel card and a DAC cable for $360, you save around 30-50$)

 

Another option, the Arista 7050T switches , 48 10gbps ports but note nowadays prices are 450-500$ because of Level1Techs video, see at the bottom.

Still, maybe you can find a 24 port version or a version without 40gbps or extra modules that will be cheaper on eBay.

There's a bunch of Artista 7050s for ~350$ : https://www.ebay.com/itm/ARISTA-DCS-7050s-52-F-NORMAL-FLOW-RACK-mts-2-DC-10GBe-SFP-TESTED/293158032434

Here's a bunch of 7124s (24port) for 300$ : https://www.ebay.com/itm/Arista-DCS-7124S-R-24-Port-10-GbE-SFP-L2-3-4-Switch-R-F-Airflow-2x-AC-PSU-HSS/372447642208

 

If you want something new, you may want to check out the 149$ Microtik switch with 4 SFP+ ports: https://mikrotik.com/product/crs305_1g_4s_in

 

It's more expensive, but you save on network cards - you could buy 29$ mellanox connectx-2 cards : https://www.unixplus.com/products/refurbished-mellanox-connectx-2-single-port-10gbe-sfp-mnpa19-xtr

Though it may be a bit difficult to set them up.

There's also Intel x520 cards for 67$  https://www.unixplus.com/products/refurbished-intel-ethernet-converged-network-adapter-x520-da1

 

 

 

[road hazard]

53 minutes ago, mariushm said:

How much do you plan to pay for the network switch?

 

Unixplus has a 24 port SFP+ + 4x1gbps Quanta LB6M for 300$ : https://www.unixplus.com/collections/network-switches/products/quanta-lb6m-24-port-10gbe-sfp-4x-1gbe-l2-l3-switch?variant=5126589710363 (or with an intel card and a DAC cable for $360, you save around 30-50$)

 

Another option, the Arista 7050T switches , 48 10gbps ports but note nowadays prices are 450-500$ because of Level1Techs video, see at the bottom.

Still, maybe you can find a 24 port version or a version without 40gbps or extra modules that will be cheaper on eBay.

There's a bunch of Artista 7050s for ~350$ : https://www.ebay.com/itm/ARISTA-DCS-7050s-52-F-NORMAL-FLOW-RACK-mts-2-DC-10GBe-SFP-TESTED/293158032434

Here's a bunch of 7124s (24port) for 300$ : https://www.ebay.com/itm/Arista-DCS-7124S-R-24-Port-10-GbE-SFP-L2-3-4-Switch-R-F-Airflow-2x-AC-PSU-HSS/372447642208

 

If you want something new, you may want to check out the 149$ Microtik switch with 4 SFP+ ports: https://mikrotik.com/product/crs305_1g_4s_in

 

It's more expensive, but you save on network cards - you could buy 29$ mellanox connectx-2 cards : https://www.unixplus.com/products/refurbished-mellanox-connectx-2-single-port-10gbe-sfp-mnpa19-xtr

Though it may be a bit difficult to set them up.

There's also Intel x520 cards for 67$  https://www.unixplus.com/products/refurbished-intel-ethernet-converged-network-adapter-x520-da1

 

I think those Dell switches go for under $100 (right around my budget for a switch),

 

My sub-system maxes out with a write speed of around 300-500MB/s so I think 10Gbps is more than enough.

 

Will read up on the Microtik switches.

 

Never heard of Unixplus.com, they good people?

 

Any comments about Linux compatibility regarding those cards?

 

[mariushm]

Yeah, those Dell switches may be cheap, but keep in mind there's only 2 10gbps ports on them (as far as I remember)

Yeah, Unixplus is OK, they also sell on eBay and have loads of volume and good rating, and they also have Youtube channel where they show off hardware and explain stuff.

I'm not really experienced with those network cards. The Intel card should be really well supported, the Mellanox may require extra steps to configure them, or uploading firmware to change between operating modes but should work.

 

Here's a bunch of videos related to switches i suggested: