10G multibit switch?

[Sarra]

I was looking at a TPLInk 5 port 10G Multibit switch, but they just pooft. I'm looking for something for home use, I don't need 64 ports, 32 ports, or even 12 ports. 8 Ports, or minimum 5 ports.

 

Going to be connecting two Ryzen 5000 based machines together for editing, plus a data storage server as a vault, and run a gigabit router (you know, for internets). I intend to run Cat6A, everything is in the same space, so cables will be shorter than 6 feet. Any suggestions?

[Falcon1986]

43 minutes ago, Sarra said:

8 Ports, or minimum 5 ports

What's your budget?

 

See this or this.

 

You'll also need NICs that can do 10Gbps.

 

But if the devices will all be in the same room, consider fiber/SFP+ and a supported switch like this one or this one. Use DAC of appropriate length.

[Sarra]

3 hours ago, Falcon1986 said:

What's your budget?

 

See this or this.

 

You'll also need NICs that can do 10Gbps.

 

But if the devices will all be in the same room, consider fiber/SFP+ and a supported switch like this one or this one. Use DAC of appropriate length.

My old server will hopefully be connected, but due to onboard connectivity limits, it'll probably be limited to 5G or 2.5G.

 

The first one you linked is what I was considering, but it's not available, and hasn't been for a while now.

[mariushm]

Trendnet makes some 2.5g switches that are cheap : Amazon.com: TRENDnet 5-Port Unmanaged 2.5G Switch, 5 x 2.5GBASE-T Ports, 25Gbps Switching Capacity, Backwards Compatible with 10-100-1000Mbps Devices, Fanless, Wall Mountable, Black, (TEG-S350) : Electronics

 

Look at some ubiquity, some microtik , qnap has some switches

 

ex 140$ for qnap  8 x 1g rj45 + 3 x 10g SFP+  : https://www.amazon.com/QNAP-QSW-308S-Switch-Gigabit-Unmanaged/dp/B07VC9RTR9/

In a pinch you could use RJ45 10g transceivers, they're around 30-50$ each though.. or just buy 10g sfp+ adapters for $50 each (see below)

 

ex $350 for 8x1g rj45 + 2 10g sfp+ only + 2 10g combo sfp+/rj45 : Amazon.com: QNAP QSW-M408-2C 10GbE Managed Switch, with 2-Port 10GbE SFP+/RJ45 Combo, 2-Port 10GbE SFP+, 2-Port 10GbE Base-T (RJ45) and 8-Port Gigabit : Everything Else

 

This looks like it would be the best mix for you. With the exception that it's not 2.5gbps / 5gbps aware (so the cards would run at 1g), you do get 4 10g ports, 2 of them RJ45

 

ex qnap 8 x 1g rj45 + 4 sfp+/rj45 for $460 : Amazon.com: QNAP QSW-M408-4C 10GbE Managed Switch, with 4-Port 10GbE SFP+/RJ45 Combo and 8-Port Gigabit : Everything Else

 

you could also go nuts with a $800 qnap 16 x 2.5g + 2 x 10g rj45 + 2 x 10g sfp+ : https://www.amazon.com/QNAP-20-Port-Managed-QSW-M2116P-2T2S-US-Management/dp/B09C1LP1L9/

 

But seriously, a  24x10g SFP+  + 4x1g rj45 switch is $230 plus shipping:   https://unixsurplus.com/quanta-lb6m/

10g sfp+ cards are < $50 ex 

full profile : https://unixsurplus.com/solarflare-sfn7022f-2-port-10gbe-sfp-pcie-3-0-server-i-oadapter-full-profile/

low profile : https://unixsurplus.com/solarflare-sfn7022f-2-port-10gbe-sfp-pcie-3-0-server-i-oadapter-low-profile/

 

DAC cables are $15-20, or you can get transceivers for $10-15 each and regular fiber.

 

You'll end up cheaper with such switch and buying fiber cards... and the cards are 2 x 10g so you could have two direct links to hardware instead of just 1.

 

 

 

[DogKnight]

 

As an option I use the following.

 

Netgear XS508M - 8 Port - 10gbe - RJ45 Switch

Intel X550-T1/T2 - 10gb RJ45 NIC's

 

The Intel NIC's can be picked up at reasonable prices on eBay. 

The Netgear switch has dropped about $200 since I purchased it. But there are definitely cheaper options available now.

 

As others have said, if you're looking to link just a couple machines in relatively close proximity, SPF connections will be a lot cheaper.

If you want to establish a 10gbe network for multiple devices now and in the future, an RJ45 based switch may be worth the expense. 

[Alex Atkin UK]

5 hours ago, DogKnight said:

 

As an option I use the following.

 

Netgear XS508M - 8 Port - 10gbe - RJ45 Switch

Intel X550-T1/T2 - 10gb RJ45 NIC's

Do you have any issues running Gigabit clients to a 10Gbit server?

 

When I went multi-gig I suddenly couldn't do Gigabit to my server any more, it was struggling at something like 300Mbit because for some bizarre reason the server was pushing data too fast to the switch (which is odd as it was TCP traffic which should have throttled down) until I enabled flow control which fixed it.  Wondering how an unmanaged switch handles that.

[Sarra]

So, my old server, which will be turned into a vault (several 10TB drives in RAID 5), has a PCIE 2.0 X4 slot. If I ran a 10Gig networking card in that slot, would it just run at 5Gig, or 8Gig, or what? The alternative is to run the GPU in the 4X slot, and put a networking card in the 16X slot. I could even run it headless, since it will be getting either Mint or Ubuntu Linux on it before the drives get installed and configured.

[mariushm]

5 hours ago, Sarra said:

So, my old server, which will be turned into a vault (several 10TB drives in RAID 5), has a PCIE 2.0 X4 slot. If I ran a 10Gig networking card in that slot, would it just run at 5Gig, or 8Gig, or what? The alternative is to run the GPU in the 4X slot, and put a networking card in the 16X slot. I could even run it headless, since it will be getting either Mint or Ubuntu Linux on it before the drives get installed and configured.

 

 

 

pci-e 2.0 is 500 MB/s per lane  x4 = 2 GB/s

pci-e 3.0 is ~985 MB/s per lane  x4 = 3.9 GB/s 

pci-e 4.0 is 2 x pci-e 3.0  , x4 =  ~7.8 GB/s

 

10 gbps is 1.25 GB/s  ... a dual 10g network card would use 2.5 GB/s  

 

So, your network card would run perfectly fine in a pci-e x4 slot, as it would use barely 1/3 of maximum bandwidth if the card is pci-e 3.0

 

If the slot bandwidth is too little, the card will still "link" at 10 gbps, but your overall throughput will be less (the card will pause accepting or transmitting new data from time to time to empty its buffers through the pci-e interface). For example, a 10gbps network card (1.25 GB/s max) in a pci-e 3.0 x1 slot (maximum 985 MB/s theoretical bandwidth) will peak at around 860-900 MB/s actual throughput, because data goes through pci-e in packets, and those packets have some overhead , so the maximum pci-e bandwidth can't really be reached.

 

 

[Sarra]

5 minutes ago, mariushm said:

 

 

 

pci-e 2.0 is 500 MB/s per lane  x4 = 2 GB/s

pci-e 3.0 is ~985 MB/s per lane  x4 = 3.9 GB/s 

pci-e 4.0 is 2 x pci-e 3.0  , x4 =  ~7.8 GB/s

 

10 gbps is 1.25 GB/s  ... a dual 10g network card would use 2.5 GB/s  

 

So, your network card would run perfectly fine in a pci-e x4 slot, as it would use barely 1/3 of maximum bandwidth if the card is pci-e 3.0

 

If the slot bandwidth is too little, the card will still "link" at 10 gbps, but your overall throughput will be less (the card will pause accepting or transmitting new data from time to time to empty its buffers through the pci-e interface). For example, a 10gbps network card (1.25 GB/s max) in a pci-e 3.0 x1 slot (maximum 985 MB/s theoretical bandwidth) will peak at around 860-900 MB/s actual throughput, because data goes through pci-e in packets, and those packets have some overhead , so the maximum pci-e bandwidth can't really be reached.

 

 

I think I would be fine to toss a single port card in my old server, then. For reliability of data transfer speeds, would it make sense to limit the link speed in software to 5gbps?

 

I found the TP-Link TL-SX105 for sale on places other than Amazon for the same price, so that sounds like it will be the way I go.

[mariushm]

No, let it stay at maximum bandwidth.

I wouldn't buy such a limited switch, with just 5 10g ports and no plain 1gbps or sfp+ ports... but if you like it, go for it

 

[DogKnight]

On 4/4/2022 at 10:14 PM, Alex Atkin UK said:

Do you have any issues running Gigabit clients to a 10Gbit server?

 

When I went multi-gig I suddenly couldn't do Gigabit to my server any more, it was struggling at something like 300Mbit because for some bizarre reason the server was pushing data too fast to the switch (which is odd as it was TCP traffic which should have throttled down) until I enabled flow control which fixed it.  Wondering how an unmanaged switch handles that.

 

Haven't had any issues. 

To give a little more info on my setup. 

 

Router is a Netgear RAX120. The 5gbe port connects to the 10gbe switch. 

Any 10gbe and multi-gig (5gbe) gear I have is connected to the 10gbe switch. 

I have one gigabit NAS attached to the 10gbe switch using two bonded links. 

Finally, the 10gbe switch is connected to a 16 port Netgear unmanaged gigabit switch which has several devices connected to it.

 

I get expected speeds out of all devices. And by expected, I mean that transfers between any devices that have the disk speed to cope with the full bandwidth. 

If I transfer to one of my NAS units, I will not get the full 5gbe speed due to the disk performance not being able to cope with the speed. 

So it comes down to the speed each device is able to handle as to the performance I get. 

Everything gigabit runs at full gigabit speeds. I have never had any issues there. Even WIFI transfer to a gigabit device will run at full gigabit speeds most of the time. 

 

My 10gbe capable devices include two Windows 10 PC's and a home lab PC running ESXI. They all run at 10gbe speeds. They are also set for jumbo frames. 

 

[Sarra]

1 hour ago, DogKnight said:

 

Haven't had any issues. 

To give a little more info on my setup. 

 

Router is a Netgear RAX120. The 5gbe port connects to the 10gbe switch. 

Any 10gbe and multi-gig (5gbe) gear I have is connected to the 10gbe switch. 

I have one gigabit NAS attached to the 10gbe switch using two bonded links. 

Finally, the 10gbe switch is connected to a 16 port Netgear unmanaged gigabit switch which has several devices connected to it.

 

I get expected speeds out of all devices. And by expected, I mean that transfers between any devices that have the disk speed to cope with the full bandwidth. 

If I transfer to one of my NAS units, I will not get the full 5gbe speed due to the disk performance not being able to cope with the speed. 

So it comes down to the speed each device is able to handle as to the performance I get. 

Everything gigabit runs at full gigabit speeds. I have never had any issues there. Even WIFI transfer to a gigabit device will run at full gigabit speeds most of the time. 

 

My 10gbe capable devices include two Windows 10 PC's and a home lab PC running ESXI. They all run at 10gbe speeds. They are also set for jumbo frames. 

 

Mm gotcha.

 

I will potentially be moving large files from a NVME Gen 4 X4 drive in one machine to another, most likely a MP600 to a MP600 Pro. This would probably be the only time I think I would max out the 10Gbe, but... My existing gigabit switch was purchased in 2010. Basically, looking for something that I can rely upon for at least a few years. I may continue to use the old switch for all gigabit devices, and my old server might just get stuck with the onboard gigabit since I don't know if I would really benefit from the surgery required to fit a 10G card to it. I might actually end up using that machine for my 3D printer, which I have yet to set up.

[DogKnight]

On 4/6/2022 at 2:08 PM, Sarra said:

Mm gotcha.

 

I will potentially be moving large files from a NVME Gen 4 X4 drive in one machine to another, most likely a MP600 to a MP600 Pro. This would probably be the only time I think I would max out the 10Gbe, but... My existing gigabit switch was purchased in 2010. Basically, looking for something that I can rely upon for at least a few years. I may continue to use the old switch for all gigabit devices, and my old server might just get stuck with the onboard gigabit since I don't know if I would really benefit from the surgery required to fit a 10G card to it. I might actually end up using that machine for my 3D printer, which I have yet to set up.

 

And that's really the only scenario where I see full speeds. Between NVME drives on the a couple Windows machines. There is always bottlenecks in many places, but for 10gbe at the moment, its mainly disk speed. 

[airborne spoon]

i use this one from Mikrotik and it works great. I use the SFP+ ports for a fiber line one to my main computer and one to my NAS and the rest i just have my bitcoin miners plugged into. Everything works great and the speeds are awesome.

There is a bit of a setup process that takes a few min to setup as a dumb switch so my Asus router controls most of it