10Gb NIC & Switch + Link Aggregation Suggestion

[Cat Graphic]

Hi,

 

I've recently talked about my situation where 6 of our computers are connected to 1 computer that host our online course development tool. The more of us editing/reading/previewing at the same time, the slower it takes. More details be seen on this topic: 

 

 

 

@Windows7ge suggested some upgrades that I'm willing to do to resolve our problem:

 

He mentionned to upgrade the Network Interface Card on the computer that host our courses (Server) to a 10Gbit one which also supports Link Aggregation. Our current switch doesn't support 10Gbit & neither does it support Link Aggregation, so we'll buy one that does. Does anyone know good Network Interface Cards & switch that support 10Gbit and Link aggregations?

 

Since all of our computers are located within a radius of 30ft, we won't need Cat6A, but Cat6 was suggested instead of Cat7. Since we''ll only require 10 cables, and the price difference of Cat 6 and 7 aren't too much, I thought of getting 10x 25ft Cat7.

 

@Windows7ge also mentionned:

On 9/10/2019 at 2:51 PM, Windows7ge said:

You can accomplish this with Cat6/Cat6A, DAC, or Fiberoptic cable and a switch & computer NIC that support appropriate ports.

Which I have no idea how DAC and Fiberoptic works.

 

Currently I'm imagining pluging a Cat6 or Cat7 from the new 10gb NIC on our server, plug it to the new Switch and then distribute Cat6/7 to every computer from the switch. Are there more to add for this to work or are there other wires that i'll have to buy?

 

Also, how will link aggregation work. Can I connect directly to the switch via ip address and by navigating a User Interface I can modify each port to limit 1Gb/s each?

 

I feel decent on computer hardware, but very new to networking field so if you have any other suggestions, please let me know.

 

Thank you very much.

 

[Windows7ge]

Did you verify the network was the bottleneck like I asked?

 

If you plan to go 10Gbit chances are you won't need LA for your number of clients. I was suggesting LA in the event of joining multiple 1Gbit links.

 

If each client by themselves don't max out a 1Gbit link then upgrading them would be a wasted expenditure. You can install a 10Gig NIC (Like a Intel X540-T1) in the server and run that to a switch with a 10Gig port. Then run 1gig to each client.

 

This again assuming you verified the bottleneck is the servers 1gig NIC.

[Electronics Wizardy]

24 minutes ago, Cat Graphic said:

Which I have no idea how DAC and Fiberoptic works.

Just look up sfp+ dac cables, there probably cheaper than going rj45, and has lower latency, and lower power. Its a better standard overall, only use rj45 if you need the backward compatbility

 

25 minutes ago, Cat Graphic said:

Also, how will link aggregation work. Can I connect directly to the switch via ip address and by navigating a User Interface I can modify each port to limit 1Gb/s each?

2 ports will have one ip. You need to set this up on a switch and client. But if your going to 10g you probably don't need it.

 

25 minutes ago, Cat Graphic said:

Can I connect directly to the switch via ip address and by navigating a User Interface I can modify each port to limit 1Gb/s each?

I wouldn't limit port speed, then whats the point of 10g.

 

26 minutes ago, Cat Graphic said:

Does anyone know good Network Interface Cards

Id probably go mellanox connect x3's, there cheap on ebay and work out of box with windows + linux

 

26 minutes ago, Cat Graphic said:

switch that support 10Gbit and Link aggregations?

Look at the mikrotik switches. Id get a https://mikrotik.com/product/crs317_1g_16s_rm

 

But there are cheap ones if you don't need that many ports.

 

 

 

Also you sure its network thats limiting you, your screen shot doen't show you maxing 1g. What are the full spces of the server? Try using performance monitor loggin as its great for this.

[mariushm]

Switches with 10gbps RJ45 ports are expensive.

You need a switch with at least 7 ports so that can be quite expensive. You're looking at around $550 or more, here's a couple examples:

590$ : NETGEAR 8-Port 10G Ethernet Smart Managed Plus Switch (XS708E) - with 1 x 10Gigabit SFP+, Desktop/Rackmount, and ProSAFE Limited Lifetime Protection : https://www.amazon.com/NETGEAR-Managed-8x10GBASE-T-Lifetime-Protection/dp/B01GTWPTJY/

Buffalo BS-MP2012 Multi Gigabit Business Switch – 12 port 10 GbE : https://www.newegg.com/p/N82E16833162157

 

540$ TRENDnet 8-Port 10G EdgeSmart Switch, 8 x 10GBASE-T Ports, Supports 2.5G/5G NBASE-T, 160Gbps Switch Capacity, 1U Rack Mountable, 10G Managed Ethernet Network Switch, Lifetime Protection, TEG-7080ES : https://www.amazon.com/TRENDnet-EdgeSmart-10GBASE-T-Protection-TEG-7080ES/dp/B07BV2VYVF/

 

 

Considering the computers are so close together, it would make more sense to go with SFP+ network cards and switches. Switches are much cheaper, network cards are just as cheap, just the network cable will be more expensive (either fiber plus two receivers or a DAC cable)

 

300$ : Quanta LB6M 24 port SFP+ 10gbps + 4 x RJ45 1 gbps : https://www.unixplus.com/collections/network-switches/products/quanta-lb6m-24-port-10gbe-sfp-4x-1gbe-l2-l3-switch

250$ : MikroTik 9-Port Desktop Switch, 1 Gigabit Ethernet Port, 8 SFP+ 10Gbps Ports (CRS309-1G-8S+IN) (can be configured either as switch or router at boot) : https://www.amazon.com/MikroTik-Desktop-Gigabit-Ethernet-CRS309-1G-8S/dp/B07NFXN4SS/

 

Network cards are cheap ...

67$ for single 10gbps SFP+ : Intel X520-DA1 : https://www.unixplus.com/collections/network-adapters/products/refurbished-intel-ethernet-converged-network-adapter-x520-da1?variant=3694983774235

108$ for 2 x 10gbps SFP+ : Intel x520-DA2 : https://www.unixplus.com/collections/network-adapters/products/refurbished-dell-intel-x520-da2-10gb-s-network-adapter

10Gtek for Intel E10G41BTDAG1P5 82599ES Chipset 10Gb Ethernet Converged Network Adapter (NIC), Single SFP+ Port, PCI Express 2.0 X8, Same as X520-DA1/X520-SR1

10Gtek Broadcom BCM57810S Chipset 10 Gigabit Ethernet Sever Adapter Card (NIC), Dual SFP+ Port PCIE

 

DAC cables are cheap:

24-38$ (1..5m) : https://www.unixplus.com/collections/network-cable/products/refurbished-10g-direct-attach-cable-1-meter-2-meter-3-meter-5-meter?variant=3861362704411

 

 

... but like the others... I think your network transfer speed isn't really the problem. You're probably fine with 1gbps, and it's probably the server software slowing down with concurrent users.

 

[Cat Graphic]

20 minutes ago, Windows7ge said:

Did you verify the network was the bottleneck like I asked?

I've checked the performance of the Ethernet in Task Manager but it didn't seem to reach too high. Are there other ways to test it?

 

11 minutes ago, Electronics Wizardy said:

Also you sure its network thats limiting you, your screen shot doen't show you maxing 1g. What are the full spces of the server? Try using performance monitor loggin as its great for this.

Perhaps it's indeed something else.

 

The specs are:

 

• i7 3.40hz CPU
• 14GB RAM
• GTX 1060 6GB vRAM
• 1TB HDD

Would replacing the Hard Drive to an SSD be more efficiant than the network?

 

[Electronics Wizardy]

1 minute ago, Cat Graphic said:

I've checked the performance of the Ethernet in Task Manager but it didn't seem to reach too high. Are there other ways to test it?

perf mon. What peak speedsare you heating.

 

2 minutes ago, Cat Graphic said:

Perhaps it's indeed something else.

 

The specs are:

 

• i7 3.40hz CPU
• 14GB RAM
• GTX 1060 6GB vRAM
• 1TB HDD

Would replacing the Hard Drive to an SSD be more efficiant than the network?

 

Get a ssd first, huge performance difference. If its small random reads the ssd will help way more than a network upgrade.

[Cat Graphic]

12 minutes ago, Electronics Wizardy said:

perf mon. What peak speedsare you heating.

Oh I didn't even know Performance Monitor was a thing! It gives way more details indeed. I've just tested 3/6 computers at the same time as most of my team left at this time, but here are the results:

 

Disc (Which correct me if I'm wrong but I think it's peaking and an SSD would solve this read/write issue):

 

 

CPU (Doesn't look at risk):

 

Network (Seems the network is ok, but what is TCP?):

 

 

 

Thank you for clearing this up!

 

 

[Electronics Wizardy]

59 minutes ago, Cat Graphic said:

Oh I didn't even know Performance Monitor was a thing! It gives way more details indeed. I've just tested 3/6 computers at the same time as most of my team left at this time, but here are the results:

Thats resource monitor, but it fine here

 

1 hour ago, Cat Graphic said:

Which correct me if I'm wrong but I think it's peaking and an SSD would solve this read/write issue):

Im guessing its disk from what I see and cause you got a hdd

 

[Cat Graphic]

17 hours ago, mariushm said:

Switches with 10gbps RJ45 ports are expensive.

You need a switch with at least 7 ports so that can be quite expensive. You're looking at around $550 or more, here's a couple examples:

590$ : NETGEAR 8-Port 10G Ethernet Smart Managed Plus Switch (XS708E) - with 1 x 10Gigabit SFP+, Desktop/Rackmount, and ProSAFE Limited Lifetime Protection : https://www.amazon.com/NETGEAR-Managed-8x10GBASE-T-Lifetime-Protection/dp/B01GTWPTJY/

Buffalo BS-MP2012 Multi Gigabit Business Switch – 12 port 10 GbE : https://www.newegg.com/p/N82E16833162157

 

540$ TRENDnet 8-Port 10G EdgeSmart Switch, 8 x 10GBASE-T Ports, Supports 2.5G/5G NBASE-T, 160Gbps Switch Capacity, 1U Rack Mountable, 10G Managed Ethernet Network Switch, Lifetime Protection, TEG-7080ES : https://www.amazon.com/TRENDnet-EdgeSmart-10GBASE-T-Protection-TEG-7080ES/dp/B07BV2VYVF/

 

 

Considering the computers are so close together, it would make more sense to go with SFP+ network cards and switches. Switches are much cheaper, network cards are just as cheap, just the network cable will be more expensive (either fiber plus two receivers or a DAC cable)

 

300$ : Quanta LB6M 24 port SFP+ 10gbps + 4 x RJ45 1 gbps : https://www.unixplus.com/collections/network-switches/products/quanta-lb6m-24-port-10gbe-sfp-4x-1gbe-l2-l3-switch

250$ : MikroTik 9-Port Desktop Switch, 1 Gigabit Ethernet Port, 8 SFP+ 10Gbps Ports (CRS309-1G-8S+IN) (can be configured either as switch or router at boot) : https://www.amazon.com/MikroTik-Desktop-Gigabit-Ethernet-CRS309-1G-8S/dp/B07NFXN4SS/

 

Network cards are cheap ...

67$ for single 10gbps SFP+ : Intel X520-DA1 : https://www.unixplus.com/collections/network-adapters/products/refurbished-intel-ethernet-converged-network-adapter-x520-da1?variant=3694983774235

108$ for 2 x 10gbps SFP+ : Intel x520-DA2 : https://www.unixplus.com/collections/network-adapters/products/refurbished-dell-intel-x520-da2-10gb-s-network-adapter

10Gtek for Intel E10G41BTDAG1P5 82599ES Chipset 10Gb Ethernet Converged Network Adapter (NIC), Single SFP+ Port, PCI Express 2.0 X8, Same as X520-DA1/X520-SR1

10Gtek Broadcom BCM57810S Chipset 10 Gigabit Ethernet Sever Adapter Card (NIC), Dual SFP+ Port PCIE

 

DAC cables are cheap:

24-38$ (1..5m) : https://www.unixplus.com/collections/network-cable/products/refurbished-10g-direct-attach-cable-1-meter-2-meter-3-meter-5-meter?variant=3861362704411

 

 

... but like the others... I think your network transfer speed isn't really the problem. You're probably fine with 1gbps, and it's probably the server software slowing down with concurrent users.

 

Thanks a lot for all the detailed info you've share! I've saved it for future when my team will be bigger and require actually a network transfer speed.