Jump to content

10Gbps Networking

Matt997
By Matt997
in Networking
 Share
Posted

Hi all, 

 

I ended up getting my self a 10Gbps switch the TL-SX3008F v1.0 but i plug in the two computers i want to be able to use the high speed but for some reason I only get 225MBs any ideas why. I have attached System information if need more let me know. Thank you for every one that has helped me in my  journey to a 10Gbps network. 

 

Computer 1: 

 

OPS: Windows 11 V. 21HC

MOB: B450 I AORUS PRO WIFI-CF

CPU: AMD Ryzen 5 3400G with Radeon Vega Graphics, 3700 Mhz, 4 Core(s), 8 Logical Processor(s)
RAM: 32GB

NIC: Mellanox ConnectX EN (MT26448) - PCIe 2.0 5GT/s, 10GigE Ethernet Adapter

Cable: Cable: OM3-LC-LC-D1m

NVMe: CT500P2SSD8

 

 

Computer 2

 

OPS: Truenas

MOB: B450 AORUS PRO WIFI

CPU: AMD Ryzen 5 3600

RAM: 64GB

NIC: Mellanox ConnectX EN (MT26448) - PCIe 2.0 5GT/s, 10GigE Ethernet Adapter

Cable: Cable: OM3-LC-LC-D1m

HardDrives:  

ada7
S61VNJ0NC03578R
232.89 GiB
boot-pool
SSD
ada8
WD-WCC7K2XRNACC
3.64 TiB
Media
HDD
ada1
WD-WCC7K3JTY0FP
3.64 TiB
Media
HDD
ada3
WD-WCC7K4RU9TCE
3.64 TiB
Media
HDD
ada4
WD-WCC7K4ZP14RX
3.64 TiB
Media
HDD
ada5
WD-WCC7K5PT09N3
3.64 TiB
Media
HDD
ada6
WD-WCC7K6FPU06E
3.64 TiB
Media
HDD
ada2
WD-WX12DA0EA55D
3.64 TiB
Media
HDD
ada0
ZDH8ACSY
3.64 TiB
Media
HDD

 

 

Link to comment
https://linustechtips.com/topic/1382546-10gbps-networking/
Share on other sites
Link to post
Share on other sites
Posted
9 minutes ago, Matt997 said:

I ended up getting my self a 10Gbps switch the TL-SX3008F v1.0 but i plug in the two computers i want to be able to use the high speed but for some reason I only get 225MBs any ideas why. I have attached System information if need more let me know. Thank you for every one that has helped me in my  journey to a 10Gbps network. 

What RAID topology (if any 😬) are your HDDs in?

 

9 minutes ago, Matt997 said:

Mellanox ConnectX EN (MT26448)

These are really old cards, and it's possible the drives may not play well with Win11.

Main System (Byarlant): Ryzen 9 5950X | Asus B550-Creator ProArt | EK 240mm Basic AIO | 32GB G.Skill DDR4 3600MT/s CL16 | XFX Speedster SWFT 210 RX 6600 | Samsung 990 PRO 2TB / Samsung 990 EVO Plus 4TB | Corsair RM750X | StarTech 4× USB 3.0 Card | Realtek RTL8127 10G NIC | Hyte Y60 Case | Dell U3415W Monitor | Keychron K12 Blue (RGB backlight)

 

Laptop (Narrative): Lenovo Flex 5 81X20005US | Ryzen 5 4500U | 16GB DDR4 3200MT/s (soldered) | Vega II 384SP Graphics | SKHynix P31 1TB NVMe SSD | Intel AX200 Wifi | Asus 2.5G USB NIC | Asus ProArt PA278QV | Keychron K4 Brown (white backlight)

 

Proxmox Server (Veda): Ryzen 7 3800XT | ASRock Rack X470D4U | Corsair H80i v2 | 128GB Micron DDR4 ECC 3200MT/s | 2× Samsung PM963a 960GB SSD / 4× WD 10TB / 4× Seagate 14TB Exos / 4× Micron MX500 2TB / 8× WD 12TB (custom external SAS enclosure) | Seasonic Prime Fanless 500W | Intel X550-T2 10G NIC | LSI 9300-8i HBA | Adaptec 82885T SAS Expander | Fractal Design Node 804 Case

 

Proxmox Server (La Vie en Rose): GMKtec Mini PC | Ryzen 7 5700U | 32GB Lexar DDR4 (SODIMM) | Vega II 512SP Graphics | Lexar 1TB 610 Pro SSD | 2× Realtek 8125 2.5G NICs


Media Center/Video Capture (Jesta Cannon): Ryzen 5 1600X | ASRock B450M Pro4 R2.0 | Noctua NH-L12S | 16GB Crucial DDR4 3200MT/s | EVGA GTX750Ti SC | UMIS NVMe SSD 256GB / TEAMGROUP MS30 1TB | Corsair CX450M | Viewcast Osprey 260e Video Capture | TrendNet (AQC107) 10G NIC | LG WH14NS40 BD-ROM | Silverstone Sugo SG-11 Case | Sony XR65A80K

 

Workbench (Doven Wolf): Lenovo m715q | Ryzen Pro 3 2200GE | 16GB Crucial DDR4 3200MT/s (SODIMM) | Vega 8 Graphics | SKHynix (OEM) 256GB NVMe SSD | uni 2.5G USB NIC | HDMI add-in module

 

Network:

Spoiler 
                       ┌─────────────── Office/Rack ───────────────────────────────────────────────┐
Google Fiber Webpass ── Cloud Gateway Max ══╦═ Pro XG 8 ══╦═ Flex 2.5-8 ══╦═ Doven Wolf
                      La Vie en Rose (DNS) ═╬═ Narrative  ╠═ Veda-NAS     ╠═ La Vie en Rose (vmbr)
                                Veda (DNS) ─┘             ╠═ Veda (vmbr)  ├─ Ptolemy (vmbr)
╔═════════════════════════════════════════════════════════╩═ Ptolemy-NAS  ├─ Veda (Mgmt)
║   ┌ Closet ┐      ┌───────── Bedroom ─────────┐                         └─ Veda (IPMI)
╚═══ Flex XG ══╦╤═══ Flex XG ══╤╦═ Byarlant
       (PoE)   ║│              │╠═ Narrative 
Kitchen Jack ══╣└─ Dual PoE ┐  │╚═ Jesta Cannon*
   (Testing)   ║┌─ Injector ┘  └── Work Laptop
     Bedroom ══╝│        ┌─────── Media Center ────────────────────────────┐
     Jack #2    └──────── Switch 8 ────────────┬─ nanoHD Access Point (PoE)
Notes:                                         ├─ Sony PlayStation 4 
─── is Gigabit / ═══ is Multi-Gigabit          ├─ Pioneer VSX-S520
* = cable passed from Bedroom to Media Center  └─ Sony XR65A80K (Google TV)
Link to comment
https://linustechtips.com/topic/1382546-10gbps-networking/#findComment-15057340
Share on other sites
Link to post
Share on other sites
Posted
  • Author
16 minutes ago, Electronics Wizardy said:

How are you testing speeds? Id get its a filecopy limit. Try using iperf to see how fast the network is and see if its a network of protocol/ drive limit.

I know I need to go in to CMD prompt for iperf but on the truenas server how do I do that. 

Link to comment
https://linustechtips.com/topic/1382546-10gbps-networking/#findComment-15057365
Share on other sites
Link to post
Share on other sites
Posted
  • Author
33 minutes ago, Electronics Wizardy said:

I think iperf is included in truenas. Just ssh in and run the command.

So did iperf took me a bit. Here is what i got back:

 

Server listening on 5201
-----------------------------------------------------------
Accepted connection from 192.168.2.139, port 61588
[  5] local 192.168.2.138 port 5201 connected to 192.168.2.139 port 35966
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-1.00   sec   111 MBytes   933 Mbits/sec
[  5]   1.00-2.00   sec   145 MBytes  1.22 Gbits/sec
[  5]   2.00-3.00   sec   162 MBytes  1.36 Gbits/sec
[  5]   3.00-4.00   sec   259 MBytes  2.17 Gbits/sec
[  5]   4.00-5.00   sec   264 MBytes  2.22 Gbits/sec
[  5]   5.00-6.00   sec   269 MBytes  2.25 Gbits/sec
[  5]   6.00-7.00   sec   232 MBytes  1.95 Gbits/sec
[  5]   7.00-8.00   sec   277 MBytes  2.32 Gbits/sec
[  5]   8.00-9.00   sec   273 MBytes  2.29 Gbits/sec
[  5]   9.00-10.00  sec   277 MBytes  2.32 Gbits/sec
[  5]  10.00-10.06  sec  15.0 MBytes  2.22 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-10.06  sec  0.00 Bytes  0.00 bits/sec                  sender
[  5]   0.00-10.06  sec  2.23 GBytes  1.91 Gbits/sec                  receiver
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
 

Link to comment
https://linustechtips.com/topic/1382546-10gbps-networking/#findComment-15057428
Share on other sites
Link to post
Share on other sites
Posted
1 minute ago, Matt997 said:

So did iperf took me a bit. Here is what i got back:

 

Server listening on 5201
-----------------------------------------------------------
Accepted connection from 192.168.2.139, port 61588
[  5] local 192.168.2.138 port 5201 connected to 192.168.2.139 port 35966
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-1.00   sec   111 MBytes   933 Mbits/sec
[  5]   1.00-2.00   sec   145 MBytes  1.22 Gbits/sec
[  5]   2.00-3.00   sec   162 MBytes  1.36 Gbits/sec
[  5]   3.00-4.00   sec   259 MBytes  2.17 Gbits/sec
[  5]   4.00-5.00   sec   264 MBytes  2.22 Gbits/sec
[  5]   5.00-6.00   sec   269 MBytes  2.25 Gbits/sec
[  5]   6.00-7.00   sec   232 MBytes  1.95 Gbits/sec
[  5]   7.00-8.00   sec   277 MBytes  2.32 Gbits/sec
[  5]   8.00-9.00   sec   273 MBytes  2.29 Gbits/sec
[  5]   9.00-10.00  sec   277 MBytes  2.32 Gbits/sec
[  5]  10.00-10.06  sec  15.0 MBytes  2.22 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-10.06  sec  0.00 Bytes  0.00 bits/sec                  sender
[  5]   0.00-10.06  sec  2.23 GBytes  1.91 Gbits/sec                  receiver
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
 

Id try using the -p 8 for 8 parallel streams to test if its a network issue or not.

 

Seems to be a network issue. Also try a direct link between the pcs if you can to rule out the switch.

Link to comment
https://linustechtips.com/topic/1382546-10gbps-networking/#findComment-15057434
Share on other sites
Link to post
Share on other sites
Posted
  • Author
57 minutes ago, Electronics Wizardy said:

Id try using the -p 8 for 8 parallel streams to test if its a network issue or not.

 

Seems to be a network issue. Also try a direct link between the pcs if you can to rule out the switch.

Was just looking at something and though this might be why I am having the issue. The NVME that i have only can handle 2500 MB's. Would this be the issue in the chain.

Link to comment
https://linustechtips.com/topic/1382546-10gbps-networking/#findComment-15057572
Share on other sites
Link to post
Share on other sites
Posted
9 minutes ago, Matt997 said:

Was just looking at something and though this might be why I am having the issue. The NVME that i have only can handle 2500 MB's. Would this be the issue in the chain.

2500MBps ≈ 25000Mbps (Bytes vs bits); not the source of your problem.

Main System (Byarlant): Ryzen 9 5950X | Asus B550-Creator ProArt | EK 240mm Basic AIO | 32GB G.Skill DDR4 3600MT/s CL16 | XFX Speedster SWFT 210 RX 6600 | Samsung 990 PRO 2TB / Samsung 990 EVO Plus 4TB | Corsair RM750X | StarTech 4× USB 3.0 Card | Realtek RTL8127 10G NIC | Hyte Y60 Case | Dell U3415W Monitor | Keychron K12 Blue (RGB backlight)

 

Laptop (Narrative): Lenovo Flex 5 81X20005US | Ryzen 5 4500U | 16GB DDR4 3200MT/s (soldered) | Vega II 384SP Graphics | SKHynix P31 1TB NVMe SSD | Intel AX200 Wifi | Asus 2.5G USB NIC | Asus ProArt PA278QV | Keychron K4 Brown (white backlight)

 

Proxmox Server (Veda): Ryzen 7 3800XT | ASRock Rack X470D4U | Corsair H80i v2 | 128GB Micron DDR4 ECC 3200MT/s | 2× Samsung PM963a 960GB SSD / 4× WD 10TB / 4× Seagate 14TB Exos / 4× Micron MX500 2TB / 8× WD 12TB (custom external SAS enclosure) | Seasonic Prime Fanless 500W | Intel X550-T2 10G NIC | LSI 9300-8i HBA | Adaptec 82885T SAS Expander | Fractal Design Node 804 Case

 

Proxmox Server (La Vie en Rose): GMKtec Mini PC | Ryzen 7 5700U | 32GB Lexar DDR4 (SODIMM) | Vega II 512SP Graphics | Lexar 1TB 610 Pro SSD | 2× Realtek 8125 2.5G NICs


Media Center/Video Capture (Jesta Cannon): Ryzen 5 1600X | ASRock B450M Pro4 R2.0 | Noctua NH-L12S | 16GB Crucial DDR4 3200MT/s | EVGA GTX750Ti SC | UMIS NVMe SSD 256GB / TEAMGROUP MS30 1TB | Corsair CX450M | Viewcast Osprey 260e Video Capture | TrendNet (AQC107) 10G NIC | LG WH14NS40 BD-ROM | Silverstone Sugo SG-11 Case | Sony XR65A80K

 

Workbench (Doven Wolf): Lenovo m715q | Ryzen Pro 3 2200GE | 16GB Crucial DDR4 3200MT/s (SODIMM) | Vega 8 Graphics | SKHynix (OEM) 256GB NVMe SSD | uni 2.5G USB NIC | HDMI add-in module

 

Network:

Spoiler 
                       ┌─────────────── Office/Rack ───────────────────────────────────────────────┐
Google Fiber Webpass ── Cloud Gateway Max ══╦═ Pro XG 8 ══╦═ Flex 2.5-8 ══╦═ Doven Wolf
                      La Vie en Rose (DNS) ═╬═ Narrative  ╠═ Veda-NAS     ╠═ La Vie en Rose (vmbr)
                                Veda (DNS) ─┘             ╠═ Veda (vmbr)  ├─ Ptolemy (vmbr)
╔═════════════════════════════════════════════════════════╩═ Ptolemy-NAS  ├─ Veda (Mgmt)
║   ┌ Closet ┐      ┌───────── Bedroom ─────────┐                         └─ Veda (IPMI)
╚═══ Flex XG ══╦╤═══ Flex XG ══╤╦═ Byarlant
       (PoE)   ║│              │╠═ Narrative 
Kitchen Jack ══╣└─ Dual PoE ┐  │╚═ Jesta Cannon*
   (Testing)   ║┌─ Injector ┘  └── Work Laptop
     Bedroom ══╝│        ┌─────── Media Center ────────────────────────────┐
     Jack #2    └──────── Switch 8 ────────────┬─ nanoHD Access Point (PoE)
Notes:                                         ├─ Sony PlayStation 4 
─── is Gigabit / ═══ is Multi-Gigabit          ├─ Pioneer VSX-S520
* = cable passed from Bedroom to Media Center  └─ Sony XR65A80K (Google TV)
Link to comment
https://linustechtips.com/topic/1382546-10gbps-networking/#findComment-15057593
Share on other sites
Link to post
Share on other sites
Posted
  • Author
1 hour ago, Electronics Wizardy said:

Id try using the -p 8 for 8 parallel streams to test if its a network issue or not.

 

Seems to be a network issue. Also try a direct link between the pcs if you can to rule out the switch.

how do you do "-p 8 for 8 parallel streams"

Link to comment
https://linustechtips.com/topic/1382546-10gbps-networking/#findComment-15057616
Share on other sites
Link to post
Share on other sites
Posted
21 minutes ago, Matt997 said:

how do you do "-p 8 for 8 parallel streams"

Add "-p 8" (without quotes) to the end of your iperf command.

Main System (Byarlant): Ryzen 9 5950X | Asus B550-Creator ProArt | EK 240mm Basic AIO | 32GB G.Skill DDR4 3600MT/s CL16 | XFX Speedster SWFT 210 RX 6600 | Samsung 990 PRO 2TB / Samsung 990 EVO Plus 4TB | Corsair RM750X | StarTech 4× USB 3.0 Card | Realtek RTL8127 10G NIC | Hyte Y60 Case | Dell U3415W Monitor | Keychron K12 Blue (RGB backlight)

 

Laptop (Narrative): Lenovo Flex 5 81X20005US | Ryzen 5 4500U | 16GB DDR4 3200MT/s (soldered) | Vega II 384SP Graphics | SKHynix P31 1TB NVMe SSD | Intel AX200 Wifi | Asus 2.5G USB NIC | Asus ProArt PA278QV | Keychron K4 Brown (white backlight)

 

Proxmox Server (Veda): Ryzen 7 3800XT | ASRock Rack X470D4U | Corsair H80i v2 | 128GB Micron DDR4 ECC 3200MT/s | 2× Samsung PM963a 960GB SSD / 4× WD 10TB / 4× Seagate 14TB Exos / 4× Micron MX500 2TB / 8× WD 12TB (custom external SAS enclosure) | Seasonic Prime Fanless 500W | Intel X550-T2 10G NIC | LSI 9300-8i HBA | Adaptec 82885T SAS Expander | Fractal Design Node 804 Case

 

Proxmox Server (La Vie en Rose): GMKtec Mini PC | Ryzen 7 5700U | 32GB Lexar DDR4 (SODIMM) | Vega II 512SP Graphics | Lexar 1TB 610 Pro SSD | 2× Realtek 8125 2.5G NICs


Media Center/Video Capture (Jesta Cannon): Ryzen 5 1600X | ASRock B450M Pro4 R2.0 | Noctua NH-L12S | 16GB Crucial DDR4 3200MT/s | EVGA GTX750Ti SC | UMIS NVMe SSD 256GB / TEAMGROUP MS30 1TB | Corsair CX450M | Viewcast Osprey 260e Video Capture | TrendNet (AQC107) 10G NIC | LG WH14NS40 BD-ROM | Silverstone Sugo SG-11 Case | Sony XR65A80K

 

Workbench (Doven Wolf): Lenovo m715q | Ryzen Pro 3 2200GE | 16GB Crucial DDR4 3200MT/s (SODIMM) | Vega 8 Graphics | SKHynix (OEM) 256GB NVMe SSD | uni 2.5G USB NIC | HDMI add-in module

 

Network:

Spoiler 
                       ┌─────────────── Office/Rack ───────────────────────────────────────────────┐
Google Fiber Webpass ── Cloud Gateway Max ══╦═ Pro XG 8 ══╦═ Flex 2.5-8 ══╦═ Doven Wolf
                      La Vie en Rose (DNS) ═╬═ Narrative  ╠═ Veda-NAS     ╠═ La Vie en Rose (vmbr)
                                Veda (DNS) ─┘             ╠═ Veda (vmbr)  ├─ Ptolemy (vmbr)
╔═════════════════════════════════════════════════════════╩═ Ptolemy-NAS  ├─ Veda (Mgmt)
║   ┌ Closet ┐      ┌───────── Bedroom ─────────┐                         └─ Veda (IPMI)
╚═══ Flex XG ══╦╤═══ Flex XG ══╤╦═ Byarlant
       (PoE)   ║│              │╠═ Narrative 
Kitchen Jack ══╣└─ Dual PoE ┐  │╚═ Jesta Cannon*
   (Testing)   ║┌─ Injector ┘  └── Work Laptop
     Bedroom ══╝│        ┌─────── Media Center ────────────────────────────┐
     Jack #2    └──────── Switch 8 ────────────┬─ nanoHD Access Point (PoE)
Notes:                                         ├─ Sony PlayStation 4 
─── is Gigabit / ═══ is Multi-Gigabit          ├─ Pioneer VSX-S520
* = cable passed from Bedroom to Media Center  └─ Sony XR65A80K (Google TV)
Link to comment
https://linustechtips.com/topic/1382546-10gbps-networking/#findComment-15057624
Share on other sites
Link to post
Share on other sites
Posted
  • Author
8 minutes ago, AbydosOne said:

Add "-p 8" (without quotes) to the end of your iperf command.

Connecting to host 192.168.2.139, port 8
[  4] local 192.168.2.138 port 57217 connected to 192.168.2.139 port 8
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   240 MBytes  2.01 Gbits/sec
[  4]   1.00-2.00   sec   242 MBytes  2.03 Gbits/sec
[  4]   2.00-3.00   sec   229 MBytes  1.92 Gbits/sec
[  4]   3.00-4.00   sec   291 MBytes  2.44 Gbits/sec
[  4]   4.00-5.00   sec   301 MBytes  2.52 Gbits/sec
[  4]   5.00-6.00   sec   230 MBytes  1.93 Gbits/sec
[  4]   6.00-7.00   sec   293 MBytes  2.46 Gbits/sec
[  4]   7.00-8.00   sec   112 MBytes   935 Mbits/sec
[  4]   8.00-9.00   sec   248 MBytes  2.08 Gbits/sec
[  4]   9.00-10.00  sec   254 MBytes  2.13 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  2.38 GBytes  2.05 Gbits/sec                  sender
[  4]   0.00-10.00  sec  2.38 GBytes  2.05 Gbits/sec                  receiver

iperf Done.

C:\Users\matth\Downloads\iperf-3.1.3-win64\iperf-3.1.3-win64>

Link to comment
https://linustechtips.com/topic/1382546-10gbps-networking/#findComment-15057642
Share on other sites
Link to post
Share on other sites
Posted
6 minutes ago, Matt997 said:

Connecting to host 192.168.2.139, port 8
[  4] local 192.168.2.138 port 57217 connected to 192.168.2.139 port 8
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   240 MBytes  2.01 Gbits/sec
[  4]   1.00-2.00   sec   242 MBytes  2.03 Gbits/sec
[  4]   2.00-3.00   sec   229 MBytes  1.92 Gbits/sec
[  4]   3.00-4.00   sec   291 MBytes  2.44 Gbits/sec
[  4]   4.00-5.00   sec   301 MBytes  2.52 Gbits/sec
[  4]   5.00-6.00   sec   230 MBytes  1.93 Gbits/sec
[  4]   6.00-7.00   sec   293 MBytes  2.46 Gbits/sec
[  4]   7.00-8.00   sec   112 MBytes   935 Mbits/sec
[  4]   8.00-9.00   sec   248 MBytes  2.08 Gbits/sec
[  4]   9.00-10.00  sec   254 MBytes  2.13 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  2.38 GBytes  2.05 Gbits/sec                  sender
[  4]   0.00-10.00  sec  2.38 GBytes  2.05 Gbits/sec                  receiver

iperf Done.

C:\Users\matth\Downloads\iperf-3.1.3-win64\iperf-3.1.3-win64>

Sorry, "-P 8" (capital P).

 

(Linux/command line people wonder why laypeople don't like command lines, and it stuff like this that I use as examples)

Main System (Byarlant): Ryzen 9 5950X | Asus B550-Creator ProArt | EK 240mm Basic AIO | 32GB G.Skill DDR4 3600MT/s CL16 | XFX Speedster SWFT 210 RX 6600 | Samsung 990 PRO 2TB / Samsung 990 EVO Plus 4TB | Corsair RM750X | StarTech 4× USB 3.0 Card | Realtek RTL8127 10G NIC | Hyte Y60 Case | Dell U3415W Monitor | Keychron K12 Blue (RGB backlight)

 

Laptop (Narrative): Lenovo Flex 5 81X20005US | Ryzen 5 4500U | 16GB DDR4 3200MT/s (soldered) | Vega II 384SP Graphics | SKHynix P31 1TB NVMe SSD | Intel AX200 Wifi | Asus 2.5G USB NIC | Asus ProArt PA278QV | Keychron K4 Brown (white backlight)

 

Proxmox Server (Veda): Ryzen 7 3800XT | ASRock Rack X470D4U | Corsair H80i v2 | 128GB Micron DDR4 ECC 3200MT/s | 2× Samsung PM963a 960GB SSD / 4× WD 10TB / 4× Seagate 14TB Exos / 4× Micron MX500 2TB / 8× WD 12TB (custom external SAS enclosure) | Seasonic Prime Fanless 500W | Intel X550-T2 10G NIC | LSI 9300-8i HBA | Adaptec 82885T SAS Expander | Fractal Design Node 804 Case

 

Proxmox Server (La Vie en Rose): GMKtec Mini PC | Ryzen 7 5700U | 32GB Lexar DDR4 (SODIMM) | Vega II 512SP Graphics | Lexar 1TB 610 Pro SSD | 2× Realtek 8125 2.5G NICs


Media Center/Video Capture (Jesta Cannon): Ryzen 5 1600X | ASRock B450M Pro4 R2.0 | Noctua NH-L12S | 16GB Crucial DDR4 3200MT/s | EVGA GTX750Ti SC | UMIS NVMe SSD 256GB / TEAMGROUP MS30 1TB | Corsair CX450M | Viewcast Osprey 260e Video Capture | TrendNet (AQC107) 10G NIC | LG WH14NS40 BD-ROM | Silverstone Sugo SG-11 Case | Sony XR65A80K

 

Workbench (Doven Wolf): Lenovo m715q | Ryzen Pro 3 2200GE | 16GB Crucial DDR4 3200MT/s (SODIMM) | Vega 8 Graphics | SKHynix (OEM) 256GB NVMe SSD | uni 2.5G USB NIC | HDMI add-in module

 

Network:

Spoiler 
                       ┌─────────────── Office/Rack ───────────────────────────────────────────────┐
Google Fiber Webpass ── Cloud Gateway Max ══╦═ Pro XG 8 ══╦═ Flex 2.5-8 ══╦═ Doven Wolf
                      La Vie en Rose (DNS) ═╬═ Narrative  ╠═ Veda-NAS     ╠═ La Vie en Rose (vmbr)
                                Veda (DNS) ─┘             ╠═ Veda (vmbr)  ├─ Ptolemy (vmbr)
╔═════════════════════════════════════════════════════════╩═ Ptolemy-NAS  ├─ Veda (Mgmt)
║   ┌ Closet ┐      ┌───────── Bedroom ─────────┐                         └─ Veda (IPMI)
╚═══ Flex XG ══╦╤═══ Flex XG ══╤╦═ Byarlant
       (PoE)   ║│              │╠═ Narrative 
Kitchen Jack ══╣└─ Dual PoE ┐  │╚═ Jesta Cannon*
   (Testing)   ║┌─ Injector ┘  └── Work Laptop
     Bedroom ══╝│        ┌─────── Media Center ────────────────────────────┐
     Jack #2    └──────── Switch 8 ────────────┬─ nanoHD Access Point (PoE)
Notes:                                         ├─ Sony PlayStation 4 
─── is Gigabit / ═══ is Multi-Gigabit          ├─ Pioneer VSX-S520
* = cable passed from Bedroom to Media Center  └─ Sony XR65A80K (Google TV)
Link to comment
https://linustechtips.com/topic/1382546-10gbps-networking/#findComment-15057664
Share on other sites
Link to post
Share on other sites
Posted
  • Author
2 hours ago, AbydosOne said:

Sorry, "-P 8" (capital P).

 

(Linux/command line people wonder why laypeople don't like command lines, and it stuff like this that I use as examples)

Lol, It happens. 

 

So info as follows: 

 

C:\Users\matth\Downloads\iperf-3.1.3\iperf-3.1.3-win64>iperf3 -c 192.168.2.139 -P 8
Connecting to host 192.168.2.139, port 5201
[  4] local 192.168.2.138 port 49190 connected to 192.168.2.139 port 5201
[  6] local 192.168.2.138 port 49191 connected to 192.168.2.139 port 5201
[  8] local 192.168.2.138 port 49192 connected to 192.168.2.139 port 5201
[ 10] local 192.168.2.138 port 49193 connected to 192.168.2.139 port 5201
[ 12] local 192.168.2.138 port 49194 connected to 192.168.2.139 port 5201
[ 14] local 192.168.2.138 port 49195 connected to 192.168.2.139 port 5201
[ 16] local 192.168.2.138 port 49196 connected to 192.168.2.139 port 5201
[ 18] local 192.168.2.138 port 49197 connected to 192.168.2.139 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.01   sec  32.6 MBytes   270 Mbits/sec
[  6]   0.00-1.01   sec  38.6 MBytes   320 Mbits/sec
[  8]   0.00-1.01   sec  33.6 MBytes   279 Mbits/sec
[ 10]   0.00-1.01   sec  26.8 MBytes   222 Mbits/sec
[ 12]   0.00-1.01   sec  23.9 MBytes   198 Mbits/sec
[ 14]   0.00-1.01   sec  35.9 MBytes   297 Mbits/sec
[ 16]   0.00-1.01   sec  38.5 MBytes   319 Mbits/sec
[ 18]   0.00-1.01   sec  32.1 MBytes   266 Mbits/sec
[SUM]   0.00-1.01   sec   262 MBytes  2.17 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   1.01-2.00   sec  38.4 MBytes   326 Mbits/sec
[  6]   1.01-2.00   sec  39.5 MBytes   335 Mbits/sec
[  8]   1.01-2.00   sec  27.4 MBytes   232 Mbits/sec
[ 10]   1.01-2.00   sec  37.6 MBytes   319 Mbits/sec
[ 12]   1.01-2.00   sec  23.6 MBytes   200 Mbits/sec
[ 14]   1.01-2.00   sec  31.6 MBytes   268 Mbits/sec
[ 16]   1.01-2.00   sec  39.5 MBytes   335 Mbits/sec
[ 18]   1.01-2.00   sec  33.0 MBytes   280 Mbits/sec
[SUM]   1.01-2.00   sec   271 MBytes  2.30 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   2.00-3.00   sec  32.9 MBytes   276 Mbits/sec
[  6]   2.00-3.00   sec  43.5 MBytes   365 Mbits/sec
[  8]   2.00-3.00   sec  43.6 MBytes   366 Mbits/sec
[ 10]   2.00-3.00   sec  33.9 MBytes   284 Mbits/sec
[ 12]   2.00-3.00   sec  36.8 MBytes   308 Mbits/sec
[ 14]   2.00-3.00   sec  29.9 MBytes   251 Mbits/sec
[ 16]   2.00-3.00   sec  41.8 MBytes   350 Mbits/sec
[ 18]   2.00-3.00   sec  39.0 MBytes   327 Mbits/sec
[SUM]   2.00-3.00   sec   301 MBytes  2.53 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   3.00-4.01   sec  26.1 MBytes   218 Mbits/sec
[  6]   3.00-4.01   sec  30.6 MBytes   255 Mbits/sec
[  8]   3.00-4.01   sec  37.9 MBytes   315 Mbits/sec
[ 10]   3.00-4.01   sec  29.6 MBytes   247 Mbits/sec
[ 12]   3.00-4.01   sec  23.4 MBytes   195 Mbits/sec
[ 14]   3.00-4.01   sec  24.4 MBytes   203 Mbits/sec
[ 16]   3.00-4.01   sec  37.5 MBytes   312 Mbits/sec
[ 18]   3.00-4.01   sec  37.5 MBytes   312 Mbits/sec
[SUM]   3.00-4.01   sec   247 MBytes  2.06 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   4.01-5.00   sec  31.1 MBytes   263 Mbits/sec
[  6]   4.01-5.00   sec  38.4 MBytes   324 Mbits/sec
[  8]   4.01-5.00   sec  36.9 MBytes   312 Mbits/sec
[ 10]   4.01-5.00   sec  32.8 MBytes   277 Mbits/sec
[ 12]   4.01-5.00   sec  35.5 MBytes   300 Mbits/sec
[ 14]   4.01-5.00   sec  32.5 MBytes   275 Mbits/sec
[ 16]   4.01-5.00   sec  30.4 MBytes   257 Mbits/sec
[ 18]   4.01-5.00   sec  32.8 MBytes   277 Mbits/sec
[SUM]   4.01-5.00   sec   270 MBytes  2.28 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   5.00-6.00   sec  29.2 MBytes   245 Mbits/sec
[  6]   5.00-6.00   sec  41.6 MBytes   349 Mbits/sec
[  8]   5.00-6.00   sec  24.0 MBytes   201 Mbits/sec
[ 10]   5.00-6.00   sec  27.2 MBytes   229 Mbits/sec
[ 12]   5.00-6.00   sec  32.1 MBytes   269 Mbits/sec
[ 14]   5.00-6.00   sec  38.0 MBytes   319 Mbits/sec
[ 16]   5.00-6.00   sec  41.4 MBytes   347 Mbits/sec
[ 18]   5.00-6.00   sec  32.6 MBytes   274 Mbits/sec
[SUM]   5.00-6.00   sec   266 MBytes  2.23 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   6.00-7.00   sec  35.4 MBytes   297 Mbits/sec
[  6]   6.00-7.00   sec  42.0 MBytes   352 Mbits/sec
[  8]   6.00-7.00   sec  34.5 MBytes   289 Mbits/sec
[ 10]   6.00-7.00   sec  31.2 MBytes   262 Mbits/sec
[ 12]   6.00-7.00   sec  28.8 MBytes   241 Mbits/sec
[ 14]   6.00-7.00   sec  37.4 MBytes   314 Mbits/sec
[ 16]   6.00-7.00   sec  46.2 MBytes   388 Mbits/sec
[ 18]   6.00-7.00   sec  36.9 MBytes   309 Mbits/sec
[SUM]   6.00-7.00   sec   292 MBytes  2.45 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   7.00-8.00   sec  29.9 MBytes   251 Mbits/sec
[  6]   7.00-8.00   sec  39.4 MBytes   330 Mbits/sec
[  8]   7.00-8.00   sec  27.8 MBytes   233 Mbits/sec
[ 10]   7.00-8.00   sec  33.4 MBytes   280 Mbits/sec
[ 12]   7.00-8.00   sec  33.2 MBytes   279 Mbits/sec
[ 14]   7.00-8.00   sec  18.9 MBytes   158 Mbits/sec
[ 16]   7.00-8.00   sec  36.1 MBytes   303 Mbits/sec
[ 18]   7.00-8.00   sec  30.2 MBytes   254 Mbits/sec
[SUM]   7.00-8.00   sec   249 MBytes  2.09 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   8.00-9.00   sec  33.8 MBytes   283 Mbits/sec
[  6]   8.00-9.00   sec  42.5 MBytes   357 Mbits/sec
[  8]   8.00-9.00   sec  34.2 MBytes   287 Mbits/sec
[ 10]   8.00-9.00   sec  31.9 MBytes   267 Mbits/sec
[ 12]   8.00-9.00   sec  30.0 MBytes   252 Mbits/sec
[ 14]   8.00-9.00   sec  34.9 MBytes   293 Mbits/sec
[ 16]   8.00-9.00   sec  42.9 MBytes   360 Mbits/sec
[ 18]   8.00-9.00   sec  39.2 MBytes   329 Mbits/sec
[SUM]   8.00-9.00   sec   289 MBytes  2.43 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   9.00-10.00  sec  25.4 MBytes   212 Mbits/sec
[  6]   9.00-10.00  sec  29.8 MBytes   249 Mbits/sec
[  8]   9.00-10.00  sec  45.2 MBytes   378 Mbits/sec
[ 10]   9.00-10.00  sec  24.0 MBytes   201 Mbits/sec
[ 12]   9.00-10.00  sec  21.6 MBytes   181 Mbits/sec
[ 14]   9.00-10.00  sec  11.5 MBytes  96.1 Mbits/sec
[ 16]   9.00-10.00  sec  33.8 MBytes   282 Mbits/sec
[ 18]   9.00-10.00  sec  24.8 MBytes   207 Mbits/sec
[SUM]   9.00-10.00  sec   216 MBytes  1.81 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec   315 MBytes   264 Mbits/sec                  sender
[  4]   0.00-10.00  sec   314 MBytes   263 Mbits/sec                  receiver
[  6]   0.00-10.00  sec   386 MBytes   324 Mbits/sec                  sender
[  6]   0.00-10.00  sec   385 MBytes   323 Mbits/sec                  receiver
[  8]   0.00-10.00  sec   345 MBytes   289 Mbits/sec                  sender
[  8]   0.00-10.00  sec   344 MBytes   289 Mbits/sec                  receiver
[ 10]   0.00-10.00  sec   308 MBytes   259 Mbits/sec                  sender
[ 10]   0.00-10.00  sec   308 MBytes   258 Mbits/sec                  receiver
[ 12]   0.00-10.00  sec   289 MBytes   242 Mbits/sec                  sender
[ 12]   0.00-10.00  sec   288 MBytes   242 Mbits/sec                  receiver
[ 14]   0.00-10.00  sec   295 MBytes   247 Mbits/sec                  sender
[ 14]   0.00-10.00  sec   295 MBytes   247 Mbits/sec                  receiver
[ 16]   0.00-10.00  sec   388 MBytes   325 Mbits/sec                  sender
[ 16]   0.00-10.00  sec   387 MBytes   325 Mbits/sec                  receiver
[ 18]   0.00-10.00  sec   338 MBytes   284 Mbits/sec                  sender
[ 18]   0.00-10.00  sec   337 MBytes   283 Mbits/sec                  receiver
[SUM]   0.00-10.00  sec  2.60 GBytes  2.23 Gbits/sec                  sender
[SUM]   0.00-10.00  sec  2.60 GBytes  2.23 Gbits/sec                  receiver

iperf Done.

C:\Users\matth\Downloads\iperf-3.1.3\iperf-3.1.3-win64>

Link to comment
https://linustechtips.com/topic/1382546-10gbps-networking/#findComment-15057885
Share on other sites
Link to post
Share on other sites
Posted
Just now, Matt997 said:

Lol, It happens. 

 

So info as follows: 

Based on all the results you've posted, there's something in the network that's limiting bandwidth to ~2.5Gbps. What kind of cables are you using? Are you using DAC cables or RJ45 transceivers? What happens if you directly connect the two 10G NICs (assign IPs manually) and run the tests again?

Main System (Byarlant): Ryzen 9 5950X | Asus B550-Creator ProArt | EK 240mm Basic AIO | 32GB G.Skill DDR4 3600MT/s CL16 | XFX Speedster SWFT 210 RX 6600 | Samsung 990 PRO 2TB / Samsung 990 EVO Plus 4TB | Corsair RM750X | StarTech 4× USB 3.0 Card | Realtek RTL8127 10G NIC | Hyte Y60 Case | Dell U3415W Monitor | Keychron K12 Blue (RGB backlight)

 

Laptop (Narrative): Lenovo Flex 5 81X20005US | Ryzen 5 4500U | 16GB DDR4 3200MT/s (soldered) | Vega II 384SP Graphics | SKHynix P31 1TB NVMe SSD | Intel AX200 Wifi | Asus 2.5G USB NIC | Asus ProArt PA278QV | Keychron K4 Brown (white backlight)

 

Proxmox Server (Veda): Ryzen 7 3800XT | ASRock Rack X470D4U | Corsair H80i v2 | 128GB Micron DDR4 ECC 3200MT/s | 2× Samsung PM963a 960GB SSD / 4× WD 10TB / 4× Seagate 14TB Exos / 4× Micron MX500 2TB / 8× WD 12TB (custom external SAS enclosure) | Seasonic Prime Fanless 500W | Intel X550-T2 10G NIC | LSI 9300-8i HBA | Adaptec 82885T SAS Expander | Fractal Design Node 804 Case

 

Proxmox Server (La Vie en Rose): GMKtec Mini PC | Ryzen 7 5700U | 32GB Lexar DDR4 (SODIMM) | Vega II 512SP Graphics | Lexar 1TB 610 Pro SSD | 2× Realtek 8125 2.5G NICs


Media Center/Video Capture (Jesta Cannon): Ryzen 5 1600X | ASRock B450M Pro4 R2.0 | Noctua NH-L12S | 16GB Crucial DDR4 3200MT/s | EVGA GTX750Ti SC | UMIS NVMe SSD 256GB / TEAMGROUP MS30 1TB | Corsair CX450M | Viewcast Osprey 260e Video Capture | TrendNet (AQC107) 10G NIC | LG WH14NS40 BD-ROM | Silverstone Sugo SG-11 Case | Sony XR65A80K

 

Workbench (Doven Wolf): Lenovo m715q | Ryzen Pro 3 2200GE | 16GB Crucial DDR4 3200MT/s (SODIMM) | Vega 8 Graphics | SKHynix (OEM) 256GB NVMe SSD | uni 2.5G USB NIC | HDMI add-in module

 

Network:

Spoiler 
                       ┌─────────────── Office/Rack ───────────────────────────────────────────────┐
Google Fiber Webpass ── Cloud Gateway Max ══╦═ Pro XG 8 ══╦═ Flex 2.5-8 ══╦═ Doven Wolf
                      La Vie en Rose (DNS) ═╬═ Narrative  ╠═ Veda-NAS     ╠═ La Vie en Rose (vmbr)
                                Veda (DNS) ─┘             ╠═ Veda (vmbr)  ├─ Ptolemy (vmbr)
╔═════════════════════════════════════════════════════════╩═ Ptolemy-NAS  ├─ Veda (Mgmt)
║   ┌ Closet ┐      ┌───────── Bedroom ─────────┐                         └─ Veda (IPMI)
╚═══ Flex XG ══╦╤═══ Flex XG ══╤╦═ Byarlant
       (PoE)   ║│              │╠═ Narrative 
Kitchen Jack ══╣└─ Dual PoE ┐  │╚═ Jesta Cannon*
   (Testing)   ║┌─ Injector ┘  └── Work Laptop
     Bedroom ══╝│        ┌─────── Media Center ────────────────────────────┐
     Jack #2    └──────── Switch 8 ────────────┬─ nanoHD Access Point (PoE)
Notes:                                         ├─ Sony PlayStation 4 
─── is Gigabit / ═══ is Multi-Gigabit          ├─ Pioneer VSX-S520
* = cable passed from Bedroom to Media Center  └─ Sony XR65A80K (Google TV)
Link to comment
https://linustechtips.com/topic/1382546-10gbps-networking/#findComment-15057893
Share on other sites
Link to post
Share on other sites
Posted
  • Author
2 minutes ago, AbydosOne said:

Based on all the results you've posted, there's something in the network that's limiting bandwidth to ~2.5Gbps. What kind of cables are you using? Are you using DAC cables or RJ45 transceivers? What happens if you directly connect the two 10G NICs (assign IPs manually) and run the tests again?

so i just uninstalled and reinstalled the drivers via device manger and got the following back: 

- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   3.00-4.00   sec  57.8 MBytes   486 Mbits/sec
[  6]   3.00-4.00   sec  49.1 MBytes   413 Mbits/sec
[  8]   3.00-4.00   sec  32.0 MBytes   269 Mbits/sec
[ 10]   3.00-4.00   sec  27.4 MBytes   230 Mbits/sec
[ 12]   3.00-4.00   sec  72.5 MBytes   610 Mbits/sec
[ 14]   3.00-4.00   sec  74.4 MBytes   626 Mbits/sec
[ 16]   3.00-4.00   sec  48.0 MBytes   404 Mbits/sec
[ 18]   3.00-4.00   sec  48.4 MBytes   407 Mbits/sec
[SUM]   3.00-4.00   sec   410 MBytes  3.45 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   4.00-5.00   sec  57.1 MBytes   477 Mbits/sec
[  6]   4.00-5.00   sec  37.0 MBytes   309 Mbits/sec
[  8]   4.00-5.00   sec  41.9 MBytes   350 Mbits/sec
[ 10]   4.00-5.00   sec  70.6 MBytes   590 Mbits/sec
[ 12]   4.00-5.00   sec  53.4 MBytes   446 Mbits/sec
[ 14]   4.00-5.00   sec  92.5 MBytes   773 Mbits/sec
[ 16]   4.00-5.00   sec  25.6 MBytes   214 Mbits/sec
[ 18]   4.00-5.00   sec  51.1 MBytes   427 Mbits/sec
[SUM]   4.00-5.00   sec   429 MBytes  3.59 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   5.00-6.00   sec  0.00 Bytes  0.00 bits/sec
[  6]   5.00-6.00   sec  51.6 MBytes   433 Mbits/sec
[  8]   5.00-6.00   sec  36.1 MBytes   303 Mbits/sec
[ 10]   5.00-6.00   sec  66.1 MBytes   554 Mbits/sec
[ 12]   5.00-6.00   sec  61.8 MBytes   518 Mbits/sec
[ 14]   5.00-6.00   sec  83.2 MBytes   698 Mbits/sec
[ 16]   5.00-6.00   sec  83.5 MBytes   700 Mbits/sec
[ 18]   5.00-6.00   sec  78.0 MBytes   654 Mbits/sec
[SUM]   5.00-6.00   sec   460 MBytes  3.86 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   6.00-7.00   sec  40.9 MBytes   343 Mbits/sec
[  6]   6.00-7.00   sec  45.1 MBytes   379 Mbits/sec
[  8]   6.00-7.00   sec  71.5 MBytes   600 Mbits/sec
[ 10]   6.00-7.00   sec  80.8 MBytes   677 Mbits/sec
[ 12]   6.00-7.00   sec  13.6 MBytes   114 Mbits/sec
[ 14]   6.00-7.00   sec  87.9 MBytes   737 Mbits/sec
[ 16]   6.00-7.00   sec  85.1 MBytes   714 Mbits/sec
[ 18]   6.00-7.00   sec  94.2 MBytes   791 Mbits/sec
[SUM]   6.00-7.00   sec   519 MBytes  4.36 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   7.00-8.00   sec  43.6 MBytes   367 Mbits/sec
[  6]   7.00-8.00   sec  18.6 MBytes   157 Mbits/sec
[  8]   7.00-8.00   sec  61.6 MBytes   519 Mbits/sec
[ 10]   7.00-8.00   sec  3.62 MBytes  30.5 Mbits/sec
[ 12]   7.00-8.00   sec  47.0 MBytes   396 Mbits/sec
[ 14]   7.00-8.00   sec  93.4 MBytes   786 Mbits/sec
[ 16]   7.00-8.00   sec  73.2 MBytes   617 Mbits/sec
[ 18]   7.00-8.00   sec  85.4 MBytes   719 Mbits/sec
[SUM]   7.00-8.00   sec   426 MBytes  3.59 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   8.00-9.00   sec  56.0 MBytes   469 Mbits/sec
[  6]   8.00-9.00   sec  42.6 MBytes   357 Mbits/sec
[  8]   8.00-9.00   sec  46.9 MBytes   392 Mbits/sec
[ 10]   8.00-9.00   sec  57.6 MBytes   482 Mbits/sec
[ 12]   8.00-9.00   sec  7.00 MBytes  58.6 Mbits/sec
[ 14]   8.00-9.00   sec  63.1 MBytes   528 Mbits/sec
[ 16]   8.00-9.00   sec  81.1 MBytes   679 Mbits/sec
[ 18]   8.00-9.00   sec  77.5 MBytes   649 Mbits/sec
[SUM]   8.00-9.00   sec   432 MBytes  3.61 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   9.00-10.01  sec  33.9 MBytes   281 Mbits/sec
[  6]   9.00-10.01  sec  40.6 MBytes   337 Mbits/sec
[  8]   9.00-10.01  sec  52.2 MBytes   433 Mbits/sec
[ 10]   9.00-10.01  sec  47.0 MBytes   390 Mbits/sec
[ 12]   9.00-10.01  sec  66.5 MBytes   552 Mbits/sec
[ 14]   9.00-10.01  sec   104 MBytes   863 Mbits/sec
[ 16]   9.00-10.01  sec  31.5 MBytes   261 Mbits/sec
[ 18]   9.00-10.01  sec  42.8 MBytes   355 Mbits/sec
[SUM]   9.00-10.01  sec   418 MBytes  3.47 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.01  sec   432 MBytes   362 Mbits/sec                  sender
[  4]   0.00-10.01  sec   432 MBytes   362 Mbits/sec                  receiver
[  6]   0.00-10.01  sec   406 MBytes   340 Mbits/sec                  sender
[  6]   0.00-10.01  sec   406 MBytes   340 Mbits/sec                  receiver
[  8]   0.00-10.01  sec   479 MBytes   401 Mbits/sec                  sender
[  8]   0.00-10.01  sec   478 MBytes   401 Mbits/sec                  receiver
[ 10]   0.00-10.01  sec   522 MBytes   437 Mbits/sec                  sender
[ 10]   0.00-10.01  sec   522 MBytes   437 Mbits/sec                  receiver
[ 12]   0.00-10.01  sec   411 MBytes   344 Mbits/sec                  sender
[ 12]   0.00-10.01  sec   411 MBytes   344 Mbits/sec                  receiver
[ 14]   0.00-10.01  sec   892 MBytes   747 Mbits/sec                  sender
[ 14]   0.00-10.01  sec   892 MBytes   747 Mbits/sec                  receiver
[ 16]   0.00-10.01  sec   602 MBytes   505 Mbits/sec                  sender
[ 16]   0.00-10.01  sec   602 MBytes   504 Mbits/sec                  receiver
[ 18]   0.00-10.01  sec   676 MBytes   566 Mbits/sec                  sender
[ 18]   0.00-10.01  sec   676 MBytes   566 Mbits/sec                  receiver
[SUM]   0.00-10.01  sec  4.32 GBytes  3.70 Gbits/sec                  sender
[SUM]   0.00-10.01  sec  4.31 GBytes  3.70 Gbits/sec                  receiver

iperf Done.

C:\Users\matth\Downloads\iperf-3.1.3-win64\iperf-3.1.3-win64>

 

Link to comment
https://linustechtips.com/topic/1382546-10gbps-networking/#findComment-15057908
Share on other sites
Link to post
Share on other sites
Posted
2 minutes ago, Matt997 said:

so i just uninstalled and reinstalled the drivers via device manger and got the following back: 

I would go to the Mellanox site and grab the "latest" drivers for the NIC (I think they might be listed as Win8.1 drivers, but they worked for my ConnectX-2s) to see if that helps.

 

What slot on the motherboard are you using? EDIT: stupid question, mITX board...

Main System (Byarlant): Ryzen 9 5950X | Asus B550-Creator ProArt | EK 240mm Basic AIO | 32GB G.Skill DDR4 3600MT/s CL16 | XFX Speedster SWFT 210 RX 6600 | Samsung 990 PRO 2TB / Samsung 990 EVO Plus 4TB | Corsair RM750X | StarTech 4× USB 3.0 Card | Realtek RTL8127 10G NIC | Hyte Y60 Case | Dell U3415W Monitor | Keychron K12 Blue (RGB backlight)

 

Laptop (Narrative): Lenovo Flex 5 81X20005US | Ryzen 5 4500U | 16GB DDR4 3200MT/s (soldered) | Vega II 384SP Graphics | SKHynix P31 1TB NVMe SSD | Intel AX200 Wifi | Asus 2.5G USB NIC | Asus ProArt PA278QV | Keychron K4 Brown (white backlight)

 

Proxmox Server (Veda): Ryzen 7 3800XT | ASRock Rack X470D4U | Corsair H80i v2 | 128GB Micron DDR4 ECC 3200MT/s | 2× Samsung PM963a 960GB SSD / 4× WD 10TB / 4× Seagate 14TB Exos / 4× Micron MX500 2TB / 8× WD 12TB (custom external SAS enclosure) | Seasonic Prime Fanless 500W | Intel X550-T2 10G NIC | LSI 9300-8i HBA | Adaptec 82885T SAS Expander | Fractal Design Node 804 Case

 

Proxmox Server (La Vie en Rose): GMKtec Mini PC | Ryzen 7 5700U | 32GB Lexar DDR4 (SODIMM) | Vega II 512SP Graphics | Lexar 1TB 610 Pro SSD | 2× Realtek 8125 2.5G NICs


Media Center/Video Capture (Jesta Cannon): Ryzen 5 1600X | ASRock B450M Pro4 R2.0 | Noctua NH-L12S | 16GB Crucial DDR4 3200MT/s | EVGA GTX750Ti SC | UMIS NVMe SSD 256GB / TEAMGROUP MS30 1TB | Corsair CX450M | Viewcast Osprey 260e Video Capture | TrendNet (AQC107) 10G NIC | LG WH14NS40 BD-ROM | Silverstone Sugo SG-11 Case | Sony XR65A80K

 

Workbench (Doven Wolf): Lenovo m715q | Ryzen Pro 3 2200GE | 16GB Crucial DDR4 3200MT/s (SODIMM) | Vega 8 Graphics | SKHynix (OEM) 256GB NVMe SSD | uni 2.5G USB NIC | HDMI add-in module

 

Network:

Spoiler 
                       ┌─────────────── Office/Rack ───────────────────────────────────────────────┐
Google Fiber Webpass ── Cloud Gateway Max ══╦═ Pro XG 8 ══╦═ Flex 2.5-8 ══╦═ Doven Wolf
                      La Vie en Rose (DNS) ═╬═ Narrative  ╠═ Veda-NAS     ╠═ La Vie en Rose (vmbr)
                                Veda (DNS) ─┘             ╠═ Veda (vmbr)  ├─ Ptolemy (vmbr)
╔═════════════════════════════════════════════════════════╩═ Ptolemy-NAS  ├─ Veda (Mgmt)
║   ┌ Closet ┐      ┌───────── Bedroom ─────────┐                         └─ Veda (IPMI)
╚═══ Flex XG ══╦╤═══ Flex XG ══╤╦═ Byarlant
       (PoE)   ║│              │╠═ Narrative 
Kitchen Jack ══╣└─ Dual PoE ┐  │╚═ Jesta Cannon*
   (Testing)   ║┌─ Injector ┘  └── Work Laptop
     Bedroom ══╝│        ┌─────── Media Center ────────────────────────────┐
     Jack #2    └──────── Switch 8 ────────────┬─ nanoHD Access Point (PoE)
Notes:                                         ├─ Sony PlayStation 4 
─── is Gigabit / ═══ is Multi-Gigabit          ├─ Pioneer VSX-S520
* = cable passed from Bedroom to Media Center  └─ Sony XR65A80K (Google TV)
Link to comment
https://linustechtips.com/topic/1382546-10gbps-networking/#findComment-15057912
Share on other sites
Link to post
Share on other sites
Posted
  • Author
1 minute ago, AbydosOne said:

I would go to the Mellanox site and grab the "latest" drivers for the NIC (I think they might be listed as Win8.1 drivers, but they worked for my ConnectX-2s) to see if that helps.

 

What slot on the motherboard are you using?

The one running Windows only has one slot PCI-E 16X and the other one that is running TrueNas is in the second slot. I will try the driver thing next. And the Cables i am using are OM3-LC-LC they are the Aqua Green color ones. 

Link to comment
https://linustechtips.com/topic/1382546-10gbps-networking/#findComment-15057918
Share on other sites
Link to post
Share on other sites
Posted
3 minutes ago, Matt997 said:

And the Cables i am using are OM3-LC-LC they are the Aqua Green color ones. 

What model transceivers are you using? (transceiver = the adapter between the SFP+ port and the fiber)

Main System (Byarlant): Ryzen 9 5950X | Asus B550-Creator ProArt | EK 240mm Basic AIO | 32GB G.Skill DDR4 3600MT/s CL16 | XFX Speedster SWFT 210 RX 6600 | Samsung 990 PRO 2TB / Samsung 990 EVO Plus 4TB | Corsair RM750X | StarTech 4× USB 3.0 Card | Realtek RTL8127 10G NIC | Hyte Y60 Case | Dell U3415W Monitor | Keychron K12 Blue (RGB backlight)

 

Laptop (Narrative): Lenovo Flex 5 81X20005US | Ryzen 5 4500U | 16GB DDR4 3200MT/s (soldered) | Vega II 384SP Graphics | SKHynix P31 1TB NVMe SSD | Intel AX200 Wifi | Asus 2.5G USB NIC | Asus ProArt PA278QV | Keychron K4 Brown (white backlight)

 

Proxmox Server (Veda): Ryzen 7 3800XT | ASRock Rack X470D4U | Corsair H80i v2 | 128GB Micron DDR4 ECC 3200MT/s | 2× Samsung PM963a 960GB SSD / 4× WD 10TB / 4× Seagate 14TB Exos / 4× Micron MX500 2TB / 8× WD 12TB (custom external SAS enclosure) | Seasonic Prime Fanless 500W | Intel X550-T2 10G NIC | LSI 9300-8i HBA | Adaptec 82885T SAS Expander | Fractal Design Node 804 Case

 

Proxmox Server (La Vie en Rose): GMKtec Mini PC | Ryzen 7 5700U | 32GB Lexar DDR4 (SODIMM) | Vega II 512SP Graphics | Lexar 1TB 610 Pro SSD | 2× Realtek 8125 2.5G NICs


Media Center/Video Capture (Jesta Cannon): Ryzen 5 1600X | ASRock B450M Pro4 R2.0 | Noctua NH-L12S | 16GB Crucial DDR4 3200MT/s | EVGA GTX750Ti SC | UMIS NVMe SSD 256GB / TEAMGROUP MS30 1TB | Corsair CX450M | Viewcast Osprey 260e Video Capture | TrendNet (AQC107) 10G NIC | LG WH14NS40 BD-ROM | Silverstone Sugo SG-11 Case | Sony XR65A80K

 

Workbench (Doven Wolf): Lenovo m715q | Ryzen Pro 3 2200GE | 16GB Crucial DDR4 3200MT/s (SODIMM) | Vega 8 Graphics | SKHynix (OEM) 256GB NVMe SSD | uni 2.5G USB NIC | HDMI add-in module

 

Network:

Spoiler 
                       ┌─────────────── Office/Rack ───────────────────────────────────────────────┐
Google Fiber Webpass ── Cloud Gateway Max ══╦═ Pro XG 8 ══╦═ Flex 2.5-8 ══╦═ Doven Wolf
                      La Vie en Rose (DNS) ═╬═ Narrative  ╠═ Veda-NAS     ╠═ La Vie en Rose (vmbr)
                                Veda (DNS) ─┘             ╠═ Veda (vmbr)  ├─ Ptolemy (vmbr)
╔═════════════════════════════════════════════════════════╩═ Ptolemy-NAS  ├─ Veda (Mgmt)
║   ┌ Closet ┐      ┌───────── Bedroom ─────────┐                         └─ Veda (IPMI)
╚═══ Flex XG ══╦╤═══ Flex XG ══╤╦═ Byarlant
       (PoE)   ║│              │╠═ Narrative 
Kitchen Jack ══╣└─ Dual PoE ┐  │╚═ Jesta Cannon*
   (Testing)   ║┌─ Injector ┘  └── Work Laptop
     Bedroom ══╝│        ┌─────── Media Center ────────────────────────────┐
     Jack #2    └──────── Switch 8 ────────────┬─ nanoHD Access Point (PoE)
Notes:                                         ├─ Sony PlayStation 4 
─── is Gigabit / ═══ is Multi-Gigabit          ├─ Pioneer VSX-S520
* = cable passed from Bedroom to Media Center  └─ Sony XR65A80K (Google TV)
Link to comment
https://linustechtips.com/topic/1382546-10gbps-networking/#findComment-15057926
Share on other sites
Link to post
Share on other sites
Posted
  • Author
14 hours ago, Matt997 said:

Mellanox MFM1T02A-SR 10GBASE-SR SFP+ Transceiver Module - FS

so this morning I moved my NIC on the TrueNas system from Pci-E slots from the second to the first and this is what i go for speed.  Could of this been what was causing my issues the whole time. 

 

 

C:\Users\matth\Downloads\iperf-3.1.3-win64\iperf-3.1.3-win64>iperf3 -c 192.168.2.139
Connecting to host 192.168.2.139, port 5201
[  4] local 192.168.2.138 port 51366 connected to 192.168.2.139 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   734 MBytes  6.15 Gbits/sec
[  4]   1.00-2.00   sec   988 MBytes  8.29 Gbits/sec
[  4]   2.00-3.00   sec  1.01 GBytes  8.67 Gbits/sec
[  4]   3.00-4.00   sec  1.00 GBytes  8.62 Gbits/sec
[  4]   4.00-5.00   sec  1002 MBytes  8.41 Gbits/sec
[  4]   5.00-6.00   sec   981 MBytes  8.23 Gbits/sec
[  4]   6.00-7.00   sec   982 MBytes  8.23 Gbits/sec
[  4]   7.00-8.00   sec   974 MBytes  8.17 Gbits/sec
[  4]   8.00-9.00   sec   990 MBytes  8.30 Gbits/sec
[  4]   9.00-10.00  sec   981 MBytes  8.23 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  9.46 GBytes  8.13 Gbits/sec                  sender
[  4]   0.00-10.00  sec  9.46 GBytes  8.13 Gbits/sec                  receiver

iperf Done.

C:\Users\matth\Downloads\iperf-3.1.3-win64\iperf-3.1.3-win64>

 

Link to comment
https://linustechtips.com/topic/1382546-10gbps-networking/#findComment-15058811
Share on other sites
Link to post
Share on other sites
Posted
  • Author

So i have a question for all of you. I have been trying for the last 2 days trying to get my 10Gb network working and thank every one that has been helping hats off to them all. I ran a iperf twice and got different results back why would that be. What you can see below is when I use the TrueNas system as the server i get close to the 10Gb connection but when I use the windows system as the server i don't even get close to it. Yes i have update the drivers to the most UpToDate ones. 

 

Windows system 192.168.2.138 

Truenas system is 192.168.2.139

 

Microsoft Windows [Version 10.0.22000.258]
(c) Microsoft Corporation. All rights reserved.

C:\Windows\system32>cd C:\Users\matth\Downloads\iperf-3.1.3-win64\iperf-3.1.3-win64

C:\Users\matth\Downloads\iperf-3.1.3-win64\iperf-3.1.3-win64>iperf3 -c 192.168.2.139
Connecting to host 192.168.2.139, port 5201
[  4] local 192.168.2.138 port 62387 connected to 192.168.2.139 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   772 MBytes  6.48 Gbits/sec
[  4]   1.00-2.00   sec   911 MBytes  7.64 Gbits/sec
[  4]   2.00-3.00   sec   914 MBytes  7.66 Gbits/sec
[  4]   3.00-4.00   sec   915 MBytes  7.68 Gbits/sec
[  4]   4.00-5.00   sec   921 MBytes  7.73 Gbits/sec
[  4]   5.00-6.00   sec   914 MBytes  7.67 Gbits/sec
[  4]   6.00-7.00   sec   918 MBytes  7.70 Gbits/sec
[  4]   7.00-8.00   sec   916 MBytes  7.69 Gbits/sec
[  4]   8.00-9.00   sec   891 MBytes  7.47 Gbits/sec
[  4]   9.00-10.00  sec   494 MBytes  4.14 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  8.37 GBytes  7.19 Gbits/sec                  sender
[  4]   0.00-10.00  sec  8.37 GBytes  7.19 Gbits/sec                  receiver

iperf Done.

C:\Users\matth\Downloads\iperf-3.1.3-win64\iperf-3.1.3-win64>iperf3 -s
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 192.168.2.139, port 56769
[  5] local 192.168.2.138 port 5201 connected to 192.168.2.139 port 32093
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-1.00   sec   192 MBytes  1.61 Gbits/sec
[  5]   1.00-2.00   sec   207 MBytes  1.73 Gbits/sec
[  5]   2.00-3.00   sec   249 MBytes  2.09 Gbits/sec
[  5]   3.00-4.00   sec   330 MBytes  2.77 Gbits/sec
[  5]   4.00-5.00   sec   328 MBytes  2.75 Gbits/sec
[  5]   5.00-6.00   sec   329 MBytes  2.76 Gbits/sec
[  5]   6.00-7.00   sec   337 MBytes  2.82 Gbits/sec
[  5]   7.00-8.00   sec   336 MBytes  2.82 Gbits/sec
[  5]   8.00-9.00   sec   335 MBytes  2.81 Gbits/sec
[  5]   9.00-10.00  sec   337 MBytes  2.82 Gbits/sec
[  5]  10.00-10.06  sec  18.2 MBytes  2.74 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-10.06  sec  0.00 Bytes  0.00 bits/sec                  sender
[  5]   0.00-10.06  sec  2.93 GBytes  2.50 Gbits/sec                  receiver
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
 

Link to comment
https://linustechtips.com/topic/1382546-10gbps-networking/#findComment-15059944
Share on other sites
Link to post
Share on other sites
Posted
21 hours ago, Matt997 said:

So i have a question for all of you. I have been trying for the last 2 days trying to get my 10Gb network working and thank every one that has been helping hats off to them all. I ran a iperf twice and got different results back why would that be. What you can see below is when I use the TrueNas system as the server i get close to the 10Gb connection but when I use the windows system as the server i don't even get close to it. Yes i have update the drivers to the most UpToDate ones. 

First off, not sure if you know but you can keep server and client the same and just turn the transfer way around using the -R command. 

 

Second, what happens if you go with multiple parallel streams with -P 2 (or more)? 

Can you check CPU load during the test on both machines? I kid you not, 10G is actually CPU heavy. 

What are real world tests like? Do you actually only see 2,5gbit?

Gaming HTPC:

R5 5600X - Cryorig C7 - Asus ROG B350-i - EVGA RTX2060KO - 16gb G.Skill Ripjaws V 3333mhz - Corsair SF450 - 500gb 960 EVO - LianLi TU100B


Desktop PC:
R9 3900X - Peerless Assassin 120 SE - Asus Prime X570 Pro - Powercolor 7900XT - 32gb LPX 3200mhz - Corsair SF750 Platinum - 1TB WD SN850X - CoolerMaster NR200 White - Gigabyte M27Q-SA - Corsair K70 Rapidfire - Logitech MX518 Legendary - HyperXCloud Alpha wireless


Boss-NAS [Build Log]:
R5 2400G - Noctua NH-D14 - Asus Prime X370-Pro - 16gb G.Skill Aegis 3000mhz - Seasonic Focus Platinum 550W - Fractal Design R5 - 
250gb 970 Evo (OS) - 2x500gb 860 Evo (Raid0) - 6x4TB WD Red (RaidZ2)

Synology-NAS:
DS920+
2x4TB Ironwolf - 1x18TB Seagate Exos X20

 

Audio Gear:

Hifiman HE-400i - Kennerton Magister - Beyerdynamic DT880 250Ohm - AKG K7XX - Fostex TH-X00 - O2 Amp/DAC Combo - 
Klipsch RP280F - Klipsch RP160M - Klipsch RP440C - Yamaha RX-V479

 

Reviews and Stuff:

GTX 780 DCU2 // 8600GTS // Hifiman HE-400i // Kennerton Magister
Folding all the Proteins! // Boincerino

Useful Links:
Do you need an AMP/DAC? // Recommended Audio Gear // PSU Tier List 

Link to comment
https://linustechtips.com/topic/1382546-10gbps-networking/#findComment-15061732
Share on other sites
Link to post
Share on other sites
Posted
  • Author
3 hours ago, FloRolf said:

First off, not sure if you know but you can keep server and client the same and just turn the transfer way around using the -R command. 

 

Second, what happens if you go with multiple parallel streams with -P 2 (or more)? 

Can you check CPU load during the test on both machines? I kid you not, 10G is actually CPU heavy. 

What are real world tests like? Do you actually only see 2,5gbit?

Thank you for the -R command will make life easer. 

 

Download From Truenas to Windows 11 Pc

Windows 11 PC
CPU- 51%
Download speed 313MBs

 

Download from Windows 11 PC to Truenas 

 

Truenas System
CPU:22%
Download speed 889MBs

 

When I eliminate the switch from the equation and use a DAC cable I get fill 10Gbps. So if I am Correct then the Switch is what is causing my problem. I even removed the switch from the Rack and did short cables from pc's to switch and used different transceivers and had the same issue.  So I attempted to do a reset on the switch today threw the port that is on the switch to do this but was unable to log in to the switch Gui to do so kept on asking me for user name and password and I never changed it from the default. So I called Tp-Link up to see if there was another way to do this and was advised no. If I was unbale to get in to the Gui I would be unable to reset it. Now this switch was order on the 15th of October so new, I called Amazon up to do a replacement and I have a new Switch coming on Monday. Hopefully it fix's my issue.  
 

 

 

Link to comment
https://linustechtips.com/topic/1382546-10gbps-networking/#findComment-15061881
Share on other sites
Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share
Go to topic listing Networking

×