Terrabit Network for LinusTechTips?

[honna1612]

Well this was not possible a few years ago but it totally is possible now to get 1Tbit network = 1000Gbit between two computers for a reasonable price.
Here is one would need:

• 3x Mellanox ConnectX-6 dual 200Gbit Ethernet Card
• 6x QSFP28 cable (copper or optic)
• Windows Server 2019
• A cpu+mainboard with at least 48PCIE Lanes

Octa Channel Ram (or very fast quad channel) to not get a memory bandwidth bottleneck. Also the NICs need to have a direct way to memory (1 NIC per socket)

Since these are normal Ethernet cards they would work with any 200gbit qsfp28 switch but if you only need point to point you could connect these directly.
Also in windows server all nics would have to be set to load balanced failover team. This creates a single nic with 1.2 Tbit minus the overhead.

So probably just over 1Tbit!

[r2724r16]

[Firewrath9]

QSFP28 can only carry 28x4 gbit.

also that costs almost 5k dollars. 

[SkyHound0202]

Quote

Terabit Ethernet has not been fully defined on Institute of Electrical and Electronics Engineers's (IEEE) roadmap. Every bandwidth beyond 400 GbE is defined as "sometime after 2020"

Well see you guys in 1 year.

[Windows7ge]

Neat but the only practical application for those speeds would be by sharing the RAM and CPU's in a cluster of other systems for applications like deep learning or computing. Once you go into forms of permanent storage even PCI_e SSD's wouldn't come close to the connection speed and since most servers would store the primary data on spinning platters (maybe with a cache) you'd be working with a network speed that's uselessly fast to your primary storage.

 

Quite frankly 40Gbit is fast enough for any application I have.

[Lurick]

11 hours ago, Windows7ge said:

Neat but the only practical application for those speeds would be by sharing the RAM and CPU's in a cluster of other systems for applications like deep learning or computing. Once you go into forms of permanent storage even PCI_e SSD's wouldn't come close to the connection speed and since most servers would store the primary data on spinning platters (maybe with a cache) you'd be working with a network speed that's uselessly fast to your primary storage.

 

Quite frankly 40Gbit is fast enough for any application I have.

So what you're saying is you want 400GbE to the server and 1TbE to the core?

[Windows7ge]

3 hours ago, Lurick said:

So what you're saying is you want 400GbE to the server and 1TbE to the core?

If the storage itself is a cluster the I suppose the "pipe" leaving the core could be 400GbE. I can't see each storage box taking more than 40GbE though.