10G LAN question

[Sarra]

I'm going to be running two machines, with one being a server, the other being my rendering/gaming machine. There will be times when I have to flush data off a 2TB or 4TB NVME gen 4 SSD to the other machine, going to another, matching 2TB or 4TB NVME gen 4 SSD. Both machines are going to be Win 10 Pro, eventually upgraded to Win 11 Pro.

 

If I got a pair of 10G NICs, could I just run a Cat6 cable from one machine to the other, with my internet still running through the onboard gigabit to my router? I'm trying to not buy a massive, expensive 10G switch, which will have tons and tons of ports that I have no use for, put extra load on my UPS, and just waste tons of room that I just don't have to begin with.

[Electronics Wizardy]

34 minutes ago, Sarra said:

 

If I got a pair of 10G NICs, could I just run a Cat6 cable from one machine to the other, with my internet still running through the onboard gigabit to my router? I'm trying to not buy a massive, expensive 10G switch, which will have tons and tons of ports that I have no use for, put extra load on my UPS, and just waste tons of room that I just don't have to begin with.

yup you can do that. Just setyp a subnet between them.

[mariushm]

Yes, you can. You just set different local IPs on each computer , ex 192.168.0.201    and 192.168.0.202  and same subnet mask, ex. 255.255.255.0

 

But it would be better to go with fiber network cards - if the computers are close to each other, let's say 3-5 meters, you can get a passive DAC cable for around 10-20$. Also, the cards are often cheaper on the used market.

 

 

 

[Sarra]

Thanks for the answers.

[CerealExperimentsLain]

I literally do this in my own.  The whole apartment is on a 1gig network that has internet and allows any communication between any PC.  The main desktop and main UnRAID server each have an additional 10gig NIC and are directly linked over a CAT6A cable.  You do have to manually set their IPs and also in Windows it can help to configure the HOST file to which IP you'd prefer the names resolve to, or sometimes Windows will randomly select which NIC the name resolves to and you want it to always be the 10GIG for those two machines.

[Sarra]

Yeah, it sounds good to me.

 

When I move, hopefully, in a few months, I'll be getting a place that has space for rackmount, and I'll invest in a switch and actual server stuff then. But for now, I just want to be able to move huge quantities of stuff from machine to machine without having to wait for hours and hours to do it @__@

[CerealExperimentsLain]

1 minute ago, Sarra said:

Yeah, it sounds good to me.

 

When I move, hopefully, in a few months, I'll be getting a place that has space for rackmount, and I'll invest in a switch and actual server stuff then. But for now, I just want to be able to move huge quantities of stuff from machine to machine without having to wait for hours and hours to do it @__@

And the parts are not expenisive now.  Copper 10gig NICs are like CAD$100 these days.  The real expenses is a 10gig copper switch, but once those come down, I'll retire my 1gig switches and just have it all as one thing again.

[mariushm]

11 minutes ago, CerealExperimentsLain said:

And the parts are not expenisive now.  Copper 10gig NICs are like CAD$100 these days.  The real expenses is a 10gig copper switch, but once those come down, I'll retire my 1gig switches and just have it all as one thing again.

You can buy dual 10g fiber cards for $25-50 each, for example here's one for $35 : https://unixsurplus.com/solarflare-sfn6122f-dual-port-sfp-10gbe-pci-e-high-profile-server-adapter-card/

 

DAC cables are cheap, 15-40$ for 3 meter ones :

3M SFP+ 10GbE Direct Attach Cable Twinax DAC

Cisco 3M Twinax 3Metre SFP+ 10GB/s 10GbE Direct Attach Cable DAC SFP-H10GB-CU3M

fs.com also has loads of them : 10Gb SFP+ Cables, SFP+ Twinax DAC Cables - FS Germany

 

 

The site that sells the $35 cards also sells a get a bundle with two intel cards and a dac cable  for $170 : https://unixsurplus.com/bundle-x520da2-intel-dual-10gb-w-2x-dac-direct-attach-cable-included/ - not really a great value but still cheap.

 

 

[CerealExperimentsLain]

9 minutes ago, mariushm said:

You can buy dual 10g fiber cards for $25-50 each, for example here's one for $35 : https://unixsurplus.com/solarflare-sfn6122f-dual-port-sfp-10gbe-pci-e-high-profile-server-adapter-card/

 

DAC cables are cheap, 15-40$ for 3 meter ones :

3M SFP+ 10GbE Direct Attach Cable Twinax DAC

Cisco 3M Twinax 3Metre SFP+ 10GB/s 10GbE Direct Attach Cable DAC SFP-H10GB-CU3M

fs.com also has loads of them : 10Gb SFP+ Cables, SFP+ Twinax DAC Cables - FS Germany

 

 

The site that sells the $35 cards also sells a get a bundle with two intel cards and a dac cable  for $170 : https://unixsurplus.com/bundle-x520da2-intel-dual-10gb-w-2x-dac-direct-attach-cable-included/ - not really a great value but still cheap.

 

 

Right, I don't see the long term appeal of this.  That's USD$170 for the 'Kit' meanwhile you can just get two copper 10gig cards for like USD$90 each + the cost of some CAT6A for whatever length you want.  You can then reintegrate that with 10 gig copper switching in the future as the cost of copper switching comes down. It will also integrate with 2.5 or 5 gig switching if you don't mind the speed drop.  You basically saved the cost of a delivery order of pizza but built something far more restrictive in terms of expansion and future proofing.

The 'cheap' you suggest is pitifully small savings at a significant cost of flexibility.

[mariushm]

19 minutes ago, CerealExperimentsLain said:

Right, I don't see the long term appeal of this.  That's USD$170 for the 'Kit' meanwhile you can just get two copper 10gig cards for like USD$90 each + the cost of some CAT6A for whatever length you want.  You can then reintegrate that with 10 gig copper switching in the future as the cost of copper switching comes down. It will also integrate with 2.5 or 5 gig switching if you don't mind the speed drop.  You basically saved the cost of a delivery order of pizza but built something far more restrictive in terms of expansion and future proofing.

The 'cheap' you suggest is pitifully small savings at a significant cost of flexibility.

 

The Solarflare cards I linked to are $35 each, and they're dual 10gbps cards, and they have drivers built into windows, linux and other operating systems..

 

If your cables are less than 30 meters, you can easily use ethernet to sfp+ transceivers to plug your 10g ethernet cable into a sfp+ port, if the switch you have has no 10g ethernet ports.

 

If it's 2 computers in the same room, it's really cheap way of connecting those.. you get two cards for the price of a single 10g ethernet card.

 

As for the other benefits ...

 

SFP+ network switches are much cheaper... try getting a 16 port or higher 10g ethernet switch at reasonable prices. You can get 48 sfp+ port switches for same price. Also you can get QSFP switches (40gbps ports which can be divided into 4 x 10gbps ports) or QSFP28 switches (100 gbps divided into 4 x 25gbps or 4x10gbps) so you could reuse the fiber in the future to get higher speeds.

 

You can get transceivers to convert RJ45 to SFP+ if you have 10g ethernet cards

You can use transceivers + fiber cable as long as you want.

 

Ethernet to SFP+ transceiver (they used to be around 20-30$ each) at around $40 : https://www.amazon.com/10GBASE-T-Transceiver-Copper-Compatible-GP-10GSFP-T/dp/B07HK871RD/

SFP+ Transceiver (LC Duplex) from 20$ : https://www.fs.com/de-en/c/10g-sfp-plus-63?sort_order=price&count=24&settab=two

They're 10$ or less on ebay ex. 6$ each : https://www.ebay.com/itm/274844675396

 

LC-LC fiber (5$ for 1m, $10 for 5m, 22$ for 15m ) : https://www.fs.com/de-en/products/40180.html?attribute=6741&id=270331

Can be found cheaper definitely.

 

Good cat6a cable, proper solid core pure copper cat6a cable, will cost you at least 200$ for a 1000ft / 300m roll, here's some examples : 

https://www.amazon.com/SolidLink-1000ft-Listed-Conductor-Ethernet/dp/B0742R9642/

https://www.amazon.com/1000ft-Unshielded-Twisted-Ethernet-trueCABLE/dp/B07FK88PX9/

https://www.amazon.com/Ethernet-Gigabit-Unshielded-Twisted-iTechCables/dp/B07XN8CN7Y/

 

so it's not that much cheaper, plus it's thicker, harder to route, more susceptible to noise and other issues.

 

[CerealExperimentsLain]

1 minute ago, mariushm said:

 

The Solarflare cards I linked to are $35 each, and they're dual 10gbps cards, and they have drivers built into windows, linux and other operating systems..

 

If your cables are less than 30 meters, you can easily use ethernet to sfp+ transceivers to plug your 10g ethernet cable into a sfp+ port, if the switch you have has no 10g ethernet ports.

 

If it's 2 computers in the same room, it's really cheap way of connecting those.. you get two cards for the price of a single 10g ethernet card.

 

As for the other benefits ...

 

SFP+ network switches are much cheaper... try getting a 16 port or higher 10g ethernet switch at reasonable prices. You can get 48 sfp+ port switches for same price. Also you can get QSFP switches (40gbps ports which can be divided into 4 x 10gbps ports) or QSFP28 switches (100 gbps divided into 4 x 25gbps or 4x10gbps) so you could reuse the fiber in the future to get higher speeds.

 

You can get transceivers to convert RJ45 to SFP+ if you have 10g ethernet cards

You can use transceivers + fiber cable as long as you want.

 

Ethernet to SFP+ transceiver (they used to be around 20-30$ each) at around $40 : https://www.amazon.com/10GBASE-T-Transceiver-Copper-Compatible-GP-10GSFP-T/dp/B07HK871RD/

SFP+ Transceiver (LC Duplex) from 20$ : https://www.fs.com/de-en/c/10g-sfp-plus-63?sort_order=price&count=24&settab=two

They're 10$ or less on ebay ex. 6$ each : https://www.ebay.com/itm/274844675396

 

LC-LC fiber (5$ for 1m, $10 for 5m, 22$ for 15m ) : https://www.fs.com/de-en/products/40180.html?attribute=6741&id=270331

Can be found cheaper definitely.

 

Good cat6a cable, proper solid core pure copper cat6a cable, will cost you at least 200$ for a 1000ft / 300m roll, here's some examples : 

https://www.amazon.com/SolidLink-1000ft-Listed-Conductor-Ethernet/dp/B0742R9642/

https://www.amazon.com/1000ft-Unshielded-Twisted-Ethernet-trueCABLE/dp/B07FK88PX9/

https://www.amazon.com/Ethernet-Gigabit-Unshielded-Twisted-iTechCables/dp/B07XN8CN7Y/

 

so it's not that much cheaper, plus it's thicker, harder to route, more susceptible to noise and other issues.

 

That's a *whole* list of parts and extra components to make your idea cheaper when copper Switches will just keep getting cheaper and you can just plug them into any existing comedy hardware instead of this miss mash of enterprise grade adaptations.