How to connect two computers with 10 gigabit?

[jt532]

Suppose I have two Windows desktop computers and I want to send files between the two at 10 gigabit speeds. Would all I need is two 10Gb PCIe cards (one for each machine, such as these [the SFP+ modules are included]) and one SFP+ cable (like this one) to connect them? Is that all it takes? Sorry I'm new to this and I'm not sure what's compatible and what isn't.

[Devryd]

I somehow doubt, that a 20$ network card can do 10GBit/s

In theory you should be able to just connect them to each other. Then you have to give each device a static IP address for that card. 
At last, you have to enable file sharing over the network.

[MrMase]

In theory yes, a 10GB card in each machine connected together.

On Each machine statically assign and IP address and Subnet to the NIC e.g.

PC1

IP Address 192.168.5.1

Subnet 255.255.255.0

Default Gateway left blank

DNS left blank

 

PC2

IP Address 192.168.5.2

Subnet 255.255.255.0

Default Gateway left blank

DNS left blank

 

You may have some fun with the firewalls to overcome and accessing the locations folder structure by setting up shares etc but would be easy enough.

 

For accessing the internet use your existing Network card where Default Gateway and DNS are assigned.

 

[Eigenvektor]

32 minutes ago, jt532 said:

Is that all it takes?

It's enough as far as network bandwidth is concerned. However to transfer files at that speed from one computer to another, the disks also have to be fast enough. It doesn't matter if your computer can send or receive data at that speed, if the disk isn't able to supply or store the data fast enough.

 

10 Gbps translates to 1.25 GB/s. A SATA SSD will (at best) be able to write at around 500 MB/s. You'll need at least a PCIe 3.0 NVMe SSD to achieve a read/write speed >1.25 GB/s, to be able to make full use of that network speed.

[Blue4130]

2 hours ago, jt532 said:

Suppose I have two Windows desktop computers and I want to send files between the two at 10 gigabit speeds. Would all I need is two 10Gb PCIe cards (one for each machine, such as these [the SFP+ modules are included]) and one SFP+ cable (like this one) to connect them? Is that all it takes? Sorry I'm new to this and I'm not sure what's compatible and what isn't.

Reading reviews, seems like many people have trouble with getting these to work on windows. These are EOL. Might be better off getting a mellanox connectx3 or similar.

 

But as @Eigenvektor, unless you are running nvme or a fast raid, you are going to be disk speed limited. down to ~150MB/s if you are using harddrives...

[jt532]

3 hours ago, Blue4130 said:

Reading reviews, seems like many people have trouble with getting these to work on windows. These are EOL. Might be better off getting a mellanox connectx3 or similar.

 

But as @Eigenvektor, unless you are running nvme or a fast raid, you are going to be disk speed limited. down to ~150MB/s if you are using harddrives...

I wasn't able to find any reviews for that card in particular (Chelsio N320-SR). Where did you find them at?

[Blue4130]

Just now, jt532 said:

I wasn't able to find any reviews for that card in particular (Chelsio N320-SR). Where did you find them at?

Amazon. Anyone running truenas (freenas) said it was ok, nearly everyone else (esxi, windows) said it didn't work. 

[Needfuldoer]

6 hours ago, Devryd said:

I somehow doubt, that a 20$ network card can do 10GBit/s

10 gig has been around in the datacenter and server markets for ages. Used cards are cheap, DAC cables are cheap, and it can all do 10 gig. Finding modern drivers for older and more obscure cards can be a challenge, but if you're using a NIC from Intel or Broadcom it's usually not a problem.

 

Even 25 gig and 40 gig gear is coming down in price on the secondhand market, as it gets sunset from production use.

 

You can go even faster for less with Infiniband, but that's a whole other can of worms.

 

None of this will run at the new "2.5 gig" consumer standards, though. They'll negotiate back down to Gigabit.

[jt532]

One thing I still don't know is if these cards will need an active or passive DAC cable. How would I check that?

[Lurick]

26 minutes ago, jt532 said:

One thing I still don't know is if these cards will need an active or passive DAC cable. How would I check that?

Doesn't matter for the card, the card could care less provided it's a supported DAC interface/manufacturer (some NICs will only work with specific brands like a dell NIC and dell branded SFP/DAC/etc)

Active DACs and passive DACs do the same thing, the only difference is active have signal conditioning components in them. Passive DACs also don't go as far compared to active DACs.

 

https://community.fs.com/blog/guide-to-10g-dac-and-aoc-cables.html

[Alex Atkin UK]

On 7/3/2022 at 9:43 AM, Devryd said:

I somehow doubt, that a 20$ network card can do 10GBit/s

A lot of data centres are upgrading to 40Gbit or higher, its not that unusual to see prices plummeting.

 

I'd be wary though, they might be cheap due to no desktop Windows drivers or something.

[jt532]

On 7/4/2022 at 11:54 AM, Alex Atkin UK said:

A lot of data centres are upgrading to 40Gbit or higher, its not that unusual to see prices plummeting.

 

I'd be wary though, they might be cheap due to no desktop Windows drivers or something.

I ended up buying them and did find a driver, although technically it wasn't for these cards. However, they show up in Windows. Now I'm just waiting for the cable to be delivered to test it and see if it actually works.

 

If I've made it this far and they show up like this in Windows, I assume they will work. But we'll see in a few days once the cable arrives.

 

[Blue4130]

18 minutes ago, jt532 said:

I ended up buying them and did find a driver, although technically it wasn't for these cards. However, they show up in Windows. Now I'm just waiting for the cable to be delivered to test it and see if it actually works.

 

If I've made it this far and they show up like this in Windows, I assume they will work. But we'll see in a few days once the cable arrives.

 

Keep the thread updated. More cheap options are always welcomed. 

[jt532]

11 hours ago, Blue4130 said:

Keep the thread updated. More cheap options are always welcomed. 

It got so close! I received the cable today, hooked them up, and both Windows computers were able to browse each other's filesystems with no problem. So on the surface it looked like it was going to work. However, as soon as I tried to copy a file from one machine to the other, the receiving machine instantly blue screened. I was able to repeat this consistently for both machines, sadly.

 

The blue screen error mentioned something about a "driver less than or equals to error" (I didn't get a picture, sorry), so I assume it won't work. The driver I used didn't list this particular card as supported, but I had been willing to try it anyway. Disappointing given it made it this far.

 

Also, the cards got extremely scalding hot. I'm not sure if newer cards would get as hot as these do.

[Blue4130]

3 hours ago, jt532 said:

.

 

Also, the cards got extremely scalding hot. I'm not sure if newer cards would get as hot as these do.

They are meant to be in servers with major airflow. At the very least, I'd be putting a fan on them. 

[jt532]

1 hour ago, Blue4130 said:

They are meant to be in servers with major airflow. At the very least, I'd be putting a fan on them. 

Unfortunately that was something I had failed to consider. One of my machines is a tiny HP server so there's not any room in that one to add a fan. But I doubt overheating was the reason for the blue screen given how it happened immediately when a file transfer started but was completely fine otherwise.

 

At least I'm not out that much money. Only about $44.

[Needfuldoer]

Don't get too discouraged! There are plenty of Intel, Broadcom, and Mellanox based 10gbe cards on eBay for around $30 apiece right now, and all of them have drivers available for Windows 10.

[jt532]

56 minutes ago, Needfuldoer said:

Don't get too discouraged! There are plenty of Intel, Broadcom, and Mellanox based 10gbe cards on eBay for around $30 apiece right now, and all of them have drivers available for Windows 10.

Would you mind sharing a few? The Intel ones I'm finding are typically in the $50 to $70 range, so I must not be looking for the right ones. Thanks!

[Blue4130]

14 minutes ago, jt532 said:

Would you mind sharing a few? The Intel ones I'm finding are typically in the $50 to $70 range, so I must not be looking for the right ones. Thanks!

Look for Mellanox ConnectX 2 or 3. Both should work in windows 10 (according to servethehome.com)

[Needfuldoer]

16 minutes ago, jt532 said:

Would you mind sharing a few? The Intel ones I'm finding are typically in the $50 to $70 range, so I must not be looking for the right ones. Thanks!

Sometimes you can find better deals if you search for a Dell or HP part number instead of a"[manufacturer] 10gbe". They're almost always SFP+ cards too, which is fine because fiber and DACs are cheap. (Don't buy an SFP+ card expecting to plug in some 10GBASE-T SFPs unless you already have them, though. They're unreasonably expensive; buying a native 10GBASE-T card usually works out cheaper.)

 

https://www.ebay.com/itm/194887839029

 

https://www.ebay.com/itm/115425890034

 

https://www.ebay.com/itm/265720899649

 

https://www.ebay.com/itm/295084573867

 

https://www.ebay.com/itm/284798378803

 

https://www.ebay.com/itm/175345431226

[jt532]

5 hours ago, Needfuldoer said:

Sometimes you can find better deals if you search for a Dell or HP part number instead of a"[manufacturer] 10gbe". They're almost always SFP+ cards too, which is fine because fiber and DACs are cheap. (Don't buy an SFP+ card expecting to plug in some 10GBASE-T SFPs unless you already have them, though. They're unreasonably expensive; buying a native 10GBASE-T card usually works out cheaper.)

 

https://www.ebay.com/itm/194887839029

 

https://www.ebay.com/itm/115425890034

 

https://www.ebay.com/itm/265720899649

 

https://www.ebay.com/itm/295084573867

 

https://www.ebay.com/itm/284798378803

 

https://www.ebay.com/itm/175345431226

Thanks, this is pretty helpful. I noticed the last card you linked was a QSFP+ card. Was that an accident, or would it still be compatible with the cable I had above?

[Needfuldoer]

1 hour ago, jt532 said:

Thanks, this is pretty helpful. I noticed the last card you linked was a QSFP+ card. Was that an accident, or would it still be compatible with the cable I had above?

It takes QSFPs because it's a 40 gig Infiniband card, but those will do 10 gigabit Ethernet as well. You just need either a QSFP DAC or a couple adapters so you can plug regular SFPs in.