Search the Community

Hello everyone! I recently bought 2 ConnectX-3 cards to improve file transfer speeds between the PC and the "server", but i don't really know how to set it up correctly. Current setup looks like this: What i need is: 1) PC, Server and the Router should be in one network; 2) PC and Server should use 40GbE for transfers between each other; 3) When accessing the internet - they both should use 100Mbit link through the router; 4) Currently, both PC and Server are also connected through SoftEther VPN server hosted on a remote VPS. I wonder if i can set something up, so if some "service" is hosted on the Server - at least the requests made from the PC (in that SoftEther network) would go straight to the Server, and not to VPS and back. I thought "Local Bridge" would do that, but didn't work out as i expected. That link through the VPN is here because i need remote access to both the PC and the Server. I also have Navidrome using that virtual network for broadcasting to remote devices. Any advice and/or instructions on how i'd have to do all of this?

After seeing Linus's exciting videos testing 40 GbE and 100 GbE networking, I decided to give things a try at home! This was especially useful since I couldn't warrant bombing $1,200 on a 10 GbE switch that would only have 2 computers using it at the time. So we decided to take the plunge and bomb $900 on a pair of Mellanox 40-Gigabit network cards (along with 40-gig rated QSFP+ cabling from FS.com). Becoming content creators in our own right, my drummer and I live in a band house together. We like to send big video files of our music/practices and movies, etc between our machines. Despite some wicked performance, we cannot seem to break 25 Gbps when copying large 10GB, 20GB, and 40GB test files. Our Seagate 520 Firecuda's consistently benchmark at 5,000 MB / sec reads with 4,250 MB / sec writes. I figured with approx 4250 MB / sec write speeds, I can expect approximately 34 Gbps speeds yet we consistently fall way short of this! So the next step, I thought, well why not take the network completely out of the equation (since I run two of these drives in my system - One for OS and one for network transfers). Even between two identical Seagate 520 Firecuda drives in my machine, the files would only copy at approx 2,200 MB / sec - Falling FAR short of the benchmarked speeds..... I was SHOCKED....Can anyone explain these discrepancies between SSD benchmark speeds vs actual transfer speeds? Why would a file copy at only 51.7% (2,200 / sec vs 4,250 MB / sec) of its benchmarked write speeds?? Also, if there is any optimization that I can do with the NICs themselves, please let me know! (Clicky maximize button in bottom right of video to see) EPIC 23 Gbps.mp4 We have the following gear: My machine: X570 motherboard AMD 16-core Ryzen 9 3950X CPU 64 GB RAM at 3200 Mhz 40-Gigabit Mellanox ConnectX-4 Lx Ethernet Adapter File-sharing Drive (OS not running on it): 2TB Seagate Firecuda 520 His machine: X570 motherboard AMD 8-core Ryzen 7 3800X CPU 32 GB RAM at 3200 Mhz 40-Gigabit Mellanox ConnectX-4 Lx Ethernet Adapter File-sharing Drive (OS not running on it): 2TB Seagate Firecuda 520

Hey guys, I'm running Proxmox 7.0-13, which seems to be based on Debian 11. Does anyone know how to get the Mellanox Drivers working? The latest driver seems to be only for 10.8. Im just wondering, if anybody else got this working. I just dont want to do the hustle of downgrading my proxmox version to 6.0 or even lower. Thanks a lot! Edit: I probably should have stated that you cannot use the lastest driver, it will simply not install, even when passed the "-force" argument.

I have a Gigabyte X570 UD motherboard (with 3 x PCIE x16 length slots) and a Mellanox ConnectX-3 10Gbe card. When I install the Decklink card, the 10Gbe card no longer appears in device manager. I have tried switching the slots between the two, but nothing changes. Both cards work perfectly when the other one is not plugged in. I have tried changing the PCIe Biufication the PCIe Gen settings in the BIOS but that doesn't help. I have tried an old Blackmagic Intensity Pro (2009) I had lying around and it works perfectly with the 10gbe card. I have also tried an Intel intel X520 10gbe card instead of the Mellanox one, and it doesn't show up if the Decklink card is installed I have upgraded the X570 bios firmware to the latest version I am stumped on this one, hoping someone here might be able to help. System: Ryzen 3800XT Gigabyte X570 UD Rev1.1 Corsair Vengeance LPX DDR4 3200 MHz- 32GB (2x16) Gigabyte RTX 3060 12GB Mellanox ConnectX-3 10Gbe card (also have a intel X520 10Gbe card which I can switch out) Blackmagic Decklink Studio 4k Windows 10 Pro 21H1 19043.1288

My currents server setup: Mainboard -> X470D4U CPU -> AMD Ryzen 7 2700 RAM -> G.Skill Aegis DIMM Kit 32GB, DDR4-3200, CL16-18-18-38 PCIe Add in Card -> Supermicro AOC-SLG3-2M2-O Slotted with -> 2X Western Digital WD_BLACK SN750 NVMe SSD 250GB, M.2 OS -> Unraid 6.9.2 The SSD's are in Slot6 (view block diagram in attachments) and in the Bios is PCIe bifrication enabled and set to 4X4. New gear to be added: Mellanox connectX-3 MCX311A-XCAT My problem is when i add the Mellanox card (no matter the slot), one of my SSD's is no longer visible/detected. I also tried every slot combination there is, but nothing worked. The Mellanox card works normal and connects succesfully to my switch. Any help or ideas why this is are very appreciated!

Hey guys, first post here, hope it's okay to immediately plead for your help. I was inspired by the LTT video to 10x your network speed on a bargain and decided to attach my main rig and my file server using mellanox connectX-2 cards i got off ebay. One of the first two that arrived was bricked with a corrupt firmware, a replacement arrived shortly thereafter from the same provider. Firmware updated to the latest for Windows 10 (2.9.1200) and software driver updated to 5.50. The cards are recognised on both systems now in the device manager. Listed as 'working correctly'. I configured the IP and subnet and as far as i can tell at this point i should have a working set up. However, on the network adaptor screen on both system it shows the status as 'network cable unplugged'. There i no light activity from the orange and green on the back. So i bought another cable from ebay figuring the first was DOA, no change. in a fit of frustration i tried to reseat the cables several times, and upon plugging in, just once, i got a brief flash of green light from the back, the only activity i've seen thusfar and i've been unable to replicate it. I attempted to clean the contacts on the sfp slot on both cards with a qtip and isoprophyl without success and then decided the second cable must also be DOA, so i bought a third off a retailer, same result. I'm totally lost at this point, the cards are clearly active and working, recognised by windows 10 and installed but the connectors on the cards are just unwilling to work. I tried using the cards on two other systems, one windows 7, the other Ubuntu with the same result. TLDR: windows displays 'network cable unplugged' with the cards, replacement cables dont work, cleaning doesn't work. Do any of you have any ideas about anything else i can try?

This summer, I started moving as much data as possible from a 2-bay NAS to an unRaid server built from a repurposed MSI X99S Gaming 7 running an i7-5930K, two cache drives, two parity drives and five data drives. Once I did that, I started planning a backup of that system, that right now is running on an ECS H55H-I motherboard and seven 3.5" drives. My plan is to continue to use WD SmartWare to make incremental backups to my little NAS and use it for nothing else. Nightly, that data will get synchronized to my primary unRaid server and I'll work out some kind of schedule to sync data to the other, figuring out how to do integrity checks and so on so that if file corruption takes place, I have some sort of window to copy it back over from a known good copy. A third level of storage will be made to an array of SSD drives and kept in a safe deposit box. Cloud storage, when I would want at least 5TB, is kind of expensive and I don't really trust the privacy of it all. After I started moving terabytes of data from my old NAS into unRaid, I decided to pick up a few Mellanox 10g fiber cards to make transfers between systems just a little faster. On all of my desktop PCs, and anywhere it's financially practical, I've done away with rotational drives and run entirely from SSDs, one PCIe and one SATA NVMe drive in my main desktop. I'm not bound by drive speed so much, but I am bound by limitations of the interface and to some degree, limitations of Windows 10. My desktop PC has three network interfaces in use. One connecting to a segregated array of Raspberry Pi (behind pfSense), one connected peer-to-peer to my unRaid server and one connected to my router to communicate with everything else. I want to streamline things and am planning to build a new pfSense router using a motherboard/cpu combination that's robust enough to operate three 2-port Mellanox cards, each running on 8 PCIe lanes. The most likely prospects at this point are an i7-5820K running on an ASRock X99 Professional X99, 2011-3. For that setup, I'm looking to use all 28 PCIe lanes (8 x 3 (fiber) + 4 (NVMe)), using one of the onboard interfaces for WAN and the other with a 24-port D-Link switch. My questions are two. Have any of you run pfSense using fiber before, and if so, can it be made to do so stable? Second, if all I'm using it for is pfSense, what power supply rating would you expect to be adequate for this kind of build? I don't know the power requirements of the Mellanox cards. I'll set it up for 10g initially but will likely move to 40g within a year. If possible, I'd like to drive the thing using a Corsair SF600 or an SF450. This will be a wall mounted server and the only wires should be power and network.

Hi Guys I have just bought a handful of Mellanox connectx-3 10gbe SFP+ cards on the cheap, I have tried the all in a Linux server to test if any of them should be DOA, but happy days all of them are registered and shows up beautifully in Unraid. But when I take the same cards and put them in a Windows 10 pc they just doesn't show up in the BIOS or in device manager? at first I thought it might not be supported, I can however find drivers for my exact Windows 10 version on Mellanox driver page? Any help or reason why I am an idiot who doesn't see the obvious problem would be greatly appreciated