Server connection speed requirement

[Leduc]

Hi everyone,
I am working for a small company and I ended up in charge of the network. My main role is in geomatics, I am not an IT guy by formation but I keep my self updated. We are planning to move to a new location soon and I wanted to upgrade the network at the same time since we start fresh.

 

Our current setup uses D-Link DGS-1510-52X (48x 1gb 4x 10gb) switch. Most computer are connected with the 1gb port with Cat6. We have the current server connected with 10gb to 2 computer that do most of the main workload. The current server uses HDD so I assumed it was not necessary to uses faster connection. 

The question I have now is that if we upgrade the server to have a smaller SSD array for processing. What is the connection speed required to avoid having a bottleneck. Also, what kind of speed should all the other computer have to be able to transfer file or work on the server at a reasonable speed.

I am afraid that if too many people work on the server at the same time it will slow down so much it become a waste of money.
 

[OddOod]

If the server has a 10gig pipe to the switch and the workstations have a 1gig connection to that same switch, you'd need 10 people maxing out the upload all at once to even see a bottleneck. That seems VERY unlikely. 
I'm not super familiar with GIS stuff, but how big are the files you're dealing with? And what software are you using for processing? Would it even benefit from >10gig speeds? Will the CPU be able to handle that? 
You can test on your current system by tossing a couple NVME drives in RAID0 to flood the pipe and check if it's a noticable speed increase

[Dutch_Master]

27 minutes ago, Leduc said:

upgrade the network at the same

I know it's tempting but that's probably the worst moment to do so. It's really frustrating to troubleshoot something when you don't even know which device introduced the error, especially as you're "just the IT guy" and not a trained sys-admin. I'd suggest transplanting the current, known working environment to the new place, make sure that works as expected and only then start swapping in new stuff. If you don't have it already, consider buying a server rack to house critical equipment, including UPS's, switches and servers. Also consider a High Availability setup (redundant servers and network stuff, basically) One important upgrade which will future-proof your network is switching over to fibre-optic cables for basically everything. Keep the Cat6e wiring in place as a backup, but the physical fibre cables are capable of much higher speeds then copper can and are only limited by the adapters on each end.

 

HTH!

[Leduc]

10 minutes ago, OddOod said:

If the server has a 10gig pipe to the switch and the workstations have a 1gig connection to that same switch, you'd need 10 people maxing out the upload all at once to even see a bottleneck. That seems VERY unlikely. 
I'm not super familiar with GIS stuff, but how big are the files you're dealing with? And what software are you using for processing? Would it even benefit from >10gig speeds? Will the CPU be able to handle that? 
You can test on your current system by tossing a couple NVME drives in RAID0 to flood the pipe and check if it's a noticable speed increase

The file size vary a lot in GIS depending on the type of file and project size, but they become very large very fast.

As an example, a point cloud can become multiple gig in size for a single file, but as soon as we talk about imagery it become multiple terabytes divided in hundred of thousands of files.

As for software it also vary a lot, most of them will load the file in ram before processing. The issues is when a project is very large, our current setup will force us to work on a HDD, which slow down the data import / exportation step it can take from several minute to several hours. That's why I wanted to uses an SSD array.

Creating a small NVME raid is a great idea to test our current setup. I can probably buy a 4x nvme adapter to put inside our current server.

[Lurick]

Once you start talking about NVMe and/or SSD arrays and bandwidth available by the server can easily allow for 100GbE or 400GbE links depending on the array setup and number of drives. I would honestly try to find something with a few 100Gb ports and then 25Gb front panel. Maybe a used Nexus N9K-C93180YC-EX or -FX (48x1/10/25 SFP ports and 6x 40/100G ports) could do the job but they are loud just keep that in mind and they're still going to run hundreds to a thousand dollars for one. They also are all CLI based configurations and if you're not familiar with Cisco's NXOS operating system then it could be a bit daunting to start with.

 

If you want to stick to 10Gb copper for the clients and 100Gb for the server then something like a Nexus N9K-C93108TC-EX or -FX would do the job with 48 front panel copper 1/10g ports and 6x40/100G ports too.

 

 

I know there are other vendors in the space and Mikrotik comes to mind and I think might have something cheaper and smaller but it's been a bit since I've looked at their copper/fiber offerings for a single box but definitely worth checking out too.

[OddOod]

52 minutes ago, Leduc said:

As an example, a point cloud can become multiple gig in size for a single file, but as soon as we talk about imagery it become multiple terabytes divided in hundred of thousands of files.

When those come in, is there any chance they are on a drive that you could just slap into the server and dump from there? multi TB dumps really shouldn't be pushed from workstations.