Processing Data on a Server

[Jonezy Lothos]

Hi all!

 

First time posting the the forum

 

I have to process data in a software package called MassHunter and it can take up to 15 min to process a large data set with the computer that came with the GC/MS (HP Z400 Xeon W3520 Quad Core Processor, 8GB of 1333 MHz ECC RAM, 500 GB SATA HDD)

 

My thought is that more cores, more RAM, and a SSD would decrease the processing time. My proof of concept PC is a HP ML350 G6 server with dual Xeon 5560 2.80GHz, 48GB of 1333 MHz ECC RAM and a Samsung 850 EVO 1TB SSD running windows 7 PRO. The SSD cost more than the whole rest of the server combined but I think it was worth it.

 

With my new server I have decreased the processing time by 50%!!! That is great but I think I can get more out of it. The software uses all 16 threads (8 per CPU) but it is only utilizing 20% +/- 5%, the RAM maxes out at 8GB and the SSD is running at most 20MB/sec.  

 

How do I squeeze more out of my "Processing Server"???

 

Thanks for the help,

 

Jonezy

[samiscool51]

@wrathoftheturkey let me quote you this

26 minutes ago, Jonezy Lothos said:

I have to process data in a software package called MassHunter

anyways, if the program has hardware acceleration  then enable it, if not but if the program is still being developed then ask the developers for hardware acceleration support for faster data processing, if they don't want to add support then you are out of luck, unless you overclock the xeon you won't get anymore performance out of it

anyways what does it process? there might be a alternative program out their that might do the work more quickly

[Jonezy Lothos]

Sadly this is the only software I can use with the GC/MS. There is other software packages but they use a 32 bit OS and I do not want to go down that path

[Jonezy Lothos]

16 minutes ago, samiscool51 said:

unless you overclock the xeon you won't get anymore performance out of it

So a faster clock speed will result in faster processing? I can update the cpu to a  X5675 3.06 GHz (6 cores/ 12 threads) to potentially increase threads and frequancy.

 

I can also purchase a better server as well but I wanted to make a cost/benefit analysis before I pull the trigger on a +$1000 server.

 

 

[Blake]

 

2 hours ago, Jonezy Lothos said:

So a faster clock speed will result in faster processing? I can update the cpu to a  X5675 3.06 GHz (6 cores/ 12 threads) to potentially increase threads and frequancy.

 

I can also purchase a better server as well but I wanted to make a cost/benefit analysis before I pull the trigger on a +$1000 server.

 

 

How often are you doing this number crunching? and how much data are you using?

 

It could be an idea to just use Azure\AWS every couple of weeks and have azure crunch through everything. https://www.troyhunt.com/the-worlds-greatest-azure-demo/

 

If it's an everyday thing, maybe not... 

 

You need to open up perfmon, and have it monitor CPU/RAM/DISKIOPS/ETC/ETC see where the bottleneck is, and upgrade/replace that part.

[samiscool51]

2 hours ago, Jonezy Lothos said:

1. So a faster clock speed will result in faster processing?

2. I can update the cpu to a  X5675 3.06 GHz (6 cores/ 12 threads) to potentially increase threads and frequancy.

1. if it worked that way i would have a super computer with a Pentium 4, but life doesn't work that way

this video explains how overclocking works, and the workloads it can benefit alltho i'm not so sure about the last one

https://www.youtube.com/watch?v=RPmDSYRKnXM

these videos are also good to watch
https://www.youtube.com/watch?v=S3I5WNHbnJ0

 

https://www.youtube.com/watch?v=wnS50lJicXc

2.does the Mb support it socket type and has the latest BIOS that has support for it? if so then go for it! i have a 6 core in my rig and it helps with CPU bound games like minecraft, also i do a lot of virtualisation and i don't wan't a xeon because the newer ones are expensive

[leadeater]

Sounds to me that best and cost effective solution would be a single socket high clock 6-8 core Xeon. From your description the software stops benefiting from more cores/threads rather quickly so the more each thread can do the better.

 

If possible you can test the thread to clock rate performance using ESXi and use resource limits to restrict the CPU cycles the VM gets. With that you could simulate 4 cores at 1Ghz vs 4 cores at 2GHz vs 8 cores at 1GHz etc.

 

The other test I would do is run 2 VMs and split the data set across them, this would get round thread scaling limits in theory.

[Jonezy Lothos]

19 hours ago, Blake said:

How often are you doing this number crunching? and how much data are you using?

We generally have 2-3 computers processing data 20-30 times a day

 

19 hours ago, samiscool51 said:

this video explains how overclocking works, and the workloads it can benefit

I do not want to overclock since I am dealing with sensitive data on a server platform.

 

18 hours ago, leadeater said:

The other test I would do is run 2 VMs and split the data set across them, this would get round thread scaling limits in theory.

Sadly I cannot separate the data sets or modify the raw data. The software makes it hard to modify the data and there is also the legality of modifying it.

 

I can't seem to find the bottleneck. None of the hardware is running even 30% of its limit. I hope I am not stuck with this problem. I guess I can always go get another cup of coffee or clean my deck while I wait....

 

Jonezy

 

 

[leadeater]

1 minute ago, Jonezy Lothos said:

We generally have 2-3 computers processing data 20-30 times a day

 

I can't seem to find the bottleneck. None of the hardware is running even 30% of its limit. I hope I am not stuck with this problem. I guess I can always go get another cup of coffee or clean my deck while I wait....

So is there a limit to the number of computers you can use to do the processing? Running 2 VMs per computer and treating them totally separately should get you closer to 60% hardware utilization from the current 30% but that's assuming you can have 4-6 computers processing data up from the current 2-3. 

[samiscool51]

4 minutes ago, Jonezy Lothos said:

I do not want to overclock since I am dealing with sensitive data on a server platform.

thats ok, i never overclock a xeon

[Mikensan]

how did you verify 20mb/s? Should also consider checking the iops during a session. You can strip some disks for performance to test if that helps at all. If this is purely computational then yea disk won't matter.

 

The operating system is probably what is trying to split the load up across all cores evenly, which is why you see it using all 16 threads. Run it again, and go into task manager and set the application's affinity to 2 and see what happens.

 

Only real difference between the two test systems is the clock rate given your results. They're the same architecture however. If you have access to an i5 or i7 based system (sandybridge or greater) with the same or greater clock rate (friend or family) see if you can run a single test on it to verify.

[Jonezy Lothos]

On 10/18/2016 at 7:39 PM, leadeater said:

So is there a limit to the number of computers you can use to do the processing?

The computers usually doing the processing are linked to the GC/MS and are relatively slow. I am trying to find another option. I am hoping to speed up the processing so my employees don't pull out their phones and start checking facebook....

7 hours ago, Mikensan said:

how did you verify 20mb/s?

Resource Monitor/Task Manager 

 

7 hours ago, Mikensan said:

If you have access to an i5 or i7 based system (sandybridge or greater) with the same or greater clock rate (friend or family) see if you can run a single test on it to verify.

the HP ML350 G6 only supports Xeon 5500 & 5600 chipsets.

 

I could try running it on my i7-6700k with 32GB of ram but I don't want to spend the money on work computers. I think I will just have to deal with the slow processing time. Thanks for the help!

 

Jonezy

[Jonezy Lothos]

UPDATE:

 

So I have made a few modifications to my processing server. Noticing that I will not be utilizing the full resources of my system I am now running two Virtual Machines and I have updated some of the components.  I replaced the old X5560 (4 core/2.80GHz) CPUs with the X5675 (6 core/3.06GHz). This has decreased my processing time another 20% ish.  

 

Another perk of building this machine on a server platform is that I have added NAS for our current server. This allows me to have nightly backup of our data. So essentially I have three computers in one for the cost of a mid range PC ($450+/-). I am in the process of creating another server to replace two more outdated workstation computers.  Let me know what I need to look out for moving forward. I don't see anything too ominous in the future but I am still new to all this.

 

-Jonezy