Linux CPU Utilization while Folding 1 vs 2 GPU

[Gorgon]

I was doing some maintenance on my folding rigs this morning and as the topic of CPU Utilization for GPU Folding seems to come up now and then I thought I'd post some graphs.

 

My primary folding rig is based on a Gigabyte Z370 Aorus Gaming 5 with 2 x 4GB GSkill RipJaws V GDDR4 2400 (6.43GB average free) and a dual core, quad thread 3.7GHz Intel Pentium Gold G5400 This CPU, while likely under powered for most productivity tasks, is more than up to the job of keeping a EVGA GTX 1070 Ti SC Black and a EVGA GTX 1060 6GB SC fed and running at 91.6% and 97% utilization respectively in their PCIe3 x16 slots both running at x8. The OS is Ubuntu 18.04.1 LTS Server. Here is the CPU utilization while pausing folding on the GTX 1070 Ti:

 

My secondary folding rig is based on a salvaged Acer AX3400 ITX nForce motherboard with an 2.9GHz AMD Athlon II X2 220 dual core processor and 3GB DDR2 1333 Memory (1.88GB average free) and a EVGA GTX 1060 6GB SC Short-Board. The OS is Ubuntu 18.04 LTS (Desktop). Here is what it's CPU Utilization looked like around the same time:

This CPU has to work a little harder to keep the GPU in it's single PCIe2 x 16 slot fed and running at 94.61% utilization.

[RollinLower]

Interesting stuff! Thanks for posting graphs aswell.

Also kinda cool how cutting off the more powerful 1070Ti, keeps CPU utilisation pretty much halved. 

These cards are of course kinda close together, but still interesting to see nonetheless!

[Gorgon]

It takes a little work but keeping both GPUs in the 60-70C range is achievable.

 

The 1070Ti was running at a 210W Power Limit at the time and the 1060 at a 100W power limit. The 1070 was at 70C and 70% Fan and the 1060 was at 60C with a 35% fan speed. One of the reasons I went with this mainboard was the 2 PCIe3 x16 slots were spaced 3-slots apart leaving a 1-slot gap. Tuning the chassis fan placement and curve and tying the curve to the PCIe3 x16 slot sensor allows for adequate airflow while folding at low and high power GPU limits.

 

The 2.5 - 2.75 slot sizing for the RTX boards makes them less desirable for dual GPU folding rigs but the lower end RTX 2070s look promising.

[Gorgon]

@IthanulI'd be interested seeing CPU graphs from a dual GPU Windows folding system while one GPU is paused and Windows isn't forcing an update down your throat

[GOTSpectrum]

On 10/25/2018 at 12:09 PM, Gorgon said:

@IthanulI'd be interested seeing CPU graphs from a dual GPU Windows folding system while one GPU is paused and Windows isn't forcing an update down your throat

I run two dual GPU folding systems, one with a GTX 1080ti/ GTX 980 and one with a GTX 980ti/ GTX 950. I can sorry this tomorrow if you like. 

[Gorgon]

10 hours ago, Ben Quigley said:

I run two dual GPU folding systems, one with a GTX 1080ti/ GTX 980 and one with a GTX 980ti/ GTX 950. I can sorry this tomorrow if you like. 

Thanks Ben,

 

You better leave your 1080 Ti shut off for a few days so I can catch up to you we can get a good comparison. 

 

Seriously. Just pause one slot for a couple of minutes (you might have to use the command line for this as I believe the advanced control pauses both) and grab a screenshot of the aggregate CPU utilization.

 

Edit: right-click on the slot in FAHControl to pause a single slot.

 

Regards,

 

Bruce

 

P.S. Figured out how to modify my Zabbix agent to report the PCIe bus utilization but I don’t think it’s going to be too useful as it appears to be just an instantaneous value reported by nvidia-smi when polled. But while I was at it I configured the Shader and Memory clocks so I can graph those over time as well.

[GOTSpectrum]

7 hours ago, Gorgon said:

Thanks Ben,

 

You better leave your 1080 Ti shut off for a few days so I can catch up to you we can get a good comparison. 

 

Seriously. Just pause one slot for a couple of minutes (you might have to use the command line for this as I believe the advanced control pauses both) and grab a screenshot of the aggregate CPU utilization.

 

Edit: right-click on the slot in FAHControl to pause a single slot.

 

Regards,

 

Bruce

 

P.S. Figured out how to modify my Zabbix agent to report the PCIe bus utilization but I don’t think it’s going to be too useful as it appears to be just an instantaneous value reported by nvidia-smi when polled. But while I was at it I configured the Shader and Memory clocks so I can graph those over time as well.

I'll be having some downtime today as I upgrade my second system CPU. Will do the pausing thing at the same time with my main system. Do you wanna see it as one graph in task manager rather than per thread graphs?

[GOTSpectrum]

7 hours ago, Gorgon said:

Thanks Ben,

 

You better leave your 1080 Ti shut off for a few days so I can catch up to you we can get a good comparison. 

 

Seriously. Just pause one slot for a couple of minutes (you might have to use the command line for this as I believe the advanced control pauses both) and grab a screenshot of the aggregate CPU utilization.

 

Edit: right-click on the slot in FAHControl to pause a single slot.

 

Regards,

 

Bruce

 

P.S. Figured out how to modify my Zabbix agent to report the PCIe bus utilization but I don’t think it’s going to be too useful as it appears to be just an instantaneous value reported by nvidia-smi when polled. But while I was at it I configured the Shader and Memory clocks so I can graph those over time as well.

 

I highlighted the time when the folding slot was stopped for you. This is an 8c/16t processor folding on 8 threads. 

[Gorgon]

6 hours ago, Ben Quigley said:

 

I highlighted the time when the folding slot was stopped for you. This is an 8c/16t processor folding on 8 threads. 

Thanks Ben,

 

Not a huge difference here but it looks like there was a lot of processes other than just GPU folding going on.  You went from about 65% aggregate CPU utilization to about 58% so a net change about 7% which is consistent with one thread of 16 (6.25%) being freed up.

 

Regards,

 

Bruce

[GOTSpectrum]

Just now, Gorgon said:

Thanks Ben,

 

Not a huge difference here but it looks like there was a lot of processes other than just GPU folding going on.  You went from about 65% aggregate CPU utilization to about 58% so a net change about 7% which is consistent with one thread of 16 (6.25%) being freed up.

 

Regards,

 

Bruce

If you want me to try anything else before the folding event please do let me know. 

[Gorgon]

Thanks,

 

I'm just waiting for some slots to finish their WUs then I’m going to take the single slot system off line and clean the dust filters.  Did the dual slot system last night and was pleasantly surprised how little dust had accumulated over 4 months considering these are in the unfinished portion of the basement.

 

The bottom case fan that pushes air up towards the GPUs in the Define S had about the same accumulation of dust as the front filters but the Power Supply area of the bottom filter had next to none which I take as an indication that the Corsair Gold Power Supply is running cool enough that it can either not run its fan or runs it at such a low speed to not attract much dust.