Jump to content
Search In
  • More options...
Find results that contain...
Find results in...

Here in Eastern Ontario we have Time of Use (ToU) Electricity Rates that increase the price of Electricity during "Mid Peak" and "On Peak" times during the day.


If your running multiple GPUs Folding you've likely noticed the increases in your Electricity Bill and here are some hints on what you can do to keep folding during the hot months and not live in a sweat box and pay through the nose.


The very first change you should contemplate is reducing the GPU's Power Limit. During normal folding in the Fall, Winter and Spring months with 10-12 GPUs I run my cards about 20% above the minimum Power Limit. I find this gives me around 90% of the PPD for 30-40% less power. NVidia Pascal and Turing GPUs, like modern CPUs, tend to run close to the edge of stability and at "stock" 100% Power Targets are not very efficient. During the summer months I run the GPUs at the minimum Power Limits for an even greater decease in Power with only a slight lessening of PPD.


While using a GUI like the NVidia Control Panel or MSI Afterburner you can lower the power limits you can also use the NVidia System Management Interface:

nvidia-smi -i 0 -pl 90

Here we are forcing the power-limit on GPU 0 to 90W. This will work in either Windows or Linux.


You will also want to enable "Persistence Mode" so the Power Limit is not reset to the Default after the current task is finished. This command:

nvidia-smi -pm 1

will enable persistence mode for all GPUs. A specific GPU can be specified also using the "-i <X>" switch where "<X> is the Device ID shown in




Scheduling Folding only during less expensive ToU periods can be accomplished but will take a bit of work to setup. Here we leverage the ability of the FAHClient program to send commands to a running "Slot" (GPU).


What we need to do is tell the GPU slots to "Finish" about half the average time it takes for that GPU to complete a Work Unit (WU) before the start of the ToU rate period. In my case my GPUs typically complete a WU in about 3 hours when running at minimum power limits so I schedule the "Finish" command to run at 05:30 as the ToU period here starts at 07:00. This results in the GPUs finishing their current WUs then "Pausing." When the ToU period ends you schedule a "Un-Pause" command to be sent to resume folding. In my case the ToU rates end at 19:00.


In Linux you could use the "cron" scheduler thus:

# use /bin/bash to run commands, instead of the default /bin/sh
# Time of Day Folding - Summer
#Mid-Peak 07:00 - 11:00
30 5 * * 1-5 /usr/bin/FAHClient --send-finish
#On-Peak 11:00-17:00
#30 9 * * 1-5 /usr/bin/FAHClient --send-finish
#Mid-Peak 17:00 - 19:00
#00 17 * * 1-5 /usr/bin/FAHClient --send-unpause
#Off-Peak 19:00-07:00
00 19 * * 1-5 /usr/bin/FAHClient --send-unpause

Windows users can utilize the Command Scheduler and the same commands:

'C:\Program Files (x86)\FAHClient\FAHClient.exe --send-finish'
'C:\Program Files (x86)\FAHClient\FAHClient.exe --send-unpause'

The result looks something like this:


Here 5 of my systems are scheduled to "Finish" at 05:30 and by 07:00 my GPU power consumption has dropped from 1kW to 300W with the last couple of Slots "Pausing" by ~07:20. Note: I am also running some BOINC tasks on the Blue and Dark Red systems accounting for the periodic power spikes.


And here's what it looks like over the last couple of weeks:


So you can see I run 24hours during weekends and holidays and about 12 hours (19:00-07:00) during weekdays when ToU rates are in effect.



Bifrost - 6 GPU Folding Rig  Linux Folding HOWTO Folding Remote Access


dcn01: Fractal Meshify C; Gigabyte Aorus ax570 Master; Ryzen 9 5950x; EVGA 240 CLC; 2 x 16GB DDR4-3200; 512GB NVMe; EVGA RTX 2070 Super XC Hybrid; Corsair TX750M

dcn02: Fractal Define S; Gigabyte ax570 Pro WiFi; Ryzen 9 3950x; Noctua NH-D15; 2 x 16GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; EVGA RTX 2070 XC Gaming; Corsair TX650M

dcn03: Fractal Meshify S2; Gigabyte Aorus ax570 Pro WiFi; Ryzen 9 3900x; Noctua NH-D15; 2 x 16GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; EVGA RTX 2060 XC Gaming; Corsair TX650M

dcn04: Fractal Meshify S2; Gigabyte z370 Gaming 5; i9-9900K; EVGA 280 CLC; 4 x 4GB DDR4-2400; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; EVGA RTX 2060 XC Ultra Gaming; Corsair TX650M

dcn05: Fractal Define R4; Gigabyte ax370 Gaming K7; Ryzen 7 2700x; Hyper 212Evo e/w Noctua NF-A12 iPPc 3000 PWM; 2 x 8GB DDR4-3200; 128GB NVMe; EVGA RTX 2070 Super XC Hybrid; EVGA GTX 1660ti XC Ultra Gaming; Corsair TX650M

dcn06: Fractal Define C; Gigabyte ax570 Gaming X; Ryzen 7 2700; 2 x 4GB DDR4-2400; Samsung 250GB SSD; Gigabyte GTX 1060 6GB; Corsair TX550M

dcn10: Supermicro SC731; Gigabyte Aorus b450m; Ryzen 5 2700; 2 x 8GB DDR4-2400; Cruical 64GB SSD; SuperMicro 300W Bronze

dcn11: Fractal Core 1100; Gigabyte Aorus b450m; Ryzen 5 2600x; 2 x 8GB DDR4-3200; Adata 256GB NVMe; Seagate ES 7200rpm 3TB HDD; EVGA GTX 1070ti SC Gaming; Corsair CX500

dcn12: Fractal Focus G; Gigabyte z370 SLI; Pentium G5500; 1 x 4GB DDR4-2400; Samsung 256GB SSD; Corsair CX450M

dcn18: Acer E3400; AMD Athlon II x64; 2 x 2GB DDR3-1600; 240GB Spinning Rust

dcn19: NUC6i3SYK; Intel i3-6100U; 2 x 8GB DDR4-2133; Samsung 120GB NVMe

Link to post
Share on other sites

Some time ago, I mashed this bash script together to set my nVidia cards to 75% of the default power draw:

for UUID in `nvidia-smi -L |cut -d: -f3 |cut -d\) -f1 |awk '{print $1}'`; do
	DEFAULT=`nvidia-smi -i $UUID -q -d POWER |grep 'Default Power' |cut -d: -f2 |awk '{print $1}'`
	MIN=`nvidia-smi -i $UUID -q -d POWER |grep 'Min Power' |cut -d: -f2 |awk '{print $1}'`
	SET=`echo $DEFAULT $MIN $PERCENT |awk '{printf "%f", ( $1 * $3 < $2 ? $2 : $1 * $3 )}'`
	nvidia-smi -i $UUID -pl $SET

It can handle any number of cards with varying power limits, as well as automatically capping the low end to the minimum supported power level.

CPU Intel Xeon X5650 | Motherboard ASUS Sabretooth X58 | RAM Corsair Vengeance 1.6GHz 9-9-9-24 24GB | GPU ASUS Dual Series GTX 1060 6GB | Case Fractal Design Define R5 |

Storage Samsung 860 Pro 512GB, WD Black 640GB, WD Green 4TB | PSU Corsair TX750 "v1" | Display ASUS VP247QG + Samsung SyncMaster T220 | Cooling Corsair H80i GT

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now