TUTORIAL FREE PPD Cloud GPU Tesla P100/V100 F@H! FINAL edition V2

[rkv_2401]

8 hours ago, Metallus97 said:

 

okay... did you guys try to delete the machine and reconfigure a new one?

Oooh, I hadn't thought of that.

 

So I deleted it and set it up again, but I'm still having the same problem.

 

I'm using deep learning image : base m42, BTW.

 

EDIT : Seems like an internet problem now, because last time I didn't get as far as having a second slot. It's so close to working

 

[rkv_2401]

Got it to run. Finally. Had to disable some firewall rules to get past the last problem.

[ConceptOW]

On 11/12/2019 at 9:01 PM, mescalin said:

Boinc is very easy to install cuz it doesn’t need drivers. Just two lines of copy paste.

 

I’ll try your tutorial on vast.ai

 

On 11/13/2019 at 10:30 AM, mescalin said:

Yes smi shows the gpu without problem. I tried it on several instances with different cards rtx 2080, gtx 1080ti, rtx 2060 but I get the same result.:/

 

 

I just found out about vast.ai.. It's very hit or miss. I have problems connecting to some. I have one working currently 

What docker image are you using? 

[jctappel67]

On 10/10/2019 at 8:42 AM, Metallus97 said:

<allow v='127.0.0.1 My.Public.ip.Address.at.home'/> 

Was I supposed to use my own IP address for this part? 

 

I'm a complete noob to SSH, and more advanced computing in general.

[sazrocks]

8 minutes ago, jctappel67 said:

Was I supposed to use my own IP address for this part? 

 

I'm a complete noob to SSH, and more advanced computing in general.

Yes, this is your own public WAN IP address (NOT your LAN ip address). You can check it with a site like https://www.whatismyip.com/what-is-my-public-ip-address/

[LFreeman]

is there a way to manually start F@H? I missed the option to autostart.

[Favebook]

1 hour ago, LFreeman said:

is there a way to manually start F@H? I missed the option to autostart.

Just type "Folding" in search bar and app will pop up.

[LFreeman]

3 hours ago, Favebook said:

Just type "Folding" in search bar and app will pop up.

Thanks. I just blew the VM out and started over.

[TheAmazingDave]

I followed this excellent guide to set up a VM on Google a few nights ago. It was chugging away nicely until yesterday morning. The server stopped responding, and with the help of Google support, it turns out it was running out of RAM. This seemed to happen when it picked up a big-boy WU 13873 with a base credit of 28.8k. This morning I changed the settings a bit, I kept it at 2 CPUs but bumped the RAM up to 4GB from the guide's suggested 1.8, and the VM started fine and didn't stop responding when it started chugging on the WU. Top shows I could probably get away with setting it at 3.25-3.5GB RAM, so I'll make that adjustment when this WU finishes.

 

Just throwing it out there in case anyone else experiences the same.

 

 

 

[jctappel67]

10 minutes ago, TheAmazingDave said:

I followed this excellent guide to set up a VM on Google a few nights ago. It was chugging away nicely until yesterday morning. The server stopped responding, and with the help of Google support, it turns out it was running out of RAM. This seemed to happen when it picked up a big-boy WU 13873 with a base credit of 28.8k. This morning I changed the settings a bit, I kept it at 2 CPUs but bumped the RAM up to 4GB from the guide's suggested 1.8, and the VM started fine and didn't stop responding when it started chugging on the WU. Top shows I could probably get away with setting it at 3.25-3.5GB RAM, so I'll make that adjustment when this WU finishes.

 

Just throwing it out there in case anyone else experiences the same.

 

 

 

The exact same thing happened to me, I just killed that instance and set up an new one with more ram

[GrockleTD]

TFW you find out your debit card won't work for the free trial...

[first2mars]

This worked great! I tested with the newest FAH version and its working just the same. New version can be found at

 

https://download.foldingathome.org/releases/public/release/fahclient/debian-stable-64bit/v7.5/fahclient_7.5.1_amd64.deb

[gramzon]

Hi Metallus97, 

 

Thank you for the great post, I've followed this through twice now and it's worked both times. Have 2 VMs on 2 accounts setup to run until they run out of credit!  

A few things may have changed since you created the thread though and I am also doing this from the UK.

 

The differences I noticed were:

1. To upgrade account: Top left menu > biling > upgrade bottom right under free trial credit.

2. Unable to find Deep learning image version you mentioned, maybe it's been upgraded. I used "Deep Learning Image: Base m44 (with CUDA 10.0)" instead of m36 and can confirm working.

3. I picked the 3.6GB version of the CPU/RAM combination from the start as other users have mentioned 1.8GB may cause application crashes, unsure if this will effect how quickly the credit runs out.

4. A word of reassurance to other users following the guide without prior goolge compute knowledge: Nvidia driver errored for me but after leaving it for a few minutes it completed and after reboot was working so if it errors give it some time to complete reboot and confirm it's installed.

5. Also I can confirm what first2mars has mentioned regarding version 7.5.1, I have also installed this version and it is working at the moment.

6. The amount of free credit you get in the UK is the free $300 which when you look in your billing section is roughly £244.16.

 

 

Edit: Taking a closer look I can see that using the 3.6GB option means it has 4 CPU cores instead of the 2, is this advisable or is there another option that would be cost efficient?

 

Edit2: For those that need to edit CPU/RAM amounts you don't need to delete the machine, you can power down the VM and click on the machine name which takes you to a page you can edit the machine.

 

Edit3: I have my personal machine with an i7 4790k + GTX1070, 4 x VMs on my server with 4 cores each, sisters desktop with 4 cores and 3 google VMs with P100's and averaging 6.2m PPD  

[DieselWeasel]

I'm having trouble getting this to work. When I install Fah and reboot the fah client doesn't seem to be working

 

Spoiler

Setting up fahclient (7.4.4) ...
The Folding@home client is now installed
You can access the Web interface by going to:
  http://folding.stanford.edu/client/
in a browser on this computer.
For information about configuring the Folding@home client for 
remote access please see the comments in:
  /etc/fahclient/config.xml
Processing triggers for mime-support (3.60) ...
Processing triggers for systemd (232-25+deb9u12) ...
jure_komac@instance-1:~$ sudo nano /etc/fahclient/config.xml
jure_komac@instance-1:~$ sudo nano /etc/fahclient/config.xml
jure_komac@instance-1:~$ sudo reboot
Connected, host fingerprint: ssh-rsa 0 6C:04:E7:60:3F:A9:0E:4A:81:C2:A1:0B:F2:02:39:3D:F3:F7:03:F8:55:1D:D3:6A:0E:B
F:EC:DE:0F:7E:E4:AD
======================================
Welcome to the Google Deep Learning VM
======================================
Version: common-cu100.m42
Based on: Debian GNU/Linux 9.11 (stretch) (GNU/Linux 4.9.0-11-amd64 x86_64\n)
Resources:
 * Google Deep Learning Platform StackOverflow: https://stackoverflow.com/questions/tagged/google-dl-platform
 * Google Cloud Documentation: https://cloud.google.com/deep-learning-vm
 * Google Group: https://groups.google.com/forum/#!forum/google-dl-platform
To reinstall Nvidia driver (if needed) run:
sudo /opt/deeplearning/install-driver.sh
Linux instance-1 4.9.0-11-amd64 #1 SMP Debian 4.9.189-3+deb9u2 (2019-11-11) x86_64
The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
jure_komac@instance-1:~$ nvidia-smi
Tue Apr  7 16:08:24 2020       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 410.104      Driver Version: 410.104      CUDA Version: 10.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Tesla P100-PCIE...  Off  | 00000000:00:04.0 Off |                    0 |
| N/A   35C    P0    27W / 250W |     10MiB / 16280MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+
jure_komac@instance-1:~$ tail /var/lib/fahclient/log.txt
16:08:01:WU00:FS00:Connecting to 65.254.110.245:8080
16:08:02:WARNING:WU00:FS00:Failed to get assignment from '65.254.110.245:8080': Empty work server assignment
16:08:02:WU00:FS00:Connecting to 18.218.241.186:80
16:08:02:WU01:FS01:Connecting to 65.254.110.245:80
16:08:02:WARNING:WU00:FS00:Failed to get assignment from '18.218.241.186:80': Empty work server assignment
16:08:02:ERROR:WU00:FS00:Exception: Could not get an assignment
16:08:02:WARNING:WU01:FS01:Failed to get assignment from '65.254.110.245:80': Empty work server assignment
16:08:02:WU01:FS01:Connecting to 18.218.241.186:80
16:08:02:WARNING:WU01:FS01:Failed to get assignment from '18.218.241.186:80': Empty work server assignment
16:08:02:ERROR:WU01:FS01:Exception: Could not get an assignment
jure_komac@instance-1:~$ 

 

 

At the start when I ssh into the vm i get 3 warnings, but the nvidia driver installs anyway.

Spoiler

WARNING: The nvidia-drm module will not be installed. As a result, DRM-KMS will not function with this

         installation of the NVIDIA driver.

 

 

WARNING: nvidia-installer was forced to guess the X library path '/usr/lib' and X module path

         '/usr/lib/xorg/modules'; these paths were not queryable from the system.  If X fails to find the NVIDIA X

         driver module, please install the `pkg-config` utility and the X.Org SDK/development package for your

         distribution and reinstall the driver.

 

 

WARNING: Unable to find a suitable destination to install 32-bit compatibility libraries. Your system may not be

         set up for 32-bit compatibility. 32-bit compatibility files will not be installed; if you wish to install

         them, re-run the installation and set a valid directory with the --compat32-libdir option.

 

I've tried installing 7.4.4 and 7.5.1 but I'm getting the same result

 

 

EDIT: I got it working somehow...

[Rowza245]

Idk why but it seems like i'm doing something wrong. I've got 2 VMs running but i'm not getting anywhere close to the PPD I would expect and i don't even know if i'm being credited with the points so i'm not sure if i've set up my identity correctly but everything seems right . The points update has been flaky at best over the last few days anyway running from my own machine so i'm unsure if its F@H's servers or me. Here's a read out from the one that's got a task on both CPU and GPU and it's config. 

 

Everything looks okay to me but i'm probably missing something obvious. If anyone could help it'd be much appreciated, otherwise if nothing seems wrong i'll ditch these VM's and start again.

Thanks

 

Edit: i forgot to mention that i did check the passkey, it is definitely correct  

 

Edit 2: Just made a completely new VM, seems to be running fine now. I think I messed around with it too much and it broke  

[WhyKickAmooCow]

Hey, though I should share this. Let me know if you wanted a separate thread about it and maybe I could do a proper guide or something.

 

I have been using the GCE "preemptible" instances with the P100 GPUs, which as you can see is quite a lot cheaper (more points for me!).

It's quite confusing, but looking at the billing page where it shows that credits are being used to run these preemptible instances even though this page says that free tier credits can't be used for preemptive instances. I guess what this means is that you need to be a paid account in order to run these instances?

 

Anyway, I digress. I use a shutdown script and a startup script of:

FAHClient --config=/etc/fahclient/config.xml --send-pause
sleep 15

and

FAHClient --config=/etc/fahclient/config.xml --send-unpause

respectively to give the instance time to safely stop folding when it inevitably does get killed (these instances last AT MOST 24hrs before they get shutdown, and likely get killed much before that (depending on time of day mine stay up from about an hour to a few hours).

 

So, that solves the safely shutting down and resuming folding problem, but manually checking for a stopped instance and starting it up again is a real drag so...

I use cloud scheduler to start the instance back up every 5 minutes (if it is already up, nothing happens) using exactly what this guide explains. I use the Cloud Scheduler -> PubSub -> Cloud Functions way. Which given this incredibly small scale, Cloud Scheduler, PubSub, and Cloud Functions are free! I have the Cloud Scheduler run the job once every 5 minutes using the cron expression '*/5 * * * *' but if you are unfamilar a quick google search should help you figure out what you should use for the frequency you want, and this site should help too.

 

So, I have my juicy 1.7MPPD-ish instance for almost an entire month! FOR FREE!

[gramzon]

I was going to give Cherry servers a go but it's asking me to put in 5 euros to get 30 euros free, can anyone confirm this is the case now?

[Metallus97]

12 hours ago, WhyKickAmooCow said:

Hey, though I should share this. Let me know if you wanted a separate thread about it and maybe I could do a proper guide or something.

 

I have been using the GCE "preemptible" instances with the P100 GPUs, which as you can see is quite a lot cheaper (more points for me!).

It's quite confusing, but looking at the billing page where it shows that credits are being used to run these preemptible instances even though this page says that free tier credits can't be used for preemptive instances. I guess what this means is that you need to be a paid account in order to run these instances?

 

Anyway, I digress. I use a shutdown script and a startup script of:

FAHClient --config=/etc/fahclient/config.xml --send-pause
sleep 15

and

FAHClient --config=/etc/fahclient/config.xml --send-unpause

respectively to give the instance time to safely stop folding when it inevitably does get killed (these instances last AT MOST 24hrs before they get shutdown, and likely get killed much before that (depending on time of day mine stay up from about an hour to a few hours).

 

So, that solves the safely shutting down and resuming folding problem, but manually checking for a stopped instance and starting it up again is a real drag so...

I use cloud scheduler to start the instance back up every 5 minutes (if it is already up, nothing happens) using exactly what this guide explains. I use the Cloud Scheduler -> PubSub -> Cloud Functions way. Which given this incredibly small scale, Cloud Scheduler, PubSub, and Cloud Functions are free! I have the Cloud Scheduler run the job once every 5 minutes using the cron expression '*/5 * * * *' but if you are unfamilar a quick google search should help you figure out what you should use for the frequency you want, and this site should help too.

 

So, I have my juicy 1.7MPPD-ish instance for almost an entire month! FOR FREE!

wowowo that’s nice! I didn’t know about that. I case you want to you could also do a little writeup (über what I quoted as the base of it) on it and i’ll include it in the post as a cheaper/longer lasting option that needs more work and configuring. Also now THANKS for sharing this!

Please send me a PM, @jctappel67 has already done modifications to the original post to reflect the change to this type of instance. What we need now is a tutorial on how to implement those shutdown and startup scripts plus the scheduler to restart the instances. I dont have the time to write it amt... but would be happy  to include it in the OP!

 

payed account: Yeha I guess that is because of the GPU. To get a GPU in the instanced that I have shown you also need to change to a payed account (but are still able to first use the free stuff)

[Metallus97]

9 hours ago, gramzon said:

I was going to give Cherry servers a go but it's asking me to put in 5 euros to get 30 euros free, can anyone confirm this is the case now?

Hm... maybe this was implemented bacause of to manny of us trying to use it for free and then never pay them anything?

[CyberHusky88]

I just got started on this, I've upgraded my account, but when I go to the quotas page,there are no Compute Engine APIs listed, and searching for GPU under Service or Metric comes up empty. 

 

Does anyone have any ideas on what I need to do at this part?

 

Spoiler

 

[gramzon]

4 hours ago, CyberHusky88 said:

I just got started on this, I've upgraded my account, but when I go to the quotas page,there are no Compute Engine APIs listed, and searching for GPU under Service or Metric comes up empty. 

 

Does anyone have any ideas on what I need to do at this part?

 

  Reveal hidden contents

 

You either didn't wait long enough after doing step 2 or you missed step 2. I just ran in to the same issue, after giving it a couple minutes for the compute part to be created the option was available.

 

Step 2: Once your in at the dashboard: Click on compute engine, then you have to wait a few minutes for google cloud to activate some stuff... (see screenshot, everything will be in spoilers form now on to keep this already big post as clean as possible)

[Shlouski]

What the best way of keeping your eye on credit usage and keep track of each instance?

[Metallus97]

12 hours ago, Shlouski said:

What the best way of keeping your eye on credit usage and keep track of each instance?

I guess you could write some nice scripts and enable email notifications. What I did is download the GCE app and keep an eye on it that way

 

13 hours ago, gramzon said:

You either didn't wait long enough after doing step 2 or you missed step 2. I just ran in to the same issue, after giving it a couple minutes for the compute part to be created the option was available.

 

Step 2: Once your in at the dashboard: Click on compute engine, then you have to wait a few minutes for google cloud to activate some stuff... (see screenshot, everything will be in spoilers form now on to keep this already big post as clean as possible)

yep!

[msvelev]

********@instance-2:~$ FAHClient --send-command ppd
13:45:15:Connecting to 127.0.0.1:36330
PyON 1 ppd
1719557.679499
---

 

Seems aight
Thanks!

 

P.S. I started getting 

"Empty work server assignment" errors for the GPU.

Is this a normal error message on linux? I've never seen it on windows.