TUTORIAL FREE PPD Cloud GPU Tesla P100/V100 F@H! FINAL edition V2

[Metallus97]

On 10/27/2019 at 6:15 AM, sazrocks said:

Thanks, @Metallus97! Got it working really easily with your instructions.

 

I also got remote control (using FAHControl) working.

I added a firewall rule under VPC Network>Firewall Rules>Create Firewall rule

I set the network to default, target to "All instances on this network", Source Ip ranges to 0.0.0.0/0 [This matches all IPs], protocols and ports to tcp port 36331 [Nonstandard port to try to prevent portscanning bots from easily guessing that FAH is running]. 

 

In the VM, I edited the FAH config.xml to include

<allow v='127.0.0.1 My.Public.ip.Address.at.home'/>
<command-port v='36331'/>
<password v='My_Super_Secret_Password'/>

Do note that the password is sent in plaintext, so don't use anything that you particularly care about. This is again just to mainly keep out bots.

 

Then restart FAH with this command (no need to reboot the machine):

sudo /etc/init.d/FAHClient restart

After this add the client to FAHControl as you normally would, making sure to use the external IP address of the VM (shown on the compute engine homepage) as the IP and also making sure to set the port to 36331 (or whatever it was that you set it to). Also make sure to enter your password. If everything went well you should see the slots show up in FAHControl just as if FAH were running on your local system.

Nice work bro!

i did not have much time the last weeks 

am I allowed to add this to the howto?

ill mention you

[sazrocks]

1 hour ago, Metallus97 said:

Nice work bro!

i did not have much time the last weeks 

am I allowed to add this to the howto?

ill mention you

Sure, go ahead.

[Favebook]

On 10/10/2019 at 3:42 PM, Metallus97 said:

The payed account was created to have the opportunity to apply for a (or several) GPU slots. Support has to unlock them for you. But this works every time. Requesting a single or maybe 2 GPUs is peanuts for them. To request GPU quota go to: Top left main menu: IAM & Adminstration -> quotas. Search for Max GPU allowed here:

  Reveal hidden contents

You have to deselect everything and then only select GPUs (all regions) leave all the other boxes as they are. Now check the box with "Compute Engine API GPUs (all Regions)" at the top of the page click "edit quota". You now have to give them your name email and phonennumer (I used 0218347348 or something but suggest taking your real one. They are trustworthy and that way you wont have complications in getting approved). You also have to tell them what you plan to to. I wrote something like "I need it for a month for a Folding event. If performance is nice ill consider letting my work run on your servers (I actually do this right now). Then send the request.

 

I just tried to make a new account at GCP and request the quotas increase. However, "Compute Engine API GPUs (all Regions)" are not showing at all. Can anyone confirm this? As it is entirely possible that they removed this feature for me (due to a new account) or maybe even entirely.

 

However, on old accounts which are already actively folding there is still the option for "Compute Engine API GPUs (all Regions)" available.

 

 

Edited October 31, 2019 by Favebook
Totally my bad, I was inpatient.

[straight_stewie]

On 10/10/2019 at 8:42 AM, Metallus97 said:

Best PPD for PPD (PerformancePerDollar for PonitsPerDay seems to be a Nvidia Tesla P100 GPU. It pulls about 1,4mPPD and is a little more than 1$/h of usage time.

If you run out of free service, the AWS EC2 p2.16xlarge Spot instance provides 16 K80 GPUs for roughly $4.00/hr.

If your number of $1/gpu/hr for Google cloud is correct, then the p2.16xlarge spot instance gives you roughly the same ppd performance per card, but you get 4 GPUs for every dollar/hr spent. This makes the cost $0.25/gpu/hr.

The caveat, again, is that this is only a better option if you have run out of free Google Cloud service.

[Metallus97]

1 hour ago, straight_stewie said:

If you run out of free service, the AWS EC2 p2.16xlarge Spot instance provides 16 K80 GPUs for roughly $4.00/hr.

If your number of $1/gpu/hr for Google cloud is correct, then the p2.16xlarge spot instance gives you roughly the same ppd performance per card, but you get 4 GPUs for every dollar/hr spent. This makes the cost $0.25/gpu/hr.

The caveat, again, is that this is only a better option if you have run out of free Google Cloud service.

THX for the info Sir. Ill try that and compare some PPD numbers. Ill add it when PPD is confirmed

[mescalin]

I use google cloud computing for boinc rosetta@home cancer research.

 

I found a gpu rental site and found a tutorial in reddit but couldnt make it work. I keep getting driver errors.

 

Vast.ai is the site. Anybody knows how to run fah on those? I dont know linux at all, all I can do is copy paste type basic things. I think I updated drivers but couldnt make it work.

[Metallus97]

24 minutes ago, mescalin said:

I use google cloud computing for boinc rosetta@home cancer research.

 

I found a gpu rental site and found a tutorial in reddit but couldnt make it work. I keep getting driver errors.

 

Vast.ai is the site. Anybody knows how to run fah on those? I dont know linux at all, all I can do is copy paste type basic things. I think I updated drivers but couldnt make it work.

Uh nice! So how are zu running boinc on GCC then? isn’t that also linux?

Vast.ai should work according to my guide as long as it has Nvidia GPUS

[mescalin]

7 hours ago, Metallus97 said:

Uh nice! So how are zu running boinc on GCC then? isn’t that also linux?

Vast.ai should work according to my guide as long as it has Nvidia GPUS

Boinc is very easy to install cuz it doesn’t need drivers. Just two lines of copy paste.

 

I’ll try your tutorial on vast.ai

[Metallus97]

2 hours ago, mescalin said:

Boinc is very easy to install cuz it doesn’t need drivers. Just two lines of copy paste.

 

I’ll try your tutorial on vast.ai

Ah ok, but then it doesn’t use the GPU. That’s the same with F@H

 

cool! Let me/us know how it goes 

[mescalin]

3 hours ago, Metallus97 said:

Ah ok, but then it doesn’t use the GPU. That’s the same with F@H

 

cool! Let me/us know how it goes 

Yes I use google for cpu wus for rosetta@home only cuz adding gpu is pricey on google cloud. (I finished my free credits and I pay for it)

 

Also you might have mentioned it but If you set on demand pricing on google you pay About the half price. For example if it’s 1$ per hour and you set it to work daily it’s $0,5. But you need to start it everyday. Just download google app and press start button and it goes on without loosing any progress.

I tried vast.ai but I get that error. Tried to install drivers again, it shows the gpu normally but when I run fah it gives me this error.

 

Failed to start core: OpenCL device matching slot 0 not found, try setting 'opencl-index' manually

[Metallus97]

2 hours ago, mescalin said:

Yes I use google for cpu wus for rosetta@home only cuz adding gpu is pricey on google cloud. (I finished my free credits and I pay for it)

 

Also you might have mentioned it but If you set on demand pricing on google you pay About the half price. For example if it’s 1$ per hour and you set it to work daily it’s $0,5. But you need to start it everyday. Just download google app and press start button and it goes on without loosing any progress.

I tried vast.ai but I get that error. Tried to install drivers again, it shows the gpu normally but when I run fah it gives me this error.

 

Failed to start core: OpenCL device matching slot 0 not found, try setting 'opencl-index' manually

Hmm does smi work and detect the GPU?

also sich GPU Modell is it?

[mescalin]

Yes smi shows the gpu without problem. I tried it on several instances with different cards rtx 2080, gtx 1080ti, rtx 2060 but I get the same result.:/

 

 

[Metallus97]

6 hours ago, mescalin said:

Yes smi shows the gpu without problem. I tried it on several instances with different cards rtx 2080, gtx 1080ti, rtx 2060 but I get the same result.:/

 

 

Ok damn.... l I’ll try it at the weekend

[Heaven]

Is this still working or I'm late to the party?
I got the trial, tried doing a compute VM, 4 cores, CPU only but it won't work: I always get: Quota 'C2_CPUS' exceeded. Limit: 0.0 in region europe-west4. Tried several regions and got the same error.
Looking in quotas seems to be all 0?

[Metallus97]

3 hours ago, Heaven said:

Is this still working or I'm late to the party?
I got the trial, tried doing a compute VM, 4 cores, CPU only but it won't work: I always get: Quota 'C2_CPUS' exceeded. Limit: 0.0 in region europe-west4. Tried several regions and got the same error.
Looking in quotas seems to be all 0?

Should still be working. For a 4core machine you have to request two more CPU slots. Maybe they have changed the default maximum CPU cores?

[Heaven]

I chose to upgrade the account and request a quota increase. They replied and told me that they are unable to increase my quota and to try again in 48 hours which is pretty funny and kind of bad since this is Google and not some random guy running some servers in his basement. LOL

I don't think I'm gonna bother them anymore and just use my slow Desktop for folding. Google should order some more Threadripper CPU's.

[Metallus97]

9 hours ago, Heaven said:

I chose to upgrade the account and request a quota increase. They replied and told me that they are unable to increase my quota and to try again in 48 hours which is pretty funny and kind of bad since this is Google and not some random guy running some servers in his basement. LOL

I don't think I'm gonna bother them anymore and just use my slow Desktop for folding. Google should order some more Threadripper CPU's.

how much did you ask for?

[Heaven]

8 cores.
But i was OK with 4 cores too, something so i can get a VM running..

[rkv_2401]

Hey guys,

 

I tried doing this on Google Cloud and got as far as actually getting folding@home to run.

 

But the GPU(a Tesla V100) isn't recognized or something.

 

In the folding log, I get : 

Spoiler

GPUs : 0

CUDA : 7.0

CUDA driver : 10000

And on running the "nvidia-smi" command, it shows 0% utilization. I followed the guide word-for-word, and I also tried reinstalling the nVidia drivers and rebooting the VM. 

 

EDIT :

 

There were some driver installation warnings that came up :

1) nVidia-KRM or something wasn't installed,

2) The installed driver was not compatible with 32-bit applications.

 

I tried to reinstall the drivers to replicate it, but it didn't work.

Can anyone help me out with this?

[mescalin]

53 minutes ago, rkv_2401 said:

Hey guys,

 

I tried doing this on Google Cloud and got as far as actually getting folding@home to run.

 

But the GPU(a Tesla V100) isn't recognized or something.

 

In the folding log, I get : 

  Hide contents

GPUs : 0

CUDA : 7.0

CUDA driver : 10000

And on running the "nvidia-smi" command, it shows 0% utilization. I followed the guide word-for-word, and I also tried reinstalling the nVidia drivers and rebooting the VM. 

 

EDIT :

 

There were some driver installation warnings that came up :

1) nVidia-KRM or something wasn't installed,

2) The installed driver was not compatible with 32-bit applications.

 

I tried to reinstall the drivers to replicate it, but it didn't work.

Can anyone help me out with this?

I couldn't run fah in the cloud. So I run boinc rosetta@home in the cloud. It's much easier to run.

 

 

 

sudo apt-get install boinc-client

 

project_attach https://boinc.bakerlab.org/  [YOURACCOUNTKEY]

 

boinccmd --get_tasks

 

 

to get your accountkey https://boinc.bakerlab.org/rosetta/weak_auth.php

 

 

 

[Metallus97]

4 hours ago, rkv_2401 said:

Hey guys,

 

I tried doing this on Google Cloud and got as far as actually getting folding@home to run.

 

But the GPU(a Tesla V100) isn't recognized or something.

 

In the folding log, I get : 

  Reveal hidden contents

GPUs : 0

CUDA : 7.0

CUDA driver : 10000

And on running the "nvidia-smi" command, it shows 0% utilization. I followed the guide word-for-word, and I also tried reinstalling the nVidia drivers and rebooting the VM. 

 

EDIT :

 

There were some driver installation warnings that came up :

1) nVidia-KRM or something wasn't installed,

2) The installed driver was not compatible with 32-bit applications.

 

I tried to reinstall the drivers to replicate it, but it didn't work.

Can anyone help me out with this?

did you choose the deep learning image? That should come prepackaged with all the necessary drivers for F@H! 

[Metallus97]

3 hours ago, mescalin said:

I couldn't run fah in the cloud. So I run boinc rosetta@home in the cloud. It's much easier to run.

 

 

 

sudo apt-get install boinc-client

 

project_attach https://boinc.bakerlab.org/  [YOURACCOUNTKEY]

 

boinccmd --get_tasks

 

 

to get your accountkey https://boinc.bakerlab.org/rosetta/weak_auth.php

 

 

 

Uh noooice! I made the guide for the F@H month but ill add in BOINC when I have more time. RL is stressful atm. But guess next month ill ad in BOINC!

But what did not work for your F@H install?

[rkv_2401]

5 hours ago, Metallus97 said:

did you choose the deep learning image? That should come prepackaged with all the necessary drivers for F@H! 

Yes, I had chosen the "Deep Learning Image: Base m43 (with CUDA 10.0)" image.

[mescalin]

7 hours ago, Metallus97 said:

Uh noooice! I made the guide for the F@H month but ill add in BOINC when I have more time. RL is stressful atm. But guess next month ill ad in BOINC!

But what did not work for your F@H install?

As I remember it was a driver issue. Fah console was giving opencl errors even after reinstalling the drivers. I had similar issues with vast.ai

[Metallus97]

16 hours ago, rkv_2401 said:

Yes, I had chosen the "Deep Learning Image: Base m43 (with CUDA 10.0)" image.

 

13 hours ago, mescalin said:

As I remember it was a driver issue. Fah console was giving opencl errors even after reinstalling the drivers. I had similar issues with vast.ai

okay... did you guys try to delete the machine and reconfigure a new one?