LTT Folding Team's Emergency Response to Covid-19

[Gustavehomme01]

2 minutes ago, HaRdLy007 said:

Sorry to interject, but how do you request a quota increase? I have a 1 GPU limit right now and would like to increase it to at least 4.

No problem see post bellow!!

 

1 hour ago, bafo_ah said:

Have you "request quota change"

click on hamburger menu (three strip menu on top left) -> IAM & Admin -> Quotas

*you need to enter to Compute Engine -> VM Instance first so they can configured your environment or something for first time, it take a time before you can create your first VM

set filter (click None first, so It clear the selection - so many quotas here)

set filter

Service -> compute engine API

Metric -> GPUs (all region) + NVIDIA T4 GPUs

Location -> Global + yourdesiredlocation

 

then you need to edit your quotas, send request to google team, and they will decided if you can do this (I request 4GPU and they approved it)

 

Then you can try to create instance again (Compute Engine -> VM Instance

 

[ChocolateCyanide]

18 minutes ago, Gustavehomme01 said:

Yep i just traded in my 2 P100 for this same setup. You get a little more PPD at half the price. Just wish i would have noticed before spent 36 hours on the P100 theoretical lost of  5-6 M points in that same 36 hours... 

 

Old setup 2 P100 = 1.2-1.7M PPD each total 2.4-3.4M PPD

New setup 8 T4= 800-900k PPD each total 6.4-7.2M PPD

Almost same cost.

I have found the V100's to be the most cost effective. Each one pulls in around 3.5M PPD for only $1.788/hr. I was also going to go with T4's but it only lets me do one GPU type per region so I can't do that many T4's.  

Edited April 8, 2020 by ChocolateCyanide

[ffbr]

8 minutes ago, Zberg said:

Desperately clawing to stay on the bottom of the graph and not fall off the page! 

Desperately flexing to get on the bottom of the graph and not stay off the page!

[HaRdLy007]

5 minutes ago, Gustavehomme01 said:

No problem see post bellow!!

 

 

Thanks, request sent

[Gustavehomme01]

So after doing my 1am cost analysts math I decided to spin down the 2 P100 for 8 T4 instead. My new army for the day...

Local Slot00 is a Ryzen 5  3600 = 100k PPD now turn off 

Local Slot 01 is a RX 590 = 550k PPD been going hard since the start and till the end.. 

So it looks like a sustained 7mPPD

 

I'v also said good bye to the following:

177Wats for a combined 50k PPD, just not worth operating now. plus the heat in the office was getting to me.

 

Oh and 420

[HaRdLy007]

Wow that Google Cloud increase request is already completed!

[bafo_ah]

45 minutes ago, ShortRouter said:

What's your PPD on this config? Btw I tried everything, 4x T4 is impossible to get atm so I got 2x P100.

Around 800k per T4 I use 4 GPU, so around 3.5M per instance...

Spoiler

 

[Gustavehomme01]

3 minutes ago, ChocolateCyanide said:

I have found the V100's to be the most cost effective. Each one pulls in around 3.5M PPD for only $1.788/hr. I was also going to go with T4's but it only lets me do one GPU type per region.  

 

You have to ask for a quota increase. That's what got me as well the fist 36 hours...

 

1 hour ago, bafo_ah said:

Have you "request quota change"

click on hamburger menu (three strip menu on top left) -> IAM & Admin -> Quotas

*you need to enter to Compute Engine -> VM Instance first so they can configured your environment or something for first time, it take a time before you can create your first VM

set filter (click None first, so It clear the selection - so many quotas here)

set filter

Service -> compute engine API

Metric -> GPUs (all region) + NVIDIA T4 GPUs

Location -> Global + yourdesiredlocation

 

then you need to edit your quotas, send request to google team, and they will decided if you can do this (I request 4GPU and they approved it)

 

Then you can try to create instance again (Compute Engine -> VM Instance)

 

I'm with ya if you look at card per card but if you look at an instance of 4 T4 to 1 P100 same cost but more the double the PPD with the 4 T4s. 

 

As you can see my P100 cost me a lot more then the advertised price of 1.09$/h. plus you still have to add the cost of the CPU/RAM and network  all in all it was roughly 4.5$/h for 2 P100s..  I dont yet have the full cost of the 8 T4s but considering the config was advertised less then the s P100 I should get more PPD. 

 

[Lavonthomon]

Do you still get credit for bad WU?

[skaullywag]

10 hours ago, GOTSpectrum said:

Heres the stats, blog post will be done when this god damn cat moves her arse!

 

https://docs.google.com/spreadsheets/d/1qg4_XMVQ18jk0CNDSgyU5Vx_doNxMAhnXraHRWl8C8M/edit?usp=sharing

Top 1k barely woohoo!

[Gustavehomme01]

4 minutes ago, Lavonthomon said:

Do you still get credit for bad WU?

Yes you do base on the amount of work that you did one them. It's usually just a fraction of the total.

[LordKitsune]

44 minutes ago, ShortRouter said:

If it's a case fan just unplug it while PC is running. If you want shutdown, make sure you pause folding so the WU doesn't get corrupted.

Normally I would but this one hooked into my PSU.

 

3-pin fan with on-board speed switch. Fan is supposed to be on high but the shaft is warped so at higher speeds the thing hits the side of its housing and makes a stupidly annoying sound. It's nowhere near as bad when on lowest setting, and I have 0 fan headers left on my mobo (Yes, I could use splitters, don't feel like dropping any more $$$ on this computer when I'm about to be building a new one.)

[ffbr]

2 minutes ago, LordKitsune said:

Normally I would but this one hooked into my PSU.

 

3-pin fan with on-board speed switch. Fan is supposed to be on high but the shaft is warped so at higher speeds the thing hits the side of its housing and makes a stupidly annoying sound. It's nowhere near as bad when on lowest setting, and I have 0 fan headers left on my mobo (Yes, I could use splitters, don't feel like dropping any more $$$ on this computer when I'm about to be building a new one.)

Nothing that a wire cutter can"t solve 

[LordKitsune]

11 minutes ago, Lavonthomon said:

Do you still get credit for bad WU?

No, You will not receive any credit for a bad WU. Even though there likely is data that can be used, a single FPC being wrong can throw the ENTIRE rest of the calculation off. 

 

Bad WUs are scrapped by the server after upload and are reassigned to a different user. Thrown out data is data that isn't contributing to the next round of WUs

[bafo_ah]

1 minute ago, LordKitsune said:

Normally I would but this one hooked into my PSU.

 

3-pin fan with on-board speed switch. Fan is supposed to be on high but the shaft is warped so at higher speeds the thing hits the side of its housing and makes a stupidly annoying sound. It's nowhere near as bad when on lowest setting, and I have 0 fan headers left on my mobo (Yes, I could use splitters, don't feel like dropping any more $$$ on this computer when I'm about to be building a new one.)

Make your own splitter, scissor and tape will do it! (NO dont do it, you could broke another fan)

Spoiler

 

[LordKitsune]

Just now, ffbr said:

Nothing that a wire cutter can"t solve 

You're right...

 

I don't want to take wire cutters to my only pc though. If anything just because I don't want to not be crunching on a WU. Screw it if I get shocked, might wake me up!

[macmanlex]

I've got 3 different systems setup to Fold, but I am not getting any Work.  Any ideas?

 

UPDATE: 2 of them actually are now, but 1 has been trying for hours to download and it's only getting "No WUs available for this configuration"

[walderston]

1 minute ago, macmanlex said:

I've got 3 different systems setup to Fold, but I am not getting any Work.  Any ideas?

This should really be pinned...

 

Add :

client-type     advanced

https://foldingathome.org/support/faq/installation-guides/configuration-guide/

 

[ChocolateCyanide]

23 minutes ago, Gustavehomme01 said:

 

You have to ask for a quota increase. That's what got me as well the fist 36 hours...

 

 

I'm with ya if you look at card per card but if you look at an instance of 4 T4 to 1 P100 same cost but more the double the PPD with the 4 T4s. 

 

As you can see my P100 cost me a lot more then the advertised price of 1.09$/h. plus you still have to add the cost of the CPU/RAM and network  all in all it was roughly 4.5$/h for 2 P100s..  I dont yet have the full cost of the 8 T4s but considering the config was advertised less then the s P100 I should get more PPD. 

 

Ah, I see...I was just increasing my overall quota for GPUs. I didn't realiseI could specify region. Thanks!

And yeah P100 definitely is worse, I should have checked it sooner oh well haha. The V100 is not too bad though, just a little bit worse than the T4's.

[ShortRouter]

47 minutes ago, HaRdLy007 said:

Sorry to interject, but how do you request a quota increase? I have a 1 GPU limit right now and would like to increase it to at least 4.

https://stackoverflow.com/questions/45227064/how-to-request-gpu-quota-increase-in-google-cloud

[Plexas]

Today is not a good day. My pump decided 20 days of usage was too much for it and decided to kick the bucket. Major pain in the ass to get it swaped with the current lockdowns in place... but we're back in the fold.  

[bafo_ah]

1 minute ago, walderston said:

This should really be pinned...

 

Add :

client-type     advanced

https://foldingathome.org/support/faq/installation-guides/configuration-guide/

 

Yeah with the consequences you got "not so stable and tested" wu.. Why not change it to beta for the risk and "bigger pool"?

[LordKitsune]

2 minutes ago, Plexas said:

Today is not a good day. My pump decided 20 days of usage was too much for it and decided to kick the bucket. Major pain in the ass to get it swaped with the current lockdowns in place... but we're back in the fold.  

Everybody is having mechanical issues now. We've been folding too hard!

 

 

[bafo_ah]

6 minutes ago, macmanlex said:

I've got 3 different systems setup to Fold, but I am not getting any Work.  Any ideas?

Can you show us your log? There is so many factor.. I even found that my internet connection can be a problem (dunno why) - my cloud folding is seem getting more wu than my folding rig.... So I setup a VPN, and viola..

[macmanlex]

7 minutes ago, walderston said:

This should really be pinned...

 

Add :

client-type     advanced

https://foldingathome.org/support/faq/installation-guides/configuration-guide/

 

Can you explain please.  Thanks