BOINC Pentathlon 2023

[Justaphf]

1 minute ago, SkillzTA said:

 

I am thinking this is a bug with the 7.16.6 client. After I cool down from this year's Pent and reorganize my fleet of computers I am going to put this to the test by trying different client versions until one doesn't experience the issue.
 

The second delay which happened today was related to a platform not being found.  Here are the surrounding event logs.

Quote

Thu 18 May 2023 02:35:02 PM EDT | Einstein@Home | Sending scheduler request: To fetch work.
Thu 18 May 2023 02:35:02 PM EDT | Einstein@Home | Reporting 2 completed tasks
Thu 18 May 2023 02:35:02 PM EDT | Einstein@Home | Requesting new tasks for NVIDIA GPU
Thu 18 May 2023 02:35:04 PM EDT | Einstein@Home | Scheduler request completed: got 0 new tasks
Thu 18 May 2023 02:35:04 PM EDT | Einstein@Home | platform 'x86_64-pc-linux-gnu' not found
Thu 18 May 2023 02:35:04 PM EDT | Einstein@Home | Project requested delay of 86400 seconds

 

3 minutes ago, SkillzTA said:

I need to get all my computers moved from the different locations they all to near my balcony door so I can open the doors and expel the heat out quicker. Two straight weeks of 90F+ temps inside is exhausting to say the least.

I just bought steeply reduced price mining frames so was able to stack them and put them right next to my basement window, which worked out nicely since the weather the past 2 weeks (other than 2 days of rain) was nice enough that I could leave the windows open night and day.  Of course I'm mostly GPU for folding, this is my first Pent and I haven't really done any Boinc since the LHC days.

 

Thanks for looking into these issues!

[Chachunka]

Third place overall! Fantastic achievement. It's been a great fun competing with you all -- teammates and opponents alike.

[Justaphf]

5 minutes ago, Justaphf said:

The second delay which happened today was related to a platform not being found.  Here are the surrounding event logs.

 

Couldn't find the 12-hour delay for the quota exceeded.  Realized it happened in the event log before I detached and reattached yesterday.  From vague recollection it was something along the lines of "not getting new tasks: daily quota exceeded (2064)".  Not sure of the number though, other that it was in the 2000's.

[GOTSpectrum]

[justpoet]

Congrats everybody on a great crunch, and an overall 3rd!  It was fun to jump in and figure this out, even though I didn't have the time to manage things like I would've liked.  I'm still sure my piddly cpu points were put to good use.  

[WiredInventor]

Just a reminder! Shutterdown for awhile, let your poor poor hardware take a break. Please keep folding/boincing at least a little bit if you can! The projects that you contribute too are very thankful.

[rkv_2401]

Great work, everyone! Not only did we manage to keep our 3rd place in the Marathon/Obstacle Run like we did in 2021, we also scored 3rd overall!

3 medals (with 1 silver, our highest position in an event so far AFAIK) is a very nice reward for our collective efforts. This was great fun!

[leadeater]

1 hour ago, SkillzTA said:

If it's currently a problem, before you reset it, can you send me a link to your host? I want to check something on it first if that's okay.

I have the issue on https://einsteinathome.org/host/12778252. Only sends me 1 WU per GPU and usually doesn't even grab another until current is finished.

[SkillzTA]

Just now, leadeater said:

I have the issue on https://einsteinathome.org/host/12778252. Only sends me 1 WU per GPU and usually doesn't even grab another until current is finished.

 

This is why:

 

2023-05-19 01:19:57.0961 [PID=683103]    [send] CUDA: req 1.00 sec, 0.00 instances; est delay 0.00

 

The client is only asking for 1 second worth of tasks.

 

edit

Your client version is 7.20.2. So the bug might be present all the way to the latest version.

[leadeater]

1 minute ago, SkillzTA said:

 

This is why:

 

2023-05-19 01:19:57.0961 [PID=683103]    [send] CUDA: req 1.00 sec, 0.00 instances; est delay 0.00

 

The client is only asking for 1 second worth of tasks.

 

edit

Your client version is 7.20.2. So the bug might be present all the way to the latest version.

I can try anything you want when I get home if you like. I'd like to see if a cause can be found like only happens when multiple projects are added. Didn't want to mess around while the event was running.

[leadeater]

@Windows7ge

Time to put all my hosts on WCG for the weekend before I shut them down

 

[SkillzTA]

1 minute ago, leadeater said:

I can try anything you want when I get home if you like. I'd like to see if a cause can be found like only happens when multiple projects are added. Didn't want to mess around while the event was running.

Ive had the issue on hosts that are only attached to E@H. So I don't think multiple projects is the issue. Something in the client is causing it to get stuck like this.

 

2x 3060Ti GPUs and a 6 core (12 thread) processor shouldn't be doing a lot of work to issue what we believe is the trigger.

 

From what I have been told the quotas each host have per day is;

512 tasks per GPU (Up to 8 GPUs counted on a host, anymore and it wont add anything additional.)

64 tasks per CPU (Up to 128 counted on a host, anymore and it wont add anything additional.)

 

That means a 2x GPU and 12 CPU (6 core, 12 threads) host should be able to have a 1792 task quota per day. 

 

Does that host do more than 1792 tasks per day (24 hour period)?

If it does and it gets the reached maximum quota message, then the client goes into a mode that stops asking for a lot of work and will only request one task per. 

 

We think that once this happens the client gets stuck in that state. Even after the 24 hours has passed where it resets its quotas for the day.

 

 

 

[leadeater]

6 minutes ago, SkillzTA said:

Does that host do more than 1792 tasks per day (24 hour period)?

I doubt it, not done a deeper check but based on WU times it's no more than 600.

 

 

Edit:

6 minutes ago, SkillzTA said:

If it does and it gets the reached maximum quota message, then the client goes into a mode that stops asking for a lot of work and will only request one task per. 

Many days ago I remember seeing such a event log message but I don't remember which host or project etc. It's likely E@H and me aborting a ton of CPU tasks it grabbed before I disabled CPU tasks. That could have thrown it in to restricted mode.

[SkillzTA]

Well, if you're bored and want to try to replicate the issue then that's what you could try.

 

The log snippet I posted can be seen on the E@H web site, on the host page of the host. Click the last contact date link and that log will be displayed. 

 

Look for the line I posted and see if you can force that to happen. 

 

I would just start a new instance to test on instead of using the current broken one though.

 

It's bed time for me.

[TylerD321]

Soooo, when do the prizes get drawn?  

[EdenThief]

3 hours ago, etv24 said:

Just a reminder! Shutterdown for awhile, let your poor poor hardware take a break. Please keep folding/boincing at least a little bit if you can! The projects that you contribute too are very thankful.

I paid for the whole gauge, I'm going to use the whole gauge!

[adm0n]

I still have over 150 tasks in pending, but they wouldn't have helped. But at least I contributed like 1.5m credits. Was fun everyone!

[Ithanul]

Big thankie to everyone.  LTT did mighty well in this BOINC Pentathlon!

[GOTSpectrum]

As far as prizes go, I will try to get it sorted this weekend 

[Windows7ge]

21 hours ago, leadeater said:

Time to put all my hosts on WCG for the weekend before I shut them down

I haven't ran WCG since they separated from IBM over a year ago. I'd love to go back and plan to if they're online again but right now isn't a great time for me. I'm not in a good position to jack my electric bill up $100 every month. Plus the heat, plus the noise, plus my landlord is already mad about how many I have while they're idling...

 

You better bet when I come back though I'm gonna come back swinging. E5-2699v3 CPUs are CHEAP and DL380 G9 servers aren't much more expensive. Plus my work can hook me up with free/cheap stuff. I'm gonna build myself a 240V cluster compute node with as many as 400 CPU cores! >:3

[leadeater]

45 minutes ago, Windows7ge said:

I'd love to go back and plan to if they're online again

WCG has been running for quite a long time now FYI. Not run it since the move either though.

[Gorgon]

On 5/18/2023 at 9:36 PM, leadeater said:

I doubt it, not done a deeper check but based on WU times it's no more than 600.

 

 

Edit:

Many days ago I remember seeing such a event log message but I don't remember which host or project etc. It's likely E@H and me aborting a ton of CPU tasks it grabbed before I disabled CPU tasks. That could have thrown it in to restricted mode.

You basically have to spam the WGC servers ever 5-7 minutes to get Open Pandemic WUs. Very hit and miss.

#0-59/5 * * * * sleep $(awk 'BEGIN{srand();print int(rand()*300)}'); /usr/bin/boinccmd --project www.worldcommunitygrid.org update

[leadeater]

1 hour ago, Gorgon said:

You basically have to spam the WGC servers ever 5-7 minutes to get Open Pandemic WUs. Very hit and miss.

 

#0-59/5 * * * * sleep $(awk 'BEGIN{srand();print int(rand()*300)}'); /usr/bin/boinccmd --project www.worldcommunitygrid.org update

Thanks, not really related to the quoted post but helpful for what I am doing now 

 

I've got the bug where non of my devices I just added show other than one, emailed support to see if they can get it fixed. Just running my systems open allowing anything but I'd like my system with GPUs in to to have a different profile so I can try out the GPU WUs. Can't do anything until it shows or use the local xml files. Going to wait for support before bothering to do anything.

[Runkies]

Just realized this whole time i wasn't part of the LTT Team for Einstein@Home... im going to go over here and cry now

[Gorgon]

5 hours ago, leadeater said:

Thanks, not really related to the quoted post but helpful for what I am doing now 

 

I've got the bug where non of my devices I just added show other than one, emailed support to see if they can get it fixed. Just running my systems open allowing anything but I'd like my system with GPUs in to to have a different profile so I can try out the GPU WUs. Can't do anything until it shows or use the local xml files. Going to wait for support before bothering to do anything.

Yeah, Good luck with that. The whole transition from IBM to Krembil has been a Sh*t Show. They have the best intentions in the world but appear to lack competent IT support. The transition was supposed to take two months back in Feb 2022 but when I gave up a couple of months ago was still  a work in progress. Their Data Center is hosted by SharcNet, a consortium of Canadian Universities in Ontario and doesn't seem to have after core business hours or weekend support.

 

Hints. Take a look at your device's stats and see if you can find the device ID off the URL Sometimes you can inject that Device ID into a request to actually configure the profile.

 

Or change the default profile, add the new device with the GPU then change the default profile back to no GPU Tasks. But you'll have to poke around the UI as the default settings are fairly well hidden.