BOINC Pentathlon 2019

[leadeater]

3 minutes ago, Ithanul said:

O darn, that sounds nice.  I be down to seeing that and/or helping.

 

I really need to get better at scripting.  Still a newbie at it.  Only been at my current job for one year so far.

Could even run up our own stats website with real time tracking that everyone, that wishes to, could submit stats to. I'm thinking directly off the clients and not pulling off project websites so it is actually real time information, might even then be able to show valid credits and total inc pending.

 

Edit:

Hell could then even do team daily targets that all clients look at and project switch based on those.

[Ithanul]

3 minutes ago, leadeater said:

Could even run up our own stats website with real time tracking that everyone, that wishes to, could submit stats to. I'm thinking directly off the clients and not pulling off project websites so it is actually real time information, might even then be able to show valid credits and total inc pending.

 

Edit:

Hell could then even do team daily targets that all clients look at and project switch based on those.

That sounds really nice.

 

I have no experience in that regards.  I be down for learning how to do it as I been curious how stat sides pull the data and how some teams run their own stat sites.

[porina]

5 hours ago, Gorgon said:

Well this is a stupid competition IMHO. Just finishing Einstein on my GPUs and over 1/2 the WUs submitted are still "pending" and haven't been validated yet. Most of those have been submitted for at least 24 hours so their double validation is obviously broken in general or specifically due to the volume from the competition and people bunkering.

Roughly 1/3 of all my output is pending. I checked some of those units, and in all observed cases they're waiting on wingman to finish. So no obvious sign of the validator having problems, just luck of outstanding. 

[leadeater]

1 hour ago, porina said:

Roughly 1/3 of all my output is pending. I checked some of those units, and in all observed cases they're waiting on wingman to finish. So no obvious sign of the validator having problems, just luck of outstanding. 

Now I think LHC is just broken, no credit validation for anyone in hours.

[GOTSpectrum]

1 hour ago, porina said:

Roughly 1/3 of all my output is pending. I checked some of those units, and in all observed cases they're waiting on wingman to finish. So no obvious sign of the validator having problems, just luck of outstanding. 

I have over 500 Units still pending on Einstein too 

[leadeater]

1 minute ago, Ben Quigley said:

I have over 500 Units still pending on Einstein too 

 

[porina]

Taking leadeater's total LHC units, roughly 1/3 have reported (valid+pending), and 2/3 are outstanding. It is quote possible the pending proportion is sat in someone else's "in progress" pile. Give it time, especially if there might be bunkering for a late dump. Can't be helped, but it more or less affects everyone equally.

[leadeater]

2 minutes ago, porina said:

Taking leadeater's total LHC units, roughly 1/3 have reported (valid+pending), and 2/3 are outstanding. It is quote possible the pending proportion is sat in someone else's "in progress" pile. Give it time, especially if there might be bunkering for a late dump. Can't be helped, but it more or less affects everyone equally.

Already table flipped and moved the big server back to WCG lol

 

Edit:

Also pending credit doesn't count, so like it's a really big problem.

[leadeater]

@Ben Quigley

I'm going to switch everything back to WCG until there some kind of sign pending work is actually getting processed, we can take two spots.

[GOTSpectrum]

5 minutes ago, leadeater said:

@Ben Quigley

I'm going to switch everything back to WCG until there some kind of sigh pending work is actually getting processed, we can take two spots.

Sounds fine

[Egg-Roll]

I have a problem... WCG isn't taking my work nothing is showing up and now it's refusing to authenticate my account  

 

All I"m getting is this:

uninstalling reinstall did nothing...

[leadeater]

15 minutes ago, Ben Quigley said:

Sounds fine

Good news, I think it's just started to actually process

 

 

[leadeater]

14 minutes ago, Egg-Roll said:

I have a problem... WCG isn't taking my work nothing is showing up and now it's refusing to authenticate my account  

 

All I"m getting is this:

uninstalling reinstall did nothing...

Remove and re-add the project

[leadeater]

@Windows7ge

What's your's looking like, my pending tasks are now thundering through. Another 1000 just got done since I last posted, just before.

 

Edit:

Also good thing I just wrote a project switching script 

[porina]

Quote

State: All (769) · In progress (86) · Validation pending (662) · Validation inconclusive (0) · Valid (21) · Invalid (0) · Error (0) 

Not seeing much progress on my LHC units... however since I didn't bunker and only did them "live", I might be more held up by others bunkering or in progress with large cache.

 

[leadeater]

4 minutes ago, porina said:

Not seeing much progress on my LHC units... however since I didn't bunker and only did them "live", I might be more held up by others bunkering or in progress with large cache.

 

It feels like it's doing it one account at a time or something, or in batches serially. Single teams will get a huge jump and everyone else is stuck at 1k or below each hourly update. We have no idea where any of the teams actually are at because of it.

 

This is going to be highly interesting as pending work starts to come in, expect to see big changes.

[Egg-Roll]

42 minutes ago, leadeater said:

Remove and re-add the project

Done, it also did this

The annoying part is I did finish projects, guess a pulsar ate them.

 

It takes my L5640 about 2.5 hours to complete per core which seems to add up with userbenchmark vs my i5.

[GOTSpectrum]

We're in 9th in the Sprint, keep it up guys!

[leadeater]

2 minutes ago, Ben Quigley said:

We're in 9th in the Sprint, keep it up guys!

Guess I'm switching all my systems back 

[Egg-Roll]

Just checked the stats again the finished projects got uploaded half seem to be valid with the other half (nearly 3 pages) validating

[leadeater]

36 minutes ago, Ben Quigley said:

We're in 9th in the Sprint, keep it up guys!

Ok so not sure why I didn't click right away about the quorum thing, thought it was something different but no different to cluster quorums. That's going to make this interesting and very hard, tasks have to be run by two people and compared before it's counted as valid. Seriously I could be a gaint ass to everyone and pull in thousands of jobs on my big server and then suspend them.

[porina]

54 minutes ago, leadeater said:

Seriously I could be a gaint ass to everyone and pull in thousands of jobs on my big server and then suspend them.

On average, it would hurt everyone else equally. For the competition, its about relative positions so wouldn't affect things. Now, if you were to scrape the wingman details you could selectively allow or disallow certain ones and that could swing things, but obviously that would be a very naughty thing to do.

 

It was actually a thing on PrimeGrid. There is the somewhat unique situation where the first to report a prime gets their name entered in the records. Like Top Gun, there are no points for 2nd place. Ok, you still get boinc credit, but that's not quite the same. You would gain an advantage to returning first if the server gives you the unit before someone else. Some person or persons actually implemented a script to check a unit, and if it was already "done" they would abort it. Didn't take long for the admins to notice this and prevent it from happening by suppressing the details until the completed work was submitted.

 

The above is also in part why I got into a habit of running with zero cache.

[leadeater]

5 minutes ago, porina said:

On average, it would hurt everyone else equally. For the competition, its about relative positions so wouldn't affect things. Now, if you were to scrape the wingman details you could selectively allow or disallow certain ones and that could swing things, but obviously that would be a very naughty thing to do.

Well I was thinking you could do that and run vbox tasks, then you're hurting all those that are sixtrack only. It's a shit thing to do but it's scarily easy to pull off, just a VM template that auto starts and pulls as many sixtrack tasks as allowed then clone that 100s or 1000s of times. Who cares if the host doing that runs slow as hell, just needs to tie up every sixtrack task it can.

 

I put vbox on my big server since it's easy to do an Windows. I would probably suggest everyone if they can do ATLAS tasks to do them, vbox required, just because of this quorum annoyance. You're just going to end the event with just so much not validated.

[Ithanul]

1 hour ago, leadeater said:

Ok so not sure why I didn't click right away about the quorum thing, thought it was something different but no different to cluster quorums. That's going to make this interesting and very hard, tasks have to be run by two people and compared before it's counted as valid. Seriously I could be a gaint ass to everyone and pull in thousands of jobs on my big server and then suspend them.

Yep....

Which has occured in the Pent before.  Big reason I prefer to vote for quorum 1 projects.  Less chance of someone either holding WUs or someone to hold and then abort WUs (had that happen to me in the past on some projects).

[leadeater]

1 minute ago, Ithanul said:

Yep....

Which has occured in the Pent before.  Big reason I prefer to vote for quorum 1 projects.  Less chance of someone either holding WUs or someone to hold and then abort WUs (had that happen to me in the past on some projects).

I think I might even be doing that unintentionally because I was trying to bunker and set my queue to pull many days worth at once. Since my systems are so resource heavy and the amount I have I get the feeling 20k tasks is a lot. Maybe I should drop that down.