BOINC Pentathlon 2019

[GOTSpectrum]

1 minute ago, Windows7ge said:

Only problem for me being each server did just a couple jobs as you described but then just stopped receiving work.

Things worked during different OS testing but now only 2/16 VMs have jobs. The rest, sitting idle.

Have your dumped WUs during testing? 

[leadeater]

4 minutes ago, Windows7ge said:

Only problem for me being each server did just a couple jobs as you described but then just stopped receiving work.

Things worked during different OS testing but now only 2/16 VMs have jobs. The rest, sitting idle.

What does the logs say? ZIKA has petty good log information compared to the other projects. From memory I had to increase the timer for checking for jobs etc in the BOINC client, ZIKA puts you on a timeout if you check in too frequently or too soon after another one.

[Windows7ge]

1 minute ago, Ben Quigley said:

Have your dumped WUs during testing? 

Nope. I made sure to let all those VMs finish their work before deleting the VM to prevent the issue leadeater & you ran into.

I set every VM to request 0.3+0.2 days of work. Lets see if it helps.

 

4.5/16 VM's have jobs (0.5 because one of them has a whole 4 WU's. I think that's WCG's way of testing the client before sending more work.)

 

6 minutes ago, leadeater said:

What does the logs say? ZIKA has petty good log information compared to the other projects. From memory I had to increase the timer for checking for jobs etc in the BOINC client, ZIKA puts you on a timeout if you check in too frequently or too soon after another one.

Don't think the data here is very helpful in this instance.

Quote

Requesting new tasks for CPU

Scheduler request completed: got 0 new tasks

Not sending work - last request too recent: 30 sec

Previously I have let this go over-night and not received work so I have no idea what the actual problem is.

[Ithanul]

31 minutes ago, Windows7ge said:

Only problem for me being each server did just a couple jobs as you described but then just stopped receiving work.

Things worked during different OS testing but now only 2/16 VMs have jobs. The rest, sitting idle.

Did the event log show anything?

That will sometimes state what the project servers are telling it for why new WUs are not given.

[leadeater]

15 minutes ago, Windows7ge said:

Not sending work - last request too recent: 30 sec

Yea you're on a timeout, asking for jobs too often.

 

Edit:

I think the default for this is 60 seconds? Try increasing this and have it staggered across your VMs so they don't do it at the same time.

[Windows7ge]

29 minutes ago, leadeater said:

Yea you're on a timeout, asking for jobs too often.

 

Edit:

I think the default for this is 60 seconds? Try increasing this and have it staggered across your VMs so they don't do it at the same time.

I staggered them 120/180 alternating. I can make it more complicated if this doesn't work like 120/180/240/300/etc. Each job takes >1hr so I wouldn't run out doing it that way if necessary.

 

7/16 now have jobs.

[Windows7ge]

I think we made good progress. All but 1 are now complaining that there's just no work to be done. So, I just added the optional projects on the WCG website. I'll give it a while and see if they all get work.

 

Unrelated, my desktop is continuing to die a slow painful death :D I can no longer boot into Windows for more than ~30mins without the system behaving very irregularly and then quickly become non-responsive followed by a BSOD. I don't have the money to rebuild it right now so I split the volume and installed Ubuntu 19.04. Why is it that Linux seems to handle unstable systems so much better? I've only had one hiccup so far today (on Linux) and it wasn't a complete crash.

[Gorgon]

On 5/27/2019 at 12:55 PM, leadeater said:

Yea you're on a timeout, asking for jobs too often.

 

Edit:

I think the default for this is 60 seconds? Try increasing this and have it staggered across your VMs so they don't do it at the same time.

Is staggering the checkpointing a performance tip or did I misread the description is it somehow related to requesting tasks?

[leadeater]

1 hour ago, Gorgon said:

Is staggering the checkpointing a performance tip or did I misread the description is it somehow related to requesting tasks?

No idea, it just seemed to help out with WCG ZIKA as that project is pretty strict about how often you ask for jobs

[GOTSpectrum]

On 5/25/2019 at 9:07 AM, RollinLower said:

any updates on this?

GUYS!

 

Colonel mortis has just gotten back to me.

 

Thanks to my message they will be looking for new community leaders for the distributed computing teams sometime in the next few days. They will be posting in the subforum and asking for your guys input on who should be appointed.

 

So I would like to announce my campaign... Jk jk.

 

But make sure to keep your eye out for the post.

 

Happy crunching,

 

Ben.

[Ithanul]

Well, not setup testing yet on my 2nd rig (1900X).  But, getting some numbers on my 1950X that runs under W10 (build 14393) which is the main rig.

 

NFS sure loves to eat up my RAM.

[Guest]

23 hours ago, Ben Quigley said:

GUYS!

 

Colonel mortis has just gotten back to me.

 

Thanks to my message they will be looking for new community leaders for the distributed computing teams sometime in the next few days. They will be posting in the subforum and asking for your guys input on who should be appointed.

 

So I would like to announce my campaign... Jk jk.

 

But make sure to keep your eye out for the post.

 

Happy crunching,

 

Ben.

He should pick by F@H points.

 

[SansVarnic]

I nominate @Ben Quigley

 

If by points, im in 20 something place.... so no for me.

[GOTSpectrum]

2 hours ago, SansVarnic said:

I nominate @Ben Quigley

 

If by points, im in 20 something place.... so no for me.

The nomination is appreciated.

 

Only for the folding opening though.

[Guest]

2 hours ago, SansVarnic said:

I nominate @Ben Quigley

 

If by points, im in 20 something place.... so no for me.

+1

Spoiler

(I was joking about the by points thing since Whaler retired so technically I'm the most points)

 

[RollinLower]

12 hours ago, Ben Quigley said:

The nomination is appreciated.

 

Only for the folding opening though.

you seem like the obvious choice, since you're basically at the front of every distributed computing event here.

[GOTSpectrum]

Guys!!!

 

Summer folding sprint is now ready for you to sign up!!!

 

[Ithanul]

On 5/31/2019 at 2:53 PM, Ben Quigley said:

The nomination is appreciated.

 

Only for the folding opening though.

Yeah, +1 for you on the F@H side.

[Gorgon]

Has the team done a Lessons Learnt exercise to close out the event?

 

I think it's important we captured what worked and what didn't while the knowledge is still fresh for next year.

 

Though I jumped in late my biggest challenge was getting the accounts set up and co ordinated with BOINCstats. Next I would say was getting the profiles working properly to not kill my daily driver and to balance resources across projects.

[Windows7ge]

I decided to do some more testing with virtual machines for bunkering purposes and it seems that there is some substantial overhead caused by Hyper-V/PROXMOX virtualization because after running BOINC natively for a few days my Project average work done skyrocketed from a little under 5,000 to just over 10,000. Then on the BOINCstats website my daily productivity stats went from 4809/day to 12,500~14,500/day so virtualization cuts performance in more than half.

 

I did find a workaround. On Linux anyways. The use of Containers. These containers run instances of an OS off the host kernel itself effectively operating natively. Not in a VM. The downside is that these containers only support CLI but BOINC has a nice feature that let me work around this and it's something leadeater might like to implement next time around.

 

For as much research as I did I could not find any data indicating that it'd be possible to run Linux containers on Windows Server. Docker didn't mention it, Hyper-V process isolation didn't mention it, I could not find it anywhere. If anybody knows a way to pull this off let me know.

 

For the process below I used a Debian distro.

 

So first there's the matter of installing BOINC:

apt-get install boinc-client

Start the service:

etc/init.d/boinc-client start

BOINC will automatically start each time you restart the container/server

 

So a feature I found that BOINC supports is remote control of a BOINC instance on a different system. (File -> New BOINC Manager window...). This feature cannot be used unless you edit the remote_hosts.cfg file on the server with any of these:

vi /var/lib/boinc/remote_hosts.cfg
nano /var/lib/boinc/remote_hosts.cfg
echo "IP of Host Controller" >> /var/lib/boinc/remote_hosts.cfg

I should note that everything is going to need a static IP for this to work. Also the BOINC controller has to be running the same/similar OS as the CLI servers. You can't have BOINC manager running on Windows with Linux CLI BOINC nodes.

 

Setting static IP's on the CLI Nodes can be done by editing your network file. They vary dependent on your specific distro but you should be able to find and edit it with either of:

vi /etc/netplan/*.yaml
nano /etc/netplan/*.yaml

Here is an example of how the file should look when you're done editing:

network:
        version: 2
        renderer: networkd
        ethernets:
                enp0s3:
                    dhcp4: no
                    addresses: [192.168.0.250/24]
                    gateway4: 192.168.0.1
                    nameservers:
                       addresses: [1.1.1.1,8.8.8.8]

Save/exit.

Now test the file to make sure you did it correctly:

netplan try

If the file tests good it will ask if you want to apply it. Hit enter.

 

Now when you remote in from the BOINC GUI computer (or VM) they will always have the same IP. The BOINC controller/GUI needs to have a static IP too as it's set in the remote_hosts.cfg files on each BOINC Node.

 

So now you can control a full stack of BOINC containers/VMs/or physical servers from one BOINC GUI. I found using containers on Linux yielded greater performance than using VMs on Linux so it does appear to be particularly beneficial and theoretically it should have almost if not identical to native performance. I think this would be the best option for bunkering. Forget VMs if you can install something like PROXMOX on your server/desktop during the next Pentathlon.