BOINC Pentathlon 2019

[leadeater]

1 minute ago, Ben Quigley said:

Yeah, durp moment, it is 3:36am after all. 

 

Have you dumped any WUs? That would make them limit you, or they simply don't have the WUs to hand out.

......... argh yea lots of sixtrack I was never going to run

[leadeater]

@Ben Quigley

New account, no difference.

[GOTSpectrum]

3 minutes ago, leadeater said:

......... argh yea lots of sixtrack I was never going to run

That could explain it, this happened to me when I accidentally dumped a bunch of Einstein, I just made an alt account. Typically it's 12-24 hours after the first update following the dump.

 

Just now, leadeater said:

@Ben Quigley

New account, no difference.

That lends to my theory of them not having the WUs to serve, or, they have some kind of IP/MAC blocking thing

[leadeater]

4 minutes ago, Ben Quigley said:

That could explain it, this happened to me when I accidentally dumped a bunch of Einstein, I just made an alt account. Typically it's 12-24 hours after the first update following the dump.

I was trying to be nice to everyone else because I had many thousands suspended across the servers so dumped all the ones that hadn't started, probably killed off 4k-6k. Figured it was bad for myself because I still had over 1k pending validation sixtrack tasks and I have no idea what my own ones were dependent on for Q2 validation, better to just release what I was never going to do.

[Windows7ge]

Finally got off work. 10:45PM right now. I got LHC@home started. Don't know why leadeater's not getting jobs because it downloaded 13 WU's for me.

Unfortunately VB only lets you use up to 50% of your CPU so on a 16 thread system I can only really put in 8.

[leadeater]

3 minutes ago, Windows7ge said:

Unfortunately VB only lets you use up to 50% of your CPU so on a 16 thread system I can only really put in 8.

Odd mine lets me run up to as many cores, not threads, as I have.

[leadeater]

Also FYI ATLAS doesn't really hit the cpu utilization hard at all but the tasks are worth a lot each, between 500 to 6000.

[GOTSpectrum]

11 minutes ago, Windows7ge said:

Finally got off work. 10:45PM right now. I got LHC@home started. Don't know why leadeater's not getting jobs because it downloaded 13 WU's for me.

Unfortunately VB only lets you use up to 50% of your CPU so on a 16 thread system I can only really put in 8.

If you got the 32bit client it will only let you just up to 8 cores afaik

[Windows7ge]

Just now, leadeater said:

Odd mine lets me run up to as many cores, not threads, as I have.

...same difference, I think. It's a 5960X so I guess VB doesn't count threads as usable "cores". That's unfortunate. My CPU is only using 10~13%. If VB just shared CPU time like ESXi or PROXMOX instead of locking cores to each VM then I would be able to do a lot more work right now. My system could easily run 6~7 of these "8 CPU" WU's at a time.

 

Agh, LHC why u do dis!?! Is so inefficient.

[leadeater]

1 minute ago, Windows7ge said:

...same difference

Oh right you said threads, was thinking it was one of your better servers with actually that many cores 

[leadeater]

10 minutes ago, Windows7ge said:

If VB just shared CPU time like ESXi or PROXMOX instead of locking cores to each VM then I would be able to do a lot more work right now. My system could easily run 6~7 of these "8 CPU" WU's at a time.

I thought vbox can core share but it's just boinc cpu scheduler not starting more tasks than cores you have? Seems like it's that to me.

 

Edit:

But on that topic, this seems like a clear case of VMs giving a huge advantage to ATLAS. With vbox nested inside.

[Windows7ge]

1 minute ago, leadeater said:

Oh right you said threads, was thinking it was one of your better servers with actually that many cores 

oof, shots fired.

 

No, I was hoping to keep my servers on WCG so I threw my unpredictable 5960X desktop on LHC:

Old server has 16 physical cores (2x 2670's)

New server has 32 physical cores (2x 2698v3's) - Speaking of buying those CPU's off some stranger from LA, California. They've been under 100% load for two weeks now with absolutely 0 issues. 10/10 may buy from that stranger again.

 

4 minutes ago, leadeater said:

I thought vbox can core share but it's just boinc cpu scheduler not starting more tasks than cores you have? Seems like it's that to me.

Years ago having a go playing with VB I can say for a fact it would only let you simultaneously run as many VMs as you have physical cores to distribute. Maybe this many years later that has changed. I don't know, but I kind of doubt it. It's a nice program for quick little shit testing but for any real productivity or VM storage type applications it's not ideal by any stretch of the imagination. Too much overhead.

 

14 minutes ago, Ben Quigley said:

If you got the 32bit client it will only let you just up to 8 cores afaik

Don't think so:

Spoiler

I wen't though an issue with x32 already. It let's you use as many cores/threads as 1 physical CPU has. If you have more than 1 CPU it ignores the 2nd, 3rd, etc CPU.

x64 does see more than 1 physical processor.

 

This doesn't explicitly say x64 but that's the download I picked.

 

[leadeater]

1 minute ago, Windows7ge said:

Years ago having a go playing with VB I can say for a fact it would only let you simultaneously run as many VMs as you have physical cores to distribute. Maybe this many years later that has changed. I don't know, but I kind of doubt it. It's a nice program for quick little shit testing but for any real productivity or VM storage type applications it's not ideal by any stretch of the imagination. Too much overhead.

Host -> VM -> BOINC -> vbox

         -> VM -> BOINC -> vbox

         -> VM -> BOINC -> vbox

         -> VM -> BOINC -> vbox

         -> VM -> BOINC -> vbox

         -> VM -> BOINC -> vbox

         -> VM -> BOINC -> vbox

 

Always a way 

[Windows7ge]

1 minute ago, leadeater said:

Host -> VM -> BOINC -> vbox

         -> VM -> BOINC -> vbox

         -> VM -> BOINC -> vbox

         -> VM -> BOINC -> vbox

         -> VM -> BOINC -> vbox

         -> VM -> BOINC -> vbox

         -> VM -> BOINC -> vbox

 

Always a way 

Actually no Oracle VM Virtualbox knows if it's running inside a VM and will refuse to work. That's the case at least of the Host -> VM is in itself Oracle VM Virtualbox. Now if I used HyperV or PROXMOX the results may be different. In either instance I would expect the performance to be terrible. Don't really know if it'd be worthwhile until you try it though I suppose.

[leadeater]

1 minute ago, Windows7ge said:

Actually no Oracle VM Virtualbox knows if it's running inside a VM and will refuse to work. That's the case at least of the Host -> VM is in itself Oracle VM Virtualbox. Now if I used HyperV or PROXMOX the results may be different. In either instance I would expect the performance to be terrible. Don't really know if it'd be worthwhile until you try it though I suppose.

You have to expose VT-x to the VM for it to work, most hypervisors don't by default.

[leadeater]

@Windows7ge OMG I think you just cracked why I'm not getting ATLAS tasks on those servers, they are VMs!. Going to try exposing VT-x and see if that works.

[Windows7ge]

5 minutes ago, leadeater said:

You have to expose VT-x to the VM for it to work, most hypervisors don't by default.

Hmn...

I think nesting VB would be terrible idea. The diminishing returns would be tremendous...but it's not like I have anything better to do maybe I'll try it and see how it goes. I get the feeling this experiment won't get very far in it's testing before it fails.

 

4 minutes ago, leadeater said:

@Windows7ge OMG I think you just cracked why I'm not getting ATLAS tasks on those servers, they are VMs!. Going to try exposing VT-x and see if that works.

You mean there's the possibility I helped by not doing anything at all? That's the best kind of helping!

[leadeater]

Just now, Windows7ge said:

You mean there's the possibility I helped by not doing anything at all? That's the best kind of helping!

Yep, well I mean credit in giving out the required information that sparks that train of thought.

 

Also FML LHC just went down, oh the timing. *shakes fist at sky*

[leadeater]

3 minutes ago, Windows7ge said:

I think nesting VB would be terrible idea. The diminishing returns would be tremendous...but it's not like I have anything better to do maybe I'll try it and see how it goes. I get the feeling this experiment won't get very far in it's testing before it fails.

Oh nah I was meaning ESXi to host VM with boinc/vbox inside it, which I'm doing on 5 now anyway. When LHC comes back online I'll be able to say if enabling VT-x actually fixes it.

 

Edit:

Or even Hyper-V, if it can expose VT-x?

[leadeater]

@Windows7ge

*High five* Works, getting ATLAS tasks now.

[Windows7ge]

8 minutes ago, leadeater said:

Edit:

Or even Hyper-V, if it can expose VT-x?

Doing some Googling it seems it is doable but not in the traditional sense of just hitting a tickbox that says "VT-x". You have to use a combination of powershell trickery and using specific versions of windows that support the "pass-through" of the VT-x function to a nested guest VM. Weird stuff. I might be interested in trying to see what would happen if I installed BOINC + VB on Linux in a Hyper-V VM and see if it will run or just spit out a bunch of errors.

 

If LHC gets picked next year it may be good information to find out if it can be run in a PROXMOX/ESXi/Hyper-V VM relatively easily. It'd allow for much more efficient bunkering.

[leadeater]

2 minutes ago, Windows7ge said:

It'd allow for much more efficient bunkering.

Even just running them overall, with these using such little CPU I would probably run up at least 10 VMs on a larger sized server, 16-31 cores.

[Windows7ge]

1 minute ago, leadeater said:

Even just running them overall, with these using such little CPU I would probably run up at least 10 VMs on a larger sized server, 16-31 cores.

It seems the moreover issue of running a VM within Hyper-V is hardware virtualization (I take it that's what VT-x does). This needs to be enabled in order for it to run a guest VM within the VM.

 

These powershell commands are apparently necessary for Hyper-V on the host (because the GUI doesn't have the option):

Get-VMProcessor -VMName vm_name | fl * // Tells you if hardware virtualization is enabled for the given VM.
Set-VMProcessor -VMName vm_name -ExposeVirtualizationExtensions $true // Enables hardware virtualization for the given VM.

After that the tutorial starts going into MAC address spoofing, traversing NAT. In theory if I bridged the Hyper-V VMs with my local network none of those instructions should be necessary.

[leadeater]

1 minute ago, Windows7ge said:

After that the tutorial starts going into MAC address spoofing, traversing NAT. In theory if I bridged the Hyper-V VMs with my local network none of those instructions should be necessary.

Yea screw that, ESXi is just a tick box

 

[Windows7ge]

27 minutes ago, leadeater said:

@Windows7ge

*High five* Works, getting ATLAS tasks now.

*Misses it because it's 12:30AM and I've been up all day*

Glad I could be of service. I look forward to unintentionally helping you again.