Repurposing My Old Desktops Into BOINC Crunching Rigs

[Windows7ge]

Gonna have to split this next part up into two posts because of the picture upload limit:

 

To my surprise I did a pretty good job of keeping track of the X79 motherboards accessories even after going on 7 years now. Still have that I/O shield (left):

 

 

Installing both I/O shields:

 

 

 

Next up is mounting the custom SSD brackets:

 

 

The SSDs here are random low capacity SSDs I had on hand. I did oriigiannly plan on booting from USB thumb drives but for reasons unknown though the X79 motherboard will boot installers from USB I could not get it to boot an installed OS from USB so we're just going to use a ADATA 64GB and a SAMSUMG 120GB.

 

Mounting them to their brackets the measurements I made aren't perfect and the cutout around the connectors really wasn't necessary (I thought it'd look cool) but they fit where they count because of the online diagram I followed for the mounting hole locations:

 

[Windows7ge]

Mounting the two SSD's:

 

 

 

I think it's a good location for it. Now to install the fans. I have a large box full of ramdom fans so I didn't want to go out and spend extra on this. The X79 rig got a Cooler Master fan that came off a Hyper 212 EVO and then 2 random fans that move enough air but I don't recall where they came from. Probably an old server chassis:

 

 

As it turns out my SSD mount location wasn't the best as the radiator cap on the H100i is touching the SSD. It lines up fine but it is making contact (not visible in picture):

 

 

On the right the same 2 mounting holes exist. I am a little worried about how hot this radd is going to get and how that may impact the life expectancy of the SSD. I may move the SSD over to the other side. Cables should reach. I think we'll check temps when we're up and running and go from there.

 

That's all for tonight. More will come tomorrow.

[Windows7ge]

Got the motherboards installed:

 

 

 

I had already assembled them beforehand. Protect the socket pins & all. Installing the RAM before-hand just makes things easier too.

 

I happen to have six of these silicone RAM slot covers from an old project. That's not enough for both motherboards but for the heck of it we can give them to the X99 board:

 

[Windows7ge]

Next up are the CPU coolers. Now I would have liked to have gone with the ever popular Cooler Master Hyper 212 EVO. Super cheap, performs very well for a low cost air cooler...but...:

 

 

You might be able to tell just from that picture alone...

 

 

She's too tall for a 4U. Which is really unfortunate because it's not too tall by a whole heck of a lot. Had to use something else. As a replacement I went for the Noctua NH-D9L:

 

 

This is a 3U compatible cooler and it uses the Noctua A9 92mm fan of which I have an extra to slap on there. I hope to overclock the 3930K when a more adequate PSU becomes available at a lower than current cost.

 

For TIM I'm using this:

 

 

This is a generic brand graphite sheet similar to IC Graphites pre-cut squares except this comes in big sheets you can cut to whatever size you want at a fraction of the cost for ~the same performance. They come from Mouser Electronics. The 3930K will be fine but I plan to O.C. the 5960X fairly aggressively. We'll see if it keeps up.

 

Now to plop the coolers on:

 

 

 

Now I tried to orient the Noctua cooler pulling/pushing air over the RAM but trying to mount the 2nd A9 fan it interfered with the RAM on both sides so I guess the VRMs are going to get a little extra incidental airflow.

 

Tomorrow we start cable management and plugging things in.

 

Also on the list is the GPU situation. This is going to be a little odd. I only have 1 spare GPU right now. Now this won't actually be a problem. The X79 board will not POST w/out a GPU, but the X99 board will. It doesn't care if it doesn't detect one it'll load the OS which is fine because I only plan to SSH in after the initial installation. They're just going to have to share it during setup.

Good night for now!

 

[Windows7ge]

I will admit the cable management could look better had I used velcro or zip ties but server chassis don't have a "behind the motherboard tray" and for our purposes we removed the 3.5" & 5.25" bays. Realistically the only important thing here is that the cables not impede airflow (not that there's anywhere near enough cables to do that).

 

For the 3930K rig this is what I came up with:

 

 

The 2 rear fans are piped right into 12V. We'll see how the motherboard does powering the front 3 fans. I might end up daisy-chaining some fan splitters off the PSU if I'm not satisfied. For this application we don't care about about noise. It's about maximum airflow.

 

This is how I did on the 5960X rig:

 

 

Now you'll notice there's a lot of PCI_e power. As I said, I don't have a GPU to give this server. It has to share the broken MSI Radeon HD 7770 that's in the 3930K rig right now. However, in the not too distant future this system will get a pair of Sapphire Tri-X R9 290Xs for F@H and/or BOINC. That will be an update I post when it's ready.

 

Provided I haven't forgotten anything, hardware wise the only other thing to address is power/reset. I'll work on that tomorrow.

[Windows7ge]

Pulling the chassis original PWR/RST switch off the front bezel it's all neatly mounted on one PCB along with some USB and LED's:

 

 

I'm prototyping a bracket for it that will possibly allow me to mount it to the mesh vent on the rear of the server:

 

 

We will see when it's done in a couple hours.

[Windows7ge]

The goofy brackets are done:

 

 

I measured something wrong. The good news is the holes in the chassis are so wide it didn't matter.

 

Here it is rigged up:

 

 

Could I have found a better way to route the wires? Probably, but I didn't want to mount it to the front and ruin my honeycomb aesthetic so this will do.

 

A shot from the other side:

 

 

It's not going anywhere. How I plan to mount these chassis up the buttons are unlikely to get pressed by accident.

 

Some very basic cable management:

 

 

That ziptie isn't set super tight. Just enough to hold the bundle together.

 

That'll be it for tonight. Tomorrow I'll do the same to the X79 rig.

[Windows7ge]

"Front" panel connectors are now setup on the X79 rig:

 

 

Only thing left hardware wise is buttoning up the lid and setting all the fans to Turbo mode.

 

One issue with the 3930K rig is it has one persistently flaky RAM channel. It's not the stick. It's either the slot or the Memory Controller in the CPU. There's also another channel that at times decides to stop working but I can give getting it going a shot.

 

WE WANT QUAD CHANNEL 32GB WORKING!

 

[Windows7ge]

So as I said, we have some problems with a couple slots:

 

 

I'm going to try to fix this and I'll see where I get.

 

Next up is just making sure everything is set to maximum fan speed:

 

CPU looks fine for the room temp and considering I'm not using thermal compound:

 

 

And everything is set to max:

 

 

 

 

Will update with the results of troubleshooting the Abnormal modules.

[Windows7ge]

And I got it going.

 

All DIMMs report:

 

 

All 32GB:

 

 

I enabled X.M.P. and got it running at factory spec. CPU won't be overclocked for now but it can still turbo up to 3.8GHz.

 

From this point on unless something goes wrong with booting the 5960X rig this is basically the end of the hardware setup for these rigs. From here I'll be focusing on the software setup. If that interests any of you you can stick around.

 

For the OS both of them will be running Ubuntu Server 18.04.4 LTS and for OpenCL compute I'll be installing the Ubuntu 18.04.4 HWE driver. A more direct route to get OpenCL support for GPU compute would be the open source ROCm project but I never got this working myself. I have gotten it working through the official AMD driver though so we will use that here.

 

First things first, I need to create a bootable thumb drive with the OS installer. I plan to use DD for this.

[Windows7ge]

The OS is now installed on both machines and was successful without error (a rarity).

 

Right now before I button up the boxes and finish the setup on both of them I'd like to verify or at least attempt setting up OpenCL.

 

Everything is handled via CLI so the plan is to just SSH in from my desktop as I've done here:

 

 

From here I first want to update all of my packages. I can do this by running the command:

sudo apt update && sudo apt upgrade

Then putting in my root password. It will check the available repos for available packages and if any of the ones I have have a new release available or if any repos have new packages I can download. It will then download and upgrade those packages. From here all I have to do is hit Y then Enter:

 

 

Although it's not explicitly necessary I like to do a reboot after getting all the packages up to date.

 

From here I already downloaded the AMD GPU driver to my desktop. I can copy it over to the server via SFTP using FileZilla:

 

 

From here I can view the driver folder in the directory I dropped it in with:

ls -l /home/boinc

 

From here before I install the driver I want to setup a couple applications:

• Htop
  • A CPU/Mem/Process management tool. Lets you view your CPU/RAM usage and kill troublesome tasks.
• lm-sensors
  • Allows for the monitoring of component temps, fan RPMs, and voltages across the system.
• boinc-client
  • This is the backend of the BOINC software. On a normal desktop this would be jointed with boinc-manager but we're going to set this up so we can manage it over the network.

All of these can be installed at once with:

sudo apt install htop lm-sensors boinc-client

 

Now from here I want to enable remote management of boinc-client so I can use the GUI utility. This can be done pretty easily by adding an entry to the var/lib/boinc-client/remote_hosts.cfg file with:

echo '192.168.0.240' >> /var/lib/boinc-client/remote_hosts.cfg

Unfortunately this requires that you be root but you can acquire that with:

sudo -i

Now from the remote client with an IP ending in .240 it can connect to this boinc-client instance and manage what the server does remotely with a GUI:

 

 

 

From here we want to look at the Event Log... and notice where it says "No usable GPUs found". This is because there's no OpenCL driver installed:

 

 

Now I will admit. I don't know if this is going to work with this exact updated version of Ubuntu Server, AMD driver, and the basic GPU in the server right now.

 

I will let you guys know when/if it works. Installing the driver is pretty simple. Just enter the driver directory and run:

./amdgpu-install --opencl=pal,legacy --headless

I'll keep you posted.

[Windows7ge]

 

Well that worked without issue. :3 Installed the driver, rebooted, and it showed right up.

 

I really expected it to give me some sort of problem.

[Windows7ge]

The systems are not currently setup in their final resting places but I have them somewhere where we can get some performance metrics and maybe do some tweaking.

 

Right now both CPUs are resting around 36~38°C:

 

 

 

Right now I should mention that I overclocked the 5960X to 4.5GHz with a 1.325V Vcore. We will see how stable it is and we will see how hot it gets.

 

I've just gotten both of them connected to my BOINC project of choice World Community Grid and started running the sub-project OpenPandemics for COVID-19.

 

I set the project to run on 10/12 of it's threads. My reason for using Linux or GNU/Linux over Windows is because there is a tangle performance benefit in this type of compute application. That and there is considerably less overhead. Ubuntu Server will run off 1C/2T fine while pegging the rest without choking on itself.

 

It's boosting up to 3.8GHz on all threads. I'm impressed:

 

 

Ignore the CPU max MHz: 5700.0000. I have no idea why it's saying that. Though I wish I could overclock it that far.

 

The 5960X is boosting to 4.5GHz as I O.C.d it to in the BIOS:

 

 

Now that things have had a chance to cook for a while temps are looking...about what I expected. The 3930K is looking fine'n'dandy at a cool 69°C:

 

 

The 5960X. Yep, I've hit the roof on how far I can O.C. it for the cooling it has:

 

 

But remember both of them are using that graphite sheet tech. There's no thermal paste on either CPU (unless you count under the IHS) which I think is really cool.

 

Also the ambient is pretty toasty. I don't have an exact number but the weathers been really hot so I'm glad to see things aren't >80°C.

 

For the time being I'd have to say that's about it for these two boxes. I will be upgrading the PSU for the 3930K eventually and the 5960X will be getting a pair of Sapphire Tri-X R9 290X's for GPU compute BOINC/F@H.

 

Until then that's the end. Thanks for following along.