[TUTORIAL] AMD GPU Linux Folding at Home | Arch, Manjaro | Sometimes more perf than Windows!!

[Metallus97]

ONLY FOR AMD GPUs!

This is the first version and I am going to continue to improve it. Suggestions? Write em below!

 

First thanks to @Chunchunmaru_ for contributing to this guide (see his post below)!

So with a bit too much time on my hands lately... I decided to finally try to fix the old Linux folding with an AMD GPU issue!

I ripped my hair out trying to get it to work on Debian. It seems that there only is a official driver form AMD that works on Ubuntu 18.04.1. If you have something else Debian based or even a newer version of Ubuntu (heck even 18.04 but updates to the newest kernel) you will have to mod some kernel stuff, mod the drivers, use custom scripts, downgrade the kernel etc. etc. -> PITA...

Nothing of this works for me and due to running a x570 and a 3900x I dont want to use old kernels. A for security and B for performance reasons (proper Zen2 support was added to a later kernel than what is “supported” for the official driver download. The same is true for other modern processors and platforms from team reed and blue).

 

BUT: If we use a rolling distro like almost everything Arch Linux Based we can do it pretty easily!

Arch: https://www.archlinux.de

This tutorial will be for Manjaro: https://manjaro.org

Verry easy to use OS! You can download it with either XFCE, Gnome or Plasma.

Personally I like Gnome but thats up to you. This is just different desktop designs and window-managers. XFCE however is recommend for old systems due to being very lightweight. The underlying subsystem is the same deal.

 

OK COOL LETZZZZ GOOO! 

 

From here on I will assume that you already have Manjaro running and connected toi the iWeb. If that is not the case there are manny tutorials out there on how to do it.

1. First we have to update everything

Terminal:

sudo pacman -Syu

If asked answer with y = yes. Let it do its thing... and then

reboot

2. Once back at the Desktop:

sudo lshw -c video

This will show you your installed GPUS. on the bottom of the output there will be: configuration: driver=

Now there can be two things:

=Radeon -> Thats the wrong (old) driver, we need to switch that. Continue with next step.
OR

=amdgpu -> Nice! Correct driver, we can continue with step 5.

 

 

3. Now we will change the driver to be the amdgpu one. That is the one with wich F@H is able access OpenCL. OpenCL is used to do processing on the GPU and that’s what OpenMM is running on. OpenMM is sort of the engine that the FAH cores run on. 

This involves a few steps but just follow through. Its not complicated:

 

3.1 MAKE SURE YOU DONT HAVE IMPORTANT STUFF ON THIS INSTALL! IF SOMETHING GOES WRONG IT WONT BOOT! Can be reverted but thats a PITA!

 

3.2 We have to check if the amdgpu drivers are enabled in the kernel 

cat /proc/config.gz | gunzip > running.config
cat running.config | grep CONFIG_DRM_AMDGPU_SI
cat running.config | grep CONFIG_DRM_AMDGPU_CIK

Do you get a xxx=y for the second and third command? -> NICE. The kernel is configured correctly

If not:

-WILL BE ADDED LATER-

 

This is not needed for new gen AMD GPU's! Only for HD7xxx or RX2xx series cards! Newer cards users go to step 5, since they should already have amdgpu drivers!

 

3.3 Now we need to edit the grub config. That is the bootloader. It has to know that we want it to from now on boot with the amdgpu drivers!

sudo nano /etc/default/grub

There should be a line with GRUB_CMDLINE_LINUX_DEFAULT="...SomeOptions..."

Depending on your card you have to add:

Southern Islands (SI): radeon.si_support=0 amdgpu.si_support=1 (https://en.wikipedia.org/wiki/Radeon_HD_7000_series)
Sea Islands (CIK): radeon.cik_support=0 amdgpu.cik_support=1 (as you correctly wrote, https://en.wikipedia.org/wiki/Radeon_Rx_200_series)

Inside the "" and along with whats already there. This will force the system to start with the amdgpu drivers.

Exit nano with ctrl+X, and save the file. DO NOT change the name or anything else!

 

3.4 Now we need to remake grub for it to read in the new instructions:

sudo grub-mkconfig -o /boot/grub/grub.cfg

This can produce warnings, however most of them can be ignored. Especcialy ones about unknown devices.

3.5 

reboot

Sometimes the first boot will fail. Wait a minute, force power of, reboot.

 

4. Once back at the desktop check if you are now using amdgpu drivers instead of radeon

sudo lshw -c video

-> Now everyone on amdgpu? NICEEEH

 

5. Launch software cenenter, go to settings and enable support for AUR and Snap. Simply google if you dont know how to do that .

6. Search for “opencl-amd” and install it via the software center 

7. reboot just to be save

8. Search for "foldingathome" from AUR and install it. There are other AUR packages for it but that one is up to date and works the best IMHO.

9. Follow this to set it up: https://wiki.archlinux.org/index.php/Folding@home

10. Want the advanced control? OK: Install "fahcontrol"

11. Want to monitor your GPU usage? OK: Install "radeontop"

To monitor the GPU go to a terminal:

sudo radeontop

CAUTION: To exit radeontop ALLLLLLWAYS press ctrl+C and DO NOT close the terminal window with radeontop still running. There is a bug so that if you do this one of your threads will be 100% all the time and you will have to manually kill radeontop. Leaving the thread on 100% can tank your PPD.

 

12. F@H should now be running as on any other Linux BUT be able to fully use the AMD GPU!

[MSMSMSM]

I'm buying you a drink.

 

[Metallus97]

2 hours ago, SpicyMustard said:

I'm buying you a drink. 

but we’re not allowed to drink together

[thomas2000]

 

Spoiler

 

Quote

18:59:32:WU01:FS01:0x22:*********************** Log Started 2020-03-21T18:59:31Z ***********************
18:59:32:WU01:FS01:0x22:*************************** Core22 Folding@home Core ***************************
18:59:32:WU01:FS01:0x22:       Type: 0x22
18:59:32:WU01:FS01:0x22:       Core: Core22
18:59:32:WU01:FS01:0x22:    Website: https://foldingathome.org/
18:59:32:WU01:FS01:0x22:  Copyright: (c) 2009-2018 foldingathome.org
18:59:32:WU01:FS01:0x22:     Author: John Chodera <john.chodera@choderalab.org> and Rafal Wiewiora
18:59:32:WU01:FS01:0x22:             <rafal.wiewiora@choderalab.org>
18:59:32:WU01:FS01:0x22:       Args: -dir 01 -suffix 01 -version 705 -lifeline 2811 -checkpoint 15
18:59:32:WU01:FS01:0x22:             -gpu-vendor amd -opencl-platform 0 -opencl-device 0 -gpu 0
18:59:32:WU01:FS01:0x22:     Config: <none>
18:59:32:WU01:FS01:0x22:************************************ Build *************************************
18:59:32:WU01:FS01:0x22:    Version: 0.0.2
18:59:32:WU01:FS01:0x22:       Date: Dec 6 2019
18:59:32:WU01:FS01:0x22:       Time: 21:20:17
18:59:32:WU01:FS01:0x22: Repository: Git
18:59:32:WU01:FS01:0x22:   Revision: f87d92b58abdf7e6bf2e173cfbc4dc3e837c7042
18:59:32:WU01:FS01:0x22:     Branch: core22
18:59:32:WU01:FS01:0x22:   Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
18:59:32:WU01:FS01:0x22:    Options: -std=gnu++98 -O3 -funroll-loops
18:59:32:WU01:FS01:0x22:   Platform: linux2 4.9.87-linuxkit-aufs
18:59:32:WU01:FS01:0x22:       Bits: 64
18:59:32:WU01:FS01:0x22:       Mode: Release
18:59:32:WU01:FS01:0x22:************************************ System ************************************
18:59:32:WU01:FS01:0x22:        CPU: AMD Ryzen 5 2600 Six-Core Processor
18:59:32:WU01:FS01:0x22:     CPU ID: AuthenticAMD Family 23 Model 8 Stepping 2
18:59:32:WU01:FS01:0x22:       CPUs: 12
18:59:32:WU01:FS01:0x22:     Memory: 15.65GiB
18:59:32:WU01:FS01:0x22:Free Memory: 11.10GiB
18:59:32:WU01:FS01:0x22:    Threads: POSIX_THREADS
18:59:32:WU01:FS01:0x22: OS Version: 5.4
18:59:32:WU01:FS01:0x22:Has Battery: false
18:59:32:WU01:FS01:0x22: On Battery: false
18:59:32:WU01:FS01:0x22: UTC Offset: 0
18:59:32:WU01:FS01:0x22:        PID: 2815
18:59:32:WU01:FS01:0x22:        CWD: /home/romus/work
18:59:32:WU01:FS01:0x22:         OS: Linux 5.4.24-1-MANJARO x86_64
18:59:32:WU01:FS01:0x22:    OS Arch: AMD64
18:59:32:WU01:FS01:0x22:********************************************************************************
18:59:32:WU01:FS01:0x22:Project: 11760 (Run 0, Clone 1199, Gen 12)
18:59:32:WU01:FS01:0x22:Unit: 0x0000001480fccb0a5e6d7cd68f3951d8
18:59:32:WU01:FS01:0x22:Reading tar file core.xml
18:59:32:WU01:FS01:0x22:Reading tar file integrator.xml
18:59:32:WU01:FS01:0x22:Reading tar file state.xml
18:59:32:WU01:FS01:0x22:Reading tar file system.xml
18:59:32:WU01:FS01:0x22:Digital signatures verified
18:59:32:WU01:FS01:0x22:Folding@home GPU Core22 Folding@home Core
18:59:32:WU01:FS01:0x22:Version 0.0.2
18:59:41:WU01:FS01:0x22:Completed 0FW528 out of 2000000 steps (0%)
18:59:41:WU01:FS01:0x22:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
18:59:54:WU00:FS00:0xa7:Completed 242500 out of 250000 steps (97%)

Thank you. Just as I broke into Top 3000 of LTT Folding Team

 

From your experience is the GPU folding working better on Linux? Windows drivers get more work put into them, I imagine, so should be more polished. 

[Favebook]

4 hours ago, thomas2000 said:

Thank you. Just as I broke into Top 3000 of LTT Folding Team

 

From your experience is the GPU folding working better on Linux? Windows drivers get more work put into them, I imagine, so should be more polished. 

Please use Spoilers when you are posting long logs or similar things.

As far as I know, Linux gives better estimate PPD.

[Metallus97]

4 hours ago, thomas2000 said:

Thank you. Just as I broke into Top 3000 of LTT Folding Team

 

From your experience is the GPU folding working better on Linux? Windows drivers get more work put into them, I imagine, so should be more polished. 

 

6 minutes ago, Favebook said:

Please use Spoilers when you are posting long logs or similar things.

As far as I know, Linux gives better estimate PPD.

Yep! At least I got an improvement of about 5% on my 290x and am expecting the same and maybe a bit more on my 5700xt. 
The *more work* on the Windows drivers usually goes into game/engine specific optimizations and towards new gaming focused features (chill,boost, etc.).

 

linux just has a better scheduler plus the advantage of not being so bloated as a full windows 10 install. -> runs a bit faster.

But the difference is higher on the CPU side of things.

[thomas2000]

24 minutes ago, Favebook said:

Please use Spoilers when you are posting long logs or similar things.

As far as I know, Linux gives better estimate PPD.

Fixed the log looks, thanks. Ima forum noob.

14 minutes ago, Metallus97 said:

 

Yep! At least I got an improvement of about 5% on my 290x and am expecting the same and maybe a bit more on my 5700xt. 
The *more work* on the Windows drivers usually goes into game/engine specific optimizations and towards new gaming focused features (chill,boost, etc.).

 

linux just has a better scheduler plus the advantage of not being so bloated as a full windows 10 install. -> runs a bit faster.

But the difference is higher on the CPU side of things.

Makes sense, thanks. The bloat on windows is real.

 

[Metallus97]

6 minutes ago, thomas2000 said:

Fixed the log looks, thanks. Ima forum noob.

Makes sense, thanks. The bloat on windows is real.

 

Oh I forgot to mention the CPU usage from the F@H slots:

I am running everything as a KVM VM with the 1c/2t of a 3900x (~4.15GHz). Only the 290X is folding on that VM.

win10: 40-50% CPU load

Manjaro: 5-10 while 5% more PPD

both on Core22 and averages about multiple projects 

[Koeshi]

Any reason that you use opencl-amd from the AUR rather than opencl-mesa from the repo?

[Metallus97]

9 hours ago, Koeshi said:

Any reason that you use opencl-amd from the AUR rather than opencl-mesa from the repo?

YES! Because -mesa is for the radeon drivers. That doesn't  work with F@H.

-amd is simply taken form the official amd drivers and repackaged to work with amdgpu

[Koeshi]

Okay thanks, I'll give that a go.

[Koeshi]

Do I need to redo the config.xml to add the GPU?  I currently have:

 <!-- Folding Slots -->
  <slot id='0' type='CPU'/>

Should that be changed to:

<!-- Folding Slots -->
  <slot id='0' type='CPU'/>

  <slot id='1' type='GPU'/>

[Metallus97]

7 minutes ago, Koeshi said:

Do I need to redo the config.xml to add the GPU?  I currently have:

 

 <!-- Folding Slots -->
  <slot id='0' type='CPU'/>

 

Should that be changed to:

 

<!-- Folding Slots -->
  <slot id='0' type='CPU'/>

  <slot id='1' type='GPU'/>

 

Yes and above that

<gpu v='true'/>

 

[Koeshi]

Where would that go in here:

 

  <!-- Slot Control -->
  <power v='light'/>

  <!-- User Information -->
  <passkey v='****************'/>
  <team v='223518'/>
  <user v='Koeshi'/>

  <!-- Folding Slots -->
  <slot id='0' type='CPU'/>
  <slot id='1' type='GPU'/>

 

[ZachTheDoge]

You are a hero. Fixed my issue just fine.

[Metallus97]

14 minutes ago, Koeshi said:

Where would that go in here:

-snip-

Think that should not matter but idd put it under the <power> line
 

[Koeshi]

Well the GPU is showing up now.  The message on it seems kinda strange ("Paused: waiting for idle") but I guess I will wait a while to see if it gets WUs.

[Favebook]

59 minutes ago, Koeshi said:

Well the GPU is showing up now.  The message on it seems kinda strange ("Paused: waiting for idle") but I guess I will wait a while to see if it gets WUs.

Move your power slider from Low to Full.

[Metallus97]

1 hour ago, Favebook said:

Move your power slider from Low to Full.

will add to FAQ

[Koeshi]

2 hours ago, Favebook said:

Move your power slider from Low to Full.

Yup, seems to be working now after doing that and manually reminding it to fold on that slot.  Think I might stop it after this WU though and look into undervolting the GPU as I just saw the Junction hit 88*C

 

EDIT: definitely will have to, has now hit 92*C, will also see what I can do about my case cooling as I know it is not optimal

[Xerosss]

Thanks for the great guide.

 

I followed all the steps but i am unable to make it work sadly...

I have a AMD 280X so the only difference is that i used "radeon.si_support=0 amdgpu.si_support=1" on the grub part to enable AMDGPU on my system since the GPU is Southern Islands (SI) and i use the last release of manjaro.

 

I get an error BAD_WORK_UNIT when the GPU try to start working on a WU.

 

I will attach the log and a screenshot of the system tab if someone want to take a look and try to help me:

Log.txt

For now i am only using the CPU.

[Metallus97]

25 minutes ago, Xerosss said:

Thanks for the great guide.

 

I followed all the steps but i am unable to make it work sadly...

I have a AMD 280X so the only difference is that i used "radeon.si_support=0 amdgpu.si_support=1" on the grub part to enable AMDGPU on my system since the GPU is Southern Islands (SI) and i use the last release of manjaro.

 

I get an error BAD_WORK_UNIT when the GPU try to start working on a WU.

 

I will attach the log and a screenshot of the system tab if someone want to take a look and try to help me:

Log.txt 4.68 kB · 0 downloads

 

For now i am only using the CPU.

I had the same problem on a 290x (wich is also SI) just enable it with cik, Works 100% for me. But I dont know why

just make sure you don't have anything important on there

[Xerosss]

1 minute ago, Metallus97 said:

I had the same problem on a 290x (wich is also SI) just enable it with cik, Works 100% for me. But I dont know why

just make sure you don't have anything important on there

When using the cik parameters it just stay using radeon drivers instead of AMDGPU one

But that's because the 280x is a rebrand of the HD7970GHz edition so it is clearly a SI card, the 290x is actually an CIK chip at seen in the wikipedia:

https://en.wikipedia.org/wiki/Radeon_Rx_200_series

• A GPU implementing Graphics Core Next 2 (Sea Islands) is found on R7 260 (Bonaire), R7 260X (Bonaire XTX), R9 290 (Hawaii Pro), R9 290X (Hawaii XT), and R9 295X2 (Vesuvius) branded products.
• A GPU implementing Graphics Core Next 1 (Southern Islands) is found on R9 270, 270X, 280, 280X, R7 240, 250, 250X, 265, and R5 240 branded products.

that generation was a bit weird in that aspect

[CWP]

I finally got my HD 7750 (GCN1) to fold in Manjaro:

• Force the use of amdgpu module. I blacklisted radeon in my setup, but both Metallus and Xerosss already provided some information on how to get that working using grub / kernel boot parameters, depending on either SI- or CIK-based GPUs.
• As Metallus said, no opencl-mesa. Uninstall it to prevent the folding client from picking it up. Otherwise, you will get "Bad platformId size" error.
• With my particular card, I ran into the same issue as @Xerosss: "clCreateCommandQueue (-6)". I was able to fix this with the information from the AMD's forum and put into the form of an add-on file by adding the following to /etc/systemd/system/foldingathome.service.d/10-envfix.conf, save, then tell systemd to load the file by running systemctl daemon-reload:

[Service]
Environment=GPU_FORCE_64BIT_PTR=1

Please note that I do not know what it actually does, or how much it will affect PPD, so I would suggest only do this if you encounter that error message (clCreateCommandQueue) in the logs. You can easily test whether you need it by using clpeak package or download FAHBench without sacrificing innocent work units. These programs will error out immediately, and can then be fixed by prefacing the command with the environment variable (leave the "export" part out): GPU_FORCE_64BIT_PTR=1 clpeak

 

EDIT 2021-09-14: opencl-amd in AUR has dropped support for Polaris-based cards. There is a comment referring to using opencl-amd-polaris, but that seems to be missing in AUR. You may have to manually downgrade opencl-amd to 20.45 to retain support for these cards.

[Chunchunmaru_]

A bit of an advice of what you are doing here for the 3.2 part 

 

You are enabling AMDGPU support for legacy GPU's who primarly use the radeon driver, the amdgpu driver is builtin at least from the last 3 years on the major distros and the only AMD GPU's that will use it are Southern Islands (SI) GPU's and Sea Islands (CIK) GPU's, and what you wrote was enabling support ONLY for CIK gpu's, you actually have to put this

Southern Islands (SI): radeon.si_support=0 amdgpu.si_support=1 (https://en.wikipedia.org/wiki/Radeon_HD_7000_series)
Sea Islands (CIK): radeon.cik_support=0 amdgpu.cik_support=1 (as you correctly wrote, https://en.wikipedia.org/wiki/Radeon_Rx_200_series)

 

This is not needed for new gen AMD GPU's