Jump to content

Linux Mint - Single GPU Passthrough setup, libvirtd.service is broken

karambyt
By karambyt
in Troubleshooting
 Share
Posted
  • Author

Okay, so I added those parts to the scripts, see below:

from vfio_startup.sh
  

#!/bin/bash

#############################################################################
##     ______  _                _  _______         _                 _     ##
##    (_____ \(_)              | |(_______)       | |               | |    ##
##     _____) )_  _   _  _____ | | _    _   _   _ | |__   _____   __| |    ##
##    |  ____/| |( \ / )| ___ || || |  | | | | | ||  _ \ | ___ | / _  |    ##
##    | |     | | ) X ( | ____|| || |__| | | |_| || |_) )| ____|( (_| |    ##
##    |_|     |_|(_/ \_)|_____) \_)\______)|____/ |____/ |_____) \____|    ##
##                                                                         ##
#############################################################################
###################### Credits ###################### ### Update PCI ID'S ###
## Lily (PixelQubed) for editing the scripts       ## ##                   ##
## RisingPrisum for providing the original scripts ## ##   update-pciids   ##
## Void for testing and helping out in general     ## ##                   ##
## .Chris. for testing and helping out in general  ## ## Run this command  ##
## WORMS for helping out with testing              ## ## if you dont have  ##
##################################################### ## names in you're   ##
## The VFIO community for using the scripts and    ## ## lspci feedback    ##
## testing them for us!                            ## ## in your terminal  ##
##################################################### #######################

################################# Variables #################################

## Adds current time to var for use in echo for a cleaner log and script ##
DATE=$(date +"%m/%d/%Y %R:%S :")

## Sets dispmgr var as null ##
DISPMGR="null"

################################## Script ###################################

echo "$DATE Beginning of Startup!"


function stop_display_manager_if_running {
    ## Get display manager on systemd based distros ##
    if [[ -x /run/systemd/system ]] && echo "$DATE Distro is using Systemd"; then
        DISPMGR="$(grep 'ExecStart=' /etc/systemd/system/display-manager.service | awk -F'/' '{print $(NF-0)}')"
        echo "$DATE Display Manager = $DISPMGR"

        ## Stop display manager using systemd ##
        if systemctl is-active --quiet "$DISPMGR.service"; then
            grep -qsF "$DISPMGR" "/tmp/vfio-store-display-manager" || echo "$DISPMGR" >/tmp/vfio-store-display-manager
            systemctl stop "$DISPMGR.service"
            systemctl isolate multi-user.target
        fi

        while systemctl is-active --quiet "$DISPMGR.service"; do
            sleep "1"
        done

        return

    fi

}

function kde-clause {

    echo "$DATE Display Manager = display-manager"

    ## Stop display manager using systemd ##
    if systemctl is-active --quiet "display-manager.service"; then
    
        grep -qsF "display-manager" "/tmp/vfio-store-display-manager"  || echo "display-manager" >/tmp/vfio-store-display-manager
        systemctl stop "display-manager.service"
    fi

        while systemctl is-active --quiet "display-manager.service"; do
                sleep 2
        done

    return

}

####################################################################################################################
## Checks to see if your running KDE. If not it will run the function to collect your display manager.            ##
## Have to specify the display manager because kde is weird and uses display-manager even though it returns sddm. ##
####################################################################################################################

if pgrep -l "plasma" | grep "plasmashell"; then
    echo "$DATE Display Manager is KDE, running KDE clause!"
    kde-clause
    else
        echo "$DATE Display Manager is not KDE!"
        stop_display_manager_if_running
fi

## Unbind EFI-Framebuffer ##
if test -e "/tmp/vfio-is-nvidia"; then
    rm -f /tmp/vfio-is-nvidia
    else
        test -e "/tmp/vfio-is-amd"
        rm -f /tmp/vfio-is-amd
fi

sleep "1"

##############################################################################################################################
## Unbind VTconsoles if currently bound (adapted and modernised from https://www.kernel.org/doc/Documentation/fb/fbcon.txt) ##
##############################################################################################################################
if test -e "/tmp/vfio-bound-consoles"; then
    rm -f /tmp/vfio-bound-consoles
fi
for (( i = 0; i < 16; i++))
do
  if test -x /sys/class/vtconsole/vtcon"${i}"; then
      if [ "$(grep -c "frame buffer" /sys/class/vtconsole/vtcon"${i}"/name)" = 1 ]; then
	       echo 0 > /sys/class/vtconsole/vtcon"${i}"/bind
           echo "$DATE Unbinding Console ${i}"
           echo "$i" >> /tmp/vfio-bound-consoles
      fi
  fi
done

sleep "1"

if lspci -nn | grep -e VGA | grep -s NVIDIA ; then
    echo "$DATE System has an NVIDIA GPU"
    grep -qsF "true" "/tmp/vfio-is-nvidia" || echo "true" >/tmp/vfio-is-nvidia
    echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/unbind

    ## Unload NVIDIA GPU drivers ##
    modprobe -r nvidia_uvm
    modprobe -r nvidia_drm
    modprobe -r nvidia_modeset
    modprobe -r nvidia
    modprobe -r i2c_nvidia_gpu
    modprobe -r drm_kms_helper
    modprobe -r drm
    modprobe -r nouveau
    modprobe -r snd_hda_intel

    echo "$DATE NVIDIA GPU Drivers Unloaded"
fi

if lspci -nn | grep -e VGA | grep -s AMD ; then
    echo "$DATE System has an AMD GPU"
    grep -qsF "true" "/tmp/vfio-is-amd" || echo "true" >/tmp/vfio-is-amd
    echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/unbind

    ## Unload AMD GPU drivers ##
    modprobe -r drm_kms_helper
    modprobe -r amdgpu
    modprobe -r radeon
    modprobe -r drm

    echo "$DATE AMD GPU Drivers Unloaded"
fi

## Load VFIO-PCI driver ##
modprobe vfio
modprobe vfio_pci
modprobe vfio_iommu_type1

echo "$DATE End of Startup!"


and vfio_teardown.sh:
  

#!/bin/bash

#############################################################################
##     ______  _                _  _______         _                 _     ##
##    (_____ \(_)              | |(_______)       | |               | |    ##
##     _____) )_  _   _  _____ | | _    _   _   _ | |__   _____   __| |    ##
##    |  ____/| |( \ / )| ___ || || |  | | | | | ||  _ \ | ___ | / _  |    ##
##    | |     | | ) X ( | ____|| || |__| | | |_| || |_) )| ____|( (_| |    ##
##    |_|     |_|(_/ \_)|_____) \_)\______)|____/ |____/ |_____) \____|    ##
##                                                                         ##
#############################################################################
###################### Credits ###################### ### Update PCI ID'S ###
## Lily (PixelQubed) for editing the scripts       ## ##                   ##
## RisingPrisum for providing the original scripts ## ##   update-pciids   ##
## Void for testing and helping out in general     ## ##                   ##
## .Chris. for testing and helping out in general  ## ## Run this command  ##
## WORMS for helping out with testing              ## ## if you dont have  ##
##################################################### ## names in you're   ##
## The VFIO community for using the scripts and    ## ## lspci feedback    ##
## testing them for us!                            ## ## in your terminal  ##
##################################################### #######################

################################# Variables #################################

## Adds current time to var for use in echo for a cleaner log and script ##
DATE=$(date +"%m/%d/%Y %R:%S :")

################################## Script ###################################

echo "$DATE Beginning of Teardown!"

## Unload VFIO-PCI driver ##
modprobe -r vfio_pci
modprobe -r vfio_iommu_type1
modprobe -r vfio

if grep -q "true" "/tmp/vfio-is-nvidia" ; then

    ## Load NVIDIA drivers ##
    echo "$DATE Loading NVIDIA GPU Drivers"
    
    modprobe drm
    modprobe drm_kms_helper
    modprobe i2c_nvidia_gpu
    modprobe nvidia
    modprobe nvidia_modeset
    modprobe nvidia_drm
    modprobe nvidia_uvm
    modprobe nouveau
    modprove snd_hda_intel

    echo "$DATE NVIDIA GPU Drivers Loaded"
fi

if  grep -q "true" "/tmp/vfio-is-amd" ; then

    ## Load NVIDIA drivers ##
    echo "$DATE Loading AMD GPU Drivers"
    
    modprobe drm
    modprobe amdgpu
    modprobe radeon
    modprobe drm_kms_helper
    
    echo "$DATE AMD GPU Drivers Loaded"
fi

## Restart Display Manager ##
input="/tmp/vfio-store-display-manager"
while read -r DISPMGR; do
  if command -v systemctl; then

    ## Make sure the variable got collected ##
    echo "$DATE Var has been collected from file: $DISPMGR"

    systemctl start "$DISPMGR.service"

  else
    if command -v sv; then
      sv start "$DISPMGR"
    fi
  fi
done < "$input"

############################################################################################################
## Rebind VT consoles (adapted and modernised from https://www.kernel.org/doc/Documentation/fb/fbcon.txt) ##
############################################################################################################

input="/tmp/vfio-bound-consoles"
while read -r consoleNumber; do
  if test -x /sys/class/vtconsole/vtcon"${consoleNumber}"; then
      if [ "$(grep -c "frame buffer" "/sys/class/vtconsole/vtcon${consoleNumber}/name")" \
           = 1 ]; then
    echo "$DATE Rebinding console ${consoleNumber}"
	  echo 1 > /sys/class/vtconsole/vtcon"${consoleNumber}"/bind
      fi
  fi
done < "$input"


echo "$DATE End of Teardown!"

 

Link to post
Share on other sites
Posted
  • Author

Rebooted, then tried to run the VM. Same issue as before, it does nothing, but this time virtual machine manager actually became unresponsive and I had to force quit it.

Running sudo systemctl status libvirtd shows status green on the three dependencies for it, whereas before those were red. But, I still get the "QEMU/KVM - Connecting..." error in Virt and I cannot start my VM or create a new one ("No active connection to install on" error).

See below:

  

● libvirtd.service - Virtualization daemon
     Loaded: loaded (/lib/systemd/system/libvirtd.service; enabled; vendor preset: enabled)
     Active: active (running) since Mon 2024-06-24 17:29:11 PDT; 7min ago
TriggeredBy: ● libvirtd-admin.socket
             ● libvirtd-ro.socket
             ● libvirtd.socket
       Docs: man:libvirtd(8)
             https://libvirt.org
   Main PID: 1361 (libvirtd)
      Tasks: 21 (limit: 32768)
     Memory: 65.6M
        CPU: 1.710s
     CGroup: /system.slice/libvirtd.service
             ├─1361 /usr/sbin/libvirtd
             ├─1516 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
             └─1517 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper

Jun 24 17:29:11 spektrum systemd[1]: Started Virtualization daemon.
Jun 24 17:29:12 spektrum dnsmasq[1516]: started, version 2.90 cachesize 150
Jun 24 17:29:12 spektrum dnsmasq[1516]: compile time options: IPv6 GNU-getopt DBus no-UBus i18n IDN2 DHCP DHCPv6 no-Lua TFTP conntrack ipset no-nftset auth cryptohash DNSSEC l>
Jun 24 17:29:12 spektrum dnsmasq-dhcp[1516]: DHCP, IP range 192.168.122.2 -- 192.168.122.254, lease time 1h
Jun 24 17:29:12 spektrum dnsmasq-dhcp[1516]: DHCP, sockets bound exclusively to interface virbr0
Jun 24 17:29:12 spektrum dnsmasq[1516]: reading /etc/resolv.conf
Jun 24 17:29:12 spektrum dnsmasq[1516]: using nameserver 127.0.0.53#53
Jun 24 17:29:12 spektrum dnsmasq[1516]: read /etc/hosts - 8 names
Jun 24 17:29:12 spektrum dnsmasq[1516]: read /var/lib/libvirt/dnsmasq/default.addnhosts - 0 names
Jun 24 17:29:12 spektrum dnsmasq-dhcp[1516]: read /var/lib/libvirt/dnsmasq/default.hostsfile

 

Link to post
Share on other sites
Posted

Okay so technically what it should happen is your desktop gets killed then your screen goes black and turns to vm right?

 

Which on yours it doesn't it just freezes your virt manager.

 

Doesn't script kill your desktop at least it should then it's actually unhooked everything like script should do.

 

Yeah i think you should find your systemd desktop service and edit that into script that mentions systemd.

 

So.. which desktop are you using?

I'm jank tinkerer if it works then it works.

Regardless of compatibility 🐧🖖

Link to post
Share on other sites
Posted

I mean you could close your graphical session and terminate the graphical server, terminating the greeter, and starting your virtual machine from a console.

This post has been ninja-edited while you weren't looking.

 

I'm a used parts bottom feeder.  Your loss is my gain.

 

I like people who tell good RGB jokes.

Link to post
Share on other sites
Posted
3 minutes ago, r00tb33r said:

I mean you could close your graphical session and terminate the graphical server, terminating the greeter, and starting your virtual machine from a console.

Yeah thats manual way to do it although he still need name of service to kill though I'm sure once he knows the name then he can add/edit script then script should technically work.

I'm jank tinkerer if it works then it works.

Regardless of compatibility 🐧🖖

Link to post
Share on other sites
Posted
  • Author
1 hour ago, BoomerDutch said:

Okay so technically what it should happen is your desktop gets killed then your screen goes black and turns to vm right?

 

Which on yours it doesn't it just freezes your virt manager.

 

Doesn't script kill your desktop at least it should then it's actually unhooked everything like script should do.

 

Yeah i think you should find your systemd desktop service and edit that into script that mentions systemd.

 

So.. which desktop are you using?

Cinnamon on Linux Mint

Link to post
Share on other sites
Posted
1 hour ago, karambyt said:

Cinnamon on Linux Mint

I did sort of pay attention to your OP, so I figured that much.

 

Give this a look.

https://forums.linuxmint.com/viewtopic.php?t=407500

This post has been ninja-edited while you weren't looking.

 

I'm a used parts bottom feeder.  Your loss is my gain.

 

I like people who tell good RGB jokes.

Link to post
Share on other sites
Posted
8 hours ago, karambyt said:

Cinnamon on Linux Mint

According chatgpt its called cinnamon.service

 

However to confirm we can list all services that's currently running.

Use sudo if needed. 

systemctl

It will show list of all services and try find name of your desktop and your display manager if there is one.

You can press down button to see more.

And ctrl + c to stop.

 

Then replace the in script.

At start of the script this part. 

DISPMGR="$(grep 'ExecStart=' /etc/systemd/system/display-manager.service | awk -F'/' '{print $(NF-0)}')"
        echo "$DATE Display Manager = $DISPMGR"

Replace display-manager.service.

Then technically it should grab it.

 

I hope that devs of mint actually put desktop and display manager into one service.

 

If you're not sure you could always cut first part and execute to see if it actually grabs your desktop seeing it's outputting in terminal.

I'm jank tinkerer if it works then it works.

Regardless of compatibility 🐧🖖

Link to post
Share on other sites
Posted
  • Author
5 hours ago, BoomerDutch said:

According chatgpt its called cinnamon.service

 

However to confirm we can list all services that's currently running.

Use sudo if needed. 

systemctl

It will show list of all services and try find name of your desktop and your display manager if there is one.

You can press down button to see more.

And ctrl + c to stop.

 

Then replace the in script.

At start of the script this part. 

DISPMGR="$(grep 'ExecStart=' /etc/systemd/system/display-manager.service | awk -F'/' '{print $(NF-0)}')"
        echo "$DATE Display Manager = $DISPMGR"

Replace display-manager.service.

Then technically it should grab it.

 

I hope that devs of mint actually put desktop and display manager into one service.

 

If you're not sure you could always cut first part and execute to see if it actually grabs your desktop seeing it's outputting in terminal.

So you're saying that I need to paste that into the startup.sh script at the beginning where it declared "dispmgr=NULL", and also replace "display-manager.service" with "cinnamon.service"?

 

Unfortunately, systemctl does not show cinnamon.service or display-manager.service...

Link to post
Share on other sites
Posted
1 hour ago, karambyt said:

So you're saying that I need to paste that into the startup.sh script at the beginning where it declared "dispmgr=NULL", and also replace "display-manager.service" with "cinnamon.service"?

 

Unfortunately, systemctl does not show cinnamon.service or display-manager.service...

 

57 minutes ago, karambyt said:

Update, the name of the service is "lightdm.service"

 

Looks like ChatGPT was wrong. You should not rely on shitty AI products, just saying.

More like you can't read.

 

Fine then you go figure it out.

 

I'll come back next day.

I'm jank tinkerer if it works then it works.

Regardless of compatibility 🐧🖖

Link to post
Share on other sites
Posted
6 hours ago, karambyt said:

shitty AI

It's called "artificial" for a reason.  It's not the real thing.

 

Unfortunately no Mint Chads came to your aid here, you will have to understand yourself the technology stack of your distro.  Or ask for help on the Mint forum.  I saw other threads on there they just point to that same guy I already linked you to.

 

My opinion is that if you wanted to first validate passthrough for yourself before investing the time into figuring out how to adapt it to your distro it will be quicker to first replicate the exact environment used in the guides you found.  Then identify the differences and adjust the procedure accordingly.

This post has been ninja-edited while you weren't looking.

 

I'm a used parts bottom feeder.  Your loss is my gain.

 

I like people who tell good RGB jokes.

Link to post
Share on other sites
Posted
1 hour ago, r00tb33r said:

My opinion is that if you wanted to first validate passthrough for yourself before investing the time into figuring out how to adapt it to your distro it will be quicker to first replicate the exact environment used in the guides you found.  Then identify the differences and adjust the procedure accordingly.

Unfortunately not all guides is working for most cases since sometimes it misses drivers or things gets changed over time.

And not many users has same desktop as others.

 

 

 

 

Technically the first part of passthrough is done 

 

All OP needs to do is find his desktop session whenever it's service or process to kill and to start however OP is lacking experience to understand.

 

So OP has done in script

  • Switch from normal drivers to vfio
  • Kill display manager

However what he needs to do is

  • Kill desktop process or service.

Then in theory everything should be unplugged from gpu and should be able to switch to VM.

 

However this method is far more difficult because screen must go blind and wondering if it's working or not.

 

To get control while its gpu is unplugged you should consider using ssh if things didn't go well.

 

But then again you'll need some basic Knowledge of linux and read little more into scripts since you're admin after all.

 

This is new process to me because i haven't done killing display manager or desktop manager because its using igpu instead.

 

however i do have certain understanding how drivers work thanks to tinkering with gpu passthrough to multiple operation systems. 

 

If you want my help then please stop with these comments and try reading better then ill happily assist any way i can.

I'm jank tinkerer if it works then it works.

Regardless of compatibility 🐧🖖

Link to post
Share on other sites
Posted

Not just killing the display manager/greeter, but shut it down (or stop), because it will restart if the process is simply killed.  That's my experience anyway.

This post has been ninja-edited while you weren't looking.

 

I'm a used parts bottom feeder.  Your loss is my gain.

 

I like people who tell good RGB jokes.

Link to post
Share on other sites
Posted
29 minutes ago, r00tb33r said:

Not just killing the display manager/greeter, but shut it down (or stop), because it will restart if the process is simply killed.  That's my experience anyway.

Yes that's how it should've done however in some different setup the greeter / display manager does launch process instead service or even doesn't have systemd after all.

I'm jank tinkerer if it works then it works.

Regardless of compatibility 🐧🖖

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share
Go to topic listing Troubleshooting

×