Linux Challenge actually helps Linux with a lasting impact: Debian and Pop_OS updating 'apt' package manager to make it more fail-safe for users

[grg994]

Summary

In the recent LTT video Linus unintentionally broke his Pop_OS system during installing Steam (https://youtu.be/0506yDSgU7M?t=597).

The cause - as in any disaster - is complex. By my thoughts it can be broken down to 3 parts:

1 - A temporary problem with Steam's package: is reported conflicts with essential system packages of Pop_OS (already fixed)

2 - Then when trying to install Steam in the command line 'apt' package manager gave a warning seemingly not clear and highlighted enough that proceeding with the installation as such will remove system components and break the system.

3 - Some human error: Linus overlooked these warnings uninstalling his desktop environment on Pop_OS.

 

The upcoming updates made by the developers of 'apt' in the last days aim to address number 2 in this chain of events, making the package manager more fail-safe.

This will help to prevent other beginner users to make the same mistake and brake their Debian-based distro.

(This has been a problem for a while, others on some forums were also reporting accidental removal of system components through apt due to bugged package conflicts)

 

Quotes

Upcoming change in Debian / apt (https://github.com/Debian/apt/blob/main/debian/NEWS)

Quote

apt (2.3.12) unstable; urgency=medium

The solver will no longer try to remove Essential or Protected packages,

any dependency problem that would need such a solution will have to be

resolved manually.

The "Yes, do as I say" prompt for removing essential packages has been

replaced by an error message. The appropriate command-line option needs

to be used instead.

Thank you to Linus Tech Tips and System76 for bringing this issue

to our attention.

-- Julian Andres Klode <jak@debian.org> Wed, 17 Nov 2021 18:26:40 +0100

 

Change in the warning message in Pop_OS's fork of apt. This was accidentally ignored by Linus on the video:

(https://github.com/pop-os/apt/pull/1/commits/0cbdc740944daa97ea5aceac52c172ada4fc0dc5)

Quote

[This is stripped from C code]

 

"You are about to do something potentially harmful."

"To continue type in the phrase [...]

 

[changed to]

 

"This operation is not permitted because it will break the system."

"Abort."

 

My thoughts

First of all I think this is absolutely amazing. A fine example how much does depend on user feedback in the open-source word.

 

I hope LTT staff will see and welcome these news.

 

Sources

Initial discussion by Pop_OS community:

https://github.com/pop-os/apt/pull/1

 

Debian community's discussion:

https://salsa.debian.org/apt-team/apt/-/merge_requests/196

 

News file of the Debian/apt repository about the update:

https://github.com/Debian/apt/blob/main/debian/NEWS

 

Explanation of the updates by a Linux youtuber:

https://youtu.be/siEIKFy1Q0I

 

[Helpful Tech Witch]

I dissagree with this fix.

Maybe hide a arg that allows you to force it, but dont make you do it manually.

Highlight it, change color, something, dont make it manual.

Something like sudo apt install package --force-install

[bcredeur97]

16 minutes ago, HelpfulTechWizard said:

I dissagree with this fix.

Maybe hide a arg that allows you to force it, but dont make you do it manually.

Highlight it, change color, something, dont make it manual.

Something like sudo apt install package --force-install

I agree with you but the argument to that is --> we'll have blogs and guides that just put /force or -force or --force-install as you mentioned in every command they list that users will blindly run and trash their systems.
 

[Giganthrax]

I applaud this change. Regular users should never be able to destroy a supposedly newcomer-friendly OS while trying to install a game launcher or any other program that isn't explicitly related to the core funtionalities of their distro. It's unintuitive to the extreme and is exactly the kind of thing that will immediately push away new users. 

[grg994]

5 minutes ago, bcredeur97 said:

I agree with you but the argument to that is --> we'll have blogs and guides that just put /force or -force or --force-install as you mentioned in every command they list that users will blindly run and trash their systems.

Apparently on debian the option will be --allow-remove-essential , a distinct one from the general --force-yes

 

Blogs and guides putting this --allow-remove-essential everywehere would be as stupid as recommending to use  rm -rf  with --no-preserve-root in general...

[RONOTHAN##]

Honestly, just changing it from "Yes, do as I say!" to something like "Yes, break my system" would fix 90% of the issues with the current system. That said, the solution I would like to see instead would be to edit a config file to be able to do this. A force flag, as said above would work, but I'd want to see it combined with changing the string to something a bit more ominous, since it seems like something that might be mildly abused.

[Forbidden Wafer]

As someone who have already borked a system like this: yay.
As someone who understands this doesn't fix the underlying issue: meh, package managers still trash in that specific aspect.

 

An A/B scheme should have been adopted from the beginning, while having a single instance of system packages should be the exception rather than the norm for desktop users, where storage isn't a huge issue.

[LAwLz]

This change wouldn't be needed if people actually read warnings instead of ignoring them... But good on Debian for catering to illiterate users.

[Eaglerino]

2 hours ago, LAwLz said:

This change wouldn't be needed if people actually read warnings instead of ignoring them... But good on Debian for catering to illiterate users.

the fact Linux types believe it's acceptable for the option to delete your desktop exists at the end of an everyday app install is exactly why you're staying in 1-3%

judging from my past experiences with open-source devs, I guarantee this got brought up earlier. it took a video from a big channel and relentless mocking to get it changed. hilarious

[xAcid9]

Why? It's part of Linux experience. 

[dizmo]

5 hours ago, LAwLz said:

This change wouldn't be needed if people actually read warnings instead of ignoring them... But good on Debian for catering to illiterate users.

I'd love for you to explain to me why a Steam installer needs to remove Essential and/or Protected packages, including the desktop environment. 

[Eigenvektor]

9 hours ago, HelpfulTechWizard said:

I dissagree with this fix.

Maybe hide a arg that allows you to force it, but dont make you do it manually.

How often do you actually need to remove essential system packages to make it a bother that they're making you jump through hoops for that?

 

I don't think I've ever been in a situation where "remove essential packages" was the correct answer.

[leadeater]

33 minutes ago, dizmo said:

I'd love for you to explain to me why a Steam installer needs to remove Essential and/or Protected packages, including the desktop environment. 

Well that, in my opinion, is the peril and issue with distro inception and every man and his dog creating them. Pop!_OS is a Fork (Ubuntu) of a Fork (Debian) so 3rd line in a chain and has it's own custom Desktop Experience packages and repositories for them as well as including the upstream Ubuntu repositories. The Steam package had dependencies that could be natively meet by the Ubuntu repositories, as these are included as sources for Pop!_OS, however Pop!_OS could not so the only valid way to resolve the dependency conflicts at that time for Pop!_OS was for APT to remove ALL the offending packages and their dependencies and then ONLY install what was required for Steam meaning every single Pop!_OS Desktop Experience package was removed, tada welcome to CLI only.

 

Now the vital criticism for me is that the Steam package comes from Pop!_OS so they added a package in to their repository without doing proper QA and doing dependency validation. Operator error is and can always be a thing but it is System76 who screwed up and introduced an issue in to their own product that for no matter how short of a time period could have given any current or potential users a very bad experience even if they didn't do a Linus.

[Lightwreather]

Hopefully, this thread won't disolve into petty arguments like quite a few other threads.
BUt good on Debian for helping people who might not understand, what is imo, somewhat Technical language. Linux needs noobs.

[Master Disaster]

Its worth pointing out that unless those downstream from Debian specifically make this change themselves (and honestly I think most of the major distros will) its going to be a while before this change goes everywhere. Because Debian is essentially the granddaddy of so many distros it has a 2 year deprecation period for major changes to essential packages, it kind of has to so they don't make big changes and destroy everything downstream in a single release.

[Master Disaster]

2 hours ago, Eigenvektor said:

How often do you actually need to remove essential system packages to make it a bother that they're making you jump through hoops for that?

 

I don't think I've ever been in a situation where "remove essential packages" was the correct answer.

Realistically, the only situation where this becomes an issue is one where the users already knows what they're doing is drastic and in that case you'd expect the user to have a certain level of competence anyway.

 

Average Jane doesn't need to swap out systemd for runit or apt for stow, they're called essential packages for a reason.

 

I love how pacman just doesn't give a fuck if I tell it to install openrc, its just like "sure man, enter your password and I'll break your system without even warning you"

[Beerzerker]

What I'm getting from this is they are indeed listening and making changes to correct things.... Which is GOOD.
How many times have we complained the makers of an OS ignoring user issues, problems in general and the like?
Plenty but at least in this case they did hear and took action.

Now, to be fair since it did involve LTT instead of just an everyday individual is probrably why they moved so fast on this and took it as seriously as they did. The fact these issues pointed out are on video just makes it more difficult to ignore, so they did what they had to do with it.

TBH in my personal instance Pop OS just isn't for me, but I do commend them for working to get this taken care of.

[Master Disaster]

1 minute ago, Beerzerker said:

What I'm getting from this is they are indeed listening and making changes to correct things.... Which is GOOD.
How many times have we complained the makers of an OS ignoring user issues, problems in general and the like?
Plenty but at least in this case they did hear and took action.

Now, to be fair since it did involve LTT instead of just an everyday individual is probrably why they moved so fast on this and took it as seriously as they did. The fact these issues pointed out are on video just makes it more difficult to ignore, so they did what they had to do with it.

TBH in my personal instance Pop OS just isn't for me, but I do commend them for working to get this taken care of.

As a Linux user its easy to look at a situation through the eyes of a Linux user, its much harder to look at the situation through the eyes of a novice. I've seen plenty of Linux channels chastising Linus for not reading the wall of text citing the fact that him having to stop and type should have been enough warning. Except that, as a new user to Linux how was he supposed to know that this message was abnormal?

 

As a community I think we tend to get drawn into complacency and the assumption that everyone should know what we consider to be basic stuff, sometimes it takes high profile newcomers to point out what we are doing wrong. Nobody can call Linus a computer noob so the fact he missed this and caught it on camera pointed out that actually, its not a pebkac issue, its a software design issue.

[Eigenvektor]

11 minutes ago, Master Disaster said:

As a Linux user its easy to look at a situation through the eyes of a Linux user, its much harder to look at the situation through the eyes of a novice. I've seen plenty of Linux channels chastising Linus for not reading the wall of text citing the fact that him having to stop and type should have been enough warning. Except that, as a new user to Linux how was he supposed to know that this message was abnormal?

I don't even think it's just beginners who can fall into this trap. I've been in situations where I blindly answered "yes" without actually reading, because my "experience" led me to believe it was asking one thing, when it was another thing entirely. Making it harder to break your system is a good thing in my eyes. If you want to mess with packages at such a level, you hopefully already know what you're doing and adding one more command line argument shouldn't be an issue.

[Paul Thexton]

If I’d been doing the same thing as Linus the sequence probably would’ve been something like

 

• Pick a distro based on general reports of packaged drivers appropriate to my hardware etc
• Make sure sound is working, game controller etc.
• Install Steam.
• Read error presented to me
• Make a coin flip decision to try a different distro, or just give it up as a bad job

I wouldn’t have removed my desktop environment, I still would have had a bad experience.

 

I still use Linux as a desktop OS, but not exclusively (only for specific things), I genuinely want the phrase “Linux is free if your time has no value” to no longer apply. I don’t think we’re there yet though I still regularly check in on that. Once we are, I will gladly abandon Windows/macOS for non work machines entirely. 

[Paul Thexton]

52 minutes ago, Eigenvektor said:

I don't even think it's just beginners who can fall into this trap. I've been in situations where I blindly answered "yes" without actually reading, because my "experience" led me to believe it was asking one thing, when it was another thing entirely. Making it harder to break your system is a good thing in my eyes. If you want to mess with packages at such a level, you hopefully already know what you're doing and adding one more command line argument shouldn't be an issue.

Agreed. An experienced user approach would be “Why isn’t apt giving me the ‘do it anyway’ option? I’ll check the man page”

 

No biggie. At all. 

[AlTech]

10 hours ago, LAwLz said:

This change wouldn't be needed if people actually read warnings instead of ignoring them... But good on Debian for catering to illiterate users.

Thank you! Everybody else in the thread seems to be acting like System76 and Debian did something wrong when the issue occured. Still don't think System76 and Debian were in the wrong but good for them for trying to combat the hate they've been getting.

[Master Disaster]

8 minutes ago, AluminiumTech said:

Thank you! Everybody else in the thread seems to be acting like System76 and Debian did something wrong when the issue occured. Still don't think System76 and Debian were in the wrong but good for them for trying to combat the hate they've been getting.

Why do people feel the need to apportion blame at all? The software was designed in a way that wasn't exactly the best for newcomers and could result in extreme situations, it was pointed out (in the most high profile way possible) and they fixed it.

 

Hindsight is always 20/20, until an issue becomes apparent how exactly is anybody supposed to know its an issue?

 

There's a legitimate argument that S76 did something wrong in this case but not because of the way apt acted, they were the ones who swapped out a library forcing steam to pull a different copy from Ubuntus repos instead but again, they weren't in the wrong in doing this, they just did the wrong thing (or more accurately missed an issue with what they did) and there's a big difference between the two.

[Nayr438]

13 hours ago, grg994 said:

Apparently on debian the option will be --allow-remove-essential , a distinct one from the general --force-yes

 

Blogs and guides putting this --allow-remove-essential everywehere would be as stupid as recommending to use  rm -rf  with --no-preserve-root in general...

And I almost guarantee they will. People don't want to read and understand what's happening, they just want something to happen. A good chunk of the Linux Community just blindly copies and pastes commands from random sites, and most of these answers/guides are written by people who have no idea what they are doing.

There thought process is, it worked for me so it will work for you.

In the LTT Video, PopOS! prevented the install by standard means, they googled a answer, blindly ran the command ignoring all output, and nuked the desktop. Changinig the command or adding a extra step won't prevent that.

 

In the Linux Community, more bad information gets absorbed and spread than good information.

 

2 hours ago, Master Disaster said:

As a Linux user its easy to look at a situation through the eyes of a Linux user, its much harder to look at the situation through the eyes of a novice. I've seen plenty of Linux channels chastising Linus for not reading the wall of text citing the fact that him having to stop and type should have been enough warning. Except that, as a new user to Linux how was he supposed to know that this message was abnormal?

 

As a community I think we tend to get drawn into complacency and the assumption that everyone should know what we consider to be basic stuff, sometimes it takes high profile newcomers to point out what we are doing wrong. Nobody can call Linus a computer noob so the fact he missed this and caught it on camera pointed out that actually, its not a pebkac issue, its a software design issue.

And you are somewhat right. As a Developer, I expect whoever is going to use my software to already know there way around Linux. If my software does something that is potentially dangerous, I will give a warning, but unless your familiar with how Linux works it's mostly meaningless to you.

I however wouldn't call it a software design issue, most of my software targets Linux Users not Windows or MacOS users where everything holds your hand every step of the way, and tries to prevent you from modifying the system.

These are things we intentionally try to avoid. There are people who are in the Linux community that go out of there way to try and adapt software to these users, and I think that's a good thing, but at the same time that's just not what a lot of us are targeting. I personally find software designed that way to be in the way and annoying.

 

And in all fairness to PopOS!, they did have a GUI that basically holds your hand and tried to prevent you from breaking the system. Linus just went out of his way to go around it.

[Master Disaster]

1 hour ago, Eigenvektor said:

I don't even think it's just beginners who can fall into this trap. I've been in situations where I blindly answered "yes" without actually reading, because my "experience" led me to believe it was asking one thing, when it was another thing entirely. Making it harder to break your system is a good thing in my eyes. If you want to mess with packages at such a level, you hopefully already know what you're doing and adding one more command line argument shouldn't be an issue.

Its kind of funny that back in the days of Windows apps being bundled with crapware, devs relied on this behaviour from the user. Instead of reading they just click next, next, next, next, install, ok, finish then wonder how they ended up with chrome, the yahoo toolbar and mcaffee AV installed on their systems.