LTT Video Error Handling Standard Operating Procedure (SOP)

[OS_dirk]

1 hour ago, imreloadin said:

Now imagine how things would have played out if THIS was Linus' response after the news broke instead of his knee-jerk emotion driven post he made a few hours after the GN video.

 

This is literally what people wanted and it really boggles my mind how this wasn't the go-to choice for the business since it's pretty common sense.

I agree and disagree on this one.

 

I don't want to see a future where Linus is no longer allowed to misspeak, or create a Faux Pas.

 

I can understand (or at least sympathize with) with the reaction from Linus the human. The week before everything blew up, a crowd sourced error checking and correction team was discussed on WAN show. It's not like Linus or LMG had been hiding the fact that there have been mistakes and errors.

 

I can see, given the timing, and given previous communications on the topic, where Linus might feel a little bit persecuted. I assume that Linus saw the expose from GN, and thought something to himself something like "I've been working on this, damn it!"

 

That said, Linus the CVO should have given himself a bit more time to cool off, and focused a bit more on the perspective of an uninformed third party, and less on his own perspective.

 

 

 

In terms of policies and procedures. I agree this should have been a priority before now, but that is considering the issue in hindsight. I doubt that anyone at LMG was making the day to day decisions (small and large) in an effort to promote false information, or to conceal various errors. (If that were the case, there are much better methods to accomplish this than discussing various problems on WAN show, or YouTube, and inviting public criticism)

 

It took the GN expose to connect the dots, and to paint the picture from there about how the various errors & omissions are starting to add up from the perspective of the uninformed consumer.

 

I still think that GN took too strong of a stance, here, but I can't fault the overall thesis. (I agree that labs is as much about marketing as it is about accuracy, but don't believe that labs was being intentionally misrepresented.) I have little doubt Linus would have roasted himself, if he were to take the time to look at the issue from Steve's perspective.

 

The criticism is largely fair, if not necessarily the magnitude.

 

---

 

Edit: Apologies for the wordy "interjection" - More or less regurgitated thoughts from mulling over this drama for the past week.

[Atra1n2]

2 hours ago, DiamondDaveL said:

Looks very reasonable and thorough to me. I imagine it will be hard to exactly see the effects of new processes and controls as when something is fixed before it's public, we (the viewers) won't know about it. 

 

EDIT: I suggested this below in response to another comment, but how should viewers report errors? Maybe LMG can setup a google for that requires an email to prevent spam, and have a text box to paste a link/title for the video in question, type of error (drop-down list), and then error details textbox. 

Google form sounds reasonable. There are ways I have personally used, to dynamically pre-fill Google forms. This could pre-fill the video URL. It's a simple modification of the form URL. When a video is uploaded, just grab the URL and paste it within the form submission URL. Limit form responses to 1 time per user, with a daily reset of the users' submission count.

[joshfrog]

7 minutes ago, Atra1n2 said:

Limit form responses to 1 time per user, with a daily reset of the users' submission count.

If you use a google form and limit it to 1 response per user you will need to submit your email address.

I never fill out shit that asks for email addresses like this because i hate giving out my email as i get enough spam.

Its a small annoyance for most id say but annoying AF for me.

That said i don't think there's any other way to block people from submitting more then 1 response at a time and the only way to stop spamming.

[joshfrog]

I think they should also add a version number so if updates or changes happen its clearly tracked and known that a video from 1 yr ago fits in with v1.0 SOP and a video from today for example is from v1.1 if it was updated.

That way if the standards get updated in time assuming they do people can look back at older videos and know it may not meet the newer standard of the SOP.

Could help cut down on people applying the wrong version of SOP to videos if in a year things change.

 

[ETHANO8225]

29 minutes ago, joshfrog said:

If you use a google form and limit it to 1 response per user you will need to submit your email address.

I never fill out shit that asks for email addresses like this because i hate giving out my email as i get enough spam.

Its a small annoyance for most id say but annoying AF for me.

That said i don't think there's any other way to block people from submitting more then 1 response at a time and the only way to stop spamming.

I'd bet there would be a way to set up a filter for spam though. And, let's say a user sent in X legit reports, they can get whitelisted from the filter to ensure they get heard. Or, cloudflare it somehow lol

[rice guru]

SOPs should always be living documents, must be reviewed regularly and Improvements must be added and reviewed at a timely manner after which should be reviewed by all relevant staff. I've worked in industrial and laboratory settings where we use SOP's and this is how we operate with them. I currently work for a geochemistry lab where I run assays for companies and individuals determining what elements are in their samples. I say all this as I am very surprised and glad to see SOPs used by larger internet media companies and is a nice look into a company that continues to grow in a space traditionally looked at as  unprofessional. 

[Atra1n2]

53 minutes ago, joshfrog said:

If you use a google form and limit it to 1 response per user you will need to submit your email address.

I never fill out shit that asks for email addresses like this because i hate giving out my email as i get enough spam.

Its a small annoyance for most id say but annoying AF for me.

That said i don't think there's any other way to block people from submitting more then 1 response at a time and the only way to stop spamming.

Yeah I think email would be a must in this case to limit spam.

[Atra1n2]

20 minutes ago, ETHANO8225 said:

I'd bet there would be a way to set up a filter for spam though. And, let's say a user sent in X legit reports, they can get whitelisted from the filter to ensure they get heard. Or, cloudflare it somehow lol

They could at least be de-prioritized until they've had previous feedback that has been vetted as good.

[joshfrog]

5 minutes ago, Atra1n2 said:

Yeah I think email would be a must in this case to limit spam.

I don't disagree but at the same time it isn't the hardest thing to bypass.

Edited August 26, 2023 by joshfrog
Spelling mistake

[moatmote]

Edited - see below (couldn't figure out how to add a second quote via edit).

[moatmote]

4 hours ago, DiamondDaveL said:

I suggested this below in response to another comment, but how should viewers report errors? Maybe LMG can setup a google for that requires an email to prevent spam, and have a text box to paste a link/title for the video in question, type of error (drop-down list), and then error details textbox. 

I've been saying this for literally years, but it makes no sense to me that there isn't a dedicated section of the forum that is regularly monitored dedicated to reporting errors and misinformation. HELL NO to a Google Form that offers 0 transparency. Reports should be publicly visible and it should be clear whether or not a report has been appropriately addressed, either via staff response justifying why it isn't an error (after which the thread should remain open for any counterarguments), or by taking the proper steps to address the error. After either course of action, the thread can be moved to a "disputed errors" or "resolved errors" forum to keep the main reporting forum clear.

 

Also, I can only pray that they aren't going to continue using "it's an unboxing not a review" as an excuse for every mistake they make in Short Circuit that could have been prevented with basic due diligence.

 

And how about not continuing to promote content you've already been informed is erroneous or misleading? Still remembering that after you read off misinformation live on the WAN Show about the S23's system utilization (which was already debunked at the time), despite my and others' multiple attempts to inform you on the topic on multiple platforms, you not only completely ignored it, but doubled down and released the clip on LMG Clips with a title and description that directly suggests that what you read is indubitably true, a video that is still up to this day: https://youtu.be/DKg5NvlJVZw

 

4 hours ago, LinusTech said:

Reactive (after the video is posted)

• Video Replace: replacing the video with a new version without re-uploading

  • This relies on YouTube and takes some time. There are fairly strict guidelines around the use of this tool and strong justification must be provided for all changes. This is preferred to a re-upload, but there is likely a soft-limit on how often we can use this resource.

• Re-upload the video: Set the original, erroneous video to Private and upload a new version. This will have algorithmic effects, but must be done if replacement is not an option

• Pinned comment: add a comment describing the correction. 

  • This will not be received by viewers who do not check the comments (common, especially for those watching on a Smart TV) and should only be used for low-severity errors.

This isn't enough. 

 

You should publish a regular corrections video (every week/2 weeks/month) that quickly summarizes all the mistakes you caught and corrected since the previous corrections video. The biggest issue with how you've been addressing the few errors that do get addressed is that it's hardly ever done in a way that anyone researching the original topic would ever find it. Having a regularly uploaded corrections video means people who follow you will at least know to check it for corrections to recent content.

 

Edit: I see that some others have discussed a regularly uploaded corrections video as well. Good.

 

Edit 2: Another factor that needs to be seriously considered—and yet another reason I'm highly against a Google Form system where reports are sent to who knows where to potentially never be seen again—is when new research/information comes to light in the future, either making something that was correct in an older video now incorrect, or giving a definitive answer to something that was questioned in an older video. It may not make sense to completely replace videos that are now simply dated, but issues like this could be reported and addressed in a corrections video as mentioned above. 

[mMontana]

I see how rare could be to take a video down.

And i don't feel that fine.

[kurangak]

4 hours ago, Absentia13 said:

The AI voice thing is ridiculous. Honestly, it's literally a 10 second section or something just get the host to record it again or delay the video. 

its not ridiculous at all. nowadays u can train an ai for voiceover. and it sounds really close to the real thing. 

[Spotty]

If you're open to suggestions, something I proposed a few weeks ago was issuing corrections during WAN show. I would still like to see this introduced in addition to the other steps you are already taking to correct errors.

When the team identifies and corrects a mistake in a video that has made it past release they can add a note in the WAN doc stating what the correction was for it to be addressed on the following WAN show by the hosts. This would likely be reserved for errors that are deemed high severity or very high severity.

My original comment:

On 7/12/2023 at 11:22 AM, Spotty said:

I would like to see LTT start issuing corrections during WAN show. Even if pinning a comment is treated as the norm on Youtube I would like to see LMG go above and beyond. People have complained about misinformation or mistakes in LMG videos from time to time. I understand that mistakes are going to happen but even if those are entirely innocent mistakes LMG should still make a fair effort to correct them. It should only take a few seconds to issue the corrections on WAN show and the writers can just add them in to the WAN show doc throughout the week (in combination with issuing a correction by pinned comment on the original video). All you would need to say is "I'd like to issue a correction for <video> earlier in the week. In that video we incorrectly stated <this>, however the correct thing is <this>. We apologise for the error".

This would be helpful for people who may have watched the video before any corrections had been made who are unlikely to revisit the video to be able to see a pinned comment or the edited/re-uploaded version.

[joshfrog]

3 minutes ago, Spotty said:

If you're open to suggestions, something I proposed a few weeks ago was issuing corrections during WAN show.

That's not a bad idea but with WAN show being 2-5 hours not everyone watches WAN show and LTT removes WAN shows from the channel for some reason so after a few days the correction wouldn't be up anymore.

 

[moatmote]

34 minutes ago, Spotty said:

If you're open to suggestions, something I proposed a few weeks ago was issuing corrections during WAN show. I would still like to see this introduced in addition to the other steps you are already taking to correct errors.

I'm very much against this. @joshfrog said exactly why. As I stated before:

1 hour ago, moatmote said:

The biggest issue with how you've been addressing the few errors that do get addressed is that it's hardly ever done in a way that anyone researching the original topic would ever find it.

To elaborate on that, the way LTT has operated thus far is something like this:

 

• Most issues: never get addressed
• Most "addressed" issues: casually mentioning in the middle of some other video (like some 3 hour build live stream or some other completely unrelated thing) "oh btw in a previous video we said X does A and B but actually it does C and D" that no one researching the original topic will ever see
• Remaining addressed issues: a pinned comment no one who already made purchasing decisions based on the original video will ever see

A regularly published dedicated corrections video ensures that viewers know where to look to potentially learn about any mistakes in recent videos, and being it's own (not-3-hour-long) video ensures that even people who aren't looking for it will view it because hey, it's another LTT video. 

[Uttamattamakin]

16 minutes ago, Spotty said:

Snip

I want to second what Spotty said that discussing errors made could be part of the WAN show. Corrections can even be content.   

For example, as this history channel does because their subject matter is controversial and fuzzy and they have to oversimplify. 

Just embrace that you are going to make errors and it's ok to point them out or have them pointed out.  

 

I would also suggest that remaking at least part of the video should be done for severity levels 4 and 5.    If people could be mislead that's to be avoided.  

Another good idea would be some organized method of receiving feedback from viewers.  A form for reporting errors or concerns with videos.  IF this exist already and I don't know about it forgive my error.  I can't be the only one who missed such a form existing If it does.  Make it more obvious if such a "complaint form" exist.  Make it part of someone's job to look at the issues reported.   The audience is the final peer review, and it is massive. 

Hire someone whose job it is to fact check the final product after it is rendered into a form for youtube upload but before it is even actually uploaded.  To be sure that final draft is error free. 

 

[moatmote]

2 minutes ago, Uttamattamakin said:

I want to second what Spotty said that discussing errors made could be part of the WAN show. Corrections can even be content.   

To quote my comment above:

1 hour ago, moatmote said:

This isn't enough. 

 

You should publish a regular corrections video (every week/2 weeks/month) that quickly summarizes all the mistakes you caught and corrected since the previous corrections video. The biggest issue with how you've been addressing the few errors that do get addressed is that it's hardly ever done in a way that anyone researching the original topic would ever find it. Having a regularly uploaded corrections video means people who follow you will at least know to check it for corrections to recent content.

 

3 minutes ago, Uttamattamakin said:

Another good idea would be some organized method of receiving feedback from viewers.  A form for reporting errors or concerns with videos.  IF this exist already and I don't know about it forgive my error.  I can't be the only one who missed such a form existing If it does.  Make it more obvious if such a "complaint form" exist.  Make it part of someone's job to look at the issues reported.   The audience is the final peer review, and it is massive. 

Currently they have nothing. People in this thread keep suggesting a Google Form, which I am vehemently against for the reason I outlined in my comment:

1 hour ago, moatmote said:

I've been saying this for literally years, but it makes no sense to me that there isn't a dedicated section of the forum that is regularly monitored dedicated to reporting errors and misinformation. HELL NO to a Google Form that offers 0 transparency. Reports should be publicly visible and it should be clear whether or not a report has been appropriately addressed, either via staff response justifying why it isn't an error (after which the thread should remain open for any counterarguments), or by taking the proper steps to address the error. After either course of action, the thread can be moved to a "disputed errors" or "resolved errors" forum to keep the main reporting forum clear.

 

[joshfrog]

I have watched the WAN show on and off for years now.
I don't think the idea of having corrections is a bad idea but there needs to be a more long term method of viewing corrections.

I also think having corrections on the WAN show opens them up to having more errors because they may miss understand the error or have a hot take on the error such as linus had with the water block.

It could open there correction up to needing to be corrected.

I think they need some form of video correction but a long term running document of corrections that can be reviewed by the community and updated if needed.

Edited August 26, 2023 by joshfrog
Spelling mistake

[costafilh0]

"this is a living document, and may change as we move forward"

 

This is the way! Always improving!

[Uttamattamakin]

4 minutes ago, moatmote said:

To quote my comment above:

 

Currently they have nothing. People in this thread keep suggesting a Google Form, which I am vehemently against for the reason I outlined in my comment:

 

I don't disagree on the Google form.  The forum might not be best since a lot of people will be afraid to complain.  This is not just an issue for this forum or this fandom.  A lot of people are not confident enough to walk up to the Kool Kid in Class and point out what they did wrong on their homework. 

 

For them an anonymous reporting form would be best.  They could square this circle by allowing for complaints or corrections to be anonymous while the content of the complaint or correction is public.  Something kin to what we see on Twitter Community Notes. 

[moatmote]

10 minutes ago, joshfrog said:

long term running document of corrections that can be reviewed by the community and updated if needed.

I have nothing against a text-based correction log and think it'd be great to have that as well. But as I've said, there must be transparency in error reports such that

• what has been reported
• which of those reports have been addressed
• which of those reports have been disputed by staff
• and which of those have yet to be addressed

is clearly visible to the public; and a video dedicated to summarizing all errors that have been addressed recently is necessary. 

 

7 minutes ago, Uttamattamakin said:

For them an anonymous reporting form would be best.  They could square this circle by allowing for complaints or corrections to be anonymous while the content of the complaint or correction is public.  Something kin to what we see on Twitter Community Notes. 

Hell, open a github. An issue tracker is exactly what's needed here. 

[Genesis8935]

44 minutes ago, joshfrog said:

That's not a bad idea but with WAN show being 2-5 hours not everyone watches WAN show and LTT removes WAN shows from the channel for some reason so after a few days the correction wouldn't be up anymore.

 

Any corrections could be clipped (as in LMG Clips) specifically and quickly so that they can be put out as an extra video for Saturday (day after) release on the main LTT channel. They could also cut a new video on Friday and play that during the WAN show as part of the show while also publishing it same time the WAN show starts (or afterwards).

[moatmote]

3 minutes ago, Genesis8935 said:

Any corrections could be clipped (as in LMG Clips) specifically and quickly so that they can be put out as an extra video for Saturday (day after) release on the main LTT channel. They could also cut a new video on Friday and play that during the WAN show as part of the show while also publishing it same time the WAN show starts (or afterwards).

I still don't think a proper corrections video fits the tone of the WAN Show at all. They've ranted about far too many things they had little to no understanding of on the WAN Show (ahem S23 storage utilization ahem) for me to feel comfortable with them discussing corrections there.

[RandomLegoBrick]

Massive props to everyone at LMG for pulling through this tough time, it takes a lot to own up to you mistakes especially when you fall this deep.  Yall messed up, ya messed up big time.  But the amount of dedication you have taken to fixing the issues has 100% earned you another chance from me.  I wish your company the best in the future!