improvement Data representation suggestion

[tristanchap]

Hi there all,

 

Just had a few suggestions on how data should be represented in videos. Of course, I understand not everyone cares for the rigor and accuracy of scientific media, but I think a discussion would be useful! 

 

One of the things I have noticed across many channels that provide data and comparisons on computer components, systems and general benchmarks is that they all lack some standards and important information in the figures produced. I am an Oxford Biology student and have been given a lot of classes and resources that are to help improve our data representation and ensure our data is unbiased and fair. This is obviously not what we should expect from free sources like LTT videos but there are some lessons that could be applied. For example:

 

• Use of colour differences between individual variables (New colour for each processor for example with different measures in a different but related shade)
• Clarity over the data used in the title. Titles are often very bare and don't tell us exactly what measure is being used. Is it the mean? is it the lowest value? Are we looking at a measure of central tendency. This sort of thing is important to know whether the examples we are being shown were cherry-picked or not!
• Some variety in the graphs used might sometimes represent the data better. If something has been tested 50 times each, you have a great sample size! So you should show that off with a boxplot that lets us know the mean, quartile ranges and outliers! This helps us to understand how variable the data is. We dont want to go a buy something because it looked super different if that is actually a rare result!
• Please, please. please! Ensure figure legends or side comments are of a reasonable size! we only see the figures for a short time and it just helps to read it. The empty space around the figure is fine to be filled with a good legend or even a little figure caption if needed!
• Sometimes its not always feasible to test something loads and loads of times, and thats okay! But that should be clear, Of course people might not understand what N = 3 means but they do understand if you give a little warning that they were only tested 3 times! we want to know how reliable your conclusions are!

Overall, the data behind the figures always sounds quite robust, and the process very well controlled! But it feels like a shame that a lot of the data is not being used as effectively as it could be. While there is a balance between how far you go, I feel there is a level to which it could be improved. A good figure can make the difference between someone buying something they truly are suited to or making a mistake due to an incorrect assumption. As an influencer, there is room to suggest that LTT has a responsibility to not mislead. Now I do feel the data could be construed to be misleading in some cases, and questions can be asked. My personal opinion is that if those questions were answered, they LTT would be providing information that no other influencer on this topic is providing, putting them ahead of the competition. There is truth in the fact that this will benefit more serious tech lovers and professionals than the average casual viewer, but that is where LTT would have to consciously decided how rigorous the figures truly are!

 

Now, I decided to post this here to see what other people think, do you agree? Do you think there are other things that they could do to improve their reliability? Do you think this is unnecesary and overly complicated and simply just a waste of time? Please, do let me know, I am interested to see what others think!

 

Thanks for getting this far if you read all that!

[LogicalDrm]

This has been discussed couple times before. So some of your points (or most of them) can be countered quite easily when you take into consideration their medium. If we would be talking about written stuff, your criticism would hold much better. But we are talking about video content where graphs are used to support spoken explanations.

 

Previous threads on subject:

https://linustechtips.com/topic/821818-lmgs-obsession-with-misleading-graphs/

https://linustechtips.com/topic/956343-thats-not-how-graphs-work-linus/

 

15 hours ago, tristanchap said:

• Use of colour differences between individual variables (New colour for each processor for example with different measures in a different but related shade)

This is used already? Or at least was last time I actually watched videos. They highlight the products that are new vs the ones that have been tested previously.

 

15 hours ago, tristanchap said:

• Clarity over the data used in the title. Titles are often very bare and don't tell us exactly what measure is being used. Is it the mean? is it the lowest value? Are we looking at a measure of central tendency. This sort of thing is important to know whether the examples we are being shown were cherry-picked or not!

This comes down to medium used. They are making videos where graphs are just addition because its easier for viewer to related numbers and bar lengths than just listen. They do often state in spoken parts what number represents. I haven't seen graph that would have left me wondering, what number does represent, maybe not shown in title, but under the graphs where it states how scale works.

 

Your point overall is valid as LMG has not shared their testing methodology in years. Other bigger reviewers do always share how tests were made (Jay, GN, OC3D).

 

15 hours ago, tristanchap said:

• Some variety in the graphs used might sometimes represent the data better. If something has been tested 50 times each, you have a great sample size! So you should show that off with a boxplot that lets us know the mean, quartile ranges and outliers! This helps us to understand how variable the data is. We dont want to go a buy something because it looked super different if that is actually a rare result!

This also comes down to medium and viewership. LMG videos are not very technical or high-value in scale of reviews. They are making short snapshots showing of the most important and best parts of things. Remember the cable tester video? That was really short snap of something that as testing and reviewing article and database would have been interesting as even longer piece. Most viewers would probably get bored and wouldn't understand scatterplots and such anyway. Only those of us who have higher education would.

 

So again, with LMG Labs and possible written content, this would and should be the way to go. Or if they ever decide to make their "turbo-nerd" series include statistical looks besides highly technical inspections and explanations.

 

15 hours ago, tristanchap said:

• Please, please. please! Ensure figure legends or side comments are of a reasonable size! we only see the figures for a short time and it just helps to read it. The empty space around the figure is fine to be filled with a good legend or even a little figure caption if needed!

Its video. You can always pause if you want to look in more detail.

 

15 hours ago, tristanchap said:

• Sometimes its not always feasible to test something loads and loads of times, and thats okay! But that should be clear, Of course people might not understand what N = 3 means but they do understand if you give a little warning that they were only tested 3 times! we want to know how reliable your conclusions are!

This comes down to testing methodology. LMG has made a video about it long time ago. Currently they state it broadly and don't go into detail. But usually tests are run only once. This is kinda standard with tech reviews. This does bring some issues, especially with those who don't have higher education and/or haven't worked with any tested data. I have had to note many times on forums when people bring up their benchmark results that there is margin of error, and something like 200 point difference between your result and reviewers result is normal.