Mild Criticism of LTT Tests

[Guest Kloaked]

 

 

 

I think you guys need to start doing a read on minimum frame rates along with max. Minimum and average fps is arguably more important than what the max fps is.

[Misanthrope]

I am happy that they tested a bit of IPC with the CS:GO test but their choice of benchmarks seems odd: I expected far more CPU bound games to be included you know, in a CPU review.

[imreloadin]

They also seem to have gotten into the practice of making most game tests GPU bound by running everything at 4K. It's kinda hard to see which of the CPUs tested really pulled ahead when you have them all paired with a GPU that can't keep up...

[wilamu]

For me LTT is more fun content. Often gives me ideas to look into other bits of tech.

 

If I want detailed information it's available in plenty of places on the web where that's their thing. 

[Guest Kloaked]

1 minute ago, wilamu said:

For me LTT is more fun content. Often gives me ideas to look into other bits of tech.

 

If I want detailed information it's available in plenty of places on the web where that's their thing. 

This is why I'm criticizing them. They have a much wider audience than these PC-focused channels yet they skim out on this important content.

[AlwaysFSX]

Just now, Kloaked said:

This is why I'm criticizing them. They have a much wider audience than these PC-focused channels yet they skim out on this important content.

That's the problem when you try to force your channel to make a video a day.

[wilamu]

I don't personally have any problems with the content, their one video per day schedule doesn't affect it imo. It has high production value and is fun to watch and if after watching the video you need more info then just go look? 

 

These guys are busy, they can't run every test under the sun then turn it into nice pretty graphs to tell you max/min/avg of everything.

[NoRomanBatmansAllowed]

7 hours ago, Kloaked said:

 

 

 

I think you guys need to start doing a read on minimum frame rates along with max. Minimum and average fps is arguably more important than what the max fps is.

The thing is that minimum FPS skewed because some game menus are locked at a really low number, or loading screens being 1fps. This doesn't demonstrate the full power of the product.

[Guest Kloaked]

Just now, wilamu said:

These guys are busy, they can't run every test under the sun then turn it into nice pretty graphs to tell you max/min/avg of everything.

I didn't know it was that hard to take a screenshot of Precision X (or something like it) that records your minimum and maximum FPS, among other useful information.

[Bouzoo]

2 minutes ago, AlwaysFSX said:

That's the problem when you try to force your channel to make a video a day.

However a proper review wouldn't hurt, even from time to time, and I'm not talking about case reviews. 

I will admit, this review had more info than usual and it's a step up, but test games in a CPU bottlenecked scenario, like QHD at least.

[wilamu]

Just now, Kloaked said:

I didn't know it was that hard to take a screenshot of Precision X (or something like it) that records your minimum and maximum FPS, among other useful information.

Like you say, no it's not hard but a screenshot wouldn't fall in line with the high production value of their videos.

[wilamu]

I don't know dude, I guess if they could fit it in then yea why not. It's always nice to get as much info as you can. Be interesting if this got mentioned by them in a future vid.

[SSL]

Just now, wilamu said:

Like you say, no it's not hard but a screenshot wouldn't fall in line with the high production value of their videos.

 

Are you joking? Production value concerns the content as much as the eye candy.

 

That said, Linus has never been good at doing objective tests. I just don't think it's something that he likes or cares enough to do properly. Which begs for the question as to why they bother at all.

[wilamu]

3 minutes ago, SSL said:

 

Are you joking? Production value concerns the content as much as the eye candy.

 

That said, Linus has never been good at doing objective tests. I just don't think it's something that he likes or cares enough to do properly. Which begs for the question as to why they bother at all.

 

That's what I'm saying. "Just" a screenshot doesn't fall high on the eye candy scale.

[dalekphalm]

7 hours ago, Kloaked said:

 

-snip-

 

I think you guys need to start doing a read on minimum frame rates along with max. Minimum and average fps is arguably more important than what the max fps is.

I 100% agree. Adding a Min and Max FPS value would not significantly increase the time to create a video, as whatever software they use to capture average FPS is probably already capturing Min and Max FPS anyway.

 

While average is incredibly important, Minimum is also very important.

 

And sure, there's the odd broken-ass-game that doesn't show proper min values, but that's a badly coded game, and they will be able to easily see if that's a problem (Or just not include that particular game in the benchmarks).

[SSL]

46 minutes ago, dalekphalm said:

I 100% agree. Adding a Min and Max FPS value would not significantly increase the time to create a video, as whatever software they use to capture average FPS is probably already capturing Min and Max FPS anyway.

 

While average is incredibly important, Minimum is also very important.

 

And sure, there's the odd broken-ass-game that doesn't show proper min values, but that's a badly coded game, and they will be able to easily see if that's a problem (Or just not include that particular game in the benchmarks).

 

Showing the min would undoubtedly paint a more accurate picture. But even then, bench-marking standards are not what they used to be a few years ago. We now have percentile-based measurements, FPS graphs, frame time graphs, and so on. Because these metrics provide a much more complete and accurate picture of performance. Mere bar charts do not really cut it these days. Consider the performance difference between generations of Intel CPUs - just a min and average FPS do not really convey the smoother game-play experience with the most recent models.

[Guest Kloaked]

5 minutes ago, SSL said:

Consider the performance difference between generations of Intel CPUs - just a min and average FPS do not really convey the smoother game-play experience with the most recent models.

Very true. PCPer has talked about this consistently and have been called Intel (or Nvidia in GPU talks) shills nearly every time they talk about this stuff, but they're practically the only ones who actually show these more informative measurements. Anandtech does as well but I find myself paying more attention to PCPer.

[dalekphalm]

5 minutes ago, SSL said:

 

Showing the min would undoubtedly paint a more accurate picture. But even then, bench-marking standards are not what they used to be a few years ago. We now have percentile-based measurements, FPS graphs, frame time graphs, and so on. Because these metrics provide a much more complete and accurate picture of performance. Mere bar charts do not really cut it these days. Consider the performance difference between generations of Intel CPUs - just a min and average FPS do not really convey the smoother game-play experience with the most recent models.

True but we all know that LTT is never going to do frametime and FCAT benchmarks. If we set aside the fact that relatively expensive equipment is required for proper analysis, they are also pretty technical benches, and LTT simply doesn't have the expertise.

 

But furthermore, they won't invest the time to do all these benchmarks. We already see them taking shortcuts and skipping bench figures, etc, to do benches as fast as possible. When adding a Min and Max FPS value into a graph would take a few minutes, and they're already excluding that, I don't think we can expect them to invest the time into doing more elaborate and technical benches.

[Guest Kloaked]

1 minute ago, dalekphalm said:

LTT simply doesn't have the expertise.

[dalekphalm]

3 minutes ago, Kloaked said:

We all know it's true. They're basically monkeys that have figured out how to record beautiful looking video

[Guest Kloaked]

Just now, dalekphalm said:

We all know it's true. They're basically monkeys that have figured out how to record beautiful looking video

Did you happen to watch the video with him and Louis Rossmann? I know not everyone is a fan of Louis for valid reasons, but Linus looked and sounded so out of place in Louis' shop.

[SSL]

7 minutes ago, dalekphalm said:

True but we all know that LTT is never going to do frametime and FCAT benchmarks. If we set aside the fact that relatively expensive equipment is required for proper analysis, they are also pretty technical benches, and LTT simply doesn't have the expertise.

 

But furthermore, they won't invest the time to do all these benchmarks. We already see them taking shortcuts and skipping bench figures, etc, to do benches as fast as possible. When adding a Min and Max FPS value into a graph would take a few minutes, and they're already excluding that, I don't think we can expect them to invest the time into doing more elaborate and technical benches.

 

A lot of these tech YT channels are in kind of a weird gray area between people like Marques Brownlee, who make vacuous, consumer-oriented drivel, and the likes of PCPer. In other words, they want to have their cake and eat it too.

[Bouzoo]

1 minute ago, Kloaked said:

Did you happen to watch the video with him and Louis Rossmann? I know not everyone is a fan of Louis for valid reasons, but Linus looked and sounded so out of place in Louis' shop.

Well he basically went there to learn how to use the oven. I know their old set was a kitchen, but that doesn't mean anything. 

[Bouzoo]

2 minutes ago, SSL said:

 

A lot of these tech YT channels are in kind of a weird gray area between people like Marques Brownlee, who make vacuous, consumer-oriented drivel, and the likes of PCPer. In other words, they want to have their cake and eat it too.

It's not like huge launches like this happen everyday, or every few months even. And they don't really review so many graphics cards as well for them to take such a huge portion of the time.

[SSL]

Just now, Bouzoo said:

It's not like huge launches like this happen everyday, or every few months even. And they don't really review so many graphics cards as well for them to take such a huge portion of the time.

 

It's an excuse for sub-par quality, is that that is. Basically, it just makes my point. As you say, they are not in the business of technical benchmarks, so why half-ass them? Stick to their strengths of outreach and communication and rely on better-equipped primary sources like Anand and PCPer to do the legwork. Remember that Linus called his company "Linus Media Group", not "Linus Tech Group".