LMG's obsession with misleading graphs

Godlygamer23 · August 15, 2017

27 minutes ago, BingoFishy said:

That's OP's point. If the scores are so similar it's hard to tell, then it doesn't matter so much.

They're not misleading graphs though - the only person being mislead is the viewer that doesn't take into account the values for the graphs. The information may very well be fully accurate.

Sauron · August 15, 2017

The graphs are correct as long as the values on the y axis are correct. I think we should take the time to read 5 numbers on a graph if we really care about what is being analyzed.

On 14/8/2017 at 5:03 PM, Enderman said:

That's not the point of a graph.

A graph is supposed to give an accurate representation of the data without having to read the numbers.

AKA

if line A is twice as long as line B, that indicates that A is performing twice as well as B.

Not that A is 62fps and B is 61fps and the graph starts at 60.

That's called misleading data.

Honestly if you're not going to read the numbers, especially when it comes to framerate, this information is completely useless - if something is twice as fast as something else it could be doing 120fps against 60 or 2fps against 1, which I would argue makes quite a bit of a difference. On top of that you're asking for the graph to be clear at first sight and then you complain when the graph is zoomed in to show the difference better... the "data" here are the numbers, not the graph. Ideally, the editor tries to keep the interval as balanced as possible between being able to tell the difference easily and not going too far with it.

And stuff like firestrike results, which are in the thousands, will seem completely identical if you don't truncate the graph, making it much harder to evaluate the difference, both at a glance and upon actually reading the numbers.

With that said, I agree that the line "value" graphs are unclear (which does not mean they are misleading) and should be explained better.

SpaceGhostC2C · August 15, 2017

On 8/14/2017 at 11:13 PM, mariushm said:

They're stupid.

There's nothing to tell you how exactly those graphs are determined...

(...)

On 8/14/2017 at 11:33 PM, Imbellis said:

This one is totally on me. I meant to specify bar graphs. I agree with you about the last four - Too many un-explained aspects(not accounted for) / not simple enough to flash by one every four seconds.

Note: I'm going to edit the previous post to specify it. Thanks.

Unfortunately, they explained how they build those "value graphs", and bad news: they are meaningless. Don't look for an explanation on those other factors because there are no other factors. They take the performance results and then aplpy a formula which does not reflect value in any meaningful way and obtain those curves. There are no underlying builds at each of those pricepoints. Instead, they just assume you get the same performance of the testbench, but somehow paid less or more for it. It doesn't matter what the money was for. In fact, it could even be that you paid more for the component under review, and that could mean said component is better value. Yes, as you read: accordign to those plots, a component could become a better value the more you pay for it...

Well, that's actually a bit facetious so let me explain: what they do in practice with that formula is simply tilting the comparison towards pure price comparison (cheapest wins) or pure performance comparison (highest performer wins). Hence, as you move to the right, you will eventually pick the better performer regarding of pricing. So, let's say you are comparing two GPUs and one is 1% better than the other. Regardless the pricing, as you move to the right (as you increase the cost of the testbecnh in a performance-irrelevant manner) the 1% fastest GPU will be "better value". It can cost half or twice, ti doesn't matter: eventually, at some "total cost" it will just be better. Because.

Similarly, if you move to the left, there will be a point at which the cheapest is better, no matter the performance difference. It can be 80% slower at 1% lower price, at some x it will be deemed "better value".

And because performance numbers come from the testbench, the assumption is that at all "total cost" points the component in question is the limiting factor...

I agree with the others in that truncating a graph isn't in itself misleading (it is much more misleading when the say "in game X, component A takes the crown" and you are seeing a 1FPS difference...). These value plots, however, are just meaningless, and the use of "diminishing returns" is indeed misleading as the curvatures they show have absolutely nothing to do with any returns to anything.

Enderman · August 16, 2017

12 hours ago, Sauron said:

The graphs are correct as long as the values on the y axis are correct. I think we should take the time to read 5 numbers on a graph if we really care about what is being analyzed.

Yes they are correct.

But they are also misleading.

This is a bad way to represent data.

If you want to show a small difference between two bars, you typically show the non-truncated graph, then add a second graph zoomed in on a portion, or make it very clear that the main graph is truncated by adding a break or truncation mark.

You will notice that in all cases, the graph starts at 0.

The break makes it obvious that the graph is not representing the total range of the values.

PS- the point of making graphs is for the people who do not take the time to read 5 numbers. If you want to teach everyone to read every number on every graph in order to prevent misinformation, then great, become an elementary school teacher like the one that taught me to find misleading information in graphs.