Is 97th percentile equal to 3% lows?

[Ezio Auditore]

I am referring to fps numbers in games to be specific.

[Ezio Auditore]

2 minutes ago, xhantus404 said:

It's another type of "average" fps, where you exclude the situations with the best performance, as I understand it.

But what does that precisely mean?

[Majestic]

5 minutes ago, Ezio Auditore said:

But what does that precisely mean?

It's the distribution of the collected data. So taking the 97th percentile means you're excluding the lower 3. Maybe read up on 'normal distributions'.

[Lurick]

97th percentile usually means the upper, so 3rd percentile would be the lower.

[Majestic]

Just now, xhantus404 said:

I think they do it the other way around, as the 97th percentile averages are lower than the total averages.

Not sure what the context is. Is it 97th percentile frametimes? Because the lower 3 would be higher numbers. So excluding those would result in lower frametimes.

[Ezio Auditore]

1 minute ago, Majestic said:

Not sure what the context is. Is it 97th percentile frametimes? Because the lower 3 would be higher numbers. So excluding those would result in lower frametimes.

watch benchmarks on ltt ryzen 5 review.

 

[Majestic]

Then it's probably a normal distr. 97th percentile (so the really low and really high is not taken into account).

[Ezio Auditore]

1 minute ago, Majestic said:

Then it's probably a normal distr. 97th percentile (so the really low and really high is not taken into account).

so they took 3% lows and highs removed them and measured the fps?

[Thony]

It means they excluded any significant FPS drops. 

[Ezio Auditore]

Just now, Thony said:

It means they excluded any significant FPS drops. 

Then how is it lower than the average?

[Ezio Auditore]

Just now, xhantus404 said:

It's not entirely clear from how they worded it. Could be that, or they only removed the highest ones. For a performance review, that would be a sensible approach as you probably do not care if your game peaks at 150fps in some moments, but you would definetly care if it dips into the 10s or so.

so if it is that is it same as 3% lows?

[Majestic]

5 minutes ago, Ezio Auditore said:

so they took 3% lows and highs removed them and measured the fps?

No, hence I said google 'normal distribution'.

 

[Ezio Auditore]

2 minutes ago, Majestic said:

No, hence I said google 'normal distribution'.

 

I didn't know about this.Am I dumb or should LTT have explained this? (you can say I am dumb if that is the case)

[Majestic]

That said, looking at LTT video's for CPU benchmarks is kinda like watching CNN for news.

 

Just now, Ezio Auditore said:

I didn't know about this.Am I dumb or should LTT have explained this? (you can say I am dumb if that is the case)

well maybe at some point they did, dunno. It's fairly standard statistics.

[Ezio Auditore]

1 minute ago, xhantus404 said:

They used to have min fps, they replaced it with 97th percentile. Judging by how the whatever bars look like next to the total averages, the answer to your original question would probably be "yes".

Maybe Linus can weigh in on that at some point and explain their new testing methodology for that. The wording leaves some room for interpretation, common sense applied to what they used to show in the graphs suggests it's a broader view on how low the fps can go - not at one single point, but during bad moments.

Think so. I'll mention @LinusTech in this post.Let's see if he replies.

BTW Thanks.

[Yoinkerman]

The way xth percentile is supposed to work, is that 97th would be the 97th/100 where 1 is low and 100 is high.

 

I guess you could plot the entire fps range and then pick arbitrary points and compare them to the range and get a percentile from there as well.

 

Whether or not LTT did that, who knows, but it doesn't look like they did either of them.

[Thony]

1 hour ago, Ezio Auditore said:

Then how is it lower than the average?

It also excludes high fps spikes. So its the most stable fps through the game that you get most of the time. 

[Dorrej]

Longtime lurker, but I just registered to clear this up because this thread is full of misinformation. 

 

I think part of the confusion is because LTT misused percentile... A percentile is a value below which a percentage of values in a dataset fall. The 97th percentile is the value that is greater than 97% of the data. I'm not sure what value ltt had in their video, but it is not the 97th percentile because that would be greater than the average. Maybe they meant 3rd percentile, which would be below 97% of the data. 

[LinusTech]

It's derived from average frametimes.

 

97% of the frames are drawn at a speed that would translate to an FPS higher than what we are reporting.

 

So in summary it's our new way of measuring minimum FPS, designed to ignore random dips from things like loading or what-have-you.

 

A video explaining the new methodology is probably in order at some point, but with all the new hardware launches and all the other crap that we always have going on around here, it's been a lower priority.

[Dorrej]

1 hour ago, LinusTech said:

It's derived from average frametimes.

 

97% of the frames are drawn at a speed that would translate to an FPS higher than what we are reporting.

 

So in summary it's our new way of measuring minimum FPS, designed to ignore random dips from things like loading or what-have-you.

 

A video explaining the new methodology is probably in order at some point, but with all the new hardware launches and all the other crap that we always have going on around here, it's been a lower priority.

Then it is 3rd percentile. A 97th percentile frametime would convert to 3rd percentile fps. 

[Ezio Auditore]

19 hours ago, Dorrej said:

Longtime lurker, but I just registered to clear this up because this thread is full of misinformation. 

 

I think part of the confusion is because LTT misused percentile... A percentile is a value below which a percentage of values in a dataset fall. The 97th percentile is the value that is greater than 97% of the data. I'm not sure what value ltt had in their video, but it is not the 97th percentile because that would be greater than the average. Maybe they meant 3rd percentile, which would be below 97% of the data. 

so third percentile is not equal to 3% lows..........right?

[Ezio Auditore]

16 hours ago, LinusTech said:

It's derived from average frametimes.

 

97% of the frames are drawn at a speed that would translate to an FPS higher than what we are reporting.

 

So in summary it's our new way of measuring minimum FPS, designed to ignore random dips from things like loading or what-have-you.

 

A video explaining the new methodology is probably in order at some point, but with all the new hardware launches and all the other crap that we always have going on around here, it's been a lower priority.

So you have converted those frametimes to their fps values, right?

Also I just saw Wendell's video regarding the issue of frametime dips and what he concluded off of it was that the Intel systems had dips in GTA V like 100ms whereas others were belows 16ms and Ryzen didn't have such huge dips although it had higher frametimes; Ryzen experience was more smoother as intel stutters were noticeable.So I feel there's going to be more to testing these cpus than just these numbers.

Thanks for the reply BTW.

[Sakkura]

22 minutes ago, Ezio Auditore said:

So you have converted those frametimes to their fps values, right?

Also I just saw Wendell's video regarding the issue of frametime dips and what he concluded off of it was that the Intel systems had dips in GTA V like 100ms whereas others were belows 16ms and Ryzen didn't have such huge dips although it had higher frametimes; Ryzen experience was more smoother as intel stutters were noticeable.So I feel there's going to be more to testing these cpus than just these numbers.

Thanks for the reply BTW.

FPS and frametime are simply the inverse of each other. Frames per time vs. time per frame.

 

It's not well worded in this situation. They should have called it 97th percentile frametime. But many viewers are more familiar with FPS than frametime.

[programster]

On 17/04/2017 at 7:22 PM, LinusTech said:

It's derived from average frametimes.

 

97% of the frames are drawn at a speed that would translate to an FPS higher than what we are reporting.

 

So in summary it's our new way of measuring minimum FPS, designed to ignore random dips from things like loading or what-have-you.

 

A video explaining the new methodology is probably in order at some point, but with all the new hardware launches and all the other crap that we always have going on around here, it's been a lower priority.

I would absolutely love such a video. One thing's for sure, it's definitely not the 97th percentile of FPS as that would tend to be much higher than the average FPS. Generally the average fps should be much closer (but not always the case) to the median of the FPS distribution (which is the 50th percentile). I had always assumed that LTT got their percentiles the wrong way round be ordering the set of readings the opposite way so that the 97th percentile was actually the 3rd percentile which would make the graphs look correct. I think that when you convert frame times to FPS and you sort the numbers the correct way around for percentiles, this will end up being the case.

 

There are two ways you can word the 3rd percentile.

97% of the frames were higher than this speed (a good minimum that removes lower extremities). 

3% of the frames were lower than this speed..

[Dylanc1500]

4 hours ago, programster said:

I would absolutely love such a video. One thing's for sure, it's definitely not the 97th percentile of FPS as that would tend to be much higher than the average FPS. Generally the average fps should be much closer (but not always the case) to the median of the FPS distribution (which is the 50th percentile). I had always assumed that LTT got their percentiles the wrong way round be ordering the set of readings the opposite way so that the 97th percentile was actually the 3rd percentile which would make the graphs look correct. I think that when you convert frame times to FPS and you sort the numbers the correct way around for percentiles, this will end up being the case.

 

There are two ways you can word the 3rd percentile.

97% of the frames were higher than this speed (a good minimum that removes lower extremities). 

3% of the frames were lower than this speed..

This is why when you are speaking about percentiles you are supposed to specify upper or lower percentiles. In this case they are referring to the frames in the lower third percentile. Otherwise it does cause confusion due to not know what end of the spectrum you are talking about.