FCAT testing Ryzen?

[CostcoSamples]

Has anyone seen any tech sites that have tested Ryzen vs Intel using FCAT?  It's expensive and a lot of work, most reviewers don't do it.  A lot of people are saying Ryzen feels smoother, even though the benchmarks don't always show it.  If these claims hold any truth, we should see them with FCAT.  

 

For those who don't know, FCAT (Frame Capture Analysis Tool) is a cool tech that Nvidia developed in 2013 to show how much better SLI is compared to Crossfire (which prompted AMD to fix CF).  The basic principle is that instead of measuring frames at the game engine (such as FRAPS or MSI Afterburner), the GPU video out is split and captured by a second system which then analyses the recorded frames.

 

The great thing about FCAT is that a lot can happen between the time the frame is rendered and the time it is displayed.  Some frames get dropped along the way, or can get held up for various reasons.  FCAT allows us to measure the user experience by measuring only what is displayed on the screen.

 

There are reports that Ryzen feels smoother than 7700k, even though average FPS is lower.  If that is true, we should be able to prove it with FCAT.  I have not seen any such testing.

[Mira Yurizaki]

FCAT is only really useful for finding microstuttering or other display artifacts like tearing. It's not as valuable as a frame rate capture tool. Most of the FPS tools can detect what's going on since all they're doing is tracking buffer swaps (which indicates a frame was rendered).

 

The whole "feeling smoother" part is due to higher minimum frame rates or less frequent stuttering. Or placebo. One of the three.

[CostcoSamples]

1 hour ago, M.Yurizaki said:

FCAT is only really useful for finding microstuttering or other display artifacts like tearing. It's not as valuable as a frame rate capture tool. Most of the FPS tools can detect what's going on since all they're doing is tracking buffer swaps (which indicates a frame was rendered).

 

The whole "feeling smoother" part is due to higher minimum frame rates or less frequent stuttering. Or placebo. One of the three.

Minimum frame rates don't mean much.  Tom's Hardware does a very thorough analysis of test data, all based on their own proprietary software.  All of their data is collected via software though, and they generally don't do FCAT.  From their analysis, Ryzen does not beat 7700k in terms of 1% low, 0.1% low, and frame time distribution is generally worse - all of which generally indicates smoothness is worse on Ryzen.  Despite all of this, many have argued that Ryzen has a certain smoothness quality that does not show up in the data.  Test benches usually don't have a lot of other software that the average user will have running in the background.  So the argument goes, that since Ryzen has more cores, that extra CPU headroom means more consistent/smooth gameplay, even if average fps is lower.  I haven't seen any data proving it.  FCAT could prove or disprove the claim.

[Mira Yurizaki]

2 minutes ago, CostcoSamples said:

Minimum frame rates don't mean much. 

So you're fine with your game dropping to 1 FPS every time it stutters rather than say 55FPS out of an average of 60?

[Mira Yurizaki]

I would edit, but those tend to be not read in time.

 

4 minutes ago, CostcoSamples said:

Test benches usually don't have a lot of other software that the average user will have running in the background.

The problem with this condition is what do you consider "software that the average user will having running in the background"? This is subjective at best. I don't have a couple dozen Chrome tabs open (I don't even use Chrome) or whatever going on in the background. I try to minimize what I have running in the background for the sake of maximizing the potential of any high performance application I run. It's also harder to isolate discrepancies. If there's a hiccup in the test, is it because the game was poorly developed or was it that one process running in the background stealing cycles for some reason?

 

It's bad enough there's no standardized hardware platform for reviewers to use. Now you want them to run a software configuration of what they think the "average user" will have running? Let's not introduce even more variables in what tends to be a heated issue on repeatable testing.

[CostcoSamples]

1 hour ago, M.Yurizaki said:

So you're fine with your game dropping to 1 FPS every time it stutters rather than say 55FPS out of an average of 60?

Educate yourself.  Minimum fps is a bad way to measure dips and stutters.  Many articles have been published on the topic.  Here's one from the Tech Report.

[CostcoSamples]

1 hour ago, M.Yurizaki said:

I would edit, but those tend to be not read in time.

 

The problem with this condition is what do you consider "software that the average user will having running in the background"? This is subjective at best. I don't have a couple dozen Chrome tabs open (I don't even use Chrome) or whatever going on in the background. I try to minimize what I have running in the background for the sake of maximizing the potential of any high performance application I run. It's also harder to isolate discrepancies. If there's a hiccup in the test, is it because the game was poorly developed or was it that one process running in the background stealing cycles for some reason?

 

It's bad enough there's no standardized hardware platform for reviewers to use. Now you want them to run a software configuration of what they think the "average user" will have running? Let's not introduce even more variables in what tends to be a heated issue on repeatable testing.

Measuring performance is a hard thing to do.  In order for results to be comparable, variables must be controlled.  Controlling variables necessarily omits other factors, which affects results.  I think a lot of reviewers are lazy in their analysis (LTT included), and at times this leads to false conclusions.  FCAT is a powerful tool that most reviewers ignore because of the cost and time needed to run it.  

 

Anyway I don't know why you are arguing against me.  All I'm saying is that I'd like to see more analysis with Ryzen to see if there is merit in the claims that it "feels smoother".