AMD Ryzen 9 review discrepancies

[Melodist]

Alright, so as I was checking out different reviews on the new 3900x, I've come across anandtech's review:

 

 

I would like to know how Gamer's Nexus' score can be so different from the review posted above:

 

 

I mean, it's pretty much across the board where other reviewers see a wider gap between the 9900k and 3900x. I wonder if that has to do with the 3900x bottlenecking the 2080 Ti or simply because they use operating systems without a clean install?

 

 

 

 

Can you please tell me how these result discrepancies came about? Is anandtech paid by AMD or are most other reviewers setting up the system on an Intel based OS with redundancies?

[AngryBeaver]

2 minutes ago, Melodist said:

Alright, so as I was checking out different reviews on the new 3900x, I've come across anandtech's review:

 

 

I would like to know how Gamer's Nexus' score can be so different from the review posted above:

 

 

I mean, it's pretty much across the board where other reviewers see a wider gap between the 9900k and 3900x. I wonder if that has to do with the 3900x bottlenecking the 2080 Ti or simply because they use operating systems without a clean install?

 

 

 

Can you please tell me how these result discrepancies came about? Is anandtech paid by AMD or are most other reviewers setting up the system on an Intel based OS with redundancies?

Well first off you are comparing 720p vs 1080p results... that being said I would expect to see the gap be more prominent on the 720p. So it just comes down to different testing mythologies and without having their complete setups on hand it is hard for me to tell you why the differences are there.

[Melodist]

Just now, AngryBeaver said:

Well first off you are comparing 720p vs 1080p results... that being said I would expect to see the gap be more prominent on the 720p. So it just comes down to different testing mythologies and without having their complete setups on hand it is hard for me to tell you why the differences are there.

to show an exaggeration of the other reviewer's much different results, I can put up the 1080 ones too.

[5x5]

Just now, Melodist said:

to show an exaggeration of the other reviewer's much different results, I can put up the 1080 ones too.

Different testing methodoly and probably different presets on the advanced settings. 

[Melodist]

4 minutes ago, 5x5 said:

Different testing methodoly and probably different presets on the advanced settings. 

That affect the CPU's performance that much? I don't think so. It's like almost 20 percent slower vs en par.

[5x5]

2 minutes ago, Melodist said:

That affect the CPU's performance that much? I don't think so. It's like almost 20 percent slower vs en par.

Nobody is forcing you to believe reality

[AngryBeaver]

1 minute ago, Melodist said:

That affect the CPU's performance that much? I don't think so. It's like almost 20 percent slower vs en par.

Well this is only 1 game. We don't know what bios settings were used. How comparable the cooling was. How comparable the motherboards were as well as other hardware.

 

So without having the full picture I cannot draw accurate conclusions. You can easily setup these tests in a way that it gimps one machine vs the other. So like I said you need to have the full picture.

[Melodist]

Just now, 5x5 said:

Nobody is forcing you to believe reality

So you aren't curious why it is so varying across several review outlets while Intel's are mostly leveled everywhere?

[Melodist]

Just now, AngryBeaver said:

Well this is only 1 game. We don't know what bios settings were used. How comparable the cooling was. How comparable the motherboards were as well as other hardware.

 

So without having the full picture I cannot draw accurate conclusions. You can easily setup these tests in a way that it gimps one machine vs the other. So like I said you need to have the full picture.

The latter is exactly what I mean. I think they use system ssds they hook up to a machine with pre installed games and tests for ease of use which they had set up with Intel machines.

[AngryBeaver]

Just now, Melodist said:

So you aren't curious why it is so varying across several review outlets while Intel's are mostly leveled everywhere?

New platform and a new architecture. So there can be big differences in drivers and support in windows/games. So you are looking at a platform that has been around for a long time and hasn't made huge changes in the architecture, vs a new process with a different architecture. So I can see that being an issue as well. Which means you will see AMD performance increase over the next few months as tweaks are made.

[Senzelian]

I think the most important note to take from this, is that average framerates are compared, which do not account for framerate dips, caused by slow access times from the CPU to the RAM or the storage, because of either early BIOS versions or chipsets or memory/storage configurations.

 

And then there are different driver revisions, different game settings, BIOS versions, microcode updates, game updates, OS updates, ... - and the list goes on.
 

Simply said; nothing is perfect.

[Melodist]

Yeah but making the CPU look en par vs 20% slower is a huge selling / not selling point for people deciding on their purchase, don't you agree?

[AngryBeaver]

1 minute ago, Melodist said:

Yeah but making the CPU look en par vs 20% slower is a huge selling / not selling point for people deciding on their purchase, don't you agree?

Well if they are posting bad review/benchmark/comparisons then don't use them anymore. That is how you handle people posting bad information you just don't use it as a source anymore.

[Senzelian]

1 minute ago, Melodist said:

Yeah but making the CPU look en par vs 20% slower is a huge selling / not selling point for people deciding on their purchase, don't you agree?

Of course, it is.
Now either people are smart and check multiple reviews and make a smart buying decision or they don't. Either way, they get what they deserve. Simple as that.

[Mister Woof]

"high detail streaming"

[Mira Yurizaki]

2 hours ago, Melodist said:

Can you please tell me how these result discrepancies came about? Is anandtech paid by AMD or are most other reviewers setting up the system on an Intel based OS with redundancies?

To start, there's other graphs in their reviews that aren't laid out at once to save space. You're supposed to click on the table below the graph being shown to see the data for the other statistics they gathered:

 

The second thing is they use different settings than Gamer's Nexus.

 

 

So even if you pulled up the "Low" category, it's still easier on the GPU than Gamer's Nexus's settings they used.

 

tl;dr, you have to look at the reviewer's testing methodology and settings before you compare what they did. If they use incompatible settings, then you can't compare the two directly, but it's still useful as a soft comparison between one GPU and another.

[Mister Woof]

Anandtech has done poorly controlled tests in the past, and they've been called out on it before.

 

Tech Jesus is all you need.

[St.Nick]

Tech Jesus also pointed out the press review drivers they got for the new chips were garbage. He was using 3200 mhz memmory, Yes it's CL14 and that helps but we all know Ryzen loves it's fast memmory speeds which might also be playing a part here. And since his initial review a lot of things about zen2 requiring some pretty specific set up to overclock has become apparent. But yeah uhh, for purely gaming, Intel just straight up still wins and the 3900x actually just doesn't do as well. Mostly if your looking at a value proposition you just want the 3600 for gaming anyway. But the hard truth here is that zen2 just doesn't win against intel if all you do is game and thats ok, it should be a lot closer to the 9900k and beat the 9700k these days if you properly optimize it but the 3900x is just not really gonna touch the i9 on gaming performance. 

[Chiyawa]

1 hour ago, St.Nick said:

Tech Jesus also pointed out the press review drivers they got for the new chips were garbage. He was using 3200 mhz memmory, Yes it's CL14 and that helps but we all know Ryzen loves it's fast memmory speeds which might also be playing a part here. And since his initial review a lot of things about zen2 requiring some pretty specific set up to overclock has become apparent. But yeah uhh, for purely gaming, Intel just straight up still wins and the 3900x actually just doesn't do as well. Mostly if your looking at a value proposition you just want the 3600 for gaming anyway. But the hard truth here is that zen2 just doesn't win against intel if all you do is game and thats ok, it should be a lot closer to the 9900k and beat the 9700k these days if you properly optimize it but the 3900x is just not really gonna touch the i9 on gaming performance.

Agree. Actually, I don't think GTA V used a lot of threads or cores to run.

 

6 hours ago, Melodist said:

Can you please tell me how these result discrepancies came about? Is anandtech paid by AMD or are most other reviewers setting up the system on an Intel based OS with redundancies?

That's not likely. You should expect about the same frame rate with a similar setup. The thing is, some of this "games" usually comes with optimisation codes for certain hardware. Take Mass Effect 2 and Mass Effect 3 as an example. Mass Effect 2 surprisingly has better graphic for me, and surely it was optimised for both ATi (or AMD) and nVidia graphic cards. However, for Mass Effect 3, if you use ATi or AMD cards, you'll notice a huge differences compared to nVidia cards. This is because Mass Effect 3 is poorly optimised for ATi or AMD cards (probably due to limited time and many games at that time favour nVidia graphics).

 

Whatever benchmark results posted on the internet, take it with a grain of salts. Because these benchmarks usually perform with some kind of 'tweaks'. If they want to list down everything, I think it would be too tedious to do so. Just imagine, a CPU runs at 1.135v and 1.141v significantly impacts the benchmark, even though they run at the same speed most of the time.

[Melodist]

9 hours ago, St.Nick said:

Tech Jesus also pointed out the press review drivers they got for the new chips were garbage. He was using 3200 mhz memmory, Yes it's CL14 and that helps but we all know Ryzen loves it's fast memmory speeds which might also be playing a part here. And since his initial review a lot of things about zen2 requiring some pretty specific set up to overclock has become apparent. But yeah uhh, for purely gaming, Intel just straight up still wins and the 3900x actually just doesn't do as well. Mostly if your looking at a value proposition you just want the 3600 for gaming anyway. But the hard truth here is that zen2 just doesn't win against intel if all you do is game and thats ok, it should be a lot closer to the 9900k and beat the 9700k these days if you properly optimize it but the 3900x is just not really gonna touch the i9 on gaming performance. 

It's not just about gaming for me, DAWs favor CPUs which are also good in gaming, means have a great single Core performance. That's why Gaming Benchmarks are a good indicator for me too.

[Melodist]

I'd actually prefer the 9900k in terms of latency for especially audio applications but having all your IO limited to just 4x PCIe is a bitter Apple to bite.

[AlpacaFan]

I have noticed it too and I do think that many reviewers are sloppy and/or biased. There are two things which I noticed -- I don't think all reviewers are using the latest BIOS updates AND many of them run on an older version of Windows 10. Windows 1903 brought scheduler improvements specifically for Ryzen and I saw some reviewers running on older versions of Windows 10. The scheduler can have a huge impact because the NUMA characteristics of the new chip are different.

 

On the other hand -- a lot of people knock synthetic benchmarks - but I view them in this light - they give an indication of what performance is theoretically available on the table which can be exploited by the OS, the drivers and the applications and games. Sure launch day performance is a good indicator, but I see a lot of performance left on the table which can still be tapped. Gaming might be single thread heavy at the moment, but that will change especially because the next console generations (Xbox and PS5) will be based on Ryzen and code will be shared between the consoles and the PCs -- developers have no choice but to find a way to exploit the potential. As the older OpenGL/DirectX 11 APIs become less important developers will start to use multithreaded code with DX12 and Vulkan. Give it 6 to 9 months and we will see better and better "real world" benchmarks for Ryzen mark my word. So my point is "synthetic benchmarks" are under-appreciated but "real world benchmarks" are probably over appreciated (especially with games). Depends on your use-case I guess.