Anandtech's Initial remarks about their unusual Ryzen 2 benchmarks

[App4that]

6 minutes ago, Brooksie359 said:

Hey if you had results that were different than other had been up for long periods of time double checking and couldn't figure out what it is you might just end up thinking the only possible explanation is that it was because of the meltdown patch. I mean you are picking them apart for making a mistake that in all honesty most of us wouldn't have been able to diagnose and if we did it would take a long time. You act like a they aren't people but benchmarking machines. 

I'm holding them accountable for the job they chose to do. 

 

When I realize something isn't right, I take responsibility for it. I don't ship it. Anandtech knew something wasn't right, I'm giving them credit that something wasn't right and not saying the did it on purpose. THAT, would be a tin foil hat. 

 

Why it's such a big deal, is evident in this topic. Publishing AMD biased reviews, feeds the issue of AMD not being able to do anything wrong. If a reviewer errors on the side of Intel, they're a shill. If they error on the side of AMD, they're truthful. 

 

That's a problem. 

[Brooksie359]

24 minutes ago, App4that said:

I'm holding them accountable for the job they chose to do. 

 

When I realize something isn't right, I take responsibility for it. I don't ship it. Anandtech knew something wasn't right, I'm giving them credit that something wasn't right and not saying the did it on purpose. THAT, would be a tin foil hat. 

 

Why it's such a big deal, is evident in this topic. Publishing AMD biased reviews, feeds the issue of AMD not being able to do anything wrong. If a reviewer errors on the side of Intel, they're a shill. If they error on the side of AMD, they're truthful. 

 

That's a problem. 

If someone had benchmarks that gave Intel more of an advantage than they should because of some small settings issue they werent aware of and then proceeded to investigate to find the root cause and then fix their benchmarks then yeah I would not call them out for it nor call them a shill. Again you act as if because it's their job they should be infallible which is completely ridiculous. Also they didn't realize it wasn't right until after they published so the whole idea of not shipping it because you know it isn't right is kinda dumb. They thought it was the meltdown patch and didn't realized that some of the other reviewers had done the same and got different results. At that point is when they started to investigate. 

[App4that]

1 minute ago, Brooksie359 said:

If someone had benchmarks that gave Intel more of an advantage than they should because of some small settings issue they werent aware of and then proceeded to investigate to find the root cause and then fix their benchmarks then yeah I would not call them out for it nor call them a shill. Again you act as if because it's their job they should be infallible which is completely ridiculous. Also they didn't realize it wasn't right until after they published so the whole idea of not shipping it because you know it isn't right is kinda dumb. They thought it was the meltdown patch and didn't realized that some of the other reviewers had done the same and got different results. At that point is when they started to investigate. 

Simply can't be true that Anandtech were unaware of the issue in their test, or that they didn't know the security patches had no impact on performance.

 

A well informed amateur knows single core performance makes the largest impact in games, they also know the security patches make no difference to performance in games. Anandtech are not amatuers, they're supposed to be professionals. As professionals they knew, both their tests being off, and that any security patch was not the cause. 

 

 

[Brooksie359]

28 minutes ago, App4that said:

Simply can't be true that Anandtech were unaware of the issue in their test, or that they didn't know the security patches had no impact on performance.

 

A well informed amateur knows single core performance makes the largest impact in games, they also know the security patches make no difference to performance in games. Anandtech are not amatuers, they're supposed to be professionals. As professionals they knew, both their tests being off, and that any security patch was not the cause. 

 

 

The meltdown patch was supposed to increase latency of the cpu memory which would have a huge impact on performance. Look at the gains ryzen gets with faster memory reducing cache memory latency. So I think it's easy for you to judge others when you don't know the whole picture. Based on their actions they had no malicious intent and they did alot to fix the issue. You just continue to judge them even after they did so much to fix their mistake. I mean generally when someone makes a mistake and they do alot on their part to make up for it most people realize they are human and made a mistake and should be treated as such. I mean you can till be mad at them for their supposed knowingly misleading review if you want.

[App4that]

Just now, Brooksie359 said:

The meltdown patch was supposed to increase latency of the cpu memory which would have a huge impact on performance. Look at the gains ryzen gets with faster memory reducing cache memory latency. So I think it's easy for you to judge others when you don't know the whole picture. Based on their actions they had no malicious intent and they did alot to fix the issue. You just continue to judge them even after they did so much to fix their mistake. I mean generally when someone makes a mistake and they do alot on their part to make up for it most people realize they are human and made a mistake and should be treated as such. I mean you can till be mad at them for their supposed knowingly misleading review if you want.

Mistakes happen, do you know the diference between an amatuer, and a professional? Hint, nothing to do with money.

 

None of the security patches affect gaming, none. Been beat to death. I've personally tested them, even the Ryzen varients. It effects some aplications, but not gaming.

 

The fact that the security patches have been tested into glue, yet people STILL talk about the security patches, is proving my point of a bias. Anandtech made the choice to feed the misinformation around the security patches, which is my point.

[Taf the Ghost]

24 minutes ago, App4that said:

Mistakes happen, do you know the diference between an amatuer, and a professional? Hint, nothing to do with money.

 

None of the security patches affect gaming, none. Been beat to death. I've personally tested them, even the Ryzen varients. It effects some aplications, but not gaming.

 

The fact that the security patches have been tested into glue, yet people STILL talk about the security patches, is proving my point of a bias. Anandtech made the choice to feed the misinformation around the security patches, which is my point.

 

Anandtech's results were "odd" in several instances and it was noted quite early, with Ryan even commenting on r/AMD about they were looking into it. The productivity and some of the game results looked within the system-to-system variance but with a few outliers. (The Rocket League DX9 results are a driver-side issue.)  You seem to be obsessing about what is really just an interesting technical insight into the way modern games respond to aspects of the system structures.

[App4that]

9 minutes ago, Taf the Ghost said:

 

Anandtech's results were "odd" in several instances and it was noted quite early, with Ryan even commenting on r/AMD about they were looking into it. The productivity and some of the game results looked within the system-to-system variance but with a few outliers. (The Rocket League DX9 results are a driver-side issue.)  You seem to be obsessing about what is really just an interesting technical insight into the way modern games respond to aspects of the system structures.

I've over explained my position, my "obsessing" has to do with how Anandtech delt with the issue.

 

The irony here is I'm not alone, even Jim over at AdoredTV noticed Anandtechs response was BS. Of course Jim was nicer than that, but he's a nice guy.

 

At this point, I'm just floored at how biased people are, and how they never learn that that bias is how companies like Intel came to behave in a way that made people biased for AMD. 

[Brooksie359]

18 minutes ago, App4that said:

I've over explained my position, my "obsessing" has to do with how Anandtech delt with the issue.

 

The irony here is I'm not alone, even Jim over at AdoredTV noticed Anandtechs response was BS. Of course Jim was nicer than that, but he's a nice guy.

 

At this point, I'm just floored at how biased people are, and how they never learn that that bias is how companies like Intel came to behave in a way that made people biased for AMD. 

It's not biased to accept when someone makes a mistake and then fixes it. From what I recall they even said that they thought it was do to the newer patches for specter and meltdown. Not the ones that had been tested alot previously. 

[App4that]

2 minutes ago, Brooksie359 said:

It's not biased to accept when someone makes a mistake and then fixes it. From what I recall they even said that they thought it was do to the newer patches for specter and meltdown. Not the ones that had been tested alot previously. 

Everyone makes mistakes, it's how we deal with them that tells the story.

 

They lied. No security patch impacts gaming performance, I've tested them all, both Intel and AMD. 

[Brooksie359]

Just now, App4that said:

Everyone makes mistakes, it's how we deal with them that tells the story.

 

They lied. No security patch impacts gaming performance, I've tested them all, both Intel and AMD. 

You test 2 cpus with the latest version of the patches that you have yet to test with. You get results that are lower than before and you rerun the test multiple times and get the same result. That would make it seem like the patch was the cause. I mean it's dumb to say that they should have know better when it most cases it would be normal to come to that conclusion. Again you act like the newer patches for meltdown could never negatively affect performance which is a huge presumption that they likely didn't make because they had data that contradicted that. If I had to go based off a presumption or data that I had collected that was repeatable I would go based off the data. That's what they did and that's how they thought it was the new patch. I would never blame someone or get mad at someone for conducting benchmarks and retesting to make sure they are consistent and then come up with a conclusion based on those results.i would even argue that you might be biased towards Intel if you think it's out of the realm of possibilities that AMD had better game benchmarks because of a Windows patch that increased cache latency on Intel cpus.

[App4that]

2 minutes ago, Brooksie359 said:

You test 2 cpus with the latest version of the patches that you have yet to test with. You get results that are lower than before and you rerun the test multiple times and get the same result. That would make it seem like the patch was the cause. I mean it's dumb to say that they should have know better when it most cases it would be normal to come to that conclusion. Again you act like the newer patches for meltdown could never negatively affect performance which is a huge presumption that they likely didn't make because they had data that contradicted that. If I had to go based off a presumption or data that I had collected that was repeatable I would go based off the data. That's what they did and that's how they thought it was the new patch. I would never blame someone or get mad at someone for conducting benchmarks and retesting to make sure they are consistent and then come up with a conclusion based on those results.i would even argue that you might be biased towards Intel if you think it's out of the realm of possibilities that AMD had better game benchmarks because of a Windows patch that increased cache latency on Intel cpus.

The patches have been tested, and were tested before the Ryzen 2 launch, yes the latest. Check the dates of tests. 

 

It's common knowledge games benefit from single core performance, over multicore. Intel had a clear advantage in single core performance, in their testing. 

 

Dude

[Brooksie359]

6 minutes ago, App4that said:

The patches have been tested, and were tested before the Ryzen 2 launch, yes the latest. Check the dates of tests. 

 

It's common knowledge games benefit from single core performance, over multicore. Intel had a clear advantage in single core performance, in their testing. 

 

Dude

show me the test that had used the intel microcode updates that were released by windows in april. to my knowledge nobody tested that as most were busy wit getting ready for the ryzen 2 testing. I could be wrong but I have yet to see any proof to the contrary. please show me these benchmarks because I cant find them which leads me to believe they don't exist.

[WMGroomAK]

39 minutes ago, Brooksie359 said:

You test 2 cpus with the latest version of the patches that you have yet to test with. You get results that are lower than before and you rerun the test multiple times and get the same result.

Based on some of the information that they compiled, it would appear that the Spectre and Meltdown patches do have an impact on any software that relies on the HPET for timing...

Quote

HPET, by the way it is invoked, is programmed by a memory mapped IO window through the ACPI into the circuit found on the chipset. Accessing it is very much an IO command, and one of the types of commands that fall under the realm of those affected by the Spectre and Meltdown patches. This would imply that any software that required HPET access (or all timing software if HPET is forced) would have the performance reduced even further when these patches are applied, further compounding the issue.

General idea I think that Anandtech was trying to do on the original testing methodology was to ensure that all the benchmark suites were using as close to the same method of calculating time as possible across platforms.  

[Misanthrope]

I am honestly not satisfied with this answer: How can they possibly assume that intel suffered a major, unprecedented performance regression and still go live with the results first then publish this explanation later? Wouldn't a professional and thorough outlet like theirs also test the older Ryzen chips and see "Ok this results are also way out of line with our own previous results something obviously happened and it obviously is not replicated by anybody else"?

 

To me this probably accomplished what they wanted: a favorable AMD review that would get raving lunatics on their side no matter what because they just want results to confirm their bias. Doesn't matter if they corrected it now, they already got tons of clicks and attention for it.

 

Either way they're too stupid or too interested in controversy for me to ever trust them again.

[App4that]

37 minutes ago, Brooksie359 said:

show me the test that had used the intel microcode updates that were released by windows in april. to my knowledge nobody tested that as most were busy wit getting ready for the ryzen 2 testing. I could be wrong but I have yet to see any proof to the contrary. please show me these benchmarks because I cant find them which leads me to believe they don't exist.

https://techreport.com/review/33299/recent-pcs-have-little-to-fear-from-intel-spectre-microcode-updates/2

 

Anandtech themselve looked at the patches, odd how they didn't visit gaming benchmarks. #odd "odd"

[Brooksie359]

13 minutes ago, App4that said:

https://techreport.com/review/33299/recent-pcs-have-little-to-fear-from-intel-spectre-microcode-updates/2

 

Anandtech themselve looked at the patches, odd how they didn't visit gaming benchmarks. #odd "odd"

I dont even see a date on that article and the fact is the last comment on it was from march meaning it would be impossible for it to have been a test of the newest microcode update released in April as march is before April....

[Trixanity]

11 hours ago, App4that said:

The facts are there, I've repeatedly posted them. Anandtech make a major mistake, one no other reviewer made. Anandtec failed to sanity check. When cornered with their benchmarks not matching any other, Anandtech went the route of blaming security patches already proven to have no impact on performance.

 

So, as the outside observer, I have two choices. Find Anandtech incompetent, or bias. 

 

Which is worse?

The facts are there and they disprove what you say. Obviously. They made a major mistake because they've used HPET because the reviewer has an overclocking background requiring him to use HPET to post valid results and besides that allowed consistent results. When cornered they said it might be the patches. At the time new patches had been released. And it seems the patches may affect HPET because of alleged increased IO load and it also showcased Intel having an impractically strict HPET implementation. In the future we'll have a full suite of non-HPET tests and it will be that way from here on out. HPET had up until that point not really been a factor. Intel was unaware because if they were they'd obviously inform each and every reviewer to stop using HPET. That's how review guides go: inform the reviewer how to best showcase the product and Intel did not disclose that. The only way to catch it was apparently to go into this storm. 

In fact they were almost publishing the review with bad AMD data through disabled PB2 but caught it just in time to re-do the benchmarks but this obviously didn't help Intel. Still there is no conspiracy against Intel because it would be a pretty damn bad conspiracy judging by the fact it was caught immediately and given other recent stories all reviews go through intense scrutiny. 

 

Your choice is to don the tinfoil. God damn it my toes are stuck.

 

Maybe we should start blaming reviewers for manually overclocking AMD CPUs when it clearly gives worse results for their OC benchmarks. In fact, the current story is that doing bclk OC and keeping specific settings can trick PB2 into running 4.4-4.5 GHz and therefore get a significant boost in many benchmarks. Although how valid that is remains to be seen because no known good source has done it and no reviewer has put it into their review. And besides that we may get back into the whole issue with bad timing which can be caused by bclk changes which can invalidate results and do we then force HPET to get some validity back? It's a can of worms but apparently if Intel isn't number one it's pretty bad because they're supposed to be number one. Always. And this is why we can't have nice things.

 

And I love the fact that you pulled the "I'm not racist, I know a black guy" defense.

 

Either way: I'd rather things like this get put forth so it can be shot down rather than reviewers remain quiet to satisfy their clients. As long as they put a disclaimer. Which got put up shortly after the review went live. Your argument is probably they should have predicted the debacle and had it beforehand but hindsight is a grand thing. Didn't they teach you at school that it's better to say something and be wrong than to say nothing at all? This is how we learn and I'm sure it was a valuable lesson for Anandtech and Ian Cutress especially. No reviewer wants to be put in this position yet you're adamant they do.

 

But really if all you're gonna do is spout gut feelings and whatnot, then you've nothing more to add. Repeating something false doesn't eventually make it right. Bring me something concrete and I'll listen. And please don't bring known biased people into this; especially not people we know are not only biased but also wrong or ignorant frequently. A broken clock is right two times a day but that doesn't really inspire confidence does it?

[mr moose]

9 hours ago, App4that said:

I've over explained my position, my "obsessing" has to do with how Anandtech delt with the issue.

 

The irony here is I'm not alone, even Jim over at AdoredTV noticed Anandtechs response was BS. Of course Jim was nicer than that, but he's a nice guy.

 

At this point, I'm just floored at how biased people are, and how they never learn that that bias is how companies like Intel came to behave in a way that made people biased for AMD. 

I'm not convinced you know what  bias is.  You are quite literally pointing to an example of scientific consensus highlighting an issue and the erroneous data being investigated and corrected. That should be applauded not made the centre of conspiracies.     It is good to see people questioning the outliers and holding them to account.   This is the exact opposite of maintaining a bias.

 

Also why have you made this about Intel and AMD?  perchance you have a bee in your bonnet about the results?

[Doobeedoo]

Yeah that was odd when I saw it the first time. But good they got it clarified what was going on. 

[Tenelia]

20 hours ago, Deus Voltage said:

Highly informative, well sourced and articulated.

 

On a side note, the gap between Intel and Ryzen seems to be closing even further. Hopefully both companies will continue to innovate and bring their best cards to the table. 

Amen people. This news post is good and should be celebrated.

[Sauron]

That was unexpected... I guess I stand corrected.

[App4that]

15 hours ago, mr moose said:

I'm not convinced you know what  bias is.  You are quite literally pointing to an example of scientific consensus highlighting an issue and the erroneous data being investigated and corrected. That should be applauded not made the centre of conspiracies.     It is good to see people questioning the outliers and holding them to account.   This is the exact opposite of maintaining a bias.

 

Also why have you made this about Intel and AMD?  perchance you have a bee in your bonnet about the results?

I am questioning the outlier, Anandtech was the outlier here. I'm met with resistance questioning the outlier, because that outlier posted results that support the AMD bias.

 

That, is why this is about Intel verses AMD. People have decided Intel is bad, and AMD is good. So nothing either do, can change that. Taht's a problem, as it give AMD a blank check, something that could take us back 5 years. 

 

16 hours ago, Trixanity said:

Didn't they teach you at school that it's better to say something and be wrong than to say nothing at all?

No, actually the opposite. Saying the wrong thing, is lying. Saying the wrong thing in science and defending it rather than admitting you got it wrong, is bad science.

 

Anandtech blamed security patches, rather than taking responsibility. Now, people are defending them that was a "good thing", so science is out the window. What a time to be alive...

[WMGroomAK]

7 hours ago, Trixanity said:

The facts are there and they disprove what you say. Obviously. They made a major mistake because they've used HPET because the reviewer has an overclocking background requiring him to use HPET to post valid results and besides that allowed consistent results. When cornered they said it might be the patches. At the time new patches had been released. And it seems the patches may affect HPET because of alleged increased IO load and it also showcased Intel having an impractically strict HPET implementation. In the future we'll have a full suite of non-HPET tests and it will be that way from here on out. HPET had up until that point not really been a factor. Intel was unaware because if they were they'd obviously inform each and every reviewer to stop using HPET.

Anandtech has an update to the article where they received a tool from Overclockers.at called TimerBench since Overclockers.at had noticed that there was an HPET bug in X299 systems...  It would appear that this HPET bug has been around since Sky Lake processors and Intel may have received information about it.

 

Quote

Matthias from Overclockers.at reached out to me and linked me to his article on how they have previously encountered the issue. The article is a nice read, and well worth clicking through:

Matthais explains how during their X299 testing, they were experiencing slowdown in their game benchmarks, and pin-pointing the problem with HPET. (We also had similar issues, and didn’t post results, but never got to the bottom of the issue.) As a result, the team over at Overclockers.at developed a tool called TimerBench in order to determine the effect of HPET. As noted, HPET has a much longer latency, but is more accurate.

In the results from overclockers.at one metric stood out: moving from Broadwell-E to Skylake-X meant that the number of theoretical peak HPET calls per second reduced from 1.4 million to 0.2 million – the latency to make a HPET call suddenly became 7x longer with Skylake-X. TimerBench, the tool developed, provides an Unreal 4.7.2 scene and measures timer calls between a system running a game, and one without

Overclockers.at Article: https://www.overclockers.at/articles/the-hpet-bug-what-it-is-and-what-it-isnt

Quote

We named it the "X299 HPET bug" as the anomaly only occured on CPUs using the X299 chipset back then. Other CPUs were not affected at the time. We contacted Intel and they didn't even bother to comment on this. When approaching an Intel engineer at a press workshop, they even knew about our bug report but denied us to show further proof. Anyway, soon after Coffee Lake S came along it became clear that all new Intel platforms are affected by the bug. We were pretty sure now that this will blow up into Intel's face at some point in the future.

...

In summary the problem is a very slow timer implementation of the High Precision Event Timer on modern platforms, that is used without care by the developers. Badly affected are Skylake X and Kaby Lake X. Impacts can also be shown on Threadripper, Coffee Lake and in some degree on Ryzen as well. It could be discussed if a slow functionality is a bug, but honestly let's just call it the "HPET bug"

 

[mr moose]

5 hours ago, App4that said:

I am questioning the outlier, Anandtech was the outlier here. I'm met with resistance questioning the outlier, because that outlier posted results that support the AMD bias.

 

That, is why this is about Intel verses AMD. People have decided Intel is bad, and AMD is good. So nothing either do, can change that. Taht's a problem, as it give AMD a blank check, something that could take us back 5 years. 

Anandtech's results were the outlier not anandtech themselves. Anandtech have behaved very much in accordance with scientific trends (I refer to science here because benchmark reviews in this industry are scientific by nature and can only be accurate if they follow a scientific method).  All that happened was they ballsed up their testing and when they found out they fixed it.  Nothing else, you are making out like they are still trying to cover up watergate.

 

Also stop talking about AMD/Intel bias and crap about blank checks.  Apart form not making any real sense it has nothing to do with the issue and only inflames the thread.

[App4that]

12 minutes ago, mr moose said:

Anandtech's results were the outlier not anandtech themselves. Anandtech have behaved very much in accordance with scientific trends (I refer to science here because benchmark reviews in this industry are scientific by nature and can only be accurate if they follow a scientific method).  All that happened was they ballsed up their testing and when they found out they fixed it.  Nothing else, you are making out like they are still trying to cover up watergate.

 

Also stop talking about AMD/Intel bias and crap about blank checks.  Apart form not making any real sense it has nothing to do with the issue and only inflames the thread.

Following scientific method would have saved Anandtech in this case, they would have contacted other reviewers and compared methodology, this would have made the posibility of finding the cause of the disparity in results happen before any drama could have happened much higher.

 

I've said that LOL. Lost count in how many times I've said that. If you find something that goes against expectations, you sanity check. Also why we have peer review. 

 

Yeah, in hindsight it was a misstep to bring up the AMD bias. Puts people in an emoptional state, something I'm not immune to so shouldn't expect anything different from others.