AMD caught cheating in Synthetic Benchmarks

[rcmaehl]

Summary

AMD has been found to be cheating in Synthetic Benchmarks for its Ryzen chips, by misreporting stats and disabling thermal protections.

 

Quotes

Quote

Benchmark cheating has a long story that goes far back for the industry... and has also been a controversial coverage topic... for several years now. Back in 2013,... shenanigans Samsung was doing on the GPU of Exynos chipsets... eventually even ended up with a successful $13.4m class-action lawsuit judgment against the company. Naming and shaming did work over the following years, as vendors quickly abandoned such methods out of fear of media backlash. In recent years however we saw a big resurgence of such methods, particularly from Chinese vendors. Most predominantly for our more western audience this happened to Huawei just a couple of generations ago with mechanisms that essentially disabled thermal throttling the of phones – letting more demanding benchmarks essentially have the SoC burn through to the maximum until thermal shutdowns. The naming and shaming here again helped, as the company had transitioned from employing invisible mechanisms to something that was a lot more honest and transparent, and a lot less problematic for follow-up devices. This time chip maker AMD has been found to be doing similar cheating on their Ryzen lineup. Preliminary testing by Phoronix who discovered the cheating, has found that Synthetic benchmarks may provide results up to 33% higher than a user may be expected to realistically encounter with their setup. AMD achieve these results by underreporting die and junction temperatures, extending CPU clocks for longer than normal, and disabling thermal protections. These methods will reduce the life of the processor using them, although it seems AMD was betting on users not running benchmarks often. Phoronix first discovered this cheating as the lifespan of its collection of Ryzen chips was abnormally short for its long-term testing of AMDs platforms. AMD has not responded for a request for comment.

 

My thoughts

Ah, it seems like everyone is getting into the benchmark cheating game lately, mobile phone makers have been doing it for decades, Intel did it back in the early 2000s and 2019, and now even AMD is doing it. Pretty sure this is just a byproduct of consumers putting too much weight on benchmarks and other numbers when they should buy what they can afford. Really, most CPUs are more than enough for the average person and unless you're being paid to do CPU calculations or get the most kills it really shouldn't matter. I hope this doesn't affect AMD too much as they're still unmatched in expandability and upgradability. Let's just see. Fortunately, it looks like AMD hasn't been cheating at their Enterprise benchmarks, so it won't hurt them there. Otherwise, Intel would probably take back some market share. Obliviously, this will affect consumer trust, but I don't think it will affect it as much as some people believe. Lastly, read the first letter of each of these sentences in the my thoughts section. Should there be a class action lawsuit over this, I don't believe so, but that's up to consumers.

 

Sources

Phoronix (charts)

Anandtech (quote source)
Userbenchmark 

Edited April 1 by rcmaehl
s/,/.

[Shimejii]

The fact that you use UserBenchmark as a source, leads me to believe that it may be completely Bullshit. That site alone has done SUCH a disservice to the PC community by spewing blatant lies and such over and over, trying their absolute best to shit on AMD just because the owner doesnt like them. Its one thing to not like a company, its another to lie about performance results and such.

 

Also April fools stories and such like this = bad idea for forum stuff

[starsmine]

1 minute ago, Shimejii said:

The fact that you use UserBenchmark as a source, leads me to believe that it may be completely Bullshit. 

look at all the links. 

Kinda rather not do april 1st jokes like this as it does get cached and passed around after the fact without it being clear. 

[RONOTHAN##]

2 minutes ago, Shimejii said:

The fact that you use UserBenchmark as a source, leads me to believe that it may be completely Bullshit. That site alone has done SUCH a disservice to the PC community by spewing blatant lies and such over and over, trying their absolute best to shit on AMD just because the owner doesnt like them. Its one thing to not like a company, its another to lie about performance results and such.

Follow the source links. 

[TetraSky]

Someone might actually believe this without looking at the source and report and the false information from an april fool's joke.

[rcmaehl]

I'll make sure the mods remove it once the forum switches back to the old, boring font.

[da na]

Wow, UserBenchMark trashing AMD's hardware again for... no apparent reason? Not the biggest AMD fan myself but I admit that UBM is biased as well as unreliable.

[Spotty]

22 minutes ago, rcmaehl said:

Lastly, read the first letter of each of these sentences in the my thoughts section.

AMPRILFOOLS?

[rcmaehl]

1 minute ago, Spotty said:

AMPRILFOOLS?

I literally double checked....

[Beerzerker]

23 minutes ago, Shimejii said:

The fact that you use UserBenchmark as a source, leads me to believe that it IS Bullshit.

Can confirm....
I've seen some weird (And impossible) results with that benchmark I've personally ran myself.

[GuiltySpark_]

The mix of people that do and don't get it in here is quite funny.

[Failure 101]

40 minutes ago, GuiltySpark_ said:

The mix of people that do and don't get it in here is quite funny.

i just saw ubm and noped out 

[starsmine]

1 hour ago, TechlessBro said:

Same shit different day.

Well every 5 years or so.

They’ve all done it before and will again.

 

53 minutes ago, Failure 101 said:

i just saw ubm and noped out 

 

2 hours ago, da na said:

Wow, UserBenchMark trashing AMD's hardware again for... no apparent reason? Not the biggest AMD fan myself but I admit that UBM is biased as well as unreliable.

[Beerzerker]

3 hours ago, da na said:

Wow, UserBenchMark trashing AMD's hardware again for... no apparent reason? Not the biggest AMD fan myself but I admit that UBM is biased as well as unreliable.

Understatement of the week.
Beyond biased against AMD and it's like this intentially pushes Intel hardware by making AMD look horrible while inflating Intel scores/results so it looks really good in comparison.

I've ran both (Amd and Intel) repeatedly before to check and then reran several times to check consistency of these results when I noted the oddness myself... Some of the scores I got were just comical on both ends of the AMD vs Intel spectrum.

I also noted it's the same for AMD/ATI cards vs Nvidia based scores too - Nvidia of course being favored.

[gamagama69]

3 hours ago, rcmaehl said:

Summary

AMD has been found to be cheating in Synthetic Benchmarks for its Ryzen chips, by misreporting stats and disabling thermal protections.

 

Quotes

 

My thoughts

Ah, it seems like everyone is getting into the benchmark cheating game lately, mobile phone makers have been doing it for decades, Intel did it back in the early 2000s and 2019, and now even AMD is doing it. Pretty sure this is just a byproduct of consumers putting too much weight on benchmarks and other numbers when they should buy what they can afford. Really, most CPUs are more than enough for the average person and unless you're being paid to do CPU calculations or get the most kills it really shouldn't matter. I hope this doesn't affect AMD too much as they're still unmatched in expandability and upgradability. Let's just see. Fortunately, it looks like AMD hasn't been cheating at their Enterprise benchmarks, so it won't hurt them there. Otherwise, Intel would probably take back some market share. Obliviously, this will affect consumer trust, but I don't think it will affect it as much as some people believe. Lastly, read the first letter of each of these sentences in the my thoughts section. Should there be a class action lawsuit over this, I don't believe so, but that's up to consumers.

 

Sources

Phoronix (charts)

Anandtech (quote source)
Userbenchmark 

damn it

[Lightwreather JfromN]

(sigh)

Does no one click on the sources anymore?

[leadeater]

Geez so much for AMD being the good company. Never going to buy them again, time to move all our ESX servers back to Intel, the CPU company we can trust to be ethical and moral.

[Arika S]

1 hour ago, Lightwreather JfromN said:

(sigh)

Does no one click on the sources anymore?

too hard..

[TVwazhere]

On 3/31/2023 at 9:37 PM, rcmaehl said:

I'll make sure the mods remove it once the forum switches back to the old, boring font.

We're pinning this to the top of TN, as well as featuring this in the main page.

 

In fact, there will be an entire subforum dedicated to just this topic.