Intel 9th Gen Paid Benchmarks Take Advantage of NDA Periods

[D13H4RD]

36 minutes ago, mr moose said:

Just be grateful the PC world can have independent benchmarks readily available, imagine if it was like the supplements industry where the ads leave you with the distinct impression they can cure cancer, depression, old age and make you attractive and immune to hangovers.

Indeed. I think this is just pure marketing to drive sales at this point.

 

Hence why I keep GN, Hardware Unboxed and other guys who do detailed benchmarks in my radar.

[Carclis]

9 minutes ago, mr moose said:

Just be grateful the PC world can have independent benchmarks readily available, imagine if it was like the supplements industry where the ads leave you with the distinct impression they can cure cancer, depression, old age and make you attractive and immune to hangovers.

The thing is there needs to be third party or independent benchmarks in order to disprove this. I think we were lucky this time in that the comparisons made included enough existing products that their absurdity was apparent at a glance. It is also the job of said third parties and independents to point out and condemn this sort of behavior because it's what makes their position as an independent relevant. The reason these tactics work and why contrived results like these work for the companies trying to sell you a product is because the tech journalists aren't doing their jobs as informants that are supposed to look out for the consumer; the reason people look to them for benchmarks. I already pointed this out in the OP but PCGamesN already failed in reporting on the benchmarks properly because they failed to recognise how many flaws it had or the methodology used despite their disclosure of it being an Intel sponsored paper.

I understand that Linus Media Group is more focused on entertaining/impractical content and news than the actual benchmark side of things but it was a real poor showing from them (and others) to not give outright lies the ridicule they deserved. Who's interests are they serving and who is paying them again?

Sorry for the rant.

[Lathlaer]

1 hour ago, Carclis said:

Who's interests are they serving and who is paying them again?

I feel like we watched different WAN Show then. I know that people wanted them to yell and shout like Steve but Steve already did that and what was left for Linus was to point out few things about the industry. He did call PT incompetent and he did call Intel incompetent. I know, he didn't show middle finger or got a stroke from anger on the WAN show but common.

 

As a side note and at the risk of stirring the hornet nest a bit, I find it a bit funny how AMD needs to be protected from themselves in those benchmarks by ignoring the option they made for their CPU

 

I mean, typical Joe sees "Gaming Mode" and wants to play games so he finds out how to enable it because it seems logical to do, does so and then suffers because of it. Then he goes to tech forums or speaks with his friends bitching how AMD CPU is shit and not giving him good fps.

[Carclis]

52 minutes ago, Lathlaer said:

I feel like we watched different WAN Show then. I know that people wanted them to yell and shout like Steve but Steve already did that and what was left for Linus was to point out few things about the industry. He did call PT incompetent and he did call Intel incompetent. I know, he didn't show middle finger or got a stroke from anger on the WAN show but common.

He didn't look into the topic before deciding to cover it. As a result blamed the consumers for pre-ordering (not entirely fair) instead of taking a stand against industry bullshit which is clearly more egregious in this case. He also said he didn't see any difference between this instance on any of the previous instances such as cherry picked benchmarks. You see there is a VERY big difference between benchmarking your product and showing performance in an instance you are always going to win, despite most other workloads not favoring your product to that extent. The issue here is that nobody will be running the AMD configuration that they showed in the benchmarks making it a result that will never be realistic. Imagine if the performance tests that AMD had shown were comparing the 8700k to the 2700x with two of the 8700k cores and hyperthreading disabled (because they can hit higher clock speeds that way). It would have and should get ridiculed because it's not something that anybody would ever do. Take a look at what Steve had to say about it:

Take a step back for a second and look at it from Steve's perspective. He works his ass off to give us some of the most comprehensive benchmarks and performance testing in the industry. In fact his last review of the he tested 35 games using 9 different GPU's and across 3 resolutions. Assuming he does three benchmark runs of each test, he has done at LEAST 2835 runs in total. Think about how much effort he puts in to ensure that the public is well in formed about the products he tests. How do you think he is going to feel when Intel commissions a third party to do testing to try and make it look as legitimate and valid as his own, but also holds him and other publications back with an embargo period (also meaning he may get less views)? Now combine that with the fact that some of these publications are refusing to do their job by looking out for the consumer. It looks kinda bad on him to be considered part of that group, don't you think?

52 minutes ago, Lathlaer said:

As a side note and at the risk of stirring the hornet nest a bit, I find it a bit funny how AMD needs to be protected from themselves in those benchmarks by ignoring the option they made for their CPU

Fair point, but I highly doubt anybody uses it. Think about the type of person to install the software. You'd have to be tech-savvy enough to be comfortable enabling and using the software as well as dumb enough to not notice that half your cores are missing (and not notice the performance hit). Enabling the setting is disingenuous and fraudulent which has been pointed out already by all of the credible reviewers. I agree that it shouldn't be on the Ryzen 3/5/7 processors though.

 

52 minutes ago, Lathlaer said:

I mean, typical Joe sees "Gaming Mode" and wants to play games so he finds out how to enable it because it seems logical to do, does so and then suffers because of it. Then he goes to tech forums or speaks with his friends bitching how AMD CPU is shit and not giving him good fps.

As said above. Typical Joe is not tech-savvy enough to use Ryzen Master or tech-savvy enough to see that it's dumb.

My simple observation is that the Patreon-backed tech journalists seem to care more about serving the interests of their consumers than the ones who have plenty of sponsors in their back pocket.

[Lathlaer]

1 hour ago, Carclis said:

As a result blamed the consumers for pre-ordering (not entirely fair)

I take it I will find you defending poor consumers who preordered RTX cards? No preorders is by far one of the most agreed-upon stances in every community and now mentioning it constitutes "consumer blaming"? WTF?

 

All he did is attribute most of what happened not to malice but incompetence based on his personal knowledge and contact with people at Intel. I guess calling PT incompetent and Intel as well because of signing off on this is not enough.

 

People confuse his take on how the events transpired (the whole part of some mid level managers covering their asses) for approval. 

 

People need to get it in their heads that just because he explains what he thinks is happening does not mean he approves of the situation. By the way he did respond to that HU comment while trying to explain his stance in greater detail. 

 

I feel like he needed some more shouting, finger showing and a little extra foam in mouth with his delivery for people to be happy. 

 

Such is the reality that most people here (on LTT) are extremely pro-AMD and because of that you will get shit not only for not blaming Intel but apparently for not hating Intel enough.

[Carclis]

2 minutes ago, Lathlaer said:

I take it I will find you defending poor consumers who preordered RTX cards? No preorders is by far one of the most agreed-upon stances in every community and now mentioning it constitutes "consumer blaming"? WTF?

I should have explained this. I'm not defending the people who pre-order exactly, although it has it's reasons in a world where we have the supply shortages that we do, particularly with Intel right now. My problem is that the criticism of the benchmarks themselves has been utterly dismissed by LTT with the excuse that the only thing that matters is that you should not pre-order. The issue is very much one that concerns the tech press as much as consumers themselves. What absolutely deserved criticising is in this thread title itself; abuse of the embargo period to push a third party benchmark as something legitimate. The only reason Intel disclosed that it was a sponsored paper by them is they were legally forced to by the FTC after their SysMark cheating many years back. Now they're trying to do the same thing by hushing reviewers with an embargo. When was the last time a comprehensive third party benchmark like this was published before an embargo was lifted? I don't understand why this was not criticised.

13 minutes ago, Lathlaer said:

All he did is attribute most of what happened not to malice but incompetence based on his personal knowledge and contact with people at Intel. I guess calling PT incompetent and Intel as well because of signing off on this is not enough.

Yes. That doesn't make sense. Intel CHOSE to publish the results it commissioned PT to find and publicly stated that the findings were identical to their own lab results. Now this means that there are one of two problems. The testing at Intel's own labs was biased in that they ordered PT to handicap the Ryzen system (as mentioned in the OP), or that the testing itself was a mistake by PT who is inexperienced in games testing and Intel still chose to publish the results which were so obviously wrong that it's not funny. In either case Intel is guilty of malice. It CHOSE to grossly mislead consumers.

20 minutes ago, Lathlaer said:

People need to get it in their heads that just because he explains what he thinks is happening does not mean he approves of the situation. By the way he did respond to that HU comment while trying to explain his stance in greater detail.

It left a lot to be desired. As I mentioned above, I can't see a way in which Intel did not deserve criticism for what they did. They chose to go to a third party because they wanted the results to look credible and they approved the publication of the benchmarks because the fraudulent results would generate some extra sales. Thinking it is incompetence doesn't really cut it.

23 minutes ago, Lathlaer said:

I feel like he needed some more shouting, finger showing and a little extra foam in mouth with his delivery for people to be happy. 

 

Such is the reality that most people here (on LTT) are extremely pro-AMD and because of that you will get shit not only for not blaming Intel but apparently for not hating Intel enough.

I'm not interested in benefiting a company. This comes from the desire to see better products and competition that is not deceptive. Call it selfish if you will but I couldn't give a rat's ass about the companies unless it's in my best interest.

[D13H4RD]

1 hour ago, Lathlaer said:

Such is the reality that most people here (on LTT) are extremely pro-AMD and because of that you will get shit not only for not blaming Intel but apparently for not hating Intel enough.

Kinda sure a lot of people will react the same way if this involved AMD as the one who’s the deceiving one.

 

If not, then they’re hardcore AMD fanboys 

[mr moose]

5 hours ago, Carclis said:

The thing is there needs to be third party or independent benchmarks in order to disprove this. I think we were lucky this time in that the comparisons made included enough existing products that their absurdity was apparent at a glance. It is also the job of said third parties and independents to point out and condemn this sort of behavior because it's what makes their position as an independent relevant. The reason these tactics work and why contrived results like these work for the companies trying to sell you a product is because the tech journalists aren't doing their jobs as informants that are supposed to look out for the consumer; the reason people look to them for benchmarks. I already pointed this out in the OP but PCGamesN already failed in reporting on the benchmarks properly because they failed to recognise how many flaws it had or the methodology used despite their disclosure of it being an Intel sponsored paper.

I understand that Linus Media Group is more focused on entertaining/impractical content and news than the actual benchmark side of things but it was a real poor showing from them (and others) to not give outright lies the ridicule they deserved. Who's interests are they serving and who is paying them again?

Sorry for the rant.

Rants are OK if your passionate about something.  I think most tech journalists are doing there jobs and the reason I said what I said was because tech does have a healthy independent review presence.   That's how we know when one is biased, because the other 10 have opposing results.   This is largely not that different to any other marketing scheme.  At worst it's just Intel being disingenuous with their marketing figures and at best its like Linus said; a fuck up in management somewhere and no one is taking the blame.   I know if I was a noob in management and gave the all clear to publish that data I would be too embarrassed to admit to it.

[leadeater]

2 hours ago, Lathlaer said:

Such is the reality that most people here (on LTT) are extremely pro-AMD

I actually don't think it's quite that, there's the pro-AMD people and the 'anything but Intel crowd' all tied in as one. A lot of people are just tired of what they deem sub par performance gains and a lack of technology progression in the CPU space, at least on the desktop side.

[mr moose]

2 minutes ago, leadeater said:

I actually don't think it's quite that, there's the pro-AMD people and the 'anything but Intel crowd' all tied in as one. A lot of people are just tired of what they deem sub par performance gains and a lack of technology progression in the CPU space, at least on the desktop side.

Which is kinda scary when you consider that almost the entire industry is struggling to improve products more than just incrementally now, yet we still get angry when it doesn't happen, and when they find another way to make things better, half the forum still gets angry because either it's different and we don't like different or they don't understand it.

[leadeater]

5 minutes ago, mr moose said:

Which is kinda scary when you consider that almost the entire industry is struggling to improve products more than just incrementally now, yet we still get angry when it doesn't happen, and when they find another way to make things better, half the forum still gets angry because either it's different and we don't like different or they don't understand it.

I get that it's a little jarring to see 20-24 core CPUs on the server side of Intel but the cores in those are so power constrained that they are not at all useful for gaming. We get 4 cores on desktop for 10 years while servers go from 6 to 24 cores in the same time, what's not understood very well is the per core performance has been going down as the cores go up.

It's actually a good thing these stupid 7980XE and W-3175X CPUs have come on the scene in the gamers sphere, very aptly shows the problem with increasing core count, power and heat along with reduction in single core and multi core boost.

 

And that's not even bringing in software issues.

[mr moose]

2 minutes ago, leadeater said:

It's actually a good thing these stupid 7980XE and W-3175X CPUs have come on the scene in the gamers sphere, very aptly shows the problem with increasing core count, power and heat along with reduction in single core and multi core boost.

Well, yes it shows the problem, but people are trying to dismiss it with claims about erroneous TDP specs or simply falling back on the old "more cores = more faster" narrative, ergo if it doesn't go faster it's because someone fucked up and not because they are battling the laws of physics with a 60 year old substrate.  

[leadeater]

7 minutes ago, mr moose said:

TDP specs

Careful now, TDP is a contentious issue .

[TechyBen]

54 minutes ago, D13H4RD2L1V3 said:

Kinda sure a lot of people will react the same way if this involved AMD as the one who’s the deceiving one.

 

If not, then they’re hardcore AMD fanboys 

I need to change businesses... because there are an awful lot of people out there that seem to think 2 wrongs make a right. XD

 

GamersNexus tries to call out errors, bugs and scams/lies when they see it regardless.

 

Also, I've seen a lot of "willful ignorance" in businesses... So while I agree the argument "Never attribute to malice that which is adequately explained by stupidity" can and does apply at times... some people/businesses choose to be stupid, if they know it will make them more money (We did not realise AMD cooler was different in performance, we did not realise the Gamers mode would not work, we did not realise the games were GPU bound, We did not realise the RAM timings would favour 1 platform etc etc).

[Carclis]

27 minutes ago, mr moose said:

Rants are OK if your passionate about something.  I think most tech journalists are doing there jobs and the reason I said what I said was because tech does have a healthy independent review presence.   That's how we know when one is biased, because the other 10 have opposing results.   This is largely not that different to any other marketing scheme.  At worst it's just Intel being disingenuous with their marketing figures and at best its like Linus said; a fuck up in management somewhere and no one is taking the blame.   I know if I was a noob in management and gave the all clear to publish that data I would be too embarrassed to admit to it.

For now it is. That was my point though. Despite the fact that we have this healthy industry where all the results disagree with the posted paper a lot of the publications didn't know better or were unwilling to dispute the results. Plus I would have expected that someone with as significant a presence in the industry as Linus would have taken the opportunity to stand up for customers as well, especially since he gets a lot of viewers that the more advanced tech channels do not. In the end though we only had two people cover the stuff as it unfolded.

 

I still don't buy the "accidental" angle. Their public statement said that their results aligned with PT's and they went with a third party for testing. I mean they obviously wanted to have their results look legitimate by going with a third party, no other reason for it. Given their recent behavior concerning the 5Ghz 28 core CPU I would characterize their motives as deception. It makes sense though. They're under pressure from the 2700x and they're unwilling to compete on price or drop the profit margins. It just seems like a desperate situation.

[D13H4RD]

31 minutes ago, leadeater said:

It's actually a good thing these stupid 7980XE and W-3175X CPUs have come on the scene in the gamers sphere, very aptly shows the problem with increasing core count, power and heat along with reduction in single core and multi core boost. 

That's probably why the single-core perf of the 8700K is actually slightly lower than that of the 7700K, which probably explains the higher boost clocks in return. 

 

Also to note is that in the PT test, Game Mode on the TR chips gave it a significant improvement in performance, which kinda proves that more cores for gaming are not simply better. At some point, the benefits just diminish.

 

Yes, I know TR isn't meant to be the ultimate gaming CPU, but it does illustrate the point 

[Carclis]

7 minutes ago, D13H4RD2L1V3 said:

Also to note is that in the PT test, Game Mode on the TR chips gave it a significant improvement in performance, which kinda proves that more cores for gaming are not simply better. At some point, the benefits just diminish.

 

Yes, I know TR isn't meant to be the ultimate gaming CPU, but it does illustrate the point 

I think that has more to do with the TR CPU's being multiple die products than large core count, or rather a combination. It has enough cores even when being cut in half and the latency improvement when only using one die is the actual performance booster.

[leadeater]

4 minutes ago, D13H4RD2L1V3 said:

That's probably why the single-core perf of the 8700K is actually slightly lower than that of the 7700K, which probably explains the higher boost clocks in return. 

 

Also to note is that in the PT test, Game Mode on the TR chips gave it a significant improvement in performance, which kinda proves that more cores for gaming are not simply better. At some point, the benefits just diminish.

 

Yes, I know TR isn't meant to be the ultimate gaming CPU, but it does illustrate the point 

It's all just a math problem really, it takes a certain amount of power to get a core stable a certain frequency. To increase the frequency the power must go up, when you have package power limits and thermal targets to meet then more cores becomes opposed to performance/frequency increases.

 

It doesn't matter the architecture or fab process the above is true for all.

[leadeater]

1 minute ago, VegetableStu said:

Taran Desktop Performance?

Which is a measure of keyboard macros per minute.

[D13H4RD]

3 minutes ago, leadeater said:

It's all just a math problem really, it takes a certain amount of power to get a core stable a certain frequency. To increase the frequency the power must go up, when you have package power limits and thermal targets to meet then more cores becomes opposed to performance/frequency increases.

 

It doesn't matter the architecture or fab process the above is true for all.

Indeed, which is why we probably shouldn't be surprised if the 9900K requires more cooling and power than the 8700K, which itself also required more of both than the 7700K.

5 minutes ago, Carclis said:

I think that has more to do with the TR CPU's being multiple die products than large core count, or rather a combination. It has enough cores even when being cut in half and the latency improvement when only using one die is the actual performance booster.

Which is probably why Game Mode exists for the TR in the first place. 

[Jito463]

7 hours ago, Carclis said:

publicly stated that the findings were identical to their own lab results.

While I honestly believe that Intel recognized how deceptive these benchmarks were prior to releasing them, one could make the argument that they were referring to the 9900k benchmarks specifically, not their comparison to the Ryzen/Threadripper benchmarks.

5 hours ago, D13H4RD2L1V3 said:

Kinda sure a lot of people will react the same way if this involved AMD as the one who’s the deceiving one.

 

If not, then they’re hardcore AMD fanboys 

I've only ever run AMD processors going back to the 486 DX/2 66MHz, and I would heartily concur to calling out AMD if they pulled a crap stunt like this.  They may cherry pick results from testing (what company doesn't?), but I'm not aware of any case where they've done something so egregious as this.

[mr moose]

9 hours ago, leadeater said:

Careful now, TDP is a contentious issue .

Then ban the people who refuse to accept how it works rather than the discussion. 

8 hours ago, Carclis said:

I still don't buy the "accidental" angle. Their public statement said that their results aligned with PT's and they went with a third party for testing.

That's what covering your arse looks like.   And the results probably were identical to their own results, Anyone can get the same results if they do thee test the same way. 

 

I had a manager once that tried to call me mentally deficient because he wanted me to have my team square out the rounded corners of a rectangle hole so a steel frame (that was rectangle in shape) could be installed.  The hole was already larger than the frame so nothing needed to be done, I tried to inform him we didn't need to waste labor squaring out the corners as the frame could just be installed and he argued I needed to see a psychologist because you can't put a rectangle in a round hole.   Nearly every foreman, supervisor and team leader in that room collectively rolled their eyes.  That is the kind of product ignorance you can get in management and the inability for those with more knowledge to change things (I put the frame in the hole with ample clearance right in front of him),  so it would be of no surprise to me at all if this is exactly how Linus claims it could be.

 

 

[pas008]

Didn't they update the article with game mode off?

 

And yes i agree all companies do this shit

Nvidia amd apple etc always have their testing of a product looking great by using half assed competitor results 

Which technically they can because it's not their product to showcase

And i dropped the tek after his amd fx shit along with his sound shit on top of that

[Curufinwe_wins]

35 minutes ago, pas008 said:

Didn't they update the article with game mode off?

 

And yes i agree all companies do this shit

Nvidia amd apple etc always have their testing of a product looking great by using half assed competitor results 

Which technically they can because it's not their product to showcase

And i dropped the tek after his amd fx shit along with his sound shit on top of that

Yes they did. Fairly quickly. It's much better of course, but also of course the 9900k ls the "best". Just not necessarily the best value. Which is also not surprising. 

[Stefan Payne]

4 hours ago, mr moose said:

Then ban the people who refuse to accept how it works rather than the discussion. 

 

That is a very bad idea...

But we should start with People who call for bans, shouldn't we?

 

Forums are to disagree with each other and discuss things we disagree upon.

13 hours ago, leadeater said:

Careful now, TDP is a contentious issue .

Because Intel doesn't accept the Intel Spec for the TDP, that is, more or less, used in the same way throughout all electronic devices - and then there is Intel right now...

And nowhere they do mention the "average usage Power" vs. the "standard definition TDP". 

 

You could say that Intel is sugarcoating the "TDP" right now because they have come under preassure...

The question is: When did they change their definition of TDP?

14 hours ago, D13H4RD2L1V3 said:

Kinda sure a lot of people will react the same way if this involved AMD as the one who’s the deceiving one.

Thing isn't just about the Paid Benchmarks, that isn't the real issue...

That is totally fine.

The issue is that they used the values on their website for advertizing without double checking and then doubling down when they were caught.

They still defended the shit...

 

With an honest apology, things wouldn't have been that hot like they are right now.

 

And an Apology that they missed this, that some processors might have been worse and that they already asked Principaled Technologies to redo the Tests and a fixed version of the document wil be up in the next couple of days.

 

But they had to make it worse with what they said...

 

Lets wait and see what happens, if the information that got leaked in German are true and that there are some thermal issues with the 9900k...