Intel wants reviewers to benchmark using windows media player instead of cinebench for low end mobile

[mr moose]

16 minutes ago, cj09beira said:

its fair to say they are more multi core than intel, but that seems to be more of a intel is milking consumers thing than different paths to performance improvements

 

How is addressing consumer demand "milking consumers"?  It is clear from each of Intel's responses to Ryzen that they are not leaps and bounds in front, it is also clear they haven't worked out 10nm yet.  We know that the vast majority of consumers still only require 2-4 cores. So Intel working on single thread performance and aiming at the larger general consumer market makes much more sense than pushing for higher core counts. 

 

Now if they pushed for higher cores counts and cut supply of all sub 6 core parts I would agree and call that milking consumers, but as it stands, not really.

 

[Euchre]

This is quite reminiscent of the debate (fuss?) over the validity of Nürburgring lap times of high end sports cars and tuner variants of cars. It is biased toward very track oriented vehicles, and not a vehicle that is comfortable in day to day types of use. However, that's really about the exact versions used for the lap times. The more common versions perform very well, even if they couldn't post the same times. So, it is a valid way to make an overall comparison, but saying your exact version of said vehicle isn't valid.

 

This is basically just the computer version of that.

[spartaman64]

5 hours ago, LAwLz said:

Seems like a pretty misleading title for this thread, no?

Intel want more focus on real world performance in applications general users actually run, rather than synthetic benchmarks.

I totally understand their point. How well a processor performs at Photoshop, WinRAR, web browsing and the likes is far more relevant to buyers than how well it performs at some synthetic benchmark like Cinebench.

 

But I think both holds value. Real world tests like the ones Intel wants more focus on directly translates to how well it actually performs for customers in those programs. I mostly use my computer for web browsing, so how a processor performs for web browsing is more important than how it performs in some synthetic benchmark which doesn't test real world workloads.

But at the same time, synthetic tests gives a much better picture of general performance, if done properly. Just because a processor performs well at let's say WinRAR doesn't mean it will perform well at Photoshop. However, if a processor performs well in lots of synthetic tests then you can be fairly safe to assume that it will perform well in real world workloads too.

i mean what do you get from this slide 

they specifically name WMP.

and even if you give them the benefit of the doubt and say they dont really mean WMP then what applications are they talking about benchmarking? i guess csgo, LoL, cinema4d and photoshop are the reasonable ones but why include the other applications in the list then and why specifically mention office applications and WMP

 

and you do you but i dont care to look at how my processor performs at web browsing because i know that its not going to have any problems

[imreloadin]

Have people seriously forgotten that Cinebench is the benchmark for the Maxon Cinema software? Like it's actually a benchmark for a real world 3d rendering software...

[leadeater]

2 hours ago, Zodiark1593 said:

Worth of cpu time <<<<<<<<<<<<<<<<<<<<<<< worth of employee time.

 

I'm rather curious as to how well mobile devices can hold up to heavier Excel workloads. Would an identical spreadsheet across different platforms represent something close to an Apples to Apples comparison?

I dunno I just made something that did what they needed it to do lol. Would be interesting to have run it on a phone and low end tablet though.

[RejZoR]

3 hours ago, imreloadin said:

Have people seriously forgotten that Cinebench is the benchmark for the Maxon Cinema software? Like it's actually a benchmark for a real world 3d rendering software...

Same with Blender. Or encoding of MP3/H.264/H.265 media. Or archiving with 7-zip/WinRAR... All the things reviewers test lol

[Bcat00]

Can’t say intel’s list there is wrong. I use Chrome the most followed by word 

[RejZoR]

28 minutes ago, Bcat00 said:

Can’t say intel’s list there is wrong. I use Chrome the most followed by word 

But really, do you really need a 9900K to run a... BROWSER?

[mr moose]

4 hours ago, imreloadin said:

Have people seriously forgotten that Cinebench is the benchmark for the Maxon Cinema software? Like it's actually a benchmark for a real world 3d rendering software...

 

It doesn't look that way.  It's more that people are saying they don't do rendering so the benchmark is kinda pointless for them.

 

Which I kinda get because for a long time I didn't do rendering and it was tiring having to wade through and separate those results from conclusions that give you an overall brief on the value/performance. 

[Thaldor]

22 minutes ago, mr moose said:

 

It doesn't look that way.  It's more that people are saying they don't do rendering so the benchmark is kinda pointless for them.

 

Which I kinda get because for a long time I didn't do rendering and it was tiring having to wade through and separate those results from conclusions that give you an overall brief on the value/performance. 

But then again if reviewers were to test with the applications Intel is saying they should test we come to the problem of relevancy and pointlessness. Like it's not a problem to make a test how fast Chrome starts but it's basicly pointless because any modern desktop CPU will do that so fast that they would be giving out results that have differences in 0.00001 seconds and if you add up something like "time until the homepage is fully loaded" it's going to be more dependable on does Googles/other servers have a hiccup today than do you have this $2000 CPU or that $100 CPU.

With Office applications you can always start opening some huge files and comparing how fast different CPUs open them. But to get to some good results that show that there is differences you need to use so huge files that you could call them synthetic benchmarks because there is very rarely anything like them in the real world. And if you are going to go to synthetic workloads or usecases you could just go to the real synthetic benchmarks where there is actual metrics (biased or not) to be used rather than you with a stopwatch waiting for that 1GB Word-file to open.

These kind of test could bring some comparable results with something like ultrabooks and 2-in-1s where there is such low performance parts in play that just normal usage is heavy use. But with something like mid- to top-tier desktop CPUs? Not a chance you get anything (totally zero, more like null) useful results testing how fast they open Chrome or how well they run CSGO. They are like testing does "Lamborghini run better 100m at 40km/h than Lada" and the result is hopefully Ferrari run it 0.001 seconds faster but the timer might have had problem to the opposite and the Lada was 0.01 seconds faster than the Ferrari. So should you try to get that $500k-1.5M Ferrari or just get that $200 Lada (yeah, we both know that the Lada can hardly run 60km/h and the Ferrari can get to the 250km/h range but the benchmark was 100m at 40km/h so that's the only data we have)?

 

(And then there comes the customer who bought the Lada and is now complaining that he was in the last place in a race with it and wants his money back because Lada couldn't even start racing against Toyotas, let alone real sportcars)

[RejZoR]

56 minutes ago, mr moose said:

 

It doesn't look that way.  It's more that people are saying they don't do rendering so the benchmark is kinda pointless for them.

 

Which I kinda get because for a long time I didn't do rendering and it was tiring having to wade through and separate those results from conclusions that give you an overall brief on the value/performance. 

It's not about what the workload is, it's about consistent becnhmarking and comparison. It's why CPU's are benchmarked in professional tools like Maxon, Blender, compression tools, video encoders, games etc. So you pick somewhat what matters the most for you. If you're a gamer, you're probably skip the rest and look at gaming benchmarks only. And if you're looking to buy the most powerful CPU to run Chrome, then what are you even doing with your life...

[TechyBen]

13 hours ago, Sakkura said:

Chrome? Okay maybe Cinebench isn't great, but Chrome?

I need 300fps in media player. Gotta play those songs FASTER on Intel chips.

[TechyBen]

10 hours ago, LAwLz said:

Seems like a pretty misleading title for this thread, no?

Intel want more focus on real world performance in applications general users actually run, rather than synthetic benchmarks.

I totally understand their point. How well a processor performs at Photoshop, WinRAR, web browsing and the likes is far more relevant to buyers than how well it performs at some synthetic benchmark like Cinebench.

 

But I think both holds value. Real world tests like the ones Intel wants more focus on directly translates to how well it actually performs for customers in those programs. I mostly use my computer for web browsing, so how a processor performs for web browsing is more important than how it performs in some synthetic benchmark which doesn't test real world workloads.

But at the same time, synthetic tests gives a much better picture of general performance, if done properly. Just because a processor performs well at let's say WinRAR doesn't mean it will perform well at Photoshop. However, if a processor performs well in lots of synthetic tests then you can be fairly safe to assume that it will perform well in real world workloads too.

9 times out of 10 the synthetic allows *repeatable* tests. This is more important than if Chrome/WinRAR/etc are "faster" because the benchmark/reviewer used a different sized zip file etc.

 

Though if WinRAR has a "benchmark" button, then that's fine. But synthetic allow you to set baselines on testable applications. Steam might be different every other week (see Gamers Nexus on the problems with driver revisions and game patches and GPU benching).

[MadmanRB]

I am going to start benchmarks with notepad..... OOOOOOOH!!!!!!!

Forget the phrase can it run Crysis, the new question is can it run freecell?

[mr moose]

19 minutes ago, Thaldor said:

But then again if reviewers were to test with the applications Intel is saying they should test we come to the problem of relevancy and pointlessness. Like it's not a problem to make a test how fast Chrome starts but it's basicly pointless because any modern desktop CPU will do that so fast that they would be giving out results that have differences in 0.00001 seconds and if you add up something like "time until the homepage is fully loaded" it's going to be more dependable on does Googles/other servers have a hiccup today than do you have this $2000 CPU or that $100 CPU.

With Office applications you can always start opening some huge files and comparing how fast different CPUs open them. But to get to some good results that show that there is differences you need to use so huge files that you could call them synthetic benchmarks because there is very rarely anything like them in the real world. And if you are going to go to synthetic workloads or usecases you could just go to the real synthetic benchmarks where there is actual metrics (biased or not) to be used rather than you with a stopwatch waiting for that 1GB Word-file to open.

These kind of test could bring some comparable results with something like ultrabooks and 2-in-1s where there is such low performance parts in play that just normal usage is heavy use. But with something like mid- to top-tier desktop CPUs? Not a chance you get anything (totally zero, more like null) useful results testing how fast they open Chrome or how well they run CSGO. They are like testing does "Lamborghini run better 100m at 40km/h than Lada" and the result is hopefully Ferrari run it 0.001 seconds faster but the timer might have had problem to the opposite and the Lada was 0.01 seconds faster than the Ferrari. So should you try to get that $500k-1.5M Ferrari or just get that $200 Lada (yeah, we both know that the Lada can hardly run 60km/h and the Ferrari can get to the 250km/h range but the benchmark was 100m at 40km/h so that's the only data we have)?

 

(And then there comes the customer who bought the Lada and is now complaining that he was in the last place in a race with it and wants his money back because Lada couldn't even start racing against Toyotas, let alone real sportcars)

The point is that sometimes those conclusions are drawn using workloads that the end user won't so the end conclusion if not properly qualified will be pointless. 

 

Some reviews are more than 50% production/tool workload based and if the conclusions and recommendations don't properly explain that (which many don't) then the only way to work out how good the CPU is for your specific task is to go through the whole review and work out which bits are irrelevant.  

 

 

13 minutes ago, RejZoR said:

It's not about what the workload is,

Yes it is, If I don't render then render benchmarks are pointless.

 

13 minutes ago, RejZoR said:

it's about consistent becnhmarking and comparison. It's why CPU's are benchmarked in professional tools like Maxon, Blender, compression tools, video encoders, games etc. So you pick somewhat what matters the most for you. If you're a gamer, you're probably skip the rest and look at gaming benchmarks only. And if you're looking to buy the most powerful CPU to run Chrome, then what are you even doing with your life...

 

You know you just contradicted yourself in that don't you?   Like I said, I have to separate the productivity component from the gaming component in the conclusions when the reviewer has just lumped it all together to give an overall recommendation.    It's great when they are more specific, but sometimes that doesn't happen. 

[Lathlaer]

20 minutes ago, mr moose said:

Yes it is, If I don't render then render benchmarks are pointless.

Agreed but the main problem with that chart is that they are not asking you whether you render - they are asking people with tablets and low powered laptops and woah, surprise that no one renders there ;-)

[Taf the Ghost]

6 hours ago, imreloadin said:

Have people seriously forgotten that Cinebench is the benchmark for the Maxon Cinema software? Like it's actually a benchmark for a real world 3d rendering software...

They have. They also miss that R20 shifted to a lot more AVX2 workload, which improve Intel vs AMD in Intel's favor. At least until Zen2 launched. 

 

Of course, the Puget Systems testing put AMD & Intel as level in the Adobe products, it's really a matter of how much you wanted to spend.

[mr moose]

1 minute ago, Lathlaer said:

Agreed but the main problem with that chart is that they are not asking you whether you render - they are asking people with tablets and low powered laptops and woah, surprise that no one renders there

I wasn't talking about Intel's list though.  I was just saying I think people understand how accurate cinebench is, it's just  they were questioning the relevance of it.  For me I like the idea of not using cinebench in some reviews because I don't (or rather didn't) do much rendering and I found it's inclusion in many reviews around the i5 mark to be more a distraction (I say this because I believe once you get that low on CPU other factors like ram size and speed play a role, Drive type etc).

 

I don't know exactly what Intel hope to get out of their desired bench marking regime, I think that is a debate in itself.  However I do think their is merit in re hashing CPU reviews so that it is much more relevant to the three main use groups (light office,  gaming and productivity).  There is zero point in testing anything above an i5/3600 for office duties. but there is merit to not muddying reviews with unnecessary tests like cinebench, blender or handbrake etc if the end user is strictly a gamer.

[TechyBen]

4 minutes ago, mr moose said:

I wasn't talking about Intel's list though.  I was just saying I think people understand how accurate cinebench is, it's just  they were questioning the relevance of it.  For me I like the idea of not using cinebench in some reviews because I don't (or rather didn't) do much rendering and I found it's inclusion in many reviews around the i5 mark to be more a distraction (I say this because I believe once you get that low on CPU other factors like ram size and speed play a role, Drive type etc).

 

I don't know exactly what Intel hope to get out of their desired bench marking regime, I think that is a debate in itself.  However I do think their is merit in re hashing CPU reviews so that it is much more relevant to the three main use groups (light office,  gaming and productivity).  There is zero point in testing anything above an i5/3600 for office duties. but there is merit to not muddying reviews with unnecessary tests like cinebench, blender or handbrake etc if the end user is strictly a gamer.

It's a bit like 0-60 stats though. I could probably count the number of people I know who do 0-60 drags in a car on one finger.  But it's a metric for comparing engines... to know the general performance of acceleration. 0-60 and horsepower and gearing give a general idea of over taking or pulling out of a junction power. They might not get it right every time, a car might have rubbish top end performance, but this is often picked up in a real use review... "it feels sluggish overtaking on a motorway" or "you have to lead in when overtaking because of turbo diesel lag, other brands solved this problem with...".

 

Most of us, and most consumers, understand that the Intel or Ryzen that scores more in a benchmark will, 99% of the time, also feel/perform better in Chrome or Word... will they "need" it? Will it be "best value"? Probably not... but those buying £300 Atom tablets, or £250 Athalon Laptops, are not checking Cinebench benchmarks.

 

[Taf the Ghost]

2 minutes ago, mr moose said:

I wasn't talking about Intel's list though.  I was just saying I think people understand how accurate cinebench is, it's just  they were questioning the relevance of it.  For me I like the idea of not using cinebench in some reviews because I don't (or rather didn't) do much rendering and I found it's inclusion in many reviews around the i5 mark to be more a distraction (I say this because I believe once you get that low on CPU other factors like ram size and speed play a role, Drive type etc).

 

I don't know exactly what Intel hope to get out of their desired bench marking regime, I think that is a debate in itself.  However I do think their is merit in re hashing CPU reviews so that it is much more relevant to the three main use groups (light office,  gaming and productivity).  There is zero point in testing anything above an i5/3600 for office duties. but there is merit to not muddying reviews with unnecessary tests like cinebench, blender or handbrake etc if the end user is strictly a gamer.

https://www.pugetsystems.com/labs/articles/Photoshop-CPU-Roundup-AMD-Ryzen-3-AMD-Threadripper-2-Intel-9th-Gen-Intel-X-series-1529/

 

Intel really shouldn't be complaining about R20. It's a real world product and an easy & repeatable benchmark that does show combined load ability. What they should really worry about is that AMD is level or better in everything else. (Hopefully we start seeing professional products rolling out multiple .exe files for the different processors, since optimizations do matter.)

[cj09beira]

9 hours ago, mr moose said:

How is addressing consumer demand "milking consumers"?  It is clear from each of Intel's responses to Ryzen that they are not leaps and bounds in front, it is also clear they haven't worked out 10nm yet.  We know that the vast majority of consumers still only require 2-4 cores. So Intel working on single thread performance and aiming at the larger general consumer market makes much more sense than pushing for higher core counts. 

 

Now if they pushed for higher cores counts and cut supply of all sub 6 core parts I would agree and call that milking consumers, but as it stands, not really.

 

today 4 cores even for 1080p 60 is not a good experience unless you have ht, people not asking for more seems to me like a lie.

I think its the other way around we haven't needed more than 4 cores all this time (a full decade) because there were none being made for consumers 

so only now can devs start to actually put more cores to use 

1 minute ago, Mr Prince said:

I have not trusted Intel ever since spectre and meltdown, i cant wait till next year to upgrade cpu and ram and everything finally.

are you on ddr3?, ddr5 is probably not out next year, zen 3 will still be ddr4 unless there are two versions of it, as it was confirmed to be ddr4 by amd, ddr5 seems more likely to be a 2021 thing

[leadeater]

18 minutes ago, TechyBen said:

It's a bit like 0-60 stats though. I could probably count the number of people I know who do 0-60 drags in a car on one finger

I like to get from stand still to the speed limit rather quickly all the time . Not in a dangerous way but the woosh is fun sooo.... WOOOOOOOoooooooooooo.

[RejZoR]

40 minutes ago, mr moose said:

The point is that sometimes those conclusions are drawn using workloads that the end user won't so the end conclusion if not properly qualified will be pointless. 

 

Some reviews are more than 50% production/tool workload based and if the conclusions and recommendations don't properly explain that (which many don't) then the only way to work out how good the CPU is for your specific task is to go through the whole review and work out which bits are irrelevant.  

 

 

Yes it is, If I don't render then render benchmarks are pointless.

 

 

You know you just contradicted yourself in that don't you?   Like I said, I have to separate the productivity component from the gaming component in the conclusions when the reviewer has just lumped it all together to give an overall recommendation.    It's great when they are more specific, but sometimes that doesn't happen. 

LMAO? Where in bloody hell I'm contradicting myself? Tests cover various tasks from professional tools, general productivity to gaming. Then it's up to YOU to pick parts of review that interests YOU. Reviewers don't have a magic crystal ball to know what you'll do with the CPU.

 

Just like I skip all the games in graphic card benchmarks that I'll never ever play or be even remotely interested in them (for example anything from Ubisoft). Or should I go on a raging rampage how reviewers dare benchmarking with games I don't play, but don't benchmark those that I do. Not to mention everyone just benchmarks framerate, but when new features are in question, they often barely touch them or just mention them. For me, extra features were always more important because I always had stupid high framerate anyway...

 

And if you're one of those idiots who skip ALL the review pages and look at the verdict, then why are you even bothering to waste time looking at a review? That's like buying a Ferrari becaue it's the fastest and verdict even says so and then complain it's shit because you're living in a fucking mountain and it's terrible to drive on a trashed gravel road...

[TechyBen]

2 minutes ago, leadeater said:

I like to get from stand still to the speed limit rather quickly all the time . Not in a dangerous way but the woosh is fun sooo.... WOOOOOOOoooooooooooo.

Yeah. I think that's where it's strange Intel is complaining... Like, I've been in those "marketing" meetings for kettles and washing machines... TVs and the likes. The presenter makes a fault with the competitors amazing top end product, then shows how the bottom of the range from them somehow "beats" it.

"This TV only outputs 20 nits, but our metrics show single mums often don't use all 200 nits of our competitor when in the kitchen listening to the radio only".

 

[Master Disaster]

Is it time to bring this old meme back...

 

 

I mean seriously Intel? There's not much I can add to this discussion that hasn't already been said but wow does this reek of desperation from Intel's marketing department.

 

Using Notebook data to compare desktop CPUs

 

I really hope Linus tears them a new one on WAN show later today.