Intel wants reviewers to benchmark using windows media player instead of cinebench for low end mobile

[leadeater]

2 minutes ago, Blademaster91 said:

Intel used a slide with mobile laptop data without any clear indication it wasn't taken from desktops, unless someone is really paying attention to the fine print then it could be very misleading.

It was clear if you were present at the conference, slides alone can be highly misleading if not designed or intended to be viewed standalone.

[pas008]

20 minutes ago, 5x5 said:

You're missing the bigger picture. A billion dollar company is acting like an angry toddler.

see below

19 minutes ago, Lathlaer said:

The previous slide was about desktop CPU's. The next one is about real world usage statistics with small print that it was taken from mobile devices. Without any kind of transition between those slides it looks like Intel is dishonestly trying to show real world usage of desktops using mobile data so people got confused and angry.

2 different data pools

didnt i link cascade x supposedly performance increase? along others mentioning small print?

 

its not that hard to determine or equate the differences

 

change a variable you change the outcome everything especially when it comes mobile vs desktop

anyone should know this change one thing you change it all

 

13 minutes ago, Blademaster91 said:

I'm going by what is mentioned here and the Der8auer video,Intel used a slide with mobile laptop data without any clear indication it wasn't taken from desktops, unless someone is really paying attention to the fine print then it could be very misleading. I'm not biased and want to see companies treated equally, Intel needs a good kicking and while I see the point of suggesting reviewers using more realistic benchmarks I like to see as many tests as possible of whatever can push the CPU hard.

fine print will always be there im not excusing intel on this but this is the norm, commercials even have this, might suffer from ed, low bp, etc etc etc

intel needs a harsh kicking in the ass for once yes but do it sanely

 

and i will say it again if you want a real benchmark someone test/benchmark these mobile or desktop cpus in real world scenarios with real back ground shit running besides windows

i dont know about you but music, videos, monitoring software, chat, etc always running while gaming

lets see if amds or intels shit is doing it better

you think testing something sterile is the answer for a multitasking system?

 

but back on point

normal multitasker could be informed on which is best for them

heavy multitasker would know which route to go

[Lathlaer]

1 minute ago, pas008 said:

2 different data pools

didnt i link cascade x supposedly performance increase? along others mentioning small print?

 

its not that hard to determine or equate the differences

 

change a variable you change the outcome everything especially when it comes mobile vs desktop

anyone should know this change one thing you change it all

Man, you are either totally missing my point or confusing me with someone else.

[pas008]

3 minutes ago, Lathlaer said:

Man, you are either totally missing my point or confusing me with someone else.

ive reread things am i missing something?

 

you can pm me too

[Master Disaster]

1 hour ago, leadeater said:

I would say that's actually highly under what Dell would be buying. If we even just stick to desktop processors not laptop etc (also not server), I wouldn't be surprised if it's closer to 100:1 than 10:1 all US retail shops.

 

Could be completely wrong but based on what I know about how many computers companies have and how often they get replaced Dell should be having tens to hundreds of thousands of purchase orders for multiple thousand computers on the go every month consistently throughout the year.

 

And before posting I'll check my assumption with some data *looking up information*.

Going to assume that includes laptops.

 

10.65 * 4 = 42.6 million per year (assumption of each quarter being equal which is wrong).

Units per month: 3.55 million

Units per week: 887.5 thousand

Units per day: 126,785.7

 

Latest firm figure I can find Intel sells 400 million CPUs per year, all SKUs.

 

Edit:

 

126,000 units per day.... (In fact 127,000 units per day really)

 

That's insane.

[TechyBen]

3 hours ago, mr moose said:

Not sure what you are trying to say, you highlighted the issue that explains all the misunderstanding yet you want to continue as if it isn't real. 

 

I think the problem here is everyone is getting way too specific and particular.  You know marketing is always going to be loose, that doesn't mean the odd ultra-book they claim can do light video processing undermines their claims that 99% of mobile users don't do any video work. Nor does it change the fact that cinebench gets used too much on devices that users clearly have no intention of using.

 

 

Again. I gave like 2 or 3 examples of my understanding, but the answer is "your still wrong, see..." with no actual answer. Where have I missed this real explanation? What?

Is Intel saying, that Media player and Excel should be used as benchmarks instead of Cinebench? Or that we should segment benchmarks between processor type/range/use?

[spartaman64]

12 hours ago, Taf the Ghost said:

https://medium.com/performance-at-intel/food-for-thought-on-our-ifa-event-395dc775de9

 

@spartaman64 Intel released something of an response to all of this.

edit: ok it seems like the presenter just didnt think about how the slides will be interpreted for people who arent at his presentation. i know ryan shrout from his pc per days and if he was the one who made the slides i know him well enough to give him the benefit of the doubt and say thats not his intention to mislead people. but i still dont think windows media player is a good benchmarker  

[5x5]

1 hour ago, spartaman64 said:

edit: ok it seems like the presenter just didnt think about how the slides will be interpreted for people who arent at his presentation. i know ryan shrout from his pc per days and if he was the one who made the slides i know him well enough to give him the benefit of the doubt and say thats not his intention to mislead people. but i still dont think windows media player is a good benchmarker  

I'd like to state a different opinion. Ryan Shrout has been less than forthcoming ever since joining intel so I am certain of two things. One, if he had seen this presentation, he most certainly knew how it appears and didn't move a muscle to correct the record until a shit storm was being locked up and two, he's th chief strategist gland as such almost certainly played a role in the creation. It's his literal job.

[mr moose]

11 hours ago, Master Disaster said:

It also doesn't mean Cinebench has no value though.

No has said that.  Not Intel nor anyone in this thread. 

11 hours ago, Master Disaster said:

Synthetic benchmarks give users a number they can look at to indicate how the device is going perform and that's valuable, especially to people who don't follow tech and don't know what to expect from a Core I5 6547864SFTR (because that's how confusing Intel's naming convention has got).

Yes, but horses for courses, just like all work load types,  some that replicate or actually test s a specific work load are not applicable to people who don't do anything like that work load. No one is saying synthetics are crap, they are saying there are situations where they are irrelevant in a review.

 

11 hours ago, Master Disaster said:

Intel is trying to push away from Cinebench because AMD is starting to win, they didn't care about it for the last 10 years while they were winning.

No they are not, they endorse it's use. they are just saying it is unhelpful to include it in reviews and draw conclusions on it for hardware (2in1's) that is never going to see that type of workload, both by design and for the end users intended purpose.

 

10 hours ago, TechyBen said:

What issue? Everyone has replied "oh, yes, that intel slide", but not actually said what about it. XD

??  many have.  there was a few journalists who took the slide out of context and told everyone they were being silly when all they wanted was for reviewers to not include cinebench on lower end mobile tech not designed for it. 

10 hours ago, TechyBen said:

How is Cinebench a problem for Intel?

It's not, they have said they endorse it for HEDT and workstations.  They have no issue with cinebench. their issue is reviewers who use include it in lower end hardware reviews and they claim that clouds the reality of the product.

10 hours ago, TechyBen said:

Is there any other metric they suggest should be used?

Lot's but no one knows how that should be implemented let alone what it would prove given even a n4000 can run youtube in the background while working on a word document and having email open at the same time (about the average workload for most laptops and 2 in 1's). 

10 hours ago, TechyBen said:

Is this mentioned in either the original slide, or the replies after?

The other testing suggestions?  I don't know, I  wasn't at the presentation. they list the most used software on these devices so I assume there is a way of testing, however I don't personally understand what they hope to achieve from that. What I interpret out of the whole thing is that Intel are primarily unhappy that reviewers are using cinebench on hardware not designed for it thus giving it a worse review outcome.

 

 

10 hours ago, TechyBen said:

I don't see how Media player is going to be a metric. I do know some sites *do* benchmark Excel. But does Excel really scale differently to Cinebench?

Maybe load times, stutter,  ability to do it while editing and saving and spreadsheet.  I don't know. What I do know is that cinebench won't tell you that either, in fact cinebench will tell you nothing except how 2 in 1's will render using that workload type.

 

About the only thing I can think you could use it for would be a constant load (have it on repeat) to test battery life.  Then it would be a useful tool to gather useful information, however you could run prime95 or something like that and get the same results.

[mr moose]

6 hours ago, spartaman64 said:

edit: ok it seems like the presenter just didnt think about how the slides will be interpreted for people who arent at his presentation. i know ryan shrout from his pc per days and if he was the one who made the slides i know him well enough to give him the benefit of the doubt and say thats not his intention to mislead people. but i still dont think windows media player is a good benchmarker  

Agree, I can't see how testing low end hardware on software that we all know it can run just fine is going to prove anything.  I think the whole thing is more about drawing attention away from irrelevant testing than it is about providing something actually useful for buyers.

[TechyBen]

1 hour ago, mr moose said:

??  many have.  there was a few journalists who took the slide out of context and told everyone they were being silly when all they wanted was for reviewers to not include cinebench on lower end mobile tech not designed for it. 

That is silly, not the reviewers. Baseline standard. Benchmark. It does not matter if it checks 0-60 in a van, that people use to transport staff around an airport. It's a metric. You mention if the car has nice cup holders, even if some people insist never to drink in the car.

 

Those benching 2 in 1s with Cinebench, will *also* note day to day use (sluggish/snappy/poor battery/brilliant screen etc). Those not... are who? Like, even PC world/Gizmodo aint making that error. Who is this imaginary boogie man Intel is afraid of?

 

Quote

Lot's but no one knows how that should be implemented let alone what it would prove given even a n4000 can run youtube in the background while working on a word document and having email open at the same time (about the average workload for most laptops and 2 in 1's). 

So don't benchmark 2 in 1s? Like, you would not care what it's baseline performance is when the core is pegged (Cinebench) high use (gaming) or average workload (un/zipping files)? The main 3 all reviewers use...??? I mean, Cinebench may not 100% translate to Windows 10 updates, but I know those can peg the CPU... and waiting an age, vs quicker, is a metric that some people care about. It's an actual performance mark, for like... performance. 

 

Quote

About the only thing I can think you could use it for would be a constant load (have it on repeat) to test battery life.  Then it would be a useful tool to gather useful information, however you could run prime95 or something like that and get the same results.

No. The math says no. Core boosts differ and voltages differ on use case. Full peg vs video vs web browsing. It won't give you battery stats like that (you could min/max it that way, but not with cinebench alone).

 

You complained it's not a use case (benchmarking cpu), then gave it as a use case (benchmarking battery)... 

 

It's a baseline for throughput, total capacity of the IPC (depending on if it's a "power virus" type load/compute). It won't tell you specifics, but it will averages.

 

Can it be gamed? Yes. But any "metric" can, or can be specifically marketed/focused on/dedicated ipc/code/hardware given (see HD video codecs, and how they kill old hardware, run with zero cpu use on new hardware, because hardware support/chip designs).

 

"Stop testing out KIAs for 0-60 and up hill climb tests, they are irrelivant... our customers never overtake, and always live in flat countries... no one uses them like that!"

[mr moose]

14 minutes ago, TechyBen said:

That is silly, not the reviewers. Baseline standard. Benchmark. It does not matter if it checks 0-60 in a van, that people use to transport staff around an airport. It's a metric. You mention if the car has nice cup holders, even if some people insist never to drink in the car.

 

Those benching 2 in 1s with Cinebench, will *also* note day to day use (sluggish/snappy/poor battery/brilliant screen etc). Those not... are who? Like, even PC world/Gizmodo aint making that error. Who is this imaginary boogie man Intel is afraid of?

You're welcome to want a cinebench score on an i3 netbook.  but the argument (and rational argument at that) is that it serves no real benefit to anyone other than to add more numbers to a review that doesn't need them.

 

Also not sure why you persist in talking about 0-60, If you require low end torque then knowing the 0-60 is a pointless metric that literally means nothing, it doesn't even provide you with useful information when comparing to other vehicles. How fast a vehicle can take off tells you nothing about how well it can tow or carry a load.  

Quote

So don't benchmark 2 in 1s? Like, you would not care what it's baseline performance is when the core is pegged (Cinebench) high use (gaming) or average workload (un/zipping files)? The main 3 all reviewers use...??? I mean, Cinebench may not 100% translate to Windows 10 updates, but I know those can peg the CPU... and waiting an age, vs quicker, is a metric that some people care about. It's an actual performance mark, for like... performance. 

People buying 2 in 1's are still in need of decent reviews and comparisons.  they just don't need to know how well a pc designed and intended for email/excel will convert an AVI to any other codec.  

 

[TechyBen]

So, here is an example:

https://www.tomshardware.co.uk/intel-atom-cpu,review-30931-8.html

 

Super in depth. Benching and checking the CPU and it's performance for those workloads.

 

Quote

The processor itself is a success – it’s affordable, consumes very little power, and while its performance is weak, it’s sufficient for its target market (low-cost PCs intended for Web use). In addition, HyperThreading is a good feature and the platform is reactive.

So, if you're looking for a workhorse, nope. If your looking for a web browser, yep. How is it not relevant? Those on Toms Hardware,might be looking for raspi pi replacement, and number cruncher server with low power use... and decide if it's got the omph for them.

 

On PCWorld/Gizmodo they will just say "It's fast/slow for web browsing".

[TechyBen]

Just now, mr moose said:

You're welcome to want a cinebench score on an i3 netbook.  but the argument (and rational argument at that) is that it serves no real benefit to anyone other than to add more numbers to a review that doesn't need them.

 

Also not sure why you persist in talking about 0-60, If you require low end torque then knowing the 0-60 is a pointless metric that literally means nothing, it doesn't even provide you with useful information when comparing to other vehicles. How fast a vehicle can take off tells you nothing about how well it can tow or carry a load.  

People buying 2 in 1's are still in need of decent reviews and comparisons.  they just don't need to know how well a pc designed and intended for email/excel will convert an AVI to any other codec.  

 

You are saying I don't need to know if the i3 is faster or slower at compute than my existing system? That installing/zipping/file transfers/render (any visual really, video etc), is not important to me, and Cinebench won't help me choose the *fastest* i3/i5/Ryzen chip for my laptop... Huh? REALLY? (I've got a snappy i7 2 core HT, I could check it's benches, and tell you if they mach chrome, even though they are benched in Cinebench. My Desktop is an ancient i5, and slightly slower than the i7, so comparing would be a nice example).

 

Quote

If you require low end torque then knowing the 0-60 is a pointless metric that literally means nothing, it doesn't even provide you with useful information when comparing to other vehicles.

Yep. The reviews then say the two. "quick to pull away" and/or "quick to overtake". But if 0-60 is 20 seconds, or 6, I know it's gonna hurt or not to drive. XD

 

Quote

 How fast a vehicle can take off tells you nothing about how well it can tow or carry a load.  

I. Am. Speechless. That's it. Done. I'm out.

[mr moose]

1 minute ago, TechyBen said:

So, here is an example:

https://www.tomshardware.co.uk/intel-atom-cpu,review-30931-8.html

 

Super in depth. Benching and checking the CPU and it's performance for those workloads.

 

So, if you're looking for a workhorse, nope. If your looking for a web browser, yep. How is it not relevant? Those on Toms Hardware,might be looking for raspi pi replacement, and number cruncher server with low power use... and decide if it's got the omph for them.

 

On PCWorld/Gizmodo they will just say "It's fast/slow for web browsing".

Apart from being over a decade old, their point is highlighted quite well in the fact that the review used cinebench, sandra, 3dmark and then claims the processor is weak performing and it's only use is web surfing. No shit?  it was never intended for any of those,  what about office, spreadsheets, website design?  all the other stuff someone might use it for?

 

Thank you for proving their point.

[TechyBen]

4 minutes ago, mr moose said:

Apart from being over a decade old, their point is highlighted quite well in the fact that the review used cinebench, sandra, 3dmark and then claims the processor is weak performing and it's only use is web surfing. No shit?  it was never intended for any of those,  what about office, spreadsheets, website design?  all the other stuff someone might use it for?

 

Again... did you read my comment? Some people use Raspberry Pies for compute. Like, actually use them. Some people check *market segmentation* to know which chip is "faster". Do I get the ATOMx765 or the ATOMqV25 laptop? I want the fastest, not the cheapest... but both are the same price, I'm not sure which has better battery life, or which is for gaming... oh, no, Mr Moose and Intel said the benchmarks are irrelevant, so I  have been told I'm not allowed to know which is an upgrade... gotta buy the one Intel insist I buy, and not make an informed choice!!!

 

It matters not if they bench Cinebench, Winzip Or SysSandra, but that they are consistent. That they update when needed (when multicore added, when AVX added etc).

 

Quote

Thank you for proving their point.

That people don't know what they are buying, and are happy to be duped?

[mr moose]

1 minute ago, TechyBen said:

You are saying I don't need to know if the i3 is faster or slower at compute than my existing system? That installing/zipping/file transfers/render (any visual really, video etc), is not important to me, and Cinebench won't help me choose the *fastest* i3/i5/Ryzen chip for my laptop... Huh? REALLY? (I've got a snappy i7 2 core HT, I could check it's benches, and tell you if they mach chrome, even though they are benched in Cinebench. My Desktop is an ancient i5, and slightly slower than the i7, so comparing would be a nice example).

Really?  I am saying people who are buying 2 in 1's are not doing it for rendering, thus those tests are not relevant. 

1 minute ago, TechyBen said:

 

Yep. The reviews then say the two. "quick to pull away" and/or "quick to overtake". But if 0-60 is 20 seconds, or 6, I know it's gonna hurt or not to drive. XD

 

I. Am. Speechless. That's it. Done. I'm out.

 

Again,  there are very different requirements in vehicles just like in computers.  you are trying to argue that knowing the torque of a truck is helpful when you want fast.  Or the other way around, it doesn't matter because the argument does not illustrate the point you are trying to make.

 

[TechyBen]

Just now, mr moose said:

Really?  I am saying people who are buying 2 in 1's are not doing it for rendering, thus those tests are not relevant. 

 

Again,  there are very different requirements in vehicles just like in computers.  you are trying to argue that knowing the torque of a truck is helpful when you want fast.  Or the other way around, it doesn't matter because the argument does not illustrate the point you are trying to make.

 

Yep. But guess what. They will post the 0-60 of a bike, car and truck... are you saying they should not?

One simple question for you. Making it very clear. Are you saying, a metric such as 0-60, should not be used, or measured?

[mr moose]

1 minute ago, TechyBen said:

 

 

Again... did you read my comment? Some people use Raspberry Pies for compute. Like, actually use them. Some people check *market segmentation* to know which chip is "faster". Do I get the ATOMx765 or the ATOMqV25 laptop? I want the fastest, not the cheapest... but both are the same price, I'm not sure which has better battery life, or which is for gaming... oh, no, Mr Moose and Intel said the benchmarks are irrelevant, so I  have been told I'm not allowed to know which is an upgrade... gotta buy the one Intel insist I buy, and not make an informed choice!!!

 

It matters not if they bench Cinebench, Winzip Or SysSandra, but that they are consistent. That they update when needed (when multicore added, when AVX added etc).

 

That people don't know what they are buying, and are happy to be duped?

That review is the exact reason Intel have said what they have said, if course people don;t know what they are buying, that is why they look at reviews. but if the reviews are not testing the product for their intended use then the duping you speak off is because the reviewers missed the mark, it has nothing to do with the consumer being ignorant.

[TechyBen]

1 minute ago, mr moose said:

That review is the exact reason Intel have said what they have said, if course people don;t know what they are buying, that is why they look at reviews. but if the reviews are not testing the product for their intended use then the duping you speak off is because the reviewers missed the mark, it has nothing to do with the consumer being ignorant.

TomsHardware are not testing it for the intended use? A development platform... for compute on low power *or* web browsing?

You do realise averages are not individual data points, right?

 

[boom]

https://www.supermicro.com/products/nfo/atom.cfm

[mr moose]

1 minute ago, TechyBen said:

Yep. But guess what. They will post the 0-60 of a bike, car and truck... are you saying they should not?

One simple question for you. Making it very clear. Are you saying, a metric such as 0-60, should not be used, or measured?

I am saying that metric means nothing to some people.  how can you not see that  0-60 figure is absolutely pointless for people who need torque. and just like in your review example, toms only gave half the information needed.

[mr moose]

9 minutes ago, TechyBen said:

TomsHardware are not testing it for the intended use? A development platform... for compute on low power *or* web browsing?

You do realise averages are not individual data points, right?

 

[boom]

https://www.supermicro.com/products/nfo/atom.cfm

you don't honestly believe that do you?  you are now trying to argue that a review presented to the public is not actually intended for the vast majority of people who are looking at that product,  but are that it is solely for the 0.22% of end users who will use sandra or 3dmark type workloads?

 

EDIT: and that is after you presented it as evidence for how the reviews are accurate and relevant for ignorant consumers.

[TechyBen]

10 hours ago, mr moose said:

I am saying that metric means nothing to some people.  how can you not see that  0-60 figure is absolutely pointless for people who need torque. and just like in your review example, toms only gave half the information needed.

I never said it's not. Intel are saying Cinebench means nothing and is pointless to some people. But it's a metric. 0-60. MPG. Even your example of torque. For some people it's useless, for others it's very helpful.

 

Complaining a single datapoint is not an average, is ridiculous. Both exist, single data points (Chrome is snappy/Windows 10 is laggy) and averages (Cinebench/Winzip/SyssofSandra), why say one is better than the other? Both are needed. And AFAIK those testing Cinbench are not testing it for the high compute alone, but as a baseline for if the other software (chrome, excel) will be as snappy.

 

If Intel make an FPGA Atom for a 2 in 1, that only computes Media player, Chrome and Excel, then yep... it would only compute those, and Cinbench would be obsolete. I don't think we have or are going to hit that any time soon. Intels sudden boost clocks do make differences for power virus vs burst loads. But as with SSDs this has been true for ages... and it still means consumers can get hit with a "it works snappy out the box 10 seconds later it chugs" because virus scans/updates/etc kill the performance (either the chip throttles, or the SSD hits it's cache limits).

 

Again, while tech can sometimes have new features, things like benchmarking can still show when and where it fails. Or when and where it succeeds. A mixed load/performance graph in Sysoft will map onto a mixed load Chrome/Excel to some degree.

[TechyBen]

10 hours ago, mr moose said:

you don't honestly believe that do you?  you are now trying to argue that a review presented to the public is not actually intended for the vast majority of people who are looking at that product,  but are that it is solely for the 0.22% of end users who will use sandra or 3dmark type workloads?

 

EDIT: and that is after you presented it as evidence for how the reviews are accurate and relevant for ignorant consumers.

No. I believe that a *general* processing unit of a CPU is for *general* purpose. If it scores low in that... it will be low for any other general use case.

 

Show me 1 CPU in a 2 in 1 that is slower in Cinebench but faster in Chrome, than any other CPU (AMD included, so that's a lot of CPU architectures!).

 

Is there a single CPU that intel makes, that is only fast in excel, Chrome, Media player, but cannot scale the same with Cinebench?

 

Quote

  but are that it is solely for the 0.22% of end users who will use sandra or 3dmark type workloads?

No. But they will do Windows 10 updates (file transfer, binary blob compute/update compiling will be done), virus scanning (multitasking), Chrome/Excel Java/apps (general compute that can often be poorly optimised).

 

As said. I have an i7 from intel, made to feel snappy, despite having "cut down" Cinebench performance. Guess what, if it had less Cinebench scores, it would also be slower in Chrome, if it had faster Cinebench scores, it would also be faster loading Steam/Compiling Linux updates 

 

Quote

EDIT: and that is after you presented it as evidence for how the reviews are accurate and relevant for ignorant consumers.

That TomsHardware article was not for 2 in 1 purchases. 

Intel complained the reviewer brought Cinebench... OK, where is this review? Where is it's fault? How do we know it was not so they could compare to the AMD chips 2 in 1 to see which was quickest for Web browsing? With Cinebench only as a baseline? How do we know it was not to check if it was a good Youtubers on the go laptop (Lots of reviews out there checking for this, as the Macbooks often do ok. Linus own videos have this).

 

Me missing the point?

[Arika]

what an absolute train-wreck of a thread.

 

Should cinebench be ignored? No

 

If someone is reviewing a CPU and giving benchmarks, they had better do more than just cinebench. because yes, while it IS a metric, it's a metric that doesn't matter to most people and there are reviews (and be extension, consumers) out there that emphasis cinebench scores waaaay to much.

 

14 minutes ago, TechyBen said:

Guess what, if it had less Cinebench scores, it would also be slower in Chrome, if it had faster Cinebench scores, it would also be faster loading Steam/Compiling Linux update

by what percentage? is there a direct line correlation between cinebench scores and chrome performance? 

 

If CPU 1 had a x% lower score than CPU 2 in cinebench, does that translate to an +/-#x% performance drops in Chrome that can be observed across the entire percentage range?