Intel wants reviewers to benchmark using windows media player instead of cinebench for low end mobile

[TechyBen]

Just now, Arika S said:

what an absolute train-wreck of a thread.

 

Should cinebench be ignored? No

 

If someone is reviewing a CPU and giving benchmarks, they had better do more than just cinebench. because yes, while it IS a metric, it's a metric that doesn't matter to most people and there are reviews (and be extension, consumers) out there that emphasis cinebench scores waaaay to much.

 

by what percentage? is there a direct line correlation between cinebench scores and chrome performance? 

 

If CPU 1 had a x% lower score than CPU 2 in cenebench, does that translate to an +/-#x% performance drops in Chrome that can be observed across the entire percentage range??

 

 

Not the entire no, as Cinebench loads the CPU 100% for so long. But there would be some correlation. Chrome don't offer a benchmark button. Cinebench do.

 

Thus one is graphed (average, Cinebench) one is typed (data point, Chrome) in reviews.

 

I'm happy to be pointed to these erronous 2 in 1 reviews with Cinebench. I mean, Linus must have done some. Hardware unboxed? Marques Brownlee? Gizmodo/TomsHardware/PCWorld/GamersNexus. You've got some really really easy targets there. Some really easy "black swans" to show I'm missing an obvious problem.

[ravenshrike]

1 minute ago, Arika S said:

If someone is reviewing a CPU and giving benchmarks, they had better do more than just cinebench.

Please point out the person(s) in this thread that argued that cinebench should be the only bench a reviewer does. Moreover, if you really want to argue that chrome et al should be included in reviews of modern processors, then it would be sufficient for reviews of all but the lowest end tablets and phones to include a geekbench run without delving further into the inanity.

[RejZoR]

10 hours ago, mr moose said:

you don't honestly believe that do you?  you are now trying to argue that a review presented to the public is not actually intended for the vast majority of people who are looking at that product,  but are that it is solely for the 0.22% of end users who will use sandra or 3dmark type workloads?

 

EDIT: and that is after you presented it as evidence for how the reviews are accurate and relevant for ignorant consumers.

Lmao, it's not about usage, it's about repeatable, comparable data points. So you know how fast something generally is within fully controlled and consistent environment. Not because people need  to run SiSoft Sandra day after day. Jesus, have you all turned stupid over night or something? That's what I explained some posts ago and you said I'm "contradicting myself". It seems more like you don't understand testing procedures at all. And I literally haven't seen a single review where they've only done synthetics and called it a day. Literally ALL of them have synthetic benchmark part, productivity in pro tools, real world stuff like compression and video encoding and gaming at various resolutions to spot bottlenecks. And same is done for graphic cards. No one really gives a shit about 3DMark, but if one graphic card scores twice as much points and individual tests have double the framerate, that's an indication of sort even if not realworld results. A repeatable and controlled, coz guess what, not all games have benchmark tools or timedemo abilities.

 

All this whining in this thread makes me wonder why no one started shitting on car specs yet, coz who really ever needs to give a fuck if Bugatti Chiron goes 480kmh when NO ONE can ever push it that fast on any public road. Heh. Omg, stupid meaningless car specs. Or times on Nurburgring. Yet manufacturers do it coz it's a consistent, repeatable demanding test "set" even though it doesn't reflect a real world case of time needed to buy some groceries... Just like synthetic benchmarks on computers.

 

Which is why this whole thread is laughable and so is Intel's whining and bitching about it. They were apparently fine with it all the past decades coz they were beating AMD, but now that AMD is slapping their ass, now it's a "problem" and needs to be addressed so people see "real world" results. Give me a break. People they are trying to target these reviews at don't read reviews anyway, they ask geek friends to tell them what to buy for X money or they blindly buy whatever is dragged through PR mills the most and they think that's the shit they need. The rest of us knows how to interpret a bloody CPU review. And they are perfectly fine the way it is. Not to mention, if Chrome is all the shit they care, they'll end up only selling Core i3's then coz that's all you ever need to run stupid Chrome. Sack the i5, i7 and i9 lines then, they are meaningless to casuals who apparently only run a browser.

[porina]

I was going to stay out of this thread but I see nothing has really moved on.

6 minutes ago, TechyBen said:

No. I believe that a *general* processing unit of a CPU is for *general* purpose. If it scores low in that... it will be low for any other general use case.

Cinebench as a benchmark is highly scaleable. The score at the end only represent loads that behave similarly. This is NOT most consumer loads.

 

One practical example I have is in the past I wanted to build a system for high fps gaming. I had recently got a Ryzen 1700 (8c16t). Slapped two 980Ti in SLI in it. Barely got above 60fps, even with overclocking. It really sucked. Moved the same GPUs into a stock 4c4t 6600k system. Bang, 120fps+ peak, 90 average in more challenging areas. Now, the title wasn't a "modern" game using high thread counts. What mattered were good cores, running fast. OG Ryzen did not provide that. If I were to repeat today with Zen 2, it would probably be a lot better.

 

The other tasks mentioned in your other reply, like virus scanning, typically run single thread anyway. Having a high thread count available doesn't make them run faster. At best, and only if you have a lot of heavy threads running at the same time which most don't, more threads on CPU could allow it to run smoother.

 

Put it another way, say you had a system with either a 3600 or 3900X in it. The 3900X would give close to 2x the Cinebench scores. Is it 2x faster at much else? Some things, maybe. Many things, no.

 

 

[mr moose]

42 minutes ago, TechyBen said:

 Intel are saying Cinebench means nothing and is pointless to some people.

No they are not it means nothing, Why do you argue these things when you clearly haven't read the article or any of the other stuff posts since?  For 5th or 6th time now Intel endorse the use of cinebench on hardware that is more appropriate for that.  And yes it is a pointless metric for some people.  Please stop pretending they are saying things they are not and please stop pretending cinebench is necessary in every review and for every person. It isn't.

42 minutes ago, TechyBen said:

But it's a metric. 0-60. MPG. Even your example of torque. For some people it's useless, for others it's very helpful.

OMG,  stop flip flopping. 

42 minutes ago, TechyBen said:

Complaining a single datapoint is not an average, is ridiculous. Both exist, single data points (Chrome is snappy/Windows 10 is laggy) and averages (Cinebench/Winzip/SyssofSandra), why say one is better than the other? Both are needed. And AFAIK those testing Cinbench are not testing it for the high compute alone, but as a baseline for if the other software (chrome, excel) will be as snappy.

 

If Intel make an FPGA Atom for a 2 in 1, that only computes Media player, Chrome and Excel, then yep... it would only compute those, and Cinbench would be obsolete. I don't think we have or are going to hit that any time soon. Intels sudden boost clocks do make differences for power virus vs burst loads. But as with SSDs this has been true for ages... and it still means consumers can get hit with a "it works snappy out the box 10 seconds later it chugs" because virus scans/updates/etc kill the performance (either the chip throttles, or the SSD hits it's cache limits).

 

Again, while tech can sometimes have new features, things like benchmarking can still show when and where it fails. Or when and where it succeeds. A mixed load/performance graph in Sysoft will map onto a mixed load Chrome/Excel to some degree.

 

I don't think you know what you are arguing anymore.  

[Murasaki]

people use WMP? 

[mr moose]

6 minutes ago, RejZoR said:

Lmao, it's not about usage, it's about repeatable, comparable data points. So you know how fast something generally is within fully controlled and consistent environment.

There is not point in having repeatable comparable results if those result mean nothing to the end user.  If they do not render then ther is no point in comparing cinebech results no matter what CPU you want to buy.  I don;t know whay that concept is so hard for people to understand.

6 minutes ago, RejZoR said:

Jesus, have you all turned stupid over night or something?

How Ironic, calling other people stupid when you haven't managed to grasp the actual issue and think your limited  understanding of testing and reviewing is better than everyone else's (including those who have worked int he industry longer than you've know what a computer is).

6 minutes ago, RejZoR said:

That's what I explained some posts ago and you said I'm "contradicting myself". It seems more like you don't understand testing procedures at all. And I literally haven't seen a single review where they've only done synthetics and called it a day.

Actually,  techyben posted one in the last 12 hours.  They ran cinbench, sandra and 3dpassmark on an atom and a pentium then concluded the processors were weak performers and only suited to web surfing.  They completely ignored everything in between.  So no, you are the one that doesn't understand and and should stop trying to explain things.

6 minutes ago, RejZoR said:

Literally ALL of them have synthetic benchmark part, productivity in pro tools, real world stuff like compression and video encoding and gaming at various resolutions to spot bottlenecks.

So your argument as to why they should is because they all do?  you're still failing to grasp the problem here. 

6 minutes ago, RejZoR said:

And same is done for graphic cards. No one really gives a shit about 3DMark, but if one graphic card scores twice as much points and individual tests have double the framerate, that's an indication of sort even if not realworld results. A repeatable and controlled, coz guess what, not all games have benchmark tools or timedemo abilities.

 

All this whining in this thread makes me wonder why no one started shitting on car specs yet, coz who really ever needs to give a fuck if Bugatti Chiron goes 480kmh when NO ONE can ever push it that fast on any public road. Heh. Omg, stupid meaningless car specs. Or times on Nurburgring. Yet manufacturers do it coz it's a consistent, repeatable demanding test "set" even though it doesn't reflect a real world case of time needed to buy some groceries... Just like synthetic benchmarks on computers.

Pointless comments that are out of context with what has happened.

6 minutes ago, RejZoR said:

Which is why this whole thread is laughable and so is Intel's whining and bitching about it. They were apparently fine with it all the past decades coz they were beating AMD, but now that AMD is slapping their ass, now it's a "problem" and needs to be addressed so people see "real world" results. Give me a break. People they are trying to target these reviews at don't read reviews anyway, they ask geek friends to tell them what to buy for X money or they blindly buy whatever is dragged through PR mills the most and they think that's the shit they need. The rest of us knows how to interpret a bloody CPU review. And they are perfectly fine the way it is. Not to mention, if Chrome is all the shit they care, they'll end up only selling Core i3's then coz that's all you ever need to run stupid Chrome. Sack the i5, i7 and i9 lines then, they are meaningless to casuals who apparently only run a browser.

So it seems your issue here is that it is Intel and not any of the actual logical reasons they put forward.  Well done.

[RejZoR]

I'm not even going to bother to respond to this dissected mess...

[mr moose]

50 minutes ago, TechyBen said:

No. I believe that a *general* processing unit of a CPU is for *general* purpose. If it scores low in that... it will be low for any other general use case.

That's nice for you, but people buy 2 in ones and low end hardware for specific purposes, and if no one gives them any comparable test results so they can decide which machine is better for them then what are they buying on?  I'll tell you, they are buying on the lack of real world information and the reviews conclusion based on how well it performed at a task the end user will never use.

50 minutes ago, TechyBen said:

Show me 1 CPU in a 2 in 1 that is slower in Cinebench but faster in Chrome, than any other CPU (AMD included, so that's a lot of CPU architectures!).

Show me an end user who cares about cinebench when they  only want to use chrome, office, email?

 

50 minutes ago, TechyBen said:

Is there a single CPU that intel makes, that is only fast in excel, Chrome, Media player, but cannot scale the same with Cinebench?

Why should there be, It's a fallacy to try and claim that because a CPU will always do better running chrome or media player that Tcinebench be should always be considered.  They are completely different work loads, you cannot tell if an A series is better than a Pentium or an i3 is better than a Ryzen3 at office work unless you actually compare bioth of them at office work.  Cinebench will tell you fuck all about it.

50 minutes ago, TechyBen said:

No. But they will do Windows 10 updates (file transfer, binary blob compute/update compiling will be done), virus scanning (multitasking), Chrome/Excel Java/apps (general compute that can often be poorly optimised).

 

As said. I have an i7 from intel, made to feel snappy, despite having "cut down" Cinebench performance. Guess what, if it had less Cinebench scores, it would also be slower in Chrome, if it had faster Cinebench scores, it would also be faster loading Steam/Compiling Linux updates 

Citation those workloads  scale in line with cinebench  scores. 

 

50 minutes ago, TechyBen said:

That TomsHardware article was not for 2 in 1 purchases. 

OMG! again,  come on, please try to keep up with the discussion.   Of course the atom and the Pentium in those were not 2 in 1's.  but it does illustrate exactly what Intel are saying and you were the one who linked it trying to argue it proved Intel wrong.

 

50 minutes ago, TechyBen said:

Intel complained the reviewer brought Cinebench... OK, where is this review? Where is it's fault? How do we know it was not so they could compare to the AMD chips 2 in 1 to see which was quickest for Web browsing? With Cinebench only as a baseline? How do we know it was not to check if it was a good Youtubers on the go laptop (Lots of reviews out there checking for this, as the Macbooks often do ok. Linus own videos have this).

 

Me missing the point?

Yep, you have, you are literally ignoring whole points made and you can;t seem to move past the concept that reviewers are not providing real world testing for mobile and low end hardware, your link proves it.

[mr moose]

2 minutes ago, RejZoR said:

I'm not even going to bother to respond to this dissected mess...

 

Claims he isn't going to respond by responding...   

[RejZoR]

1 minute ago, mr moose said:

 

Claims he isn't going to respond by responding...   

By responding point by point. What are you, freaking 5 years old? dafaq...

[mr moose]

Just now, RejZoR said:

By responding point by point. What are you, freaking 5 years old? dafaq...

So do you want to participate in the discussion or just make dumb comments then insult people when you no longer have an argument?

[RejZoR]

14 minutes ago, mr moose said:

So do you want to participate in the discussion or just make dumb comments then insult people when you no longer have an argument?

What discussion, you're consistently and repeatedly failing to grasp reality. There is no frigging review that only uses Cinebench. NONE. So where is the frigging problem? You're a gamer? Great, then fucking skip the Cinebench part of review and look at games results only. That's literally why CPU reviews cover so many different things. And that's why all this whining is laughable and just straight up retarded. People (and Intel) whining over non issue things that are already done in place. And if something can run Cinebench reasonably fast, I'm pretty sure it'll also run some stupid browser. LMAO

[ravenshrike]

54 minutes ago, mr moose said:

For 5th or 6th time now Intel endorse the use of cinebench on hardware that is more appropriate for that. 

Had the slide been clearly labeled in the presentation that kicked this shit off no one would give a damn. Instead, after a very clearly labeled slide about desktop computers comes a bunch of metrics with very tiny fine print that this is in regards to tablets and 2in1s. It's a bait and switch usage of the statistics that without the brouhaha the vast majority of people who saw the slides would assume that Intel is saying Cinebench is an inappropriate benchmark for desktop class processors, when in fact it is chrome et al that are pretty useless metrics for those processors.

[RejZoR]

It literally doesn't matter how anyone changes the methodology of tests, if people don't know how to interpret it, it's pointless no matter what you do. What are you going to do with Chrome that's apparently most used and super important? Oh, so it opens a webpage I don't really ever use 0.3 seconds faster than that lower CPU model. Wow, that tells me everything I need to know now. That tells normies absolutely nothing. LOL And then people will run that on some crappy ass 5400RPM HDD or DRAM-less SSD coz they are cheap, ruining everything anyone has worked for in the review. Or look how there is no difference in Chrome performance and then try video encoding and realize their CPU sucks. But it worked just as fast in Chrome as that 800€ CPU! What's going on!? LMAO

 

I was on a lookout for 3900X. But since 90% of stuff I do is gaming, after picking up only gaming parts of the reviews (that also had Cinebench, Blender and the lot), I realized it's not worth spending so much money because in games, the difference was just few frames compared to my old Core i7. Big brain ignored synthetics and went for the games. Mind blown technique CPU makers don't want you to know about!

[TechyBen]

3 hours ago, mr moose said:

That's nice for you, but people buy 2 in ones and low end hardware for specific purposes, and if no one gives them any comparable test results so they can decide which machine is better for them then what are they buying on?

So. Chrome, has specific code that can be boosted by the Intel chip in 2 in 1s faster than any other general process code? That Google has some special algorithm that is not using ints/floats/memory? That Chrome cpu usage does not scale to Sisoft Sandra benches, and Intels boost, works via boosting only for 1 second at a time (Chrome) and this does not show in any benchmarks?

 

That having a Cinebench score in LTT/GamersNexus/TomsHardware, for tech understanding and competent viewers interested in the IPC and sustained workload management of Intel chips, or thermal/memory/system management on sustained loads in 2 in 1s (which they do sometimes review) is a distraction to those reading PCworld/Gizmodo reveiws that say "lol, it's green, best 2 in 1 ever!"?

 

[Edit]

I'm asking for proof of the "thing in your hand your saying is a problem", not the imaginary impossible ones. There are, AFAIK no benchmarks agreed about Chrome/Media player etc, other than those mixed workloads things like Sisoft Sandra do. So if there is, I'm happy to be shown them (that should not be like asking for a teapot around the sun, as far as I know).

 

Here is my example, no Cinebench scores:

https://gizmodo.com/c/reviews/laptops-tablets/2-in-1s

If you find them, then yes... big problem, Intel is being rubbished for no good reason... Or is it just tech tubers/sites/magazines, checking Intels ipc/cpu structure, posting Cinebench scores, for informed consumers?

 

[TechyBen]

3 hours ago, porina said:

I was going to stay out of this thread but I see nothing has really moved on.

Cinebench as a benchmark is highly scaleable. The score at the end only represent loads that behave similarly. This is NOT most consumer loads.

 

One practical example I have is in the past I wanted to build a system for high fps gaming. I had recently got a Ryzen 1700 (8c16t). Slapped two 980Ti in SLI in it. Barely got above 60fps, even with overclocking. It really sucked. Moved the same GPUs into a stock 4c4t 6600k system. Bang, 120fps+ peak, 90 average in more challenging areas. Now, the title wasn't a "modern" game using high thread counts. What mattered were good cores, running fast. OG Ryzen did not provide that. If I were to repeat today with Zen 2, it would probably be a lot better.

 

The other tasks mentioned in your other reply, like virus scanning, typically run single thread anyway. Having a high thread count available doesn't make them run faster. At best, and only if you have a lot of heavy threads running at the same time which most don't, more threads on CPU could allow it to run smoother.

 

Put it another way, say you had a system with either a 3600 or 3900X in it. The 3900X would give close to 2x the Cinebench scores. Is it 2x faster at much else? Some things, maybe. Many things, no.

 

 

If you have a single core, virus scanner or not, it will be slower than a multicore.

You then don't "lag" in Chrome when scanning, because more cores to run other things (Video + Chrome + update + scanner etc). Cinebench would show this, that a multicore would score higher points than a single core, right? Will the average metric of Cinebench show *specific* code/performance metrics? No. It's not a benchmark *suit*, not is it tailored to those specifics. Where is Intel's "burst technology" benchmark we should be using for 2 in 1s? They make (OEM whatever) them, so they'd know best... right?

 

I am not sure how the example of your gaming rig shows anything about the topic? On average the Ryzen will perform better. "Average" (Cinebench example). A specific, your game with 4 threaded clocks preferring code will perform better on the i5 (Chrome example). Yep. No one is saying not to also use other metrics. How does having a Cinebench and game FPS score make it more difficult to chose or know how a CPU will perform?

[colonel_mortis]

This topic seems to have descended into a flame war, and it doesn't look like there's much more constructive discussion to be had. As such, I am locking it here.