Early benchmarks (Geekbench) of an Asus Windows 10 on ARM device got leaked — and it sucks

[captain_to_fire]

Sources: Winfuture.de via Slashgear

 

Quote

Following last week’s revelation of a now deleted Geekbench entry comes one that drags ASUS’ name into the news. Sadly, this Windows 10 on ARM sighting might be the worst we’ve seen so far.

 

This isn’t a case of Windows 10 on ARM vs Windows 10 on x86. It is, after, arguable that a quad-core Intel Core i7, even a fanless one, could outperform an octa-core Qualcomm Snapdragon 835 under certain condition. No, this is a case of Windows 10 on ARM versus, say, Android on ARM, running on the exact same chipset and closely similar hardware.

 

An Android device running on a Snapdragon 835 averages 2,200 on single-core and 7,700 on multi-core tests on Geekbench. Last week’s mysterious “Qualcomm CLS” device 1,202 and 4,263 on those same scores. This ASUSTeK TP370QL, in comparison, yields an embarrassing 889 in single-core performance and 3,174 in multi-core at its highest. Granted, the ASUS device has only 4 GB of RAM, versus the Qualcomm CLS’ 8 GB. And, granted, these are most likely development boards with very early versions of Windows 10 on ARM. There are, however, still no signs of progress, nearly one year after the initial announcement.

That is disappointing. Just look at the Galaxy Note 8 with the same SoC:

 

Quote

The model number also reveals a few details. So she suggests that it is a 2-in-1 device from the ZenBook Flip series, as it has "traditional" model numbers with the letter sequence "TP" at the beginning. The numbers in the model number provide information: the "3" stands for the display diagonal, which will probably be around 13 inches. The "7" in turn suggests that it could probably be more of a premium segment device. In both cases, however, this would presuppose that ASUS adhered to its usual scheme for the model numbers. 

 

The benchmark results of the new ASUS notebook with Qualcomm CPU are about the same level as in the previously seen prototypes of Qualcomm and HP - about one third of the results reported by current smartphones with Android and the same processor. Apparently, one is currently still in development, since the different measurement results each have widely varying version numbers of the firmware. 

I know how people in the forum tend to be dismissive of synthetic benchmarks including Geekbench but remember for the most part, devices scoring high in Geekbench are the ones that actually perform very well. Since Geekbench is an x86 application, it must be running on the x86 emulation layer of W10 on ARM and Microsoft promised users that many of their existing x86/64 applications will run just fine on an ARM Windows 10 device.

 

I don't think with emulation an application can run on its native speed nor can use the hardware efficiently but if those abysmal Geekbench scores is an indication on how a typical x86/64 app will run on a Windows 10 tablet running on Snapdragon 835, I think this might be headed for another Windows RT flop unless Microsoft has some other tricks up its sleeves that can change things. Windows 10 on ARM looks just like Windows 10 on an Intel/AMD PC and both can run Windows Store apps and x86/64 apps that aren't on the store and yet the performance is way better on an Intel/AMD tablet. It's obvious that Microsoft is positioning these upcoming ARM based PCs running on Windows 10 to go head to head with iOS, Android and Chrome OS but I think they're giving way too much promises here. While an iPad can only run apps from the app store, as a user I am sure that the thousands of iPad apps will run just fine and take advantage of the hardware because it's running natively unlike an ARM Windows 10 PC which can run UWP apps and some x86/64 applications but not as fast as a laptop with AMD/Intel inside. 

 

Also, there are many games in the Windows Store that are x64 exclusive.

 

Are these incompatible apps will appear on the screens of Windows 10 on ARM users? They might get the wrong impression just because their device can run Photoshop means it can run games found on the Windows Store. I could be wrong here but I think Microsoft is creating once again a hot mess they tried to erase when they ceased development of Windows RT by trying to be everything at once.

 

Edited November 25, 2017 by hey_yo_

[TheCherryKing]

This is not surprising. It would be great if the real x86 processors could be made more energy efficient. Unfortunately, that is very difficult. 

[captain_to_fire]

Just now, TheCherryKing said:

This is not surprising. It would be great if the real x86 processors could be made more energy efficient. Unfortunately, that is very difficult. 

Intel/AMD can shrink the transistors further to 10 nm or even less

[TheCherryKing]

Just now, hey_yo_ said:

Intel/AMD can shrink the transistors further to 10 nm or even less

That would be nice. I have a felling it will never be done. Just look at how many times Cannonlake has been delayed.

[captain_to_fire]

Just now, TheCherryKing said:

That would be nice. I have a felling it will never be done. Just look at how many times Cannonlake has been delayed.

I'd rather see Microsoft taking a few pages of Apple's playbook by adding a 15W ULV core i5 for resource intensive tasks and a Snapdragon 835 for lighter taks which will greatly improve battery life. The question is can Microsoft do that kind of hand off of tasks between chips?

[TheCherryKing]

1 minute ago, hey_yo_ said:

I'd rather see Microsoft taking a few pages of Apple's playbook by adding a 15W ULV core i5 for resource intensive tasks and a Snapdragon 835 for lighter taks which will greatly improve battery life. The question is can Microsoft do that kind of hand off of tasks between chips?

Resource intensive tasks is the main reason why x86 tablets and laptops aren't going away anytime soon. 

[captain_to_fire]

Just now, TheCherryKing said:

Resource intensive tasks is the main reason why x86 tablets and laptops aren't going away anytime soon. 

I think for the non techie people who would buy these ARM powered Windows 10 tablets and laptops, they might get the impression that they can run something like DaVinci Resolve off of it and expect it to run well which I think is just fantasy land. 

[DildorTheDecent]

Real benchmarks only pls.

 

Geekbench sucks.

[captain_to_fire]

5 minutes ago, DildorTheDecent said:

Real benchmarks only pls.

 

Geekbench sucks.

At the moment, the only performance benchmarking apps available on the Windows Store are AnTuTu, GFXBench, and Relative Benchmark but then some of these Windows Store apps are also for x86/64 only so benchmarking a Windows 10 on ARM device might pose some challenges. 

[DildorTheDecent]

23 minutes ago, hey_yo_ said:

At the moment, the only performance benchmarking apps available on the Windows Store are AnTuTu, GFXBench, and Relative Benchmark but then some of these Windows Store apps are also for x86/64 only so benchmarking a Windows 10 on ARM device might pose some challenges. 

I'd take AnTuTu over Geekbench any day.

 

It's hilarious to me that a reviewer would honestly use Geekbench as a performance metric. Only becasue it's probably the easiest to understand.

[Bit_Guardian]

1 hour ago, hey_yo_ said:

I'd rather see Microsoft taking a few pages of Apple's playbook by adding a 15W ULV core i5 for resource intensive tasks and a Snapdragon 835 for lighter taks which will greatly improve battery life. The question is can Microsoft do that kind of hand off of tasks between chips?

It certainly can, just like offloading to a GPU for hardware acceleration in encoding.

[Trixanity]

2 hours ago, hey_yo_ said:

I'd rather see Microsoft taking a few pages of Apple's playbook by adding a 15W ULV core i5 for resource intensive tasks and a Snapdragon 835 for lighter taks which will greatly improve battery life. The question is can Microsoft do that kind of hand off of tasks between chips?

It'll drive up costs. It would be better if Intel or AMD integrated ARM cores on die.

[Sauron]

DUH, if you emulate it it performs poorly... nobody with any understanding of how emulation works would have expected anything different. "Just fine" for microsoft word doesn't mean it will be fast.

2 hours ago, DildorTheDecent said:

Real benchmarks only pls.

 

Geekbench sucks.

Emulated x86 geekbench I might add.

[Bit_Guardian]

25 minutes ago, Trixanity said:

It'll drive up costs. It would be better if Intel or AMD integrated ARM cores on die.

I'm sure Intel will if the demand appears. It has a full ARM license after all.

 

11 minutes ago, Sauron said:

DUH, if you emulate it it performs poorly... nobody with any understanding of how emulation works would have expected anything different. "Just fine" for microsoft word doesn't mean it will be fast.

Emulated x86 geekbench I might add.

Emulation does not have to be slow. What do you think the Java JVM is exactly? It is the emulation of a theoretical machine. Java can in fact be just as fast or faster than optimized C++ at some things (if you don't have an equally good lazy garbage collection runtime in your C++ program).

[Dabombinable]

Its Geekbench, its not a reliable metric.

[MyName13]

Why do so many people consider geekbench unreliable but not cinebench?

[Bit_Guardian]

9 minutes ago, MyName13 said:

Why do so many people consider geekbench unreliable but not cinebench?

Cinebench is very consistent. Geekbench deviates 5% or more per run. It's also incredibly ARM-biased in its optimisation. It still hasn't taken AVX512 into account, whereas SiSoft Sandra has, which is why Skylake-x has double the per-clock performance of Skylake in the multimedia benchmark.

[LAwLz]

4 hours ago, DildorTheDecent said:

I'd take AnTuTu over Geekbench any day.

You got to be kidding. AnTuTu is even worse than Geekbench. At least Geekbench gives you a somewhat comprehensive breakdown of what was tested.

 

4 hours ago, DildorTheDecent said:

It's hilarious to me that a reviewer would honestly use Geekbench as a performance metric. Only becasue it's probably the easiest to understand.

AnTuTu is easier to understand... One of the main reasons why people use Geekbench is because it runs on so many platforms and it's strictly a CPU test.

[Sauron]

2 hours ago, Bit_Guardian said:

 

Emulation does not have to be slow. What do you think the Java JVM is exactly? It is the emulation of a theoretical machine. Java can in fact be just as fast or faster than optimized C++ at some things (if you don't have an equally good lazy garbage collection runtime in your C++ program).

You misunderstand what the JVM does. Your Java code is universal, but the JVM itself is compiled individually for each architecture. It's an interpreter with sandbox features to put it simply. It is nothing like an emulator.

 

Emulation, on the other hand, translates precompiled executables from one assembly language to another. Depending on what it's translating, performance may vary, but going from a cisc architecture, with HUNDREDS of valid commands, to a risc one is inevitably a demanding operation and will degrade performance. It is particularly ridiculous to use it on a synthetic benchmark, which is (in theory, because geekbench fails at it) optimized to squeeze every last drop of performance out of a given chip and can't possibly work as intended if you have to translate its requests every time. It's also worth noting that different emulations have different requirements, as is clearly demonstrated by the dozens of console emulators that sometimes underperform even on machines that are hundreds of times more powerful than the original hardware.

3 hours ago, Bit_Guardian said:

It certainly can, just like offloading to a GPU for hardware acceleration in encoding.

While in theory it may be possible, it would definitely require specific hardware support as opposed to off the shelf products. A gpu doesn't pick up where the cpu left off, it is assigned a task and reports back when it's finished. What @hey_yo_ is asking, if I'm not mistaken, is whether alternating between the two chips on the fly is possible, and with that hardware it is not. Even just shared memory access would be incredibly problematic.

[Drak3]

1 hour ago, MyName13 said:

Why do so many people consider geekbench unreliable but not cinebench?

Because Cinebench is consistant and is actually comparable with all platforms it runs on.

Whereas Geekbench scores can be manipulated easily, and deviate too much on consrquitive runs on clean installs. 

[Bananasplit_00]

honestly not sure which way i want this to go, but being able to put our most loved pice of spyware on more things isnt that bad i guess?

[Matu20]

Every time I see Geekbench, a little part of me dies.

[Dabombinable]

9 hours ago, hey_yo_ said:

Sources: Winfuture.de via Slashgear

 

That is disappointing. Just look at the Galaxy Note 8 with the same SoC:

 

 

 

These are the results for Tegra 3. The first quad core ARM CPU used in tablets from what I understand, and which are notorious for bad single threaded performance. This is why people call bullshit when they see Geekbench:

 

Edit: Got the CPU wrong, but the point still stands.

[MageTank]

9 hours ago, hey_yo_ said:

I know how people in the forum tend to be dismissive of synthetic benchmarks including Geekbench but remember for the most part, devices scoring high in Geekbench are the ones that actually perform very well. Since Geekbench is an x86 application, it must be running on the x86 emulation layer of W10 on ARM and Microsoft promised users that many of their existing x86/64 applications will run just fine on an ARM Windows 10 device.

 

We are not dismissive of Geekbench because it is synthetic, we are dismissive of it because it's an incredibly poor, inaccurate benchmark. You can have identical CPU's at identical clocks, but different memory speeds will drastically alter the results of the score. Why is that a problem? Geekbench only reports default DMI strings, so you never really know the actual clock speeds of the hardware that is listed. Not only that, but clean OS install's score much higher than bloated OS's. So while you can see the CPU is identical, and the clock speed on the DMI string might be identical, knowing the exact clock speed/state of the OS is next to impossible

 

You simply cannot compare two opposing scores on Geekbench unless you have an absolute control over both devices being tested.

[Bit_Guardian]

4 hours ago, Sauron said:

You misunderstand what the JVM does. Your Java code is universal, but the JVM itself is compiled individually for each architecture. It's an interpreter with sandbox features to put it simply. It is nothing like an emulator.

 

Emulation, on the other hand, translates precompiled executables from one assembly language to another. Depending on what it's translating, performance may vary, but going from a cisc architecture, with HUNDREDS of valid commands, to a risc one is inevitably a demanding operation and will degrade performance. It is particularly ridiculous to use it on a synthetic benchmark, which is (in theory, because geekbench fails at it) optimized to squeeze every last drop of performance out of a given chip and can't possibly work as intended if you have to translate its requests every time. It's also worth noting that different emulations have different requirements, as is clearly demonstrated by the dozens of console emulators that sometimes underperform even on machines that are hundreds of times more powerful than the original hardware.

While in theory it may be possible, it would definitely require specific hardware support as opposed to off the shelf products. A gpu doesn't pick up where the cpu left off, it is assigned a task and reports back when it's finished. What @hey_yo_ is asking, if I'm not mistaken, is whether alternating between the two chips on the fly is possible, and with that hardware it is not. Even just shared memory access would be incredibly problematic.

You couldn't be farther from the truth. Every emulator on the planet does exactly what the JVM does: interpret code and respond with expected values. How it gets to those values is irrelevant (hence many JVMs, but same behavior). The JVM is in fact a more complicated emulator than just about any other, video game consoles included. You're trying to elevate the problem, but I'm sorry, it's reducible to code interpretation.

 

Java programs also are precompiled executibles. They're compiled to Java bytecode.

 

ARM is not true RISC, and modern x86 processors use RISCy micro-ops anyway. There's an interpretation (read: emulation) layer literally built in. In short, that's not the performance bottleneck on merit.

 

If the emulation layer programmers did their work correctly, you won't be interpreting requests every time. You'll be caching quite a few of them. This is how the JavaScript interpreter in Chrome works actually. Please see the GoTo 2017 presentation on emulating the Atari in JavaScript

 

No, it does not require specific hardware support, not unless that console had something truly exotic such as fixed-function pipeline hardware or a special DSP. This is exactly why languges like C++ are so powerful. They continually prove every point you've made here wrong.

 

And actually while we're on this subject, the JVM is a monolithic cross-aechitecture codebase (aka a NoArch executable, which you will observe if you install it on Linux).

 

HSA (and in fact OpenCL 2.1 and the last 3 versions of CUDA do not require the GPU report back. It can literally pick up and run. C++ SyCl, OpenACC, and HPX do not require this either).

 

It is possible. It's a solved problem. It's baked into the most recent standards of every major parallel programming framework today. Get started on OpenMP and work your way up and out.