Exynos Galaxy S9 significantly underperforms in comparison to Snapdragon models - Anandtech

[D13H4RD]

Given the impressive benchmark scores, one would definitely expect the Exynos to crush the Snapdragon 845 in the S9. While it certainly does so in benchmarks, in the real world, it's a completely different story that flips the page completely.

 

In their in-depth review, Anandtech found that the Exynos 9810 had significant worse real-world performance in their testing in comparison to the SD845 variant, with overall improvement being only marginally better than the Exynos 8895 in its predecessor. 

Quote

AnandTech is usually data-driven when making claims about performance so the stark contrast between the Exynos’ synthetic performance and the system benchmarks more than ever question the validity of both. There are two questions to answer here: are the benchmarks still working as intended and representative, and if they are, what happened to the Exynos 9810’s raw performance?

 

For the first question, I haven’t seen any evidence to contradict the results of our system benchmarks. The Exynos 9810 variant of the Galaxy S9 simply isn’t any faster in most workloads and in one-on-one comparisons against the Snapdragon 845 variant it was indeed the less consistent one in performance and losing out in terms of responsiveness, even if that difference in absolute terms is very minor.

 

As to why this is happening on the Exynos is something that I attribute to scheduler and DVFS. Samsung’s new scheduler and DVFS just seems absolutely atrociously tuned for performance. I tested an interactive workload on both Snapdragon and Exynos devices and the contrast couldn’t be any greater. On the Snapdragon 845 Galaxy S9 a steady state workload thread will seemingly migrate from a full idle state of the little CPUs onto the big CPUs after 65ms. At the migration moment the big CPUs kick into full gear at 2803MHz and will maintain that frequency for as long as the workload demands it.

 

On the Exynos 9810 Galaxy S9 the same workload will also migrate around at the 60ms time from the little cores up to the big cores, however once on the big cores the thread starts at the lowest frequencies of the big cluster – 650-741MHz. It takes the workload a whole 370ms until it reaches the 2314MHz state of the M3 cores – which according to the SPEC benchmarks is around the maximum single-threaded performance of the Snapdragon 845’s performance cores. To reach the full 2703MHz of the M3 cores the workload needs to have been active for a staggering 410ms before the DVFS mechanism starts switching to that frequency state.

 

UI workloads are highly transactional and very rarely is there something which takes longer than a few frames. The fact that the Exynos 9810 takes over 5x longer to reach the maximum performance state of the Snapdragon 845 basically invalidates absolutely everything about the performance of its cores. For workloads which are shorter than 400ms (which is a *lot* of time in computing terms) the Snapdragon will have already finished the race before the Exynos warms up. Only at higher workload durations would the Exynos then finally catch up. Acceleration vs maximum speed being the key aspects here. This is Samsung’s first EAS based scheduler for Exynos devices, and the way the schedutil governor is tuned here is a great disappointment for performance.

 

Beyond the Exynos’ overzealous “slow-and-steady” DVFS approach I’m also not happy how the core count/maximum frequency mechanism is implemented. This is a simple HR timer task that checks the CPU runqueues and based on a threshold of heavy threads it simply offlines or onlines the CPUs. The fixed interval here is 15ms when in a quad-core state and 30ms in dual- and single-core states. Beyond the fact that the whole offlineing/onlineing of the cores is extremely inefficient as a scheduler mechanism, it’s worrisome that when the SoC is in dual or single-mode and there’s suddenly a burst of threads, the CPUs will be highly underprovisioned in terms of capacity up to 30ms until the mechanism turns back on the other cores.

 

The fact that the DVFS mechanism is so slow completely invalidates the benefit of such a mechanism in the first place as the one use-case where single-threaded performance trumps everything is web-browsing and heavy JavaScript workloads, which by nature, are short and bursty. Samsung should have simply completely ignored frequencies above 2.1-2.3GHz (matching the Snapdragon in ST Performance), ignored this whole variable maximum frequency mechanism, and instead concentrated on getting performance through scheduler and DVFS response-time, something which Qualcomm seems to have learned to master. In the end S.LSI investment in a performant custom CPU core is sabotaged by very questionable software, and the Exynos’ CPU performance goals go largely unfulfilled in real interactive workloads.

Source: https://www.anandtech.com/show/12520/the-galaxy-s9-review/5

 

Even more damning of which is that when the Exynos processor is allowed to flex its muscles, it seems consumes much more power than its predecessor 

Quote

The Exynos 9810 Galaxy S9 absolutely fell flat on its face in this test and posted the worst results among our tracking of the latest generation devices, lasting 3 hours less than the Exynos 8895 Galaxy S8. This was such a terrible run that I redid the test and still resulted in the same runtime.

 

I investigated the matter further to try to see if this was caused by the high energy usage of the M3 cores – and it seems it is. Enabling the “CPU limiter” (S9 PS result in the graphs) which is found in the battery optimisation options of Samsung’s firmware greatly throttles the M3 cores down to 1469 MHz, memory controller to half speed and also seemingly changes some scheduler settings to make them more conservative. This results in peak performance equal to the Exynos 8895- however the scheduler alterations also noticeably slow down UI responsiveness so it’s actually a worse experience. Nevertheless, backing off on performance results in regaining almost 3 hours.

 

This is such a terrible battery performance of the Exynos 9810 variant that it again puts even more clout into the new SoC. My theory as to why this happens is that not only do the higher frequency state require more energy per work done than competing SoCs – because this is a big CPU complex there’s also lots of leakage at play. The DVFS system being so slow might actually be bad for energy here as we might be seeing the opposite of race-to-sleep, walk-to-waste. The fact that Apple’s SoCs don’t have any issues with battery life in this test showcases that it’s not an inherent problem of having a high-power micro-architecture, but rather something specific to the Exynos 9810.

Source: https://www.anandtech.com/show/12520/the-galaxy-s9-review/8

 

Granted, it should be noted that the reviewer's own experience isn't that negative, but it still paints a pretty cloudy picture. 

Quote

In my personal every-day usage I can’t saw that I noticed a massive disadvantage in battery life on the Galaxy S9, however my everyday usage is relatively light and I haven’t had enough time with the phone yet as a daily driver to make a final judgment. I did notice that the Exynos 9810 does shows signs of suffering in heavy tasks. Instances of Gmail syncing my inbox with a new account did once result in a warm phone while the Snapdragon 845 Galaxy S9 did not showcase this characteristic.

Source: https://www.anandtech.com/show/12520/the-galaxy-s9-review/8

 

It's entirely possible that these can be improved with a future software upgrade but for the Exynos 9810 SoC, it's not looking too bright. Which is a shame because the last 2 managed to outperform some of Qualcomm's offerings. This one doesn't seem to be able to carry that torch. 

[8uhbbhu8]

Wasn't there some new a while back saying that Samsung was going to artificially limit their cpus to not cannibalize their snapdragon models? That would explain why but the power consumption factor IS worrying.

[D13H4RD]

14 minutes ago, 8uhbbhu8 said:

Wasn't there some new a while back saying that Samsung was going to artificially limit their cpus to not cannibalize their snapdragon models? That would explain why but the power consumption factor IS worrying.

That's exactly what I was worried about. That the Exynos was made to be super powerful at the expense of efficiency, and not necessarily making sure it's powerful while efficiently managing that sort of power. 

[Sniperfox47]

48 minutes ago, D13H4RD2L1V3 said:

It's entirely possible that these can be improved with a future software upgrade but for the Exynos 9810 SoC, it's not looking too bright. Which is a shame because the last 2 managed to outperform some of Qualcomm's offerings. This one doesn't seem to be able to carry that torch. 

If it's mostly scheduler issues it shouldn't be that big of an issue to fix. For something like that it could even potentially come through as part of a security fix for most phones (not sure if Knox has issues with that).

 

The past couple Exynos chips have performed well in benchmarks but been taken down in real world performance. This is much of the same. I really hope they can get a handle on this, because it should legitimately be a lot more powerful and efficient than the Snapdragon offerings for the first time, outside of benchmarks.

[D13H4RD]

5 minutes ago, Sniperfox47 said:

The past couple Exynos chips have performed well in benchmarks but been taken down in real world performance. This is much of the same. I really hope they can get a handle on this, because it should legitimately be a lot more powerful and efficient than the Snapdragon offerings for the first time, outside of benchmarks.

The 8890 and 8895 actually performed better in the real world compared to their Snapdragon counterparts AFAIK except in GPU.

 

Especially the 8890 in the S7. I believe that was when they started going custom. The gap is mostly closed with the 8895, and smaller still with Oreo. 

[Morgan MLGman]

I've got quite a bad luck, I recently got my first Samsung Galaxy phone which is the Galaxy S9 (Exynos model, I'm from EU) and now this happens, despite early benchmarks stating the opposite

 

Though I found this video uploaded yesterday:

In general they trade blows and it's really hard to notice any difference in the real world. Scheduler issues on the Exynos will probably be partially or fully fixed by firmware updates so I'm not too worried, this phone is blistering-fast anyway so I'd actually prefer to have more power "under the hood" for the future. Especially if the difference is this insignificant.

[Zodiark1593]

This seems awfully reminisce of Nvidia's Denver cpu where the gisnt core performed exceptionally well in benchmarks, but was otherwise.lackluster outside. Granted, one runs native ARM code, the other runs some form of code morphing, so it's probably not directly comparable, but it's still interesting to see the similarities.

[michaelocarroll007]

7 hours ago, D13H4RD2L1V3 said:

The 8890 and 8895 actually performed better in the real world compared to their Snapdragon counterparts AFAIK except in GPU.

 

Especially the 8890 in the S7. I believe that was when they started going custom. The gap is mostly closed with the 8895, and smaller still with Oreo. 

the S6 Didnt come in snapdragon chips at all 100% Samsung chips. They were the only android phone worth buying that year due to the 810 Being terrible.

 S7-9 they have offered both flavors with the samsung chips in past years sort of being the better option overall by most peoples standards but hasn't had the same lead as the year they Fully Dropped the SD lineup.

[D13H4RD]

25 minutes ago, michaelocarroll007 said:

the S6 Didnt come in snapdragon chips at all 100% Samsung chips. They were the only android phone worth buying that year due to the 810 Being terrible.

 S7-9 they have offered both flavors with the samsung chips in past years sort of being the better option overall by most peoples standards but hasn't had the same lead as the year they Fully Dropped the SD lineup.

I think that has a lot to do with how bad the 810 was

[Doobeedoo]

Yeah I even remember reading it before that it's cuase of scheduler and latency to ramp up clocks to max. Very odd yes, though also they shouldn't worry much about staying closer to SD version and nerf Exynos one. 

[Castdeath97]

WWHHYY CANT WE GET THE 845 IN THE UK SAMSUNG, WHHHHYYYY!?

[DocSwag]

17 hours ago, D13H4RD2L1V3 said:

The 8890 and 8895 actually performed better in the real world compared to their Snapdragon counterparts AFAIK except in GPU.

 

Especially the 8890 in the S7. I believe that was when they started going custom. The gap is mostly closed with the 8895, and smaller still with Oreo. 

True of 8890, but IIRC the 835 did better than the 8895.

16 hours ago, Morgan MLGman said:

I've got quite a bad luck, I recently got my first Samsung Galaxy phone which is the Galaxy S9 (Exynos model, I'm from EU) and now this happens, despite early benchmarks stating the opposite

Only benchmark that out the exynos ahead that I've seen is geek bench. Unfortunately, tech outlets LOVE to use geek bench and absolutely nothing else.

16 hours ago, Morgan MLGman said:

In general they trade blows and it's really hard to notice any difference in the real world. Scheduler issues on the Exynos will probably be partially or fully fixed by firmware updates so I'm not too worried, this phone is blistering-fast anyway so I'd actually prefer to have more power "under the hood" for the future. Especially if the difference is this insignificant.

Speed tests are much more RAM and NAND based, not cpu or gpu dependent. The places you might notice it more are animations, web browsing, games, etc.

[Morgan MLGman]

9 hours ago, DocSwag said:

Only benchmark that out the exynos ahead that I've seen is geek bench. Unfortunately, tech outlets LOVE to use geek bench and absolutely nothing else.

Speed tests are much more RAM and NAND based, not cpu or gpu dependent. The places you might notice it more are animations, web browsing, games, etc.

Hmm, from the videos that I watched comparing different phones to both the Exynos and Snapdragon models it doesn't seem like it:

The Exynos version was not only faster on average than the iPhone X but also had better results in his testing methodology than the Snapdragon S9+ model that he tested previously.

It was a lot faster when opening multiple different apps for the first time (2:27 vs 3:04) probably due to a more powerful chip, but was a bit slower when re-opening them again later (0:58 vs 0:44) so it probably might be caused by the scheduler issue.

 

I'm not sure how reliable are results like that, but they do paint a rough picture of how those two CPUs stack up to one another in terms of real-world performance.

 

Link to the Snapdragon model comparison to the iPhone X from the same channel: https://www.youtube.com/watch?v=nxFTgOfIEqw

[DocSwag]

14 hours ago, Morgan MLGman said:

Hmm, from the videos that I watched comparing different phones to both the Exynos and Snapdragon models it doesn't seem like it:

The Exynos version was not only faster on average than the iPhone X but also had better results in his testing methodology than the Snapdragon S9+ model that he tested previously.

It was a lot faster when opening multiple different apps for the first time (2:27 vs 3:04) probably due to a more powerful chip, but was a bit slower when re-opening them again later (0:58 vs 0:44) so it probably might be caused by the scheduler issue.

 

I'm not sure how reliable are results like that, but they do paint a rough picture of how those two CPUs stack up to one another in terms of real-world performance.

 

Link to the Snapdragon model comparison to the iPhone X from the same channel: https://www.youtube.com/watch?v=nxFTgOfIEqw

AFAIK those tests don't test raw cpu performance though.... Rather, they test NAND performance and DRAM performance. The differences are more likely to be due to better/worse IO and a better/worse IMC than the cpu cores.

 

It's especially evident if you look at all the real world tests anandtech did that the exynos just doesn't do well. It takes the exynos way too long to turbo up, making it really fail at the short bursty workloads that honestly are the main reason single threaded performance is useful at all in smartphones.

[DrMacintosh]

Apple should just start selling the A series of SoCs to Android manufacturers. 

 

Though Android would only benefit from the raw power, not many of the optimization’s priopriatary to the OS. 

 

The S9 is wicked fast, but mostly because how the skin is designed to have fairly quick animations. The S9 also has more RAM so if you hit the RAM limit on the iPhone X in a side by side test the iPhone should lose. 

 

Apple still is leading the industry with graphics and raw multi core power. 

[Zodiark1593]

12 minutes ago, DrMacintosh said:

Apple should just start selling the A series of SoCs to Android manufacturers. 

 

Though Android would only benefit from the raw power, not many of the optimization’s priopriatary to the OS. 

 

The S9 is wicked fast, but mostly because how the skin is designed to have fairly quick animations. The S9 also has more RAM so if you hit the RAM limit on the iPhone X in a side by side test the iPhone should lose. 

 

Apple still is leading the industry with graphics and raw multi core power. 

On the board level, I tend to see a lot more in the way of capacitors and other such components on Apple's devices than many Android devices. Beyond just the cost of the chip, I suspect a robust power delivery is required to feed the big CPU cores, which is probably going to be more expensive than what many OEMs would be willing to pay for.

[DrMacintosh]

15 minutes ago, Zodiark1593 said:

On the board level, I tend to see a lot more in the way of capacitors and other such components on Apple's devices than many Android devices. Beyond just the cost of the chip, I suspect a robust power delivery is required to feed the big CPU cores, which is probably going to be more expensive than what many OEMs would be willing to pay for.

That’s true, Apple’s PCBs are works of art. 

 

Black logic baords are so cool despite almost nobody getting to look at them. 

[captain_to_fire]

1 hour ago, DrMacintosh said:

Apple should just start selling the A series of SoCs to Android manufacturers. 

NO! Apple will lose what makes the iPhone and iPad special, very fast SoCs. Besides, that's like saying Apple should license macOS to PC OEMs which I hope  they don't consider. 

[DrMacintosh]

Just now, hey_yo_ said:

NO! Apple will lose what makes the iPhone and iPad special, very fast SoCs. Besides, that's like saying Apple should license macOS to PC OEMs which I hope  they don't consider. 

Kinda

 

The A series is just so much better than what Qualcomm is doing it would just be interesting to see if Apple could market a sub series of A SoCs for use in Android products. 

 

Now of course this means that Apple could tweak the Android variants and make the slower than their iOS counterparts to still give the iPhone an advantage. 

 

Its really more of a thought experiment than it is an actual suggestion. 

 

And yeah, don’t worry, macOS is staying on the Mac  

[Zodiark1593]

26 minutes ago, DrMacintosh said:

Kinda

 

The A series is just so much better than what Qualcomm is doing it would just be interesting to see if Apple could market a sub series of A SoCs for use in Android products. 

 

Now of course this means that Apple could tweak the Android variants and make the slower than their iOS counterparts to still give the iPhone an advantage. 

 

Its really more of a thought experiment than it is an actual suggestion. 

 

And yeah, don’t worry, macOS is staying on the Mac  

Die shrunk Apple A9 with reduced power requirements for simpler board designs.

 

 

Since when does Apple ever willingly share though? One would have far better chances yanking teeth from a wide-awake Crocodile or Shark. Neither of which I'm keen on getting close too, mind you.

[RagnarokDel]

12 minutes ago, Zodiark1593 said:

Shark.

It's actually really easy, just get bit.

[D13H4RD]

2 hours ago, DrMacintosh said:

Apple should just start selling the A series of SoCs to Android manufacturers. 

 

Though Android would only benefit from the raw power, not many of the optimization’s priopriatary to the OS. 

 

The S9 is wicked fast, but mostly because how the skin is designed to have fairly quick animations. The S9 also has more RAM so if you hit the RAM limit on the iPhone X in a side by side test the iPhone should lose. 

 

Apple still is leading the industry with graphics and raw multi core power. 

I don’t think they would

 

Because then they’re throwing away one of their competitive advantages 

[Trixanity]

7 hours ago, DrMacintosh said:

Apple should just start selling the A series of SoCs to Android manufacturers. 

 

Though Android would only benefit from the raw power, not many of the optimization’s priopriatary to the OS. 

 

The S9 is wicked fast, but mostly because how the skin is designed to have fairly quick animations. The S9 also has more RAM so if you hit the RAM limit on the iPhone X in a side by side test the iPhone should lose. 

 

Apple still is leading the industry with graphics and raw multi core power. 

False. Anandtech has showed time and time again how efficient Adreno graphics are. Sometimes Apple's graphics wins in benchmarks (often due to them using Metal and Adreno still being forced to run on OpenGL) but they do so while consuming more power and the size is also bigger. Apple doesn't allow measurements but they run into thermal constraints; that's proof enough.

[Clanscorpia]

2 hours ago, Trixanity said:

False. Anandtech has showed time and time again how efficient Adreno graphics are. Sometimes Apple's graphics wins in benchmarks (often due to them using Metal and Adreno still being forced to run on OpenGL) but they do so while consuming more power and the size is also bigger. Apple doesn't allow measurements but they run into thermal constraints; that's proof enough.

Efficient =/= better

[Trixanity]

2 minutes ago, Clanscorpia said:

Efficient =/= better

It is better though. Adreno wins in the vast majority of metrics including most absolute performance benchmarks. 

I'll give Apple that they're still new to the GPU game so it may improve but it doesn't appear like Qualcomm is twiddling their thumbs either.

 

It's well documented that Adreno is ahead of everyone else. And consistently too. 

 

Efficiency has proven to be a very important metric anyway. Especially in a battery-powered device.