Jump to content

UserBenchmark Adjusts Algorthym to Favor Single-Thread Performance, Causing Strange Results

infinitytec

Thanks for all the feedback people have shared on this!

I have updated the OP to reflect the changed FAQ page UB now has up, and  some of my thoughts about it.

Link to comment
Share on other sites

Link to post
Share on other sites

5 hours ago, infinitytec said:

Thanks for all the feedback people have shared on this!

I have updated the OP to reflect the changed FAQ page UB now has up, and  some of my thoughts about it.

And also, tell Linus to include about this in his new video

Link to comment
Share on other sites

Link to post
Share on other sites

On 7/26/2019 at 11:31 AM, emosun said:

So we just need every programmer from 2003 to work their way out of the industry before it can move on

More like new programmers need to be taught how to use multithreading well. I wish my degree taught me how to make a thread safe event system in C++ but alas that looks like something I'll need to figure out on my own or steal someone else's event system.

Link to comment
Share on other sites

Link to post
Share on other sites

I will be honest, even their current formula seems to be a bit out of wack. Since if you take the raw scores from the cpu page and apply what they claim their formula is you get different results. However looking at the numbers and seeing everything while playing around with different weights it becomes clear that with the more recent releases of higher core count CPUs, it has shown a flaw in how userbenchmark ranks an scores the CPUs as well as the test they use which only check for singe core, quad core, and 64 core. Hopefully this change in their weighting is only temporary while they are building a new CPU test suite to better show more nuance between varying core counts without lumping everything above 4 into multi core.

 

I have created a spreadsheet that shows that and so that my claims can be validated by the community, I did leave a custom option so people can play around with their own ideas of better balanced weights as well as some ideas I had for various work loads, use cases, and finding middle ground between their old algorthym and their new one.

 

Here is the link to the public editable google sheet: https://docs.google.com/spreadsheets/d/13xZ8yLktz_cQZVkd7dRAxtKRAzlGIqR1hU29CLl7re0/edit?usp=sharing

 

edit: Public viewable(just to keep a static copy) only: https://docs.google.com/spreadsheets/d/1CjRsaBx5Z5BThjlF-jipkqRj9vshCkYoIHDHZWyPgXk/edit?usp=sharing

Edited by Orthusaku
Link to comment
Share on other sites

Link to post
Share on other sites

25 minutes ago, Orthusaku said:

I will be honest, even their current formula seems to be a bit out of wack. Since if you take the raw scores from the cpu page and apply what they claim their formula is you get different results. However looking at the numbers and seeing everything while playing around with different weights it becomes clear that with the more recent releases of higher core count CPUs, it has shown a flaw in how userbenchmark ranks an scores the CPUs as well as the test they use which only check for singe core, quad core, and 64 core. Hopefully this change in their weighting is only temporary while they are building a new CPU test suite to better show more nuance between varying core counts without lumping everything above 4 into multi core.

 

I have created a spreadsheet that shows that and so that my claims can be validated by the community, I did leave a custom option so people can play around with their own ideas of better balanced weights as well as some ideas I had for various work loads, use cases, and finding middle ground between their old algorthym and their new one.

 

Here is the link to the public google sheet: https://docs.google.com/spreadsheets/d/13xZ8yLktz_cQZVkd7dRAxtKRAzlGIqR1hU29CLl7re0/edit?usp=sharing

Wow. That took some time to put together.

Link to comment
Share on other sites

Link to post
Share on other sites

My biggest issue, is that whenever I need to look up two different CPUs for a comparison between them, userbenchmark is essentially always the first few results...

And that website is trash, barely indicative of real results... wasting your time trying to find results... this news just add fuel to the dumpster fire that is that website.

CPU: AMD Ryzen 3700x / GPU: Asus Radeon RX 6750XT OC 12GB / RAM: Corsair Vengeance LPX 2x8GB DDR4-3200
MOBO: MSI B450m Gaming Plus / NVME: Corsair MP510 240GB / Case: TT Core v21 / PSU: Seasonic 750W / OS: Win 10 Pro

Link to comment
Share on other sites

Link to post
Share on other sites

So its weighted for minecraft server cpus?

Resident Mozilla Shill.   Typed on my Ortholinear JJ40 custom keyboard
               __     I am the ASCIIDino.
              / _)
     _.----._/ /      If you can see me you 
    /         /       must put me in your 
 __/ (  | (  |        signature for 24 hours.
/__.-'|_|--|_|        
Link to comment
Share on other sites

Link to post
Share on other sites

Might wanna fix the title:

 

Screenshot_2019-07-28-14-00-04.png

I spent $2500 on building my PC and all i do with it is play no games atm & watch anime at 1080p(finally) watch YT and write essays...  nothing, it just sits there collecting dust...

Builds:

The Toaster Project! Northern Bee!

 

The original LAN PC build log! (Old, dead and replaced by The Toaster Project & 5.0)

Spoiler

"Here is some advice that might have gotten lost somewhere along the way in your life. 

 

#1. Treat others as you would like to be treated.

#2. It's best to keep your mouth shut; and appear to be stupid, rather than open it and remove all doubt.

#3. There is nothing "wrong" with being wrong. Learning from a mistake can be more valuable than not making one in the first place.

 

Follow these simple rules in life, and I promise you, things magically get easier. " - MageTank 31-10-2016

 

 

Link to comment
Share on other sites

Link to post
Share on other sites

On 7/27/2019 at 4:59 AM, pas008 said:

But for my needs cad and gaming

Single core performance is king for my needs in both

Neither of these is actually single thread which is a huge flaw when looking at a test that gives a result for a single thread workload and then is used to say that this type of CPU is better for lightly threaded workloads. Almost no CPU in stock configuration can attain single thread performance because any more than one active thread, aka literally almost always, means you aren't getting it.

 

It does you no good when looking at a CPU with single core boost to say 5Ghz when it's 2-4 core boost is only 4.6Ghz as an example. These mythical single thread applications don't really exist and the ones that do are because there is no performance requirement.

 

Unless you are running out of stock with MCE or fixed multipliers the single thread benchmark is pretty useless, people doing this don't need these types of sites that rank CPUs.

 

Further to the above point the CPU industry is changing which will make single thread test irrelevant. With the move to dynamic boost across all cores to non-fixed values based on multiple metrics and boundaries every single core will be operating as fast as it can for the point in time load being run. This means which ever application you are running, what ever background tasks are doing, you are getting the best performance, the best clocks possible.

 

Single thread benchmarking is well on it's way to obsolescence which goes to show just how un-useful and inkeeping with the times userbenchmark is.

Link to comment
Share on other sites

Link to post
Share on other sites

The only reason single-thread performance is still so relevant is because game creators are still not scaling multi-threaded capabilities as high as they could or should. Draw calls are still mostly single threaded and that is why you see single thread performance matter so much when it comes to max fps in 1080p games. I understand why this is being done, but it doesn't mean it should be done.

 

Once the market actually decides to truly move to high core counts the advantage single thread performance has will start to diminish. I mean yes it matters because stronger cores make for even better multi-core setups... but when you look at 8 cores vs 12 cores there is no reason to see the higher core machine with similar IPC fall behind due to a few hundred MHZ difference.

 

There are some games and API's out there that don't have these problems and those are the games where you see the higher core count cpus hang with or even surpass the higher single thread ones. Just look at how things scale on the productivity side... they have full adapted. We just need the gaming side to follow suit and for things like dx12 to successfully spread draw calls out to more cores.

Link to comment
Share on other sites

Link to post
Share on other sites

3 hours ago, leadeater said:

Neither of these is actually single thread which is a huge flaw when looking at a test that gives a result for a single thread workload and then is used to say that this type of CPU is better for lightly threaded workloads. Almost no CPU in stock configuration can attain single thread performance because any more than one active thread, aka literally almost always, means you aren't getting it.

 

It does you no good when looking at a CPU with single core boost to say 5Ghz when it's 2-4 core boost is only 4.6Ghz as an example. These mythical single thread applications don't really exist and the ones that do are because there is no performance requirement.

 

Unless you are running out of stock with MCE or fixed multipliers the single thread benchmark is pretty useless, people doing this don't need these types of sites that rank CPUs.

 

Further to the above point the CPU industry is changing which will make single thread test irrelevant. With the move to dynamic boost across all cores to non-fixed values based on multiple metrics and boundaries every single core will be operating as fast as it can for the point in time load being run. This means which ever application you are running, what ever background tasks are doing, you are getting the best performance, the best clocks possible.

 

Single thread benchmarking is well on it's way to obsolescence which goes to show just how un-useful and inkeeping with the times userbenchmark is.

So in gaming and my uses for cad

 Single core means nothing? Lol

Cause obviously they are slowing to perform better. 7700k same as 8700k in cad in the area i use

And many games to until it starts using much more threads

 

And ofc they are all using light multicore

 

Link to comment
Share on other sites

Link to post
Share on other sites

5 hours ago, leadeater said:
On 7/26/2019 at 11:59 AM, pas008 said:

 

Neither of these is actually single thread which is a huge flaw when looking at a test that gives a result for a single thread workload and then is used to say that this type of CPU is better for lightly threaded workloads. Almost no CPU in stock configuration can attain single thread performance because any more than one active thread, aka literally almost always, means you aren't getting it. 

CAD is primarily single threaded. That is, the stuff that's multithreaded isn't that important. That being said, the amount of applications apart from CAD which are purely serial by their underlying nature and not by outdated programming mentality is pretty small which makes UBs weighting insane. If they wanted to do a proper weighting, you would have single core at 10%, quad core at 50%, octo core at 30%, and Multi-Core(9 and above) at 10%

 

Link to comment
Share on other sites

Link to post
Share on other sites

On 7/25/2019 at 6:04 AM, Blademaster91 said:

Except if Intel were involved in Userbench being so skewed towards fewer cores, they wouldn't be showing the as already mentioned result of a i3-9350KF being "better" than a i9-9980XE.

I've never found Userbench very useful though, even before they changed the scoring, it was obviously a site someone shouldn't use to make a decision without looking at real benchmarks first.

Yes, they would because the end result is AMD gets hurt, but at least you're still buying an Intel CPU. 

CPU i7 6700 Cooling Cryorig H7 Motherboard MSI H110i Pro AC RAM Kingston HyperX Fury 16GB DDR4 2133 GPU Pulse RX 5700 XT Case Fractal Design Define Mini C Storage Trascend SSD370S 256GB + WD Black 320GB + Sandisk Ultra II 480GB + WD Blue 1TB PSU EVGA GS 550 Display Nixeus Vue24B FreeSync 144 Hz Monitor (VESA mounted) Keyboard Aorus K3 Mechanical Keyboard Mouse Logitech G402 OS Windows 10 Home 64 bit

Link to comment
Share on other sites

Link to post
Share on other sites

On 7/25/2019 at 11:41 AM, poochyena said:

I want to know how much faster one cpu is over another. I can't do that with reviews since reviews never compare current CPUs with much older ones. A review telling me the ryzen 3700 gets 200 fps in CSGO doesn't tell me anything if idk how much my current cpu gets.

A 7600k is a refresh of the 6600K that's sightly faster. Go watch hardware unboxed recent ryzen review. Your CPU is there

CPU i7 6700 Cooling Cryorig H7 Motherboard MSI H110i Pro AC RAM Kingston HyperX Fury 16GB DDR4 2133 GPU Pulse RX 5700 XT Case Fractal Design Define Mini C Storage Trascend SSD370S 256GB + WD Black 320GB + Sandisk Ultra II 480GB + WD Blue 1TB PSU EVGA GS 550 Display Nixeus Vue24B FreeSync 144 Hz Monitor (VESA mounted) Keyboard Aorus K3 Mechanical Keyboard Mouse Logitech G402 OS Windows 10 Home 64 bit

Link to comment
Share on other sites

Link to post
Share on other sites

3 hours ago, AngryBeaver said:

The only reason single-thread performance is still so relevant is because game creators are still not scaling multi-threaded capabilities as high as they could or should. Draw calls are still mostly single threaded and that is why you see single thread performance matter so much when it comes to max fps in 1080p games. I understand why this is being done, but it doesn't mean it should be done.

 

Once the market actually decides to truly move to high core counts the advantage single thread performance has will start to diminish. I mean yes it matters because stronger cores make for even better multi-core setups... but when you look at 8 cores vs 12 cores there is no reason to see the higher core machine with similar IPC fall behind due to a few hundred MHZ difference.

 

There are some games and API's out there that don't have these problems and those are the games where you see the higher core count cpus hang with or even surpass the higher single thread ones. Just look at how things scale on the productivity side... they have full adapted. We just need the gaming side to follow suit and for things like dx12 to successfully spread draw calls out to more cores.

That and FPS is time/latency dependent, and often not massively (on the scale of a render farm/cpu cluster) scalable currently. So getting those last few FPS out to the GPU to render can help. But it's diminishing returns at this point.

 

That is to say. It matters, at certain price/performance points. Once you hit the med/high end, it's no longer the bottleneck, and GPU/throughput else where is.

Link to comment
Share on other sites

Link to post
Share on other sites

2 hours ago, ravenshrike said:

CAD is primarily single threaded. That is, the stuff that's multithreaded isn't that important. That being said, the amount of applications apart from CAD which are purely serial by their underlying nature and not by outdated programming mentality is pretty small which makes UBs weighting insane. If they wanted to do a proper weighting, you would have single core at 10%, quad core at 50%, octo core at 30%, and Multi-Core(9 and above) at 10%

Every CAD application I have used has used, and will use more than a single thread. Be it SolidWorks, AutoCAD + Inventor and another one I forget the name of all handle more than a single thread.

 

The problem isn't that these are often lightly threaded, they are not single threaded. If single thread performance is 500 and 2 thread is 450, when are you getting 500? Basically never without MCE. You do not get single thread performance without MCE. Bolding that because the point seems to be missed.

Link to comment
Share on other sites

Link to post
Share on other sites

12 minutes ago, leadeater said:

Every CAD application I have used has used, and will use more than a single thread. Be it SolidWorks, AutoCAD + Inventor and another one I forget the name of all handle more than a single thread.

 

The problem isn't that these are often lightly threaded, they are not single threaded. If single thread performance is 500 and 2 thread is 450, when are you getting 500? Basically never without MCE. You do not get single thread performance without MCE. Bolding that because the point seems to be missed.

There are old ones. Etc. Most of my gaming/CAD is older games/ancient (Win XP) CAD. where single core throughput is key. So I'm likely for my next build/upgrade to go classic high power CPU, but lots of RAM and SSD speed or something, to try and optimise for fast FPS/loading screens on old games.

 

But yeah, 4 to 8 core (or 4 core 8 thread) is realy nice to err on the side of the 8 core options. It's really really hard to know what is best, i7 2 core 4 thread super turboing Intel CPU, or a low i3 4 core? Without actual benchmarks, individual like for like comparisons (perhaps the i3 PC has a GTX710, but the i7 a 1050ti), it's very hard to know. Everyone's "workload" is unique these days if not 100% facebook, 100% [a specific] gaming or 100% 4k video editing. Those 3 off the top of my head are generic workloads. For anything else? Who knows which specific CPU is fastest for a specific price point till we check. :P

 

Link to comment
Share on other sites

Link to post
Share on other sites

I can’t recall the last game I played that was actually single core. Even some of the truly questionable PC ports I have like Dynasty Warriors 8 XL used more than one core. PC games aren’t using 16 threads, but most of them are definitely not single core either. 

 

If this is for gaming, measuring 4 and 6 core performance would make a lot more sense to me. Many games use 4-6 cores while not utilizing SMT very well. “Single threaded performance” is important but it’s not really because a single thread is being used, it’s because 6 threads are being used efficiently instead of 12.

AMD Ryzen 7 3700X | Thermalright Le Grand Macho RT | ASUS ROG Strix X470-F | 16GB G.Skill Trident Z RGB @3400MHz | EVGA RTX 2080S XC Ultra | EVGA GQ 650 | HP EX920 1TB / Crucial MX500 500GB / Samsung Spinpoint 1TB | Cooler Master H500M

Link to comment
Share on other sites

Link to post
Share on other sites

16 hours ago, AngryBeaver said:

The only reason single-thread performance is still so relevant is because game creators are still not scaling multi-threaded capabilities as high as they could or should. Draw calls are still mostly single threaded and that is why you see single thread performance matter so much when it comes to max fps in 1080p games. I understand why this is being done, but it doesn't mean it should be done.

 

Once the market actually decides to truly move to high core counts the advantage single thread performance has will start to diminish.

Don't worry, they will... when someone's willing to foot the R&D bill (as well as retraining involved) for game engines and developers to fully utilize those 12-16 cores (24-32 threads).

 

Perhaps they'll fund-raise with more surprise mechanics? ?

Link to comment
Share on other sites

Link to post
Share on other sites

18 minutes ago, leadeater said:

I'm sure when Star Citizen comes out all these game engine threading issues will be solved.

By the time SC comes out most computers won't even use cores as we currently define them.  We'll all be using a single mass of artificial synapses. 

 

EDIT: you'll just plug the game into the side of your head and twitch on the ground for 5 minutes all the time mentally experiencing a few hours of intergalactic space flight and trading.

Grammar and spelling is not indicative of intelligence/knowledge.  Not having the same opinion does not always mean lack of understanding.  

Link to comment
Share on other sites

Link to post
Share on other sites

Guest
This topic is now closed to further replies.


×