Userbenchmark is completely biased or BS???

[chnapo]

So the fact that Intel vs AMD is as biased as possible (on userbenchmark) is now a well known fact (changing impacts of single core performance, hiding multi-core performance comparisons, no comparison beyond 64 threads etc.). But now watch this. I compare 2 GPUs and just for fun, I hit "Nvidia vs AMD Bottleneck" and the first thing I see is that RTX 2060 Super is 22% better (in CS GO) than RX 5700 XT. (source: https://www.userbenchmark.com/EFps/,2060S,,_,5700-XT,,_CSGO,,9600K, ) Weird, all reviewers pretty much said that 5700 XT can beat 2070 (that´s why 2070S was released right?) in some games. SO now let´s have more fun. Comparing 2060S vs 2070 gives me result of 8% in favor of 2060 Super (what??? - source: https://www.userbenchmark.com/EFps/,2060S,,_,2070,,_CSGO,,9600K, ). So obviously from now on I know for sure that the comparison is wrong. But let´s have more fun. Let´s compare 2060S vs the new 6800XT (GPU that pretty much competes with 3080). Oh no, 2060S is 15% better (what in all heavens - source: https://www.userbenchmark.com/EFps/,2060S,,_,6800-XT,,_CSGO,,9600K,). Naturally, their amazing 2060 Super also beats RTX 3080 (lol, source: https://www.userbenchmark.com/EFps/,2060S,,_,3080,,_CSGO,,9600K,). Now wanna see something ridiculous? Let´s try RX 580? It beats 6800 XT by 3% (like, seriously? source: https://www.userbenchmark.com/EFps/,6800-XT,,_,580,,_CSGO,,9600K,).

I am using pre-defined CPU (for sure the site like userbenchmark knows that GPUs are supposed to be compared without CPU bottleneck so I am going to trust them LOOOL ehm 9600K ehm).

So guys, if someone knows where do I buy that 2060 Super that not only beats 2070, but also 6800 XT and is on par with 3080 - please let me know. I am interested in that RX 580 (that also beats 6800 XT). I think this could potentially solve all GPU shortages as the new RTX and RX series would be no longer needed for gamers - at least according to userbenchmark.

EDIT: In that comparison, no other CPU than 9600K can be selected for 6800 XT and to top it off, all good gaming CPUs can only be combined with 2060S for that comparison.

[WhitetailAni]

6 minutes ago, chnapo said:

Userbenchmark is completely biased or BS???

It's both.

[The Torrent]

its benchmarks r bs but its good for troubleshooting

[Fasauceome]

Old news

 

 

The website is utterly and completely unreliable. Not only does it have bias bit it also hosts a pretty terrible and inconsistent "benchmark" program that doesn't produce consistent results across systems.

 

 

 

[Radium_Angel]

5 minutes ago, Fasauceome said:

The website is utterly and completely unreliable.

Would that include same vs same (nvidia vs nvidia, or intel vs intel)?

[Fasauceome]

7 minutes ago, Radium_Angel said:

Would that include same vs same (nvidia vs nvidia, or intel vs intel)?

Yeah because of the inconsistency of data. Old versions of drivers, lack of control for variables in user systems. Look at OP's example:

 

50 minutes ago, chnapo said:

Comparing 2060S vs 2070 gives me result of 8% in favor of 2060 Super

Wack

[Radium_Angel]

1 minute ago, Fasauceome said:

Yeah because of the inconsistency of data. Old versions of drivers, lack of control for variables in user systems. Look at OP's example:

 

Wack

Fair enough, any site that is more accurate?

[cm992]

14 minutes ago, Fasauceome said:

The website is utterly and completely unreliable. Not only does it have bias bit it also hosts a pretty terrible and inconsistent "benchmark" program that doesn't produce consistent results across systems.

Yea that post of yours does make me agree that their main effective speed index is probably biased but do you think the individual measurements of 1core 2core 4core 8core 64core are biased too? Because I always take a look only those figures as a quick and dirty comparison before digging for game benchmarks on youtube. And what about userbenchmark for GPUs?

[Fasauceome]

2 minutes ago, Radium_Angel said:

Fair enough, any site that is more accurate?

I tend to recommend against aggregate data websites like this, especially ones that try to conveniently wrap the comparison into a single percentage.

 

Benchmarks that share their testing methodology are the most reliable ones, so tech tubers like hardware unboxed and gamers nexus, who describe how their tests are carried out, will have the most accurate data. In general, base your info off the results of any source you can verify did the proper testing.

1 minute ago, cm992 said:

do you think the individual measurements of 1core 2core 4core 8core 64core are biased too? Because I always take a look only those figures as a quick and dirty comparison before digging for game benchmarks on youtube

I do think those test scores are biased, because the testing suite is extremely limited and favors intel a lot (in terms of software optimization) and if you're looking at YouTube benchmarks anyway, why bother checking UB in the first place?

 

2 minutes ago, cm992 said:

And what about userbenchmark for GPUs?

Same story.

[cm992]

5 minutes ago, Fasauceome said:

if you're looking at YouTube benchmarks anyway, why bother checking UB in the first place?

Because its a lot more convenient.

 

I have an idea related to this. It would be really great if steam updated their hardware survey and fps counter into a system where you could opt-in to sending performance telemetry data while you play games that includes average fps and 1% and 0.1% fps lows along with your hardware info and game graphics settings. Then steam could have a performance browser where you can filter by game and any hardware you want.

[Fasauceome]

1 minute ago, cm992 said:

Because its a lot more convenient.

 

I have an idea related to this. It would be really great if steam updated their hardware survey and fps counter into a system where you could opt-in to sending performance telemetry data while you play games that includes average fps and 1% and 0.1% fps lows along with your hardware info and game graphics settings. Then steam could have a performance browser where you can filter by game and any hardware you want.

It would have similar problems. The bias is only one issue, another issue is the bad data pool. Without a way to control for variables (buggy drivers, extraneous software, thermal throttling, etc.) then the data itself can't really be trusted.

 

Don't prioritize convenience, prioritize accuracy 

[cm992]

10 minutes ago, Fasauceome said:

It would have similar problems. The bias is only one issue, another issue is the bad data pool. Without a way to control for variables (buggy drivers, extraneous software, thermal throttling, etc.) then the data itself can't really be trusted.

 

Don't prioritize convenience, prioritize accuracy 

Drivers could easily be included in the hardware info collection and implemented as a filter. But you're right extraneous software and thermal throttling would "pollute" the dataset but considering the size and flexabillity of such a dataset I don't think it would be worthless just because there may be a minority of people with dust build up and 50 chrome tabs open during their game. Understanding that it wouldnt be able to control for extraneous software and thermal throttling just means that you should keep in mind that your performance if you were to buy the same parts and take care of your system should be a bit higher. I still think that such a massive and elegant system would provide some value.

 

EDIT: Now that I think of it cpu and gpu temperatures could be included in the fps telemetry and very hot outliers could be automatically excluded from the browser.

[Fasauceome]

6 hours ago, cm992 said:

but considering the size and flexabillity of such a dataset I don't think it would be worthless just because there may be a minority of people with dust build up and 50 chrome tabs open during their game. Understanding that it wouldnt be able to control for extraneous software and thermal throttling just means that you should keep in mind that your performance if you were to buy the same parts and take care of your system should be a bit higher

Not quite, there's more to it than that. You have CPU choice to worry about as well, requiring a whole other data set to factor in, then you have the types of software that might be used in the background. Some users record with OBS. Some record with geforce experience. Some use the CPU. Some use NVENC. Some use radeon's OpenCL acceleration which has a rather large performance hit. You would have to take into account these as well.

 

Streaming online? Discord in the background? Watching YouTube on a second monitor? Depending on how many cores your CPU has, these can disproportionately affect performance. Not to mention, malware or software that the user doesn't even know about affecting performance.

 

It's really not a viable system to design. Userbenchmark's sole purpose is aggregate data comparison for gamers and they still can't pull it off, I don't think a steam hardware survey implementation could spring up and provide a more accurate experience. Just way too many moving parts.

[cm992]

9 minutes ago, Fasauceome said:

It's really not a viable system to design. Userbenchmark's sole purpose is aggregate data comparison for gamers and they still can't pull it off, I don't think a steam hardware survey implementation could spring up and provide a more accurate experience. Just way too many moving parts.

Good points, I didn't think about streaming or malware. So it wouldn't be perfect for sure but wouldn't it still provide some useful insight? Of course you would have to keep in mind that it isn't a perfect representation of performance ie if you keep your OS clear of malware and arent streaming you might get a bit more performance. It would definitely be better than userbenchmark and it would be super cool. Of course I highly doubt it'll ever happen it was just an idea.