Why is everyone saying userbenchmark is bad?

[TommyTwommy]

As someone who has been into computer his whole life, and consider my self to be pretty knowledgeable in that area and up to date with all the trends, I am really surprised that all these tech youtubers are saying it is a bad site.. how is a site that has hundreds of thousands of benchmarks bad? I understand it might not be accurate because of different sets of hardware, overclocks, benchmarking conditions, etc. But I think it is a great way to approximately check how good a component is compared to some other ones.. I know that for someone who doesn't understand how computers work it might not be that useful but for someone who can understand what each number means i think it is perfect and I've been personally using for many months now when picking new components.. That is how I ended up with a 8700k and a 1070.. It was best price to performance ratio at the time of purchase for what I need it..

[Fasauceome]

easy to read and accurate are far from the same thing. While some maintain that "it's easy to understand therefore it's useful," the data collected is just wrong.

 

I keep referring to Steve Burke's comments on the site, and how he states the controlling for variables is entirely nonexistant, do you have any specific counter remarks for that?

[Mnky313]

1 minute ago, berny22 said:

As someone who has been into computer his whole life, and consider my self to be pretty knowledgeable in that area and up to date with all the trends, I am really surprised that all these tech youtubers are saying it is a bad site.. how is a site that has hundreds of thousands of benchmarks bad? I understand it might not be accurate because of different sets of hardware, overclocks, benchmarking conditions, etc. But I think it is a great way to approximately check how good a component is compared to some other ones.. I know that for someone who doesn't understand how computers work it might not be that useful but for someone who can understand what each number means i think it is perfect and I've been personally using for many months now when picking new components.. That is how I ended up with a 8700k and a 1070.. It was best price to performance ratio at the time of purchase for what I need it..

https://cpu.userbenchmark.com/Compare/Intel-Core-i3-8350K-vs-AMD-Ryzen-TR-1950X/3935vs3932
This... This is why...

But in all seriousness, it's because it favours intel and single core performance significantly meaning that a lot of the time the +x% means absolutely nothing despite people thinking it does, if you look down in the description of single/multicore performance it can be somewhat more useful but still not great.

[BuckGup]

It's absolutely terrible. The pricing is always wrong which then affects the performance to price ratio. You have no clue if the results are based upon OCed results, if it's the stock cooler, if there was a CPU bottleneck, and so on. It's pretty bad 

[GrockleTD]

It is vaguely useful if you are trying to compare your GPU OC imo, or am I wrong in thinking this?

[geo3]

5 minutes ago, fasauceome said:

I keep referring to Steve Burke's comments on the site, and how he states the controlling for variables is entirely nonexistant, do you have any specific counter remarks for that?

Having a sample size of just 1 is also unscientific. Most of these guys are just defensive because they feel they own the realm of benchmarking.  What they are not considering is that large sample sizes tend to average out the variance and arrive at a good result in the end anyway.

 

User bench is flawed, but so is their method. I think both are useful to provide a more complete picture.

 

I do think Userbechmarks method of score aggregation is very wrong. However they don't hide the score break down from you. It's right there below the aggregate so you can draw your own conclusions. They also show the distribution of user scores so you can tell where you are likely to fall as well. 

[Fasauceome]

1 minute ago, geo3 said:

Having a sample size of just 1 is also unscientific

That's why you watch more than one review before coming to a conclusion

[geo3]

1 minute ago, fasauceome said:

That's why you watch more than one review before coming to a conclusion

There are not 100's of bench markers out there.

[Fasauceome]

Just now, geo3 said:

There are not 100's of bench markers out there.

I'll take 5 benchmarks with actual control for external variables than hundreds of average Joes benchmarking with an exe file. It's just far more precise.

[geo3]

1 minute ago, fasauceome said:

It's just far more precise.

But not necessarily more accurate, or more representative of what a person would get in the real world. Those reviewers are not using the same hardware as me or the same software or running both at the same settings. Nor are they running the same stuff as each other. Taking their results as the definitive truth is no more valid than Userbench.

 

 Plus I don't have the patients to watch 5 reviews of the same part.

 

Now if there is a reviewer who tests 500 samples of each part he gets and tests with every combination of other hardware, levels of over clock, application and application setting and does each combination for 100 runs, then I will say there is no use for something like userbench.

[GrockleTD]

8 minutes ago, geo3 said:

But not necessarily more accurate, or more representative of what a person would get in the real world. Those reviewers are not using the same hardware as me or the same software or running both at the same settings. Nor are they running the same stuff as each other. Taking their results as the definitive truth is no more valid than Userbench.

 

 Plus I don't have the patients to watch 5 reviews of the same part.

 

Now if there is a reviewer who tests 500 samples of each part he gets and tests with every combination of other hardware, levels of over clock, application and application setting and does each combination for 100 runs, then I will say there is no use for something like userbench.

so perhaps the problem is you are too lazy to do proper research? it has been shown many times that there is little performance variance between AIB models, excluding things such as the kingpin cards, so there is no real reason why they should be discounted for this. by that logic, you should also be against Ubenchmark

[Fasauceome]

18 minutes ago, geo3 said:

But not necessarily more accurate, or more representative of what a person would get in the real world

I would contest that this actually is the case.

 

18 minutes ago, geo3 said:

 Plus I don't have the patients to watch 5 reviews of the same part.

That's not a reason aggregate benchmark sites would be better. That's just an excuse to use them and hope their information is reliable

[geo3]

Just now, fasauceome said:

That's not a reason aggregate benchmark sites would be better.

I never said they are better.  I said both are flawed and incomplete and that user bench provides a different side of the picture that individual reviewers are incapable of.

[Fasauceome]

1 minute ago, geo3 said:

I never said they are better.  I said both are flawed and incomplete and that user bench provides a different side of the picture that individual reviewers are incapable of.

Aggregate benchmark sites are far more flawed, that's my point. For example, R9 290X vs 780 ti, or 4770K vs 4790. These results are flat out wrong. 

[TommyTwommy]

1 hour ago, fasauceome said:

easy to read and accurate are far from the same thing. While some maintain that "it's easy to understand therefore it's useful," the data collected is just wrong.

 

I keep referring to Steve Burke's comments on the site, and how he states the controlling for variables is entirely nonexistant, do you have any specific counter remarks for that?

as I said "I understand it might not be accurate because of different sets of hardware, overclocks, benchmarking conditions, etc."

 

but it can really help to get a better picture of what each component is capable of.. there is so much info that no other site has and so many universal benchmarks and if you want to you can go through each of them to see what components were used at what clocks..

 

anyways the point is, i honestly thought it was a great site with a lot of useful information, if you know what that information means and if you don't interpret it wrong

[Fasauceome]

4 minutes ago, berny22 said:

but it can really help to get a better picture of what each component is capable of.

But it really can't, because it's not painting a realistic picture if the data is erroneous (see previous mention of the 780 ti vs R9 290X)

 

5 minutes ago, berny22 said:

i honestly thought it was a great site with a lot of useful information, if you know what that information means and if you don't interpret it wrong

Therein lies the second problem, most people have no idea how to interpret the userbenchmark figures and therefore couldn't draw a meaningful conclusion even if the data was accurate

[TommyTwommy]

12 minutes ago, fasauceome said:

But it really can't, because it's not painting a realistic picture if the data is erroneous (see previous mention of the 780 ti vs R9 290X)

 

Therein lies the second problem, most people have no idea how to interpret the userbenchmark figures and therefore couldn't draw a meaningful conclusion even if the data was accurate

for those certain scenarios where the data is wrong you can always dig deeper and look through reviews from reputable sources.. I am just saying, it is a good starting point that contains so much information.. some of it is correct and some is wrong.. when you're buying something you definitely shouldn't rely on information solely from that site.. but it is my first stop always to compare multiple products, see their marker share and performance (of course taken with a grain of salt, until i look at actual benchmarks)

 

the site is very user friendly, has tons of (maybe not so accurate) data.. and i dont think it deserves the hate it is getting atm

 

i am not saying you should go there, compare products and immediately make a choice on your purchase.. 

[Fasauceome]

7 minutes ago, berny22 said:

for those certain scenarios where the data is wrong you can always dig deeper and look through reviews from reputable sources

But you won't know that unless you know that userbenchmark is wrong, and laymen who use userbenchmark as a reliable source of info won't know that at all

[TommyTwommy]

1 minute ago, fasauceome said:

But you won't know that unless you know that userbenchmark is wrong, and laymen who use userbenchmark as a reliable source of info won't know that at all

if you read the rest of my post you would see that I stated many times it is only a good starting point.. not your first and last point of research into a product you are planning to buy.. regardless of the userbenchmark being correct or incorrect (mostly they are correct) you always will and should watch multiple reviews before spending your money.. also if you go to their ranking and sort  it by "avg benchmark" you will see that it mostly shows it in the correct order..

 

so if someone is looking for the BEST GPU FOR GAMING regardless of the price  their research ends right there.. you don't need much brain to know that you do not need to waste money on a Quadro RTX 8000 and that if you're gaming an RTX Titan or 2080 Ti is what you need.. and there you go.. you got yourself the best gaming card on the market without any hassle

 

if someone wants the best bang for their buck i'd say an RX 570 (#1 value on userbenchmark) is a great choice and if they want the best card based on user rating, 1070 is a great choice.. (even I agree as I own one