"Fair" Overclocking Leaderboard Discussion

[For Science!]

After what I can only describe as an epic sh*tstorm with the Userbenchmark Overclocking Competition. It got me thinking as to exactly what would be a good way to quantify and score an overclocking board. Perhaps those interested in making such a thread in future can contribute and learn from this thread to make a OC competition thread that is widely accepted by the community @Storm-Chaser.

 

In my opinion, an overclocking competition should be exactly that, a competition who is able to squeeze out more performance out of a certain chip. For this reason, while it is probably tempting and the easiest to just rank by score, it is important to quantify the score as a percentage over stock for a certain chip.

 

For example, no matter how much you bend backwards, an i7 2600K is not going to beat a R9 5950X in Cinebench Multi (or anything, perhaps). However, the point of an OC competition is the OCing and not whether you score higher (in my opinion). So I think that if you can get 150% performance out of a i7-2600K, that should be credited in some way. This would then mean that then chips that have a lot of overclocking headroom would be unfairly favored over more recent chips that don't have a lot of headroom (squeezing 150% out of a 2600K may be a lot easier than a 5950X, for example). So therefore the average "overclockability" would need to be also considered as a metric. 

 

Obviously this would mean that the leaderboard would be very segmented with each chip needing its own table. I think that would be too fine and for this reason I would say just like how wrestlers are seperated by weight-class, you could group chips by a certain silicon generation (maybe?). And to give a metric that's comparable between weight-classes, I think some number that describes how good your OCing was relative to other OC's in that weight-class will let you demonstrate your "OCing skills".

 

How I could see it at the moment would be....

 

- Each user would need to submit a "stock" score and "OC" score along with details of their chip.

- Stock scores for an individual chip would need to be averaged with 3 sigma rejection to compute an "average stock score" for a certain chip, alternatively an autheticated user would serve as the "baseline stock"

- "OC" scores would need to be computed first as a percentage increase over stock (OC score / average stock score * 100) and then tabulated into the weight-class (e.g. Intel 14 nm, or AMD 7 nm)

- The percentage increase overstock can then be calculated as a sigma value (maybe?), to basically signify how much better your OC is relative to other OC's in the weight-class

- The leaderboard would actually be sorted by sigma value

 

I can foresee many issues at the moment as the board will depend heavily on how the weight-class are sorted, but for now this is probably enough to get a discussion started.

[porina]

I don't know about the userbenchmark competition, but I have been active on hwbot and other overclocking forums in the past. What is described sounds a lot like hwbot in the sense that there is a leaderboard for each CPU. GPU, whatever. They don't require any baseline requirement. Verification is obviously a factor and depending on the benchmark there are various measures you have to take. That's quite a pain as it is easy to make mistakes in the submission, and they may get removed long after submission so you might not have the hardware any more to re-do the testing. As such the submission overhead is not insignificant and takes away from some of the fun. They also sort by cooling class too: Extreme which I think is LN2, Amateur which is anything sub-ambient (chillers, ice), Enthusiast/Novice which covers pretty much everyone else. I only went to Amateur as it removes the requirement for a photo of the system being benched in most cases.

 

The other thing is, it isn't purely about getting the highest clocks (in most cases), but about optimising the system for the benchmark. GPU benches will still be affected by CPU for example. CPU benches may be affected by ram, some more by latency like the older Pi benches, some more by ram bandwidth like y-cruncher. As a result, it is still a bit pay-to-win. Want the best GPU score? Regardless of the GPU model, you'll want a fast CPU and fast ram too.

[Guest]

It's a tech forum, not an OC forum. 

 

GeekBench is my suggestion, but StormChaser refused any help from myself when I had contacted him in PM.

[For Science!]

1 minute ago, ShrimpBrime said:

It's a tech forum, not an OC forum. 

 

GeekBench is my suggestion, but StormChaser refused any help from myself when I had contacted him in PM.

I think OCing is within the scope of tech, so don't see a problem with an OC challenge per se, but yes, navigating through social interaction is also important, which is another problem all together

 

2 hours ago, porina said:

I don't know about the userbenchmark competition, but I have been active on hwbot and other overclocking forums in the past. What is described sounds a lot like hwbot in the sense that there is a leaderboard for each CPU. GPU, whatever. They don't require any baseline requirement. Verification is obviously a factor and depending on the benchmark there are various measures you have to take. That's quite a pain as it is easy to make mistakes in the submission, and they may get removed long after submission so you might not have the hardware any more to re-do the testing. As such the submission overhead is not insignificant and takes away from some of the fun. They also sort by cooling class too: Extreme which I think is LN2, Amateur which is anything sub-ambient (chillers, ice), Enthusiast/Novice which covers pretty much everyone else. I only went to Amateur as it removes the requirement for a photo of the system being benched in most cases.

 

The other thing is, it isn't purely about getting the highest clocks (in most cases), but about optimising the system for the benchmark. GPU benches will still be affected by CPU for example. CPU benches may be affected by ram, some more by latency like the older Pi benches, some more by ram bandwidth like y-cruncher. As a result, it is still a bit pay-to-win. Want the best GPU score? Regardless of the GPU model, you'll want a fast CPU and fast ram too.

Yes, I think its complicated with GPU benches and whole system benchmarks, but I guess the generally unless if its a bench that uses the GPU heavily, generally speaking CPU and RAM are the primary players and I guess that can be bundled into one.

[porina]

I only just saw the other thread. It is a shame it got derailed so fast and I think shows the worst side of the community here. It is a reality in the competitive benchmarking world that a particular benchmark may work better or worse on certain models than others without any intentional bias. It seems many of the respondents wouldn't have been happy if it was anything other than Cinebench which is a near ideal scaling case, but there are so many more benchmarks that are more interesting than that.

[Guest]

1 minute ago, porina said:

I only just saw the other thread. It is a shame it got derailed so fast and I think shows the worst side of the community here. It is a reality in the competitive benchmarking world that a particular benchmark may work better or worse on certain models than others without any intentional bias. It seems many of the respondents wouldn't have been happy if it was anything other than Cinebench which is a near ideal scaling case, but there are so many more benchmarks that are more interesting than that.

None of it is Ideal on Windows 10 without benchmate and you know that. The RTC will skew benchmarks, and in some cases really bad. 

 

I had thought of one time maybe just simple WPrime comp. Again, needs the wrapper.... which doesn't launch WPrime because it can't launch it or doesn't launch it as administrator. Without the wrapper, still need to be launched via admin, and then we'll just have skewed scoring. For example, a lot of the scores I've seen with Cinebench in that thread, you can make out the ones that look funny. Complaints of lower scores than average actually. That's a problem for any comparisons. 

 

And lastly, good luck getting it pinned. Pretty sure that was what SC dude was trying to do. That's the at most I've seen him contribute, either about his overclocks and rigs or starting a bench thread. Which is nothing wrong with it, but he talks too much in the thread. Just put a bench up, explain how to run it, give an example screen shot and let the thread take it's course. It's not hard really.

[porina]

15 minutes ago, ShrimpBrime said:

None of it is Ideal on Windows 10 without benchmate and you know that. The RTC will skew benchmarks, and in some cases really bad. 

Potential cheating is not even a consideration at this point. If the goal was to introduce people to more competitive benchmark running, it will be encountered at some point, but isn't the biggest thing to worry about at the start. Also Benchmate isn't the only solution, benchmarks like 3DMark family and y-cruncher for example have their own checks.

 

15 minutes ago, ShrimpBrime said:

Complaints of lower scores than average actually. That's a problem for any comparisons. 

An opportunity for people to learn how to optimise (legally) benchmark results.

 

15 minutes ago, ShrimpBrime said:

And lastly, good luck getting it pinned.

Was that aimed at me? Just throwing random thoughts in this. I think my competitive benching days are past, outside of the occasional one I might find interesting. People will come and go. I'll leave it to the next generation to drive it forwards.

[Guest]

23 minutes ago, porina said:

Potential cheating is not even a consideration at this point. If the goal was to introduce people to more competitive benchmark running, it will be encountered at some point, but isn't the biggest thing to worry about at the start. Also Benchmate isn't the only solution, benchmarks like 3DMark family and y-cruncher for example have their own checks.

 

An opportunity for people to learn how to optimise (legally) benchmark results.

 

Was that aimed at me? Just throwing random thoughts in this. I think my competitive benching days are past, outside of the occasional one I might find interesting. People will come and go. I'll leave it to the next generation to drive it forwards.

Not aimed AT YOU specifically, just to anyone in general. 

I see a lot of content on this forum that "Should" be stickied, it never is. (that aside from benching threads)

 

Yes opportunity. It's there, guys like us are here, most inquires at this forum.... #1 is, "help it turns on but black screen".... (just saying I notice this)

 

And it's not about cheating. The RTC skews the benchmark all by it's self. No DLL tweaks necessary. 

[Guest]

How about this thought.....

 

Start a how to overclock your AMD / Intel processor thread first.... Then perhaps a bench thread. (My 2 cents)

[porina]

26 minutes ago, ShrimpBrime said:

Not aimed AT YOU specifically, just to anyone in general. 

Since it was in a reply to me, I was unclear hence asking.

 

26 minutes ago, ShrimpBrime said:

And it's not about cheating. The RTC skews the benchmark all by it's self. No DLL tweaks necessary. 

Is it that bad? Got a pointer on the subject from a benchmarking perspective?

[Guest]

2 minutes ago, porina said:

Since it was in a reply to me, I was unclear hence asking.

 

Is it that bad? Got a pointer on the subject from a benchmarking perspective?

Gotcha, I'm not here to offend, just being honest and forthcoming.

 

RTC

Well it's bad enough that the operating system W8 and up was banned from use at HWBot. You can still DLL tweak on W7 however, but we'll leave that aside.

 

SO certain benchmarks do better on certain operating systems. It's always been that way. 

PiMod = Win XP

PiMod 32M W7

WPrime W7

Cinebench W7

 

Everything with W8 and up Benchmated. 

I remember when they (hwbot) wrapped Aquamark lol. That was a joke all by it's self. This one... Win XP (W7 depending on how new the HW is)

These are very common Cpu benchmarks. Shit I think PiMod dates back to 1995 if I'm not mistaken. So W95, W98, ME maybe 2000 and of course XP.

 

____

 

We fall in a timeline of overclocking when the system does it for you. Thus almost all benchmarks should be very close to identical and the minor tweaks don't bring that 50% and higher Cpu overclocks that once where to provide a wide range of benchmark scores. This with a combination of W8++ killed competitive benching. Everyone has the same binned chip these days. Everyone is getting the same performance across 2 identical systems. 

Unless you run LN2 SS or DryIce on modern hardware, there's really no point in competition. 

 

If they would bring back those 500mhz Cpu OC back to the table, I think overclocking manually for arbitrary points could become a thing again.

 

 

[Guest]

Here is a link you can read about RTC and AM4 platform. 

 

https://hwbot.org/newsflash/4335_ryzen_platform_affected_by_rtc_bias_w88.110_not_allowed_on_select_benchmarks

[For Science!]

4 hours ago, porina said:

The other thing is, it isn't purely about getting the highest clocks (in most cases), but about optimising the system for the benchmark. GPU benches will still be affected by CPU for example. CPU benches may be affected by ram, some more by latency like the older Pi benches, some more by ram bandwidth like y-cruncher. As a result, it is still a bit pay-to-win. Want the best GPU score? Regardless of the GPU model, you'll want a fast CPU and fast ram too.

I guess what I would like to get to is some kind of scoring system that is relatively chip agnostic, if you are able to produce an 99th percentile OC for a 7700K, you should be "on-par" with somebody that can squeeze out a 99th percentile OC for a 6600K. If a chip has no or little OC headroom, I guess it becomes difficult but more rewarding to get into higher percentile brackets?

 

I guess it makes sense to seperate out ambient and sub-ambient cooling, but then we've doubled the sampling space again...difficult difficult

[Guest]

40 minutes ago, For Science! said:

I guess what I would like to get to is some kind of scoring system that is relatively chip agnostic, if you are able to produce an 99th percentile OC for a 7700K, you should be "on-par" with somebody that can squeeze out a 99th percentile OC for a 6600K. If a chip has no or little OC headroom, I guess it becomes difficult but more rewarding to get into higher percentile brackets?

 

I guess it makes sense to seperate out ambient and sub-ambient cooling, but then we've doubled the sampling space again...difficult difficult

You just keep it simple as possible. That's all I can say about a bench thread.

 

Post your GeekBench Scores!!!

 

DL from here:

make submission in this format : User / Cpu / VGA / Cooling / score.

Example screen shot: here.

Have fun good luck!

 

No reason to reinvent the wheel here.

 

 

[For Science!]

1 minute ago, ShrimpBrime said:

You just keep it simple as possible. That's all I can say about a bench thread.

 

Post your GeekBench Scores!!!

 

DL from here:

make submission in this format : User / Cpu / VGA / Cooling / score.

Example screen shot: here.

Have fun good luck!

 

No reason to reinvent the wheel here.

 

 

But this would be essentially a pay-to-win competition no? I was trying to think if there was a way to normalize further.

[Guest]

1 minute ago, For Science! said:

But this would be essentially a pay-to-win competition no? I was trying to think if there was a way to normalize further.

All competition is pay to win. Just like free video games!! 

If you wanted a fair fight, you would divide by platform.

AM4 Geekbench 5000 series cpu / 3000 series cpu /2000 ect ect.

 

Then like you said becomes big and complicated. 

 

The Cinebench thread seems to be popular with just "post your CB scores" and it's a done deal.

I often forget this is not an old school "overclock forum". 

Most users here, "enabled XMP and it won't post" is a very common thread posted.

Very many much less... "how do I overclock my AM4 processor"

 

I'd say at least 50% of the threads opened in the cpu/motherboard topic section should be moved to trouble shooting.....

 

Not sure. It can be very complicated. But for this forum, not worth all of anyone's trouble. These kids just want to post their rig and start gaming more so than anything.

 

[For Science!]

1 minute ago, ShrimpBrime said:

All competition is pay to win. Just like free video games!! 

If you wanted a fair fight, you would divide by platform.

AM4 Geekbench 5000 series cpu / 3000 series cpu /2000 ect ect.

 

Then like you said becomes big and complicated. 

 

The Cinebench thread seems to be popular with just "post your CB scores" and it's a done deal.

I often forget this is not an old school "overclock forum". 

Most users here, "enabled XMP and it won't post" is a very common thread posted.

Very many much less... "how do I overclock my AM4 processor"

 

I'd say at least 50% of the threads opened in the cpu/motherboard topic section should be moved to trouble shooting.....

 

Not sure. It can be very complicated. But for this forum, not worth all of anyone's trouble. These kids just want to post their rig and start gaming more so than anything.

 

Yes, and the CB thread fulfills that niche, so another thread to post another score from another benchmarking suite serves no additional value. I think there is some value if there could be a way to demonstrate OC skill that would preferably be hardware agnostic. I mean, if you have the tecnical knowhow and the patience to bring a 6700K to a 99th percentile OC, you can probably do the same for a 9900K or a 3950X. 

 

I think there could be a market for such a thread as you might be like "why is my OC score for my 9900K lower than the person with a 2700K??" and then this may promote people to take a deeper dive into actual tuning

[Guest]

1 hour ago, For Science! said:

Yes, and the CB thread fulfills that niche, so another thread to post another score from another benchmarking suite serves no additional value. I think there is some value if there could be a way to demonstrate OC skill that would preferably be hardware agnostic. I mean, if you have the tecnical knowhow and the patience to bring a 6700K to a 99th percentile OC, you can probably do the same for a 9900K or a 3950X. 

 

I think there could be a market for such a thread as you might be like "why is my OC score for my 9900K lower than the person with a 2700K??" and then this may promote people to take a deeper dive into actual tuning

There is quite a bit to competitive benchmarking and in turn a quite a bit to learn tweaking a system for a score vs running a daily clock speed and there really isnt a line in between that I know of.

 

Really for LTT forums, I think a tweaking or OC thread with some benchmarks for comparisons would be useful. 

Benching is a moot point if some OC training is missing.

 

But how do we make it quick easy and appealing? 

 

EDIT: example

Tweaking - AMD - 

Nothing do I ever read about FID and DID overclocking. 

SenseMi offset and SenseMi Skew settings in bios.

This is on top of PBO, FXR and IF meanings differences and values of OC.

Ratio's need attention 1:1:1 FCLK BCLK MCLK and the correlation to performance.

Memory tweaking. Very important. Set XMP and go is one thing. Tweaking for numbers is a different ball game 100%

Repercussions of OC the bus clocks. Yeah, can eek out a little performance. At a risk no doubt. 

(I could go on and on and on lol.)

[Fast_N_Curious]

On 2/4/2021 at 4:30 AM, For Science! said:

After what I can only describe as an epic sh*tstorm with the Userbenchmark Overclocking Competition. It got me thinking as to exactly what would be a good way to quantify and score an overclocking board. Perhaps those interested in making such a thread in future can contribute and learn from this thread to make a OC competition thread that is widely accepted by the community @Storm-Chaser.

 

In my opinion, an overclocking competition should be exactly that, a competition who is able to squeeze out more performance out of a certain chip. For this reason, while it is probably tempting and the easiest to just rank by score, it is important to quantify the score as a percentage over stock for a certain chip.

 

For example, no matter how much you bend backwards, an i7 2600K is not going to beat a R9 5950X in Cinebench Multi (or anything, perhaps). However, the point of an OC competition is the OCing and not whether you score higher (in my opinion). So I think that if you can get 150% performance out of a i7-2600K, that should be credited in some way. This would then mean that then chips that have a lot of overclocking headroom would be unfairly favored over more recent chips that don't have a lot of headroom (squeezing 150% out of a 2600K may be a lot easier than a 5950X, for example). So therefore the average "overclockability" would need to be also considered as a metric. 

 

Obviously this would mean that the leaderboard would be very segmented with each chip needing its own table. I think that would be too fine and for this reason I would say just like how wrestlers are seperated by weight-class, you could group chips by a certain silicon generation (maybe?). And to give a metric that's comparable between weight-classes, I think some number that describes how good your OCing was relative to other OC's in that weight-class will let you demonstrate your "OCing skills".

 

How I could see it at the moment would be....

 

- Each user would need to submit a "stock" score and "OC" score along with details of their chip.

- Stock scores for an individual chip would need to be averaged with 3 sigma rejection to compute an "average stock score" for a certain chip, alternatively an autheticated user would serve as the "baseline stock"

- "OC" scores would need to be computed first as a percentage increase over stock (OC score / average stock score * 100) and then tabulated into the weight-class (e.g. Intel 14 nm, or AMD 7 nm)

- The percentage increase overstock can then be calculated as a sigma value (maybe?), to basically signify how much better your OC is relative to other OC's in the weight-class

- The leaderboard would actually be sorted by sigma value

 

I can foresee many issues at the moment as the board will depend heavily on how the weight-class are sorted, but for now this is probably enough to get a discussion started.

So I am down for a discussion here with specifics, I don't have time to dive into this right now, but I will say we had a "horrific" time getting userbenchmark.com benching competition off the ground on another forum I attend. Same problems. Exactly. After the mods cleaned it up and allowed it to proceed as fit based on some various input, it turned out to be one of my favorite benchmark comps to date. Not just saying that. I think everyone had a good bit of fun with it and turned out to be a quite detailed study of users systems and components. I will show you the leaderboard with some of the data blocked out but you should still get the idea.... thoughts???

 

[For Science!]

8 hours ago, Storm-Chaser said:

 

Is the reason why you persist to Userbenchmark so that you don't have to do the computations with regards to percentile ranges? It's just such a infamous suite, that I think its not worth pursuing.

[Fast_N_Curious]

6 hours ago, For Science! said:

Is the reason why you persist to Userbenchmark so that you don't have to do the computations with regards to percentile ranges? It's just such a infamous suite, that I think its not worth pursuing.

You wanted to have a discussion about leaderboards so I indulged you... now this? Why are you so opposed to userbenchmark? it's just another tool one can use when measuring system performance. 

 

Userbenchmark is unique in that it compares your hardware to a "so called" gold standard, which is pretty interesting and an idea I like.

[For Science!]

1 hour ago, Storm-Chaser said:

You wanted to have a discussion about leaderboards so I indulged you... now this? Why are you so opposed to userbenchmark? it's just another tool one can use when measuring system performance. 

 

Userbenchmark is unique in that it compares your hardware to a "so called" gold standard, which is pretty interesting and an idea I like.

Well, community reputation is also important, and you have no reason to push user benchmark so. You know the cpu-z thread was much better received, so why not stick to that? I am not telling you what to, but I do think that using a well-perceived app will liven up the contest.

 

anyway, with respect to the above, can you address whether the standard is OCed or not and how easy it is in particular to OC a particular chip. How do you not bias the ranking towards systems that have a lot of headroom vs stock.

[Fast_N_Curious]

18 hours ago, For Science! said:

Well, community reputation is also important, and you have no reason to push user benchmark so. You know the cpu-z thread was much better received, so why not stick to that? I am not telling you what to, but I do think that using a well-perceived app will liven up the contest.

 

anyway, with respect to the above, can you address whether the standard is OCed or not and how easy it is in particular to OC a particular chip. How do you not bias the ranking towards systems that have a lot of headroom vs stock.

The userbenchmark comp is still pretty popular with over 3K views so it's doing alright for itself.

 

And don't get me wrong, I'm all for livening up the contest. I didn't want to do the CPUz thing again for a couple months so userbenchmark was a substitute to buy a little more time... I thought people could get on board with. The sad thing is that I'm sure there would have some interest if people had not swooped in and bashed the entire idea before it got off the ground. Makes everyone timid is what it does. I am going to give you some background on userbenchmark and how effective speed is measured feel free to bounce questions off me I know their systems pretty well.

 

Effective Speed (abbreviated ES in the leaderboard) is included as well. This is your processors performance profile relative to the 9900K, the "control" chip that userbenchmark.com is using as the gold standard to measure all other chips... The percentile result is based on how well your CPU stacks up within the entire sample group of identical processors, as measured by userbenchmark.com. I have included a link below so you know exactly how userbenchmark.com is evaluating and formulating your processors performance statistics.

 

A couple pointers to de-mystify this experience:
*In listing the effective CPU speed above in the leaderboard, know that it is being measured relative to a 9900K 8 core CPU.
*In other words, the 8 core 9900K is the CPU performance baseline and the standard to which all other CPUs are measured.
*The Nvidia 2060s is the GPU performance baseline to which all other GPUs are measured.
*The Samsung 850 Pro is the SSD performance baseline to which all other SSDs are measured.

 

 

 

 

 

 

 

[Fast_N_Curious]

 

Some of my personal thoughts on using UBM for pre "binning" and CPU overclocking ceilings in general... 

We have become heavily dependent on synthetic benchmarks to give us confidence, and I think another way in which userbenchmark brings key statistics to the fore is the way they compile and display your CPU's actual benchmarking results in visual terms. First, you have a sliding scale at the bottom that indicates relative performance against the 9900K. Then, you can see exactly how other people with the same CPU as you stack up, again, in visual terms. Once you have a large enough benchmark pool of results to draw accurate comparisons, you can begin to interpret the visual scale in the context of "overclocking potential" or overclocking headroom determination. In the case of the six core i5 9600KF, you can see results simply don't "fall off" at a certain point, indicating the chip should have a decent overclocking "potential". What I mean by this is not harsh judgment against the AMD 5800X, its just merely another tool or resource you can use to aid in "optimizing" your next overclocking build. To be more specific, you can see the AMD benchmark pool has a "overclocking" result (represented by this case in the visual "profile") that is less "desirable" than that of the 9600KF. The AMD processor results on the scale at the bottom resemble a "cliff" past a certain point. In other words, the AMD processor benchmark results are grouped much more around a specific MHz speed with not as much potential or headroom to go above that. Versus the 9600KF which has results that continue to string on to higher and higher speeds, albeit at a lower rate. But it just serves to show you that if the cooling solution is good and the overclocking is competent, the performance potential ceiling of the 9600KF is generally higher than that of the Ryzen 7 5800X. This interpretation should be considered "presumptive" only but may aid in your search in finding a CPU with the ideal overclocking "ceiling" that you are seeking to build. This would not be a tool used to "needle" other people, it would be a tool you could use to measure the performance capabilities of your next system before your put it together. Or finding a really really great CPU with a really really high performance potential. Interested to see what you guys think about this. Obviously limited by the submissions themselves, i.e. less "overclockers" running submissions with one or the other here could taint the true results.