Windows 8 Benchmarks No Longer Accepted at HWBOT

[raxx]

http://hwbot.org/news/9824_breaking_windows_8_benchmark_results_no_longer_accepted_at_hwbot

 

This isn't good news. I am not sure what I can ad to this article since I don't do benchmarks and I have

yet to use Win8 as of right now. But this won't make Linus and anyone else in the hardware testing 

business happy.

[Exotath]

That is an incredible find! I can't believe I haven't heard of this before....pretty big flaw with Win 8 imo.

[raxx]

That is an incredible find! I can't believe I haven't heard of this before....pretty big flaw with Win 8 imo.

Your avatar says it all ;)

[miljaker]

meaning win8 is even better then we tought? or worse?

 

I see that an underclocked windows 8 machine gives better scores then its supposed to. Does an overclocked windows 8 give worse scores then its supposed to?

[VAITO]

not such a big issue, anyone who is serious about this stuff isn't dumb enough to downgrade to Windows 8 anyway

[Eyal]

I didn't expect this at all. I wonder if the same affect will occur with AMD based system as well as it did on Intel systems...

Why did I buy windows 8?

 

 

I wonder if Microsoft will make a fix for it... because if they wont, I'll be super pissed.

I might even do the testing myself just because it's just not cool... I have a coppy of windows 7 laying around...

[GoodBytes]

Who cares. Oh no, you benchmark that no one cares to look won't be buried in some site. Oh the humanity.

Also, I don't see the problem with this. It's not Microsoft faulty if they made a critical design flaw in their benchmark software.

 

To add more salt on the wound, Benchmark scores were and always was meaningless. Drivers of hardware detects benchmark software and does special optimization, and even downright not drawing things that are hard to see, or force reduction of polygons on some objects, to get a higher score. It's just as bad as the meaningless Windows Experience Index score, which Microsoft will remove in Windows 8.1

 

Real world performance folks is what maters. The rest, is some benchmark software.

[Exotath]

Yea that is true...but a jacked up clock seems like it could cause some problems for people. 

[raxx]

Yea, no big deal. It's not like Linus uses Win8 for benchmarking. Or does he?

[Anult]

Yea, no big deal. It's not like Linus uses Win8 for benchmarking. Or does he?

 

That is a question we may never know at this present time.

[raxx]

That is a question we may never know at this present time.

Why? Did Linus get run over by a bus or something?

[GoodBytes]

Yea, no big deal. It's not like Linus uses Win8 for benchmarking. Or does he?

The software works... it's just that you can't submit the score.

[Anult]

Why? Did Linus get run over by a bus or something?

Dunno. Maybe his Windows 8 PC got run over or something.

[raxx]

I am not sure why this is "no big deal" and "we knew this all along" because I am pretty sure people have asked the question "Which OS is better for games?" and up

until now, nobody has said that the benchmarks are off and they are invalid for Win8, so stick with Win7 for accurate benchmark results. But I could be wrong.

[Ashiataka]

The original post has been deleted, could someone who read it explain what it said? 

 

Thanks.

[raxx]

Still works for me, but here it is: http://hwbot.org/news/9824_breaking_windows_8_benchmark_results_no_longer_accepted_at_hwbot

[ajoy39]

Yea that is true...but a jacked up clock seems like it could cause some problems for people. 

windows syncs their clock over the internet every day by default, it's fine. 

[raxx]

I will copy+paste if you can't read the article:

As the result of weekend-time research, the HWBOT staff has decided to invalidate all benchmark records established with the Windows8 operating system. Due to severe validity problems with the Windows8 real time clock (“RTC”), benchmarks results achieved with Windows8 cannot be trusted. The main problem lies with the RTC being affected when over- or underclocking under the operating system. The operating system uses the RTC as reference clock, and benchmarks use it to reference (benchmark) time.

 

Background Information – Remember Heaven?

 

At the moment of writing, we do not have the full technical what’s and how’s figured out. Since this problem affects everyone who is passionate about overclocking, it is important to provide an explanation. It is far from the complete story, but it should be enough for you to understand why we have decided to ban Windows 8 from HWBOT.

Do you remember the history of Unigine Heaven at HWBOT? About three years ago, we launched a wrapper for the back-then brand new DX11 benchmark software Unigine Heaven. The wrapper featured an easy and secure way to submit benchmark scores to HWBOT via data files. On November 2, 2010 we posted a response to an on-going discussion about downclocking in Windows affecting the benchmark score. To make a very long story short, by downclocking in Windows the Heaven benchmark time runs slower than it really is. One second in the Heaven benchmark is equal to 1,x second of real time. Because there is more time within a second, the system can render more frames. The benchmark itself is unaware of all this – for the benchmark, one second is still exactly one second. In the end, the system renders a higher amount of frames in a longer timeframe. In the result calculation for frames per second (“FPS”), the “frames” have increased but the “second” remains the same. You get a higher score.

Later that week, we published a V1.03b version of the wrapper, which fixed the downclock issue. Without going too much into detail, the wrapper uses a second source to verify the benchmark duration and takes the relative measured time difference into account when calculating the final score. Problem solved.

 

Windows8 – “Support for all devices”

 

As you know, Microsoft is trying to come up with a unified operating system and user interface for a wide range of devices, including tablets, smartphones, Xbox One, and the desktop PC. Building this unified platform is not easy. It is not just a matter of creating an interface that can be used with a multitude of input devices (finger, mouse, controller), but it also needs to support as many devices as possible. Getting the software to run out of the box on as much hardware as possible is the challenge they are facing.

Of the many aspects to fine-tune, one feature in particular is causing Windows8 to be practically useless for (competitive) overclocking: the RTC. Quoting Wikipedia, “A real-time clock (RTC) is a computer clock (most often in the form of an integrated circuit) that keeps track of the current time. Although the term often refers to the devices in personal computers, servers and embedded systems, RTCs are present in almost any electronic device which needs to keep accurate time.” Sparing you the details of development process, compared to Windows7 and previous versions, Microsoft made changes to how it measures time to be compatible with embedded or low cost PCs that do not have a fixed RTC clock. After all, having a fixed RTC clock adds cost to a platform.

Your PC system uses the RTC for many things. For example, it ensures the Windows Time on your machine is accurate. For most benchmark applications, the RTC is used as reference clock when executing the benchmark code. By synchronizing with the RTC, the benchmark knows exactly how much time has passed, and takes that value into account when calculating the performance of your system.

[raxx]

There is more, but there are charts and stuff.

[Ashiataka]

Thank you.

 

So Windows has a software RTC that changes the rate it ticks at depending on the clock speed of the processor?

 

Would that be a very simplified but close-enough one line explanation of the problem?

[mgsstar]

Guess that people will now have to use Windows 7 for benchmarking and wait till Windows 8 fixes their RTC.

[ajoy39]

I am not sure why this is "no big deal" and "we knew this all along" because I am pretty sure people have asked the question "Which OS is better for games?" and up

until now, nobody has said that the benchmarks are off and they are invalid for Win8, so stick with Win7 for accurate benchmark results. But I could be wrong.

Linus did the his comparisons in real world games, not benchmarks, so his Win 7 vs Win 8 results still hold true. 

[GoodBytes]

Guess that people will now have to use Windows 7 for benchmarking and wait till Windows 8 fixes their RTC.

 

There is nothing to fix. The benchmark software score needs to analyst and measure based on the RTC level. It's hard to do, I am not going to deny this, but its all part of the work.

It's not the first time benchmark software moan.. they did with Vista, and if I recall correctly with 2000 or XP too.

[mgsstar]

There is nothing to fix. The benchmark software score needs to analyst and measure based on the RTC level. It's hard to do, I am not going to deny this, but its all part of the work.

It's not the first time benchmark software moan.. they did with Vista, and if I recall correctly with 2000 or XP too.

So Windows 7 is unaffected in most benchmarks but Windows 8 is affected in most benchmarks and thats why they banned Windows 8?

[GoodBytes]

No. The site HWBOT decided to block Windows 8. They think that the value is wrong is what affect scores. We don't know if because of RTC changes it ends up affecting 1-2 points in some benchmarking software or something larger. But seeing that, at no point in time, we saw that benchmark score in Windows 8 magically is completely over Windows 7 with large margin, else it would have been talked about on how benchmark software get a drastic boost in performance in WIndows8, but gaming performance is marginal. This suggests, in my opinion, that HWBOT freaking out for the 1-2 point difference.

 

I guess it maters for world record benchmark score, as, I would assume, as it could be that someone beats someone else by a few points. But again, benchmark software makers can fix this.