cpu Skylake CPUs in some cases more than 50% slower than the ealier generations

[Anghammarad]

This is from a Blog entry from A. Kraus: https://aloiskraus.wordpress.com/2018/06/16/why-skylakex-cpus-are-sometimes-50-slower-how-intel-has-broken-existing-code/

 

In short, the architecture has changed so that certain code causes the perfomance break in the current skylake cpus. 

 

Some Programs seem to run 50% 

 

pause

 

The problem regarding the massive performance loss lies in the pause command interpretation. This for short. 

 

Quote

CPU	Pause Duration In ns
Xeon E5 1620v3 3.5GHz	4
Xeon(R) Gold 6126 CPU @ 2.60GHz	43

 

Due to this change you can see the time spiking up more and more every time the code goes for a pause cycle. 

 

Intel says they implemented the change due to the massive instruction loads todays processors may receive and that PAUSE is now more vital then ever.

 

Quote

 

The .NET enviroment already has a fix/update at hand, which may not reside on all systems hit by that issue.

 

The full extend of the search why a new cpu runs code at only half the speed than to the previous version, while the rest stays the same is a really good read.

 

Have fun reading.

 

 

My opinion with this... well this might hit more systems than the so called y2k bug due to well code getting interpreted differently. If this is not known, no workaround done, well you wonder why your new highend hardware underperfoms that much against its predecessors... So due to the Impact this would have to be communicated better by Intel, so that fixes or workarounds in the code interpreter for all the languages could be accessable in time.

 

 

Cheers

Ang

Edited June 18, 2018 by Anghammarad
Because it was needed... me slacker...

[Bananasplit_00]

Pretty interesting issue, wonder what made Intel increase the pause delay

Edited June 18, 2018 by Crunchy Dragon

[porina]

A bit too abstract for me. Are there end user examples which show a big difference in performance? Or is this a case of "if you do only this thing it is a lot slower"?

[Anghammarad]

1 minute ago, porina said:

A bit too abstract for me. Are there end user examples which show a big difference in performance? Or is this a case of "if you do only this thing it is a lot slower"?

In short, every program whos code uses the "Pause" function, will slow down a Skylake system until it has gotten a workaround installed, like for .NET the released fixed version. 

 

But software today is written an many different languages, not only .NET... for example java, c#, e, c++, assembler, cobol, comal, basic, python, pearl, only to name a few...

[porina]

55 minutes ago, Anghammarad said:

In short, every program whos code uses the "Pause" function, will slow down a Skylake system until it has gotten a workaround installed, like for .NET the released fixed version. 

The question remains, what software is noticeably impacted? For example, are there any examples where IPC of say Haswell is higher than Skylake? Also the linked article was written around Skylake-X/Xeon, does this apply to Skylake mainstream too? What about Kaby Lake, Coffee Lake?

 

What I'm trying to understand is what is the real world impact of this? Skylake has been out for nearly 3 years now. If there was anything approaching 50% loss in applications compared to previous generations, someone would have noticed long ago.

[Anghammarad]

4 minutes ago, porina said:

The question remains, what software is noticeably impacted? For example, are there any examples where IPC of say Haswell is higher than Skylake? Also the linked article was written around Skylake-X/Xeon, does this apply to Skylake mainstream too? What about Kaby Lake, Coffee Lake?

 

What I'm trying to understand is what is the real world impact of this? Skylake has been out for nearly 3 years now. If there was anything approaching 50% loss in applications compared to previous generations, someone would have noticed long ago.

Yes it is a feature in the whole skylake architecture. from mid 2017 on.

 

IRRC there were some cases here on the forum where people asked why their new system is giving such a bad performance... This could be the impact of this "feature". 

[porina]

13 minutes ago, Anghammarad said:

Yes it is a feature in the whole skylake architecture. from mid 2017 on.

 

IRRC there were some cases here on the forum where people asked why their new system is giving such a bad performance... This could be the impact of this "feature". 

Skylake was launched summer 2015. Vague performance claims don't cut it. Where is the evidence this impacts in practice? That's not to say it doesn't or can't have any impact, but in nearly 3 years since it came out, if there were any major performance regressions you'd think someone would have mentioned it long ago.

[Syntaxvgm]

28 minutes ago, porina said:

Skylake was launched summer 2015. Vague performance claims don't cut it. Where is the evidence this impacts in practice? That's not to say it doesn't or can't have any impact, but in nearly 3 years since it came out, if there were any major performance regressions you'd think someone would have mentioned it long ago.

I think we're looking at non gaming aspects here. They literally multiplied the pause time by more than 14x times the cycles and that certainly does something. I have no idea why they did this, there's certainly a good reason, but nice of them to break existing code, which imo should never be done unless absolutely necessary. 
From my understanding, it's up to 140 cycles, but it can be specified lower. I think the problem comes with programs expecting a max time of somewhere around 10 cycles and maybe not specifying a number of cycles? So were looking at a problem that will be patched, but has to be patched per program and per platform they use I think, so it could be a small problem basically forever for some things that dont receive updates. I also think that some programs cannot specify the number of pause cycles for some sort of security reason. 
I believe Spinlock functions loop this wait time basically, which..it's waiting on something from another thread to finish? But more threads multiplies the problem, that is for sure. 

But i could really be wrong about all of this, my understanding is very weak and from memory. 

The blog mentions that MS may have had some advanced warning based on server 2016 not having the problem. 

[DKL]

Okay so my questions, as someone with an i3 6100...

1) What should I/can I do as the end user to eventually fix this? Anywhere specific I should keep an eye out for a fix being released?
2) As mentioned earlier, is there any hard evidence of this affecting real-world performance?
3) What mainstream applications are affected by this?

[porina]

33 minutes ago, DKL said:

Okay so my questions, as someone with an i3 6100...

1) What should I/can I do as the end user to eventually fix this? Anywhere specific I should keep an eye out for a fix being released?

As someone with 7 systems that are Skylake or newer, I'm currently doing... nothing about this. This is even lower on the radar as far as I'm concerned than the Spectre/Meltdown fixes.

[leadeater]

5 hours ago, porina said:

Skylake was launched summer 2015. Vague performance claims don't cut it. Where is the evidence this impacts in practice? That's not to say it doesn't or can't have any impact, but in nearly 3 years since it came out, if there were any major performance regressions you'd think someone would have mentioned it long ago.

Sounds more related to Skylake-SP to me given the change to mesh architecture and all the examples being Xeons. I would Imagine it has to do with the mesh change and the reconfiguration of more L2 cache and less L3 cache, pausing and ignoring the OS might be better than dumping data out of the cache then having to go back to ram and reload to cache again then try again. 

[vanished]

It sounds like this is just skylake specifically but can anyone confirm that?  Or is this potentially an issue for skylane and anything newer (kaby, etc.)?

[Mira Yurizaki]

The PAUSE instruction that this slide deck is talking about is for a thread to notify the CPU that it's going to do a spinlock type of wait. For those that don't program, a spinlock is the thread doing nothing until some resource is free. Like if a thread is waiting for the hard drive to be free because someone else is using it, it does this:

while (hard_drive_available == false):
  do_nothing

Alternatively, there's always the Wikipedia page on the subject if you want to read it.

 

According to a Stack Overflow post about what the PAUSE instruction does:

Quote

PAUSE notifies the CPU that this is a spinlock wait loop so memory and cache accesses may be optimized. See also pause instruction in x86 for some more details about avoiding the memory-order mis-speculation when leaving the spin-loop.

 

PAUSE may actually stop CPU for some time to save power. Older CPUs decode it as REP NOP, so you don't have to check if its supported. Older CPUs will simply do nothing (NOP) as fast as possible.

 

So this additional latency is likely a power saving measure. Instead of waking up 10 cycles later to check something, it waits up to 140 cycles later.

 

Note that spin locking tends to be looked down upon in multithreaded programming because it wastes CPU time. It has its perks, in that it's simple and keeps the thread on the CPU, but unless you know the time to acquire the lock is extremely low and you're not doing big context switches, it's better to not use it. To quote another Stack Overflow post:

 

Quote

EDIT: A question came up: "Does that mean I should use spinlocks wherever possible?" and I'll try to answer it:

As I mentioned Spinlocks are only useful in places where anticipated waiting time is shorter than a quantum (read: milliseconds) and preemption doesn't make much sense (e.g. kernel objects aren't available).

 

If waiting time is unknown, or if you're in user mode Spinlocks aren't efficient. You consume 100% CPU time on the waiting core while checking if a spinlock is available. You prevent other threads from running on that core till your quantum expires. This scenario is only feasible for short bursts at kernel level and unlikely an option for a user-mode application.

 

Here is a question at SO addressing that: Spinlocks, How Useful Are They?

 

So is there a good predictor of which software this affects most? Not really.

[Dabombinable]

#devil'scanyonMR

[Ryujin2003]

7 hours ago, Anghammarad said:

In short, every program whos code uses the "Pause" function, will slow down a Skylake system until it has gotten a workaround installed, like for .NET the released fixed version. 

 

But software today is written an many different languages, not only .NET... for example java, c#, e, c++, assembler, cobol, comal, basic, python, pearl, only to name a few...

So this would be for multithreaded applications that have to wait on a background thread to complete?

[Mira Yurizaki]

22 minutes ago, gabrielcarvfer said:

It benefits heavy parallel workloads with shit ton of threads with message passing/shared variables, that is not the case for the majority of software running on desktops. Probably hurts programs that used pause without a good reason. That stack overflow post saying that it's not efficient in user more is not really accurate, that will depend on how the thread is implemented on the system (library based or built into the kernel). 

I believe the SO post mentioned it's not good to spinlock in userland space. In kernel space it's not as bad since the context switch isn't as painful.

 

19 minutes ago, Ryujin2003 said:

So this would be for multithreaded applications that have to wait on a background thread to complete?

No. It's only if the application is using a spinlock to wait for something. That something is not necessarily completion of a thread.

[AlTech]

7 hours ago, porina said:

The question remains, what software is noticeably impacted? For example, are there any examples where IPC of say Haswell is higher than Skylake? Also the linked article was written around Skylake-X/Xeon, does this apply to Skylake mainstream too? What about Kaby Lake, Coffee Lake?

I'm going to hazard a guess that this only applies to Skylake-X and anything using Intel's Mesh Core Communication Architecture. 

Quote

What I'm trying to understand is what is the real world impact of this? Skylake has been out for nearly 3 years now. If there was anything approaching 50% loss in applications compared to previous generations, someone would have noticed long ago.

Yeah, which leads me to believe it's a Skylake-X issue.

[Princess Luna]

Fix title please, this is according to the source a Skylake-X HEDT exclusive issue.

[Arika]

I'm sorry but when we're talking an increase of 10 cycles to 140 cycles in processors running in excess of 2 to 3 thousand million cycles per second this hardly seems like an issue

[Mira Yurizaki]

3 minutes ago, Sierra Fox said:

I'm sorry but when we're talking an increase of 10 cycles to 140 cycles in processors running in excess of 2 to 3 thousand million cycles per second this hardly seems like an issue

The slide deck says "as many as."

 

Also are you from the UK or abouts?

[Arika]

Just now, M.Yurizaki said:

The slide deck says "as many as."

 

Also are you from the UK or abouts?

negative.

 

I'm from their prison island

[Mira Yurizaki]

Just now, Sierra Fox said:

negative.

 

I'm from their prison island

I only ask because I noticed the way you said the number 1,000,000,000

[Arika]

Just now, M.Yurizaki said:

I only ask because I noticed the way you said the number 1,000,000,000

yeah. i could have said billion as a regular number

 

When i was in school my programming/IT teacher always taught us that when talking about clock speeds and gigahertz that it should be referred to as thousand million instead of billion. no idea why, and i just ran with it.

[HunterAP]

2 hours ago, Princess Cadence said:

Fix title please, this is according to the source a Skylake-X HEDT exclusive issue.

I tried the test program that came from the article, supposedly my 6700K has this issue with .NET Core & Framework, I'm about to test it after installing .NET Core 2.1 and Framework 4.7.2

[leadeater]

4 hours ago, HunterAP said:

I tried the test program that came from the article, supposedly my 6700K has this issue with .NET Core & Framework, I'm about to test it after installing .NET Core 2.1 and Framework 4.7.2

Any news? No issue, has the issue?