Intel processor bug leads to Windows and Linux kernel updates and possible performance hits

[Stefan Payne]

3 minutes ago, ariz said:

So this flaw has been going on for years and years and no one at Intel knew about this until now??

Yip...

Possibly since Pentium M...

 

ANd the question is if there really was nobody who knew about it or if there was somebody who found it but was ignored by the higher ups because they thought that nobody will find it and because of the performance hit the fix in hardware would have had, they decided  to go for the performance instead.

 

 

 

As far as I understand it, the reason is a far too agressive prediction unit of the CPU. And it went to far...

[MikeSK]

According to this source,"Intel chips that were made in the last decade suffer from a serious security issue". For more details, check out:
http://bgr.com/2018/01/03/intel-chips-security-issue-mac-windows/

 

[mrkaru]

I really wish we had more info on how this effects regular users. The people doing all the testing are coming from a sysadmin perspective, and this seems to effect them hugely. I don't like waiting around for weeks for this to be rolled out to everybody to find out whether or not my computer is now in dire need of a very expensive upgrade or not. 

[leadeater]

19 minutes ago, Stefan Payne said:

Phoronix made benchmarks.

IF we are talking about Database Servers like the one for this forum.

It's only limited testing and the database test only shows for that data database engine. What about MSSQL? or Oracle?

 

What about VM performance impact?

 

They tested like less than 1% of what is required to fully understand the impact of this.

 

22 minutes ago, Stefan Payne said:

I was talking about Desktop systems, here in this Forum, not about Servers in this case.

Because whoever gets a server won't ask here in this forum...

But I care about servers, I mentioned because you mentioned throwing away Intel servers. We have a few hundred physical servers in 3 different data centers and run roughly 1500 VMs, throwing away isn't an option and there isn't even an alternative to switch to yet. Kinda sucks since I really want to test an AMD EPYC based HPE server but they only got one and it's the wrong kind for us.

 

Anyway the good news, kind of, is that we won't have to wait that long to get a lot more information on this.

[Stefan Payne]

2 minutes ago, leadeater said:

It's only limited testing and the database test only shows for that data database engine. What about MSSQL? or Oracle?

Could be a license violation to publish benchmarks with those.

But why do you think that would change? 

Essentially they do the same stuff...

2 minutes ago, leadeater said:

They tested like less than 1% of what is required to fully understand the impact of this.

Yes, just the neccessary stuff to give you some idea what performance you can expect with the patch.

In short it seems like the more syscalls you do, the worse the performance will get.

 

2 minutes ago, leadeater said:

But I care about servers, I mentioned because you mentioned throwing away Intel servers.

Yes, I should have been clear about that, my bad.

 

2 minutes ago, leadeater said:

We have a few hundred physical servers in 3 different data centers and run roughly 1500 VMs, throwing away isn't an option and there isn't even an alternative to switch to yet. Kinda sucks since I really want to test an AMD EPYC based HPE server but they only got one and it's the wrong kind for us.

Expect a performance hit of around 50% for now. If it is less, be happy. If it is more, well you know...

And with throwing away I meant that you are looking to replace the Intel Servers as far as possible, especially on loads that require a high amount of syscalls wich could be the case for what you are doing.

2 minutes ago, leadeater said:

Anyway the good news, kind of, is that we won't have to wait that long to get a lot more information on this.

Yeah, but that could be the only good news you will get in this regard...

 

If AMD really isn't affected, that kills every Intel Chip in serious Server applications like Database and so on...

[Gdog]

Linus Tarvolds talking about the potential impact:

 

https://lkml.org/lkml/2018/1/2/703

[leadeater]

14 minutes ago, Stefan Payne said:

But why do you think that would change? 

Essentially they do the same stuff...

Because the way they interact with the system is different, they use different memory allocation models and different I/O schemes, plus MSSQL is Windows not Linux so that is a factor.

 

There's already big performance differences between different database engines as it is before this patch.

 

14 minutes ago, Stefan Payne said:

Yes, just the neccessary stuff to give you some idea what performance you can expect with the patch.

In short it seems like the more syscalls you do, the worse the performance will get.

Problem is that amount is generally unknown so the impact is unknown. Like how much will this impact ESXi or Hyper-V or KVM. Is there a difference in how they work which will impact this more, does ESXi do more syscalls than KVM?

 

What about our finance software, or backup software. Are storage arrays going to get patched with this or not? Are we about to lose a massive amount of storage performance meaning literally everything is about to grind to a halt.

 

Those tests are interesting but not that helpful.

 

14 minutes ago, Stefan Payne said:

Expect a performance hit of around 50% for now. If it is less, be happy. If it is more, well you know...

And with throwing away I meant that you are looking to replace the Intel Servers as far as possible, especially on loads that require a high amount of syscalls wich could be the case for what you are doing.

50% is very unlikely, the quoted 30% hit by the people working on the patch was done using a network loop back test which is very much worst case possible. Even so if it's above 5% that's potentially 7 or more servers we need to buy to cover it. But that doesn't make the application  performance get any of that speed back, much of the application performance issue cannot be resolved it'll just be slower now which sucks.

[DarkSmith2]

2 hours ago, GoodBytes said:

Ok.. and no one else has problems. Ok.

yepp for sure, and thats why the web is full of complains about win10 and gaming performance and its so loud that in the last year microsoft officially stated to make it better in the future

[su /bin/bash]

2 hours ago, Stefan Payne said:

Why don't you say:

 

Hey, my computer is open for attack and part of a BOTNET. Because that's what you are saying. You don't seem to understand what Security is about and for.

I say this but computer dont undestand me  

I have no any security software.

If I know how to disable the built-in security measures in my operating system and my hardware, then I would have turned them off, or if there was one, I would have deleted them.

[mrkaru]

5 minutes ago, su /bin/bash said:

I say this but computer dont undestand me  

I have no any security software.

If I know how to disable the built-in security measures in my operating system and my hardware, then I would have turned them off, or if there was one, I would have deleted them.

You don't happen to work for Equifax, do you? 

 

On a serious note though, you're either a troll or an incredibly irresponsible admin. While patching this bug will decrease performance, the majority of security services don't effect performance in the slightest, and even basic first line of defence security is better than nothing. 

[wojwen]

Is there any news about which tasks will take the biggest hit? I've seen that it doesn't affect gaming, but that's not surprising since GPUs make the biggest difference there anyway. How about 3D rendering, video rendering etc?

[porina]

2 hours ago, su /bin/bash said:

I care about me is maximum productivity, security does not any matter. I think too much productivity is wasted in the name of security. The problem of loss of performance exchanged for security is so great that it slows the progress of all mankind.

I might not quite go as far, but I do this where appropriate. My internet facing system is patched up with bare minimum defensive software, but I have a bunch of compute nodes and they're stripped down as far as reasonable. AV is one of the worst offenders and I continuously wonder why I bother. Number of detected valid attacks/infections/whatever: 0. Number of false positives: more than 0. AV is costing me time and performance and has saved me exactly zero times ever. It is only FUD-mongers that mean I reluctantly leave it interfering in the background.

 

When this patch goes live, I will have to make a decision on updates, and my thinking right now is to freeze pre-patch on the compute nodes to ensure max performance going forwards. I might apply the patch if the performance drop in what I care about is not detectable within expected benchmark variation.

 

[MikeSK]

For the Intel fix, I have read they are updating the kernel for Linux and Windows.   AMD may not be affected by the current exploit, however, the question is if they are going to be running the same OS kernels and will it have a similar effect on their performance?

[xteron]

12 minutes ago, MikeSK said:

For the Intel fix, I have read they are updating the kernel for Linux and Windows.   AMD may not be affected by the current exploit, however, the question is if they are going to be running the same OS kernels and will it have a similar effect on their performance?

 

After what I've seen the earlier version of kernel to linux affected all CPU's, but amd has submitted a patch to exclude this workaround if it's an AMD cpu. Dont know if it has been approved yet.

[Sir_Loin_Ofsteak]

Don't worry guys, it can't be that bad. It's not like the Intel CEO sold all his shares in the weeks leading up to the reveal or anyth...wait. Shit...

 

https://www.americanbankingnews.com/2017/12/01/intel-corporation-intc-ceo-brian-m-krzanich-sells-889878-shares.html

[Tellos]

I am worried if now I have to stop my plans for a newm oniter with higher refresh and higher resoultion cause my chip will get ganked by this update. Until I knwo I cannot make plans for such a purchase.

[MikeSK]

If the patch is not applied to OS with AMD CPUs, can exploit software fake the CPU information for an intel to appear as an AMD and exploit the Intel? 

[xteron]

6 minutes ago, MikeSK said:

If the patch is not applied to OS with AMD CPUs, can exploit software fake the CPU information for an intel to appear as an AMD and exploit the Intel? 

 

I checked the fix that was submitted, and it looks like this is determined under boot. I dont think that's very easy to do without access or other vulnerabilities. Think you have to get another bootloader or something else to run before kernel is loaded.

[Neokolzia]

27 minutes ago, MikeSK said:

If the patch is not applied to OS with AMD CPUs, can exploit software fake the CPU information for an intel to appear as an AMD and exploit the Intel? 

from my understanding this is a issue occuring at basically a hardware level, making the chips without the patch vulnerable.   It would be like a physical memory access glitch that is found to exploit say PsVita's or 3DS's unless they re-engineer how things access each other and what permissions everything has, even if its fixed at the firmware level/kernels etc it is still exploitable theoretically.  

 

However if it was engineered differently so its physically not possible to do that it would never be exploitable regardless of spoofing etc.

[GoodBytes]

Microsoft releases a statement that the security issues affects AMD and ARM CPUs as well, this is confirmed by Intel as well:

 

Microsoft says:

Quote

We're aware of this industry-wide issue and have been working closely with chip manufacturers to develop and test mitigations to protect our customers. We are in the process of deploying mitigations to cloud services and have also released security updates to protect Windows customers against vulnerabilities affecting supported hardware chips from Intel, ARM, and AMD. We have not received any information to indicate that these vulnerabilities had been used to attack our customers.

 

Intel statement:

Quote

Intel and other technology companies have been made aware of new security research describing software analysis methods that, when used for malicious purposes, have the potential to improperly gather sensitive data from computing devices that are operating as designed. Intel believes these exploits do not have the potential to corrupt, modify or delete data.

 

Recent reports that these exploits are caused by a “bug” or a “flaw” and are unique to Intel products are incorrect. Based on the analysis to date, many types of computing devices — with many different vendors’ processors and operating systems — are susceptible to these exploits.

 

Intel is committed to product and customer security and is working closely with many other technology companies, including AMD, ARM Holdings and several operating system vendors, to develop an industry-wide approach to resolve this issue promptly and constructively. Intel has begun providing software and firmware updates to mitigate these exploits. Contrary to some reports, any performance impacts are workload-dependent, and, for the average computer user, should not be significant and will be mitigated over time.

 

Microsoft says that Windows 10 users will start getting the update at 5pm EST time (so, out now).

And Windows 7 and 8 users will get it on the next Tuesday Patch.

 

Sources from the Verge says that Skylake architecture based CPUs won't have much of an impact on performance, but older CPUs can have more visible one.

Quote

The firmware updates and software patches could cause some systems to run slower. Sources familiar with the situation tell The Verge that Intel processors that are based on Skylake or newer architecture won’t see a significant performance degradation. However, older processors could slow down more significantly due to the firmware and software updates.

 

Sources:

https://www.theverge.com/2018/1/3/16846784/microsoft-processor-bug-windows-10-fix

https://www.theverge.com/2018/1/3/16846540/intel-processor-security-flaw-bug-response

 

Update

As you noticed, ARM CPUs are affected as well, and Google confirmed that Android and ChromeOS are affected by this, and that they working on a fix as well (although no info on release time). Also, ARM confirmed that iPhones and iPad ( Cortex-A series) are affected as well.

https://www.theverge.com/2018/1/3/16846840/intel-arm-processor-flaw-chipocalypse-windows-macos-linux

 

[Sir_Loin_Ofsteak]

AMD have refuted Intel's claim that they are equally affected:

 

https://www.cnbc.com/2018/01/03/amd-rebukes-intel-says-flaw-poses-near-zero-risk-to-its-chips.html

[Sir_Loin_Ofsteak]

Intel PR bullshit translation now available on the Register. This just gets better and better.

 

*Cooks popcorn*

*Eats popcorn*

 

https://www.theregister.co.uk/AMP/2018/01/04/intels_spin_the_registers_annotations/

[mulkman]

Just ran a couple of benchmarks at work POST and PRE the Microsoft Patch.

Windows 7 x64 Build Running Skylake 6700 at Work

 

CineBench R15

PRE

POST

 

Geekbench

 

PRE

 

 

POST

 

[Gdog]

Apparently, this is going to have a negative hit on nVidia cards when coupled with intel CPU's. Will not effect AMD GPU + Intel CPU combo as AMD GPU's (since GCN 1.1) has a dedicated pipeline for syscalls whereas nVidia do not.

 

Any benchmarks can offer insight on this? If anyone has pre/post results on nVida/Intel combo hardware please post.

[Stefan Payne]

 

Can it get any worse?

YES, IT CAN!

 

 

 

@leadeater FYI

@GoodBytes