Intel Cutting AVX-512 Support from Alder Lake CPUs.

[RejZoR]

I don't quite understand how is this an issue. It was never officially supported and if you so dearly need AVX-512, you'll either use some Alder Lake based CPU for heavy loads without E cores or you'll use some older Intel CPU that has AVX-512.

 

For 99% of users, this is a non issue. Most, even compute intensive apps don't use AVX-512 and games most certainly don't. There are few that use AVX-2, but for the most part they only use some variant of SSE.

[leadeater]

18 minutes ago, James Evens said:

Currently there are none/the only serious player in this market is Intel.

No? And even more no if we are talking about laptops.

 

5700G is barely slower than a 5800X so unless you are wanting a desktop CPU with more performance than a 5800X and also don't mind a slower iGPU, or specifically need Intel QuickSync, then the 5700G is the better option for CPUs with iGPUs.

 

This is even more the case for laptops, Ryzen Mobile is vastly better than anything Intel has on the market now as there are no Alder Lake mobile CPUs yet.

 

What it really boils down to is do you have a heavy performance reliance on AVX or not, if yes go Intel, if not go AMD. If you need strong general GPU compute and only from a iGPU go AMD.

 

Both offer highly competitive products that will suit people in different ways, there is no automatic "Intel is the only serious player".

[porina]

The points have been covered at some point, but the summary position is:

 

AVX-512 was never supported in Alder Lake in the first place. Access to it was found by mobo manufacturers and enabled. In the same way remember the PCIe 4.0 support on older AMD chipsets when Zen 2 was launched? Mobo manufacturers found a way to get it working, but AMD killed it off with bios updates.

 

The main reason for disabling AVX-512 in Alder Lake is Windows and software in general. If you move AVX-512 code from a P core to an E core, bad things happen. The simplest solution with lowest impact is to not have AVX-512 in P-cores. I had thought of other solutions but they would require software support which would negatively impact more software than not having AVX-512.

 

There is likely no legal ground for action against Intel since AVX-512 support was never official. There may be some grounds for legal action with either the seller of the system or the mobo manufacturers depending on how they advertised it and the laws of a particular region.

 

AVX-512 on ADL being more efficient than Rocket Lake might be due to the process update. I don't have test data I'd trust to make a determination. If anyone has an ADL system with AVX-512 enabled and is willing to run some custom Prime95 benchmarks for me, please get in touch!

 

There was also talk before launch that the E-cores provided more benefit overall than AVX-512 did. That would very much depend on the workload so may not apply in all cases.

 

I do not think it is a market segmentation thing, since AVX-512 has been present in at least two mobile generations (Ice Lake, Tiger Lake), and one mainstream desktop generation (Rocket Lake). If you include HEDT that would bring in Skylake-X and Cascade Lake-X too.

 

For my personal interests, code like Prime95 is about 40% faster (IPC) with AVX-512 than AVX2 on Rocket Lake vs Skylake family. Skylake-X has a better implementation is about 80% faster. On AMD side with AVX2, Zen 2 slightly surpassed Skylake by 4%, with Zen 3 another 10% faster on top of that.

 

There is speculation that Zen 4 might have AVX-512 support, so we might have a strange reversal in positions since AMD have most of the time had worse FP64 performance compared to Intel. For Intel to return AVX-512 with hybrid CPUs will probably require some degree of AVX-512 support on E-cores. It doesn't have to be good support. They just need to do like older AMD and provide instruction support without any significant performance behind it. Then AVX-512 can remain enabled and it is up to the OS to keep it on P-cores for performance.

[Forbidden Wafer]

5 hours ago, porina said:

For Intel to return AVX-512 with hybrid CPUs will probably require some degree of AVX-512 support on E-cores.

Just don't schedule stuff that uses AVX512 to E-cores. You can check the instructions before loading (only a few formats have the flag to indicate avx use) in the binaries indicating it's being used...  

 

Also, why not let users choose if they want to use AVX-512 or E-cores (which are useless for a bunch of people)? Really. At this point, Intel can continue burning for being jerks. 

[RejZoR]

People talk like E cores are absolute trash. Those E cores are actually on par with "performance" cores found in Skylake CPU's just few years ago.

 

Der8auer even tried gaming exclusively on E cores and games actually ran great. I have a quad core laptop that runs Gracemont cores which are essentially an older revision of E cores found on Alder Lake. The thing is amazing and it runs fully passive.

[porina]

40 minutes ago, Forbidden Wafer said:

Just don't schedule stuff that uses AVX512 to E-cores. There's a freaking flag in the binaries indicating it's being used...  

Not a programmer, is that the case? If so, it would be a step to implement in Windows or other OSes. Could it be more than that though? For example, I know of binaries that will use AVX-512 if present, but AVX2 or other codepaths if not. The software wouldn't use AVX-512 at all if not present. I guess the complication is that you might only have small parts of software that can use AVX-512 and the rest can run on P and E cores for best performance. What then? You lose the E-cores for that other time? Performance is just more consistent for unknown consumer workloads the route Intel have taken. It may be a question how much use E cores provide vs extra P-cores with or without AVX-512, 

 

I love AVX-512, but at the end of the day it is about processing throughput, and the E-cores may offset that in more cases than not.

 

40 minutes ago, Forbidden Wafer said:

Also, why not let users choose if they want to use AVX-512 or E-cores (which are useless for a bunch of people)?

There is a user choice. If you primarily want to run AVX-512 workloads, there is still a bunch of choice from the outgoing Rocket Lake generation, to workstation Xeons. For the enthusiast, the X299 platform is a bargain right now as used CPUs for it are very cheap, and have a superior AVX-512 implementation compared to the mainstream consumer parts.

[leadeater]

1 hour ago, Forbidden Wafer said:

Just don't schedule stuff that uses AVX512 to E-cores. You can check the instructions before loading (only a few formats have the flag to indicate avx use) in the binaries indicating it's being used...  

 

Also, why not let users choose if they want to use AVX-512 or E-cores (which are useless for a bunch of people)? Really. At this point, Intel can continue burning for being jerks. 

If it were that simple then it would have been done and how it works. Intel already said it's not that simple and that is the reason for why Alder Lake doesn't have support for AVX-512.

 

The issue isn't where a process starts, it's every single check that the scheduler does to see where to place a thread ongoing and there is a massive difference between a simple utilization check and looking down to the instruction level.

 

From what I read this might have been possible if Alder Lake were only supported on Windows 11, outright no workarounds, but I'm still a little dubious if the Thread Director and the Windows 11 scheduler really is as smart as being made out.

[Forbidden Wafer]

1 hour ago, leadeater said:

If it were that simple then it would have been done and how it works. 

It is that simple unless they're doing something weird at the processor level. 

 

1 hour ago, leadeater said:

The issue isn't where a process starts, it's every single check that the scheduler does to see where to place a thread ongoing

Sorry, but CPU masks are a thing. You do it once and the scheduler won't attempt to schedule that program on the wrong cores.

1 hour ago, leadeater said:

there is a massive difference between a simple utilization check and looking down to the instruction level.

objdump -d library/executable | grep %zmm0

If it is not empty, it contains AVX512 code, so mask out cores that do not support it during process setup. Done.

[Mark Kaine]

Every time i wanna go intel they pull something like this... literally since Pentium 1, which was my last intel desktop processor...

("proud" owner of an i5u 7200, but that thing is rather abysmal,  for the price especially, which probably isnt entirely intels fault tbf, thanks Lenovo!)

 

 

17 hours ago, Forbidden Wafer said:

Too bad for them, because if it got out, and it worked, then they disabled for no good reason, it still looks bad on them.

just like sony and linux support (which they heavily advertised, mind you) they dont care, $$$ is still rolling. sony didn't even need to remove linux support,  there was no hack, they were just *scared* there would be,  few months later ps3 was fully hacked by the overflow guy (iirc) *without linux*, including "dev store" access and remains to this day (i don't know if they fixed the store access,  i suppose they just shut it down now, but they didn't for years, even though they knew what was going on)

 

ie. intel doesn't care, also if in doubt play the "security" card, that always works

 

 

 

 

[grg994]

5 hours ago, Forbidden Wafer said:

Just don't schedule stuff that uses AVX512 to E-cores.

I am so curious since the release what would happen if in this case. I mean with E-cores enabled as normal, what do the P-cores do if you put AVX-512 code on them manually? Would they just give you SIGILL? Please someone try this

[grg994]

On 1/1/2022 at 8:35 PM, CommanderAlex said:

Intel will be killing off AVX-512 enablement through a microcode update

 

Btw if someone wants to run eg. scientific computing on Alder Lake / Linux setup with E-cores disabled + AVX-512 in the future - just disable the updates for the intel-ucode (or something similar for your disro) package in the package manager's config file.

 

Or if you building the system only in the future, force an older version of the intel-ucode package to be installed then. At least on Linux you cannot be forcefully affected by this as the OS loads up the microcode during every boot by it's own discretion.

[cj09beira]

12 hours ago, porina said:

The points have been covered at some point, but the summary position is:

 

AVX-512 was never supported in Alder Lake in the first place. Access to it was found by mobo manufacturers and enabled. In the same way remember the PCIe 4.0 support on older AMD chipsets when Zen 2 was launched? Mobo manufacturers found a way to get it working, but AMD killed it off with bios updates.

 

The main reason for disabling AVX-512 in Alder Lake is Windows and software in general. If you move AVX-512 code from a P core to an E core, bad things happen. The simplest solution with lowest impact is to not have AVX-512 in P-cores. I had thought of other solutions but they would require software support which would negatively impact more software than not having AVX-512.

Pcie 4.0 kinda worked, some people had stability issues with it on though, so it was kinda of a mess.

While in this case, it was already not a problem, if E cores are enabled avx512 is disabled, if E cores are disabled avx 512 can be enabled, it worked well enough, at stock avx512 wouldn't be available but if for some reason you really need it you could choose to enable it, so disabling it completely only prevents people from accessing it.

[WolframaticAlpha]

16 hours ago, TrigrH said:

However I draw the line at taking features already fucking released away.

But the feature was already announced to be removed before launch. AVX512 on Alder Lake was an ugly hack.

[leadeater]

9 hours ago, Forbidden Wafer said:

It is that simple unless they're doing something weird at the processor level. 

But it's not, saying it is doesn't make it so.

 

9 hours ago, Forbidden Wafer said:

Sorry, but CPU masks are a thing. You do it once and the scheduler won't attempt to schedule that program on the wrong cores.

 

9 hours ago, Forbidden Wafer said:

If it is not empty, it contains AVX512 code, so mask out cores that do not support it during process setup. Done.

How about you go read the information about what the Thread Director is and how it works again before speaking about this again. Try and do that every 30 microseconds, you can't which is why it's done within the CPU. Also consider the fact that CPU features are reported to the OS as a whole, not broken down to cores so how do you prevent Windows from placing a thread on an E core when as far as it knows it supports AVX-512? Every core presents the same feature set as there is no precedent for them to have different capabilities.

 

The basic framework to support this actually already is in Alder Lake and the Thread Director, the problem is I suspect OS schedulers or something else outside of the CPU. Process/Thread classification is being done and part of that already is whether or not AVX is being used.

 

Quote

If there’s an obvious potential for better IPC or better efficiency, then it suggests the thread is moved. Workloads are broadly split into four classes:

• Class 3: Bottleneck is not in the compute, e.g. IO or busy loops that don’t scale
• Class 0: Most Applications
• Class 1: Workloads using AVX/AVX2 instructions
• Class 2: Workloads using AVX-VNNI instructions

Anything in Class 3 is recommended for E-cores. Anything in Class 1 or 2 is recommended for P cores, with Class 2 having higher priority. Everything else fits in Class 0, with frequency adjustments to optimize for IPC and efficiency if placed on the P-cores. The OS may force any class of workload onto any core, depending on the user.

https://www.anandtech.com/show/16959/intel-innovation-alder-lake-november-4th/3

 

If it really were that simple AVX-512 would be supported and working on Alder Lake, Intel has been the most hot on this and uses it in their provided benchmarks to show how they are better. Some future HEDT product is not the reason for it and is not a good reason at all, neither is that supported by existing products either.

[YoungBlade]

Based on the timing of this, I think the release of the CPUs without E cores is a factor here. I think some motherboard makers might have been preparing to enable AVX-512 by default on CPUs without E cores, which would have made the higher SKU parts look bad in some situations.

 

Imagine some reviewers post results where a 12400 stock does better than even an overclocked 12600K, because the workload happens to take advantage of AVX-512. At first glance, it would look like the E cores are tanking performance, and unless the reviewer thought it through, that would be the logical conclusion to come to.

 

I do think product segmentation is a real factor here, but I don't think it's the only reason. as originally, AVX-512 was a hacky workaround that their business customers probably wouldn't have undertaken anyway. But anything that makes the E cores look bad hurts Intel's marketing in a big way.

[TrigrH]

45 minutes ago, WolframaticAlpha said:

But the feature was already announced to be removed before launch. AVX512 on Alder Lake was an ugly hack.

There is a big difference between physically in the silicon and an ugly hack. It works and it worked a lot better than the crap on rocket lake.

 

If it wasn't planned to go in, it wouldn't exist at all. Sounds like to me intel failed to get it working how they wanted in time and decided to pull it - too little too late. 

[WolframaticAlpha]

20 minutes ago, TrigrH said:

There is a big difference between physically in the silicon and an ugly hack. It works and it worked a lot better than the crap on rocket lake.

 

If it wasn't planned to go in, it wouldn't exist at all. Sounds like to me intel failed to get it working how they wanted in time and decided to pull it - too little too late. 

It is an ugly hack. You have to disable the E cores. 

 

> If it wasn't planned to go in, it wouldn't exist at all

Yeah, because they didn't have time to actually reengineer a die without AVX512. If you bought adl for avx512, then it is (unfortunately) on you.

 

Would I like AVX512 on my processor? sure. Is it a dick move by intel? sure. Should you have bought a processor where a feature was disabled and works thru a hack? Nope.

[Forbidden Wafer]

2 hours ago, leadeater said:

How about you go read the information about what the Thread Director is and how it works again before speaking about this again. 

 

It collects data and provides more info for the OS scheduler. Aka the OS scheduler still makes the decisions and could easily mask out incompatible cores...  

 

2 hours ago, leadeater said:

Also consider the fact that CPU features are reported to the OS as a whole, not broken down to cores so how do you prevent Windows from placing a thread on an E core when as far as it knows it supports AVX-512? Every core presents the same feature set as there is no precedent for them to have different capabilities.

Every OS has different configurations for specific CPU families. Just look at the Linux source code.

 

From the kernel mailing list:

 

Quote

Introduce a new struct x86_hybrid_pmu to save the unique capabilities
  from different PMUs. 

https://lkml.org/lkml/2021/2/8/1142

 

Now on the code: yeah, they pick different capabilities based on the smp_processor_id(). 

 

https://github.com/torvalds/linux/commit/61e76d53c39bb768ad264d379837cfc56b9e35b4

 

Scheduling? Yup, they check it too. See the cpu masks? The smt and x86 core flags? Told ya. 

 

https://github.com/torvalds/linux/commit/cabdc3a8475b918e55744f43719b26a82dc8fa6b

[Forbidden Wafer]

29 minutes ago, WolframaticAlpha said:

It is an ugly hack. You have to disable the E cores. 

Because they don't want to properly schedule stuff and are too lazy to disable the silicon before selling it...  

[jagdtigger]

15 hours ago, RejZoR said:

People talk like E cores are absolute trash.

Because they are and pretty much pointless in a desktop.....

[porina]

9 hours ago, cj09beira said:

Pcie 4.0 kinda worked, some people had stability issues with it on though, so it was kinda of a mess.

That's the thing about unsupported features. It might or might not work. If it does, or at least appears to, great. If not, you turn it off and move on. You have no support to fall back on. If people think Intel should leave AVX-512 option open in an unsupported role, then that would similarly apply to AMD PCIe 4.0, and also AMD's unsupported ECC support on consumer AM4. People who really care about a feature will ensure they get a supported version of that feature. The proportion who would use an unsupported feature because it is there is miniscule.

 

4 hours ago, TrigrH said:

It works and it worked a lot better than the crap on rocket lake.

Got a link to some data for that? If I try to search for it now all I get are variations of articles on the removal news.

 

In my testing of the Rocket Lake implementation using Prime95-like workloads, it gives about 40% IPC increase over Comet Lake, at near enough same perf/W. This is weaker than Skylake-X at about 80% IPC increase vs Skylake (Skylake = Comet Lake for this purpose). I don't have power data for that as my system doesn't report it in a usable way.

 

As such I'm really curious if the ADL implementation behaves much differently in either performance and/or efficiency. There may be some efficiency benefit from the upgraded process alone. If there is a perf difference, the question then is, is it actually due to the AVX-512 implementation or other changes elsewhere in the CPU?

 

4 hours ago, TrigrH said:

If it wasn't planned to go in, it wouldn't exist at all. Sounds like to me intel failed to get it working how they wanted in time and decided to pull it - too little too late. 

Based on some drama around Alder Lake launch, it does sound like the decision to not enable AVX-512 was made internally at Intel relatively late in the day but still way before launch.

[leadeater]

5 hours ago, Forbidden Wafer said:

It collects data and provides more info for the OS scheduler. Aka the OS scheduler still makes the decisions and could easily mask out incompatible cores...  

 

Every OS has different configurations for specific CPU families. Just look at the Linux source code.

 

From the kernel mailing list:

 

https://lkml.org/lkml/2021/2/8/1142

 

Now on the code: yeah, they pick different capabilities based on the smp_processor_id(). 

 

https://github.com/torvalds/linux/commit/61e76d53c39bb768ad264d379837cfc56b9e35b4

 

Scheduling? Yup, they check it too. See the cpu masks? The smt and x86 core flags? Told ya. 

 

https://github.com/torvalds/linux/commit/cabdc3a8475b918e55744f43719b26a82dc8fa6b

No because you didn't actually read what I said, CPU features are presented and common to ALL cores in Windows, prior to Windows 11. Don't go applying the capabilities of the Linux scheduler to every single OS.

 

Further to that you still have the same issue, unless you are up to date then CPU support is only for x version and up only which leads to bad user experience if they cannot update because they have now unsupported hardware only and an existing install or some specific version they have to run which can't be updated.

 

The flaw in your evidence and argument is trying to prove your case using Linux when I was referring to Windows.

 

"But Linux can do it", cool story but Linux != every computer OS. I have not seen anything in the Windows world that would allow any of this to work properly and certainly not anything before Windows 11. Having a hardware product that only works correctly in Linux, or Linux and Windows 11 only, is a real quick way to make it a DOA product.

 

How is that Windows 11 adoption rate going?

[leadeater]

1 hour ago, porina said:

Based on some drama around Alder Lake launch, it does sound like the decision to not enable AVX-512 was made internally at Intel relatively late in the day but still way before launch.

If it were ever going to work then the CPU would have to not present AVX-512 to Windows 10 and prior but then do so for Windows 11 and later, somehow. If they were intending on doing this then for whatever reason they weren't able to achieve it. Or alternatively didn't care to since Windows 10 is going out of support soon and will just sunset offical support for that OS when that happens and by then E cores will probably have AVX-512 anyway.

 

Effort versus reward balancing, probably just wasn't worth it.

[leadeater]

1 hour ago, porina said:

Got a link to some data for that? If I try to search for it now all I get are variations of articles on the removal news.

 

In my testing of the Rocket Lake implementation using Prime95-like workloads, it gives about 40% IPC increase over Comet Lake, at near enough same perf/W. This is weaker than Skylake-X at about 80% IPC increase vs Skylake (Skylake = Comet Lake for this purpose). I don't have power data for that as my system doesn't report it in a usable way.

 

As such I'm really curious if the ADL implementation behaves much differently in either performance and/or efficiency. There may be some efficiency benefit from the upgraded process alone. If there is a perf difference, the question then is, is it actually due to the AVX-512 implementation or other changes elsewhere in the CPU?

Maybe not what you are looking for? Not sure but here it is anyway.

 

 

 

https://www.anandtech.com/show/17047/the-intel-12th-gen-core-i912900k-review-hybrid-performance-brings-hybrid-complexity/2

 

 

Quote

The Core i9-11900K in our test peaks up to 296 W, showing temperatures of 104ºC, before coming back down to ~230 W and dropping to 4.5 GHz. The Core i7-11700K is still showing 278 W in our ASUS board, tempeartures of 103ºC, and after the initial spike we see 4.4 GHz at the same ~230 W.

https://www.anandtech.com/show/16495/intel-rocket-lake-14nm-review-11900k-11700k-11600k/5

 

So looks to me ADL is using about 80% of the power compared to RKL for AVX-512 with about 6% more performance. Unless I'm wrong? This is rough though since the power is from different AVX-512 workloads, performance is common workload.

 

From what I understood about Golden Cove should be equivalent implementation to SKL-X, so I suspect ram bandwidth is a limiter for ADL? What do you think? Maybe it's not the same though.

 

Golden Cove

 

Skylake Server

 

 

 

[porina]

1 hour ago, leadeater said:

If it were ever going to work then the CPU would have to not present AVX-512 to Windows 10 and prior but then do so for Windows 11 and later, somehow. If they were intending on doing this then for whatever reason they weren't able to achieve it. Or alternatively didn't care to since Windows 10 is going out of support soon and will just sunset offical support for that OS when that happens and by then E cores will probably have AVX-512 anyway.

I shouldn't have used "enable" but "support". The current unofficial situation could have been official, but AVX-512 was dropped from support to the surprise of some working on it within Intel.

 

I don't think Win10 is a big factor in this. Win10 will still be supported by MS at the time Alder Lake is replaced by a few generations.

 

1 hour ago, leadeater said:

Maybe not what you are looking for? Not sure but here it is anyway.

Thanks for trying, but that isn't what I'm looking for. One "problem" with AVX-512 is that it is not just one thing but a family of instructions. Personally I'm interested in AVX-512F which is mandatory, and even then, particularly on the parts relating to FP64 performance. The best tool I know of for that is Prime95, but it requires an understanding of how it works in order to take meaningful data from it. Can't press a button and get a result.

 

I have no idea what 3DPM does, but it does feel like very unique software. It represents itself, and I'm not sure what else. Dr Wafer Eater described original versions of 3DPM which he developed as part of his PhD as representative of scientist code. That is, it works but wont be optimised. The version that is available to the public does not have AVX-512 support. The software first made my radar when results were presented showing >>50% speedup from SMT. I don't think I've encountered other software that does that. For lack of a better term, it is probably very low density code and offers significant speedups. It is unclear to me just how much "AVX-512" stuff is going on. With Prime95, it is a LOT.

 

1 hour ago, leadeater said:

So looks to me ADL is using about 80% of the power compared to RKL for AVX-512 with about 6% more performance. Unless I'm wrong? This is rough though since the power is from different AVX-512 workloads, performance is common workload.

If we assume the AVX-512 implementation by itself isn't different, then an efficiency improvement may be expected just from process update alone. Of course with lots of variables not considered here.

 

1 hour ago, leadeater said:

From what I understood about Golden Cove should be equivalent implementation to SKL-X, so I suspect ram bandwidth is a limiter for ADL? What do you think? Maybe it's not the same though.

Ram bandwidth shortage has been on my radar for AVX2 workloads at least as far back as Haswell, and probably Sandy Bridge too (AVX) but I just didn't understand it then. For my use cases multi-threading has taken some edge off that, where cores work on shared data that is more easily kept on CPU cache. The higher end Skylake-X implementations were two unit, so offering potentially double the throughput compared to single unit implementations seen in consumer hardware. I assume ADL is single unit.

 

Ram bandwidth limiting equally affects Ryzen, especially since Zen 2 when its FP performance finally crossed Haswell levels. But AMD are more generous with cache so if you can work within the fragmentated nature you can get decent throughput out of it. It forces you to have multiple slower tasks than one faster one like with monolithic-like designs.