AMD anounces new Epyc cpus and new datacenter compute Card

[cj09beira]

Today, November 8th 2021, amd announced in a live-stream what their next generation data center products will look like.

Starting out with their newest server cpu code named Milan-X, which essentially is Milan with a truck load of cache added on top of it, 512MB worth (8*64MB), as they showed a few months ago it looks exactly like the old milan processors, keeping the same 64 core maximum, same socket etc, and current servers can support it with just a bios update.

 

Then on the graphics side they announced the MI250x and MI250, a dual die, octa channel HBM 2e, with 3.2TB/s of memory bandwidth, 128GB of vram, and 220 Compute units based on tsmc's 6nm node with a monster tdp to match of 560W.

 

then by the end they also gave us a sneak peak into what's next, confirming MLID's leak to a teeth, with zen 4 having 2 versions, one code named GENOA with up to 96 cores, and another code named BERGAMO with up to 128 Cores, the latter optimized for density and less so general compute both will be made on tsmc's 5nm and launch around late 2022, and interestingly its already sampling to customers, (Bergamo to come later in 2023).

along the expected specs like DDR5, PCIe 5.0, is a very important one CXL

 

Quote

CDNA2:

58B transistors

220 Compute units in MI250X / 208 Compute units in the MI250

Double and single precision: 47.9 Tflops 

half precision: 383Tflops

880 matrix cores

400GB/s of bi directional bandwidth between both dies 

 

Another things announced:

New partnership with facebook's meta and SAP S/4HANA,

The new milan-X cpus allow a 66% performance increase in RTL verification, that amd uses internally to verify cpu/gpu designs,

milan-X on Azure HBv3 instances

 

 

 

 

My thoughts

I love too see packaging tech continue to be pushed by amd, a small detail of the cdna 2 announcement is that its actually using embedded silicon interposers to connect the gpus to the hbm stacks, and to connect the two gpu dies, this means that hbm will now become much more flexible as one no longer needs to have a silicon interposer the size of the entire substrate, added to the fact that hbm 3 is near its prime time things are looking good for the standard, can't wait for all this tech to trickle down to us mortals.

Just noticed they also announced CXL support with zen 4 epyc, so we might be close to getting a more efficient use of the pcie lanes our cpus have as well as finally a interconnect where optane can thrive.

 

Sources

 

 

[igormp]

There's a nice overview on AT:

https://www.anandtech.com/show/17054/amd-announces-instinct-mi200-accelerator-family-cdna2-exacale-servers

 

I wonder if they'll still wait until january to announce the consumer Zen3D, or if they'll do it this year.

32 minutes ago, cj09beira said:

Then on the graphics side they announced the MI250x and MI250, a dual die, octa channel HBM 2e, with 3.2TB/s of memory bandwidth, 128GB of vram, and 220 Compute units based on tsmc's 6nm node with a monster tdp to match of 560W.

Those are really cool, specially given the OAM form-factor. I can see those being important for supercomputers and scientific workstations.

As for hyperscalers and other usages? Well, I doubt it'll get any traction there, but it doesn't seen to be their focus anyway.

[cj09beira]

3 minutes ago, igormp said:

There's a nice overview on AT:

https://www.anandtech.com/show/17054/amd-announces-instinct-mi200-accelerator-family-cdna2-exacale-servers

 

I wonder if they'll still wait until january to announce the consumer Zen3D, or if they'll do it this year.

Those are really cool, specially given the OAM form-factor. I can see those being important for supercomputers and scientific workstations.

As for hyperscalers and other usages? Well, I doubt it'll get any traction there, but it doesn't seen to be their focus anyway.

their focus seems to be getting large customers to buy for very specific workloads, then slowly improving the general performance and software stack

[StDragon]

512MB of cache.... You know, there was a time when you could easily game on 512MB of RAM. Obviously you don't need textured assets there, but the core engine fitting in all cache would probably be a big boost.

[Quackers101]

finally it was posted, lets go, please some AMD AI stuff too ;()

[igormp]

4 minutes ago, Quackers101 said:

finally it was posted, lets go, please some AMD AI stuff too ;()

Sadly, I doubt this is going to be a thing soon, there's no signs of AMD putting the effort on their software stack for ML stuff, and those announced products are clearly targeted at more scientific workloads (hence all of the full-speed FP64 stuff).

[CarlBar]

5 hours ago, StDragon said:

512MB of cache.... You know, there was a time when you could easily game on 512MB of RAM. Obviously you don't need textured assets there, but the core engine fitting in all cache would probably be a big boost.

 

He, my first ever PC had 128mb, (though it quickly got upgraded, to 512mb ironically).

 

Also given the core count difference between the two versions of Zen4 i have to winder is the lower core count one is another extreme cache form?

[leadeater]

10 hours ago, cj09beira said:

Today, November 8th 2021,

LIES!!

 

 

[leadeater]

10 hours ago, igormp said:

Those are really cool, specially given the OAM form-factor. I can see those being important for supercomputers and scientific workstations.

As for hyperscalers and other usages? Well, I doubt it'll get any traction there, but it doesn't seen to be their focus anyway.

If Nvidia SXM was attractive to the hyperscalers then this will be equally as much, so probably not a whole lot. I think the 4 GPU SXM designs do get used there, maybe the 8 GPU but not the 16 GPU designs far as I know.

 

10 hours ago, cj09beira said:

their focus seems to be getting large customers to buy for very specific workloads, then slowly improving the general performance and software stack

Software really needs to come first though. Only so much raw compute power can be attractive before the issue of practically using it becomes the large factor, talking GPUs here.

 

EPYC CPUs are basically a given for new deployments now, Intel Xeon is MIA until Sapphire Rapids gets announced and there still is no guarantee that's going to be be as all round competitive as EPYC 7003 is.

 

7 hours ago, igormp said:

Sadly, I doubt this is going to be a thing soon, there's no signs of AMD putting the effort on their software stack for ML stuff, and those announced products are clearly targeted at more scientific workloads (hence all of the full-speed FP64 stuff).

AI/ML is still scientific workloads as well. Weka is a widely used tool in this area, written here in NZ by Waikato University. Example of ongoing and just starting projects now using these computation types and frameworks is Maori voice recognition and translation, seismic ground movement tracking using thousands of IoT sensors, orthophoto and ground radar analysis of pasture quality.

[cj09beira]

5 hours ago, CarlBar said:

 

He, my first ever PC had 128mb, (though it quickly got upgraded, to 512mb ironically).

 

Also given the core count difference between the two versions of Zen4 i have to winder is the lower core count one is another extreme cache form?

there are 2 ways i see the difference between the two manifest, 1 is the cloud version simply having less cache, the second is having denser slower cache but the same amount (as amd showed with zen 3d they can double density on the same node)

10 hours ago, igormp said:

Sadly, I doubt this is going to be a thing soon, there's no signs of AMD putting the effort on their software stack for ML stuff, and those announced products are clearly targeted at more scientific workloads (hence all of the full-speed FP64 stuff).

they are working on it, its just quite slow, for example rocm now supports rdna2, it took them 1 year to get there though.

[igormp]

4 hours ago, leadeater said:

AI/ML is still scientific workloads as well. Weka is a widely used tool in this area, written here in NZ by Waikato University. Example of ongoing and just starting projects now using these computation types and frameworks is Maori voice recognition and translation, seismic ground movement tracking using thousands of IoT sensors, orthophoto and ground radar analysis of pasture quality.

As someone who works in the area and had to use weka, I hate it. All of my homies hate it too.

 

It's perfect for people who actually have no technical knowledge of ML and want to try out stuff in a graphical way, but it's awful for more "complicated" stuff.

Anyway, weka doesn't work with AMD GPUs (barely any ML framework does), and its requirements are a bit different from your usual scientific workloads (uses low-precision data for faster results instead of double precision).

 

1 hour ago, cj09beira said:

they are working on it, its just quite slow, for example rocm now supports rdna2, it took them 1 year to get there though.

ROCm still doesn't support the first RDNA, and it's not available for RDNA as of today. Is it going to be announced with ROCm 5.0? I saw this on their github repo:

[BuckGup]

I've been working with the MI200s firsthand for a month or two now and they have insanely high failure rates. Like 1:2 

[cj09beira]

25 minutes ago, BuckGup said:

I've been working with the MI200s firsthand for a month or two now and they have insanely high failure rates. Like 1:2 

that's sad but cool as i just want a dead one, they look so good

hopefully i will find one on ebay eventually.

 

Any idea why they are dying?, i am guessing emib failures

[AlTech]

To everybody that says MLID is unreliabe and never gets things right as a leaker: He was right on the money with Bergamo.

 

Also 220 CUs is insane. I imagine the price tag will be equally insane though that's an issue for the corporate customers of the MI200 series.

[tim0901]

15 hours ago, igormp said:

Sadly, I doubt this is going to be a thing soon, there's no signs of AMD putting the effort on their software stack for ML stuff, and those announced products are clearly targeted at more scientific workloads (hence all of the full-speed FP64 stuff).

Lots of scientific computing relies on ML these days. ATLAS at CERN - for example - uses it to analyse the exabytes of data they create each day in the trigger system, as well as to explore the big data sets stored in the Worldwide LHC Computing Grid. Machine learning is an absolute requirement to be able to cope with the huge amounts of data outputted by these experiments and is only becoming more important as time goes on.

 

Of course, lots of these ML algoritms are now being run on FPGAs, rather than traditional compute hardware, as they are much more efficient than CPUs or GPUs. One test by CERN found that an Arria 10 PAC FPGA is roughly as fast as a GTX 1080 when running a ML workload, but only uses 1/5 of the power.

[saltycaramel]

Beyond excited for the future of multi-story 3D stacked CPUs. 

 

AMD+TSMC look like they’re all-in on it.

 

Looking forward to see Apple+TSMC 3D-stacked SoiCs, hopefully. 

[leadeater]

4 hours ago, igormp said:

It's perfect for people who actually have no technical knowledge of ML and want to try out stuff in a graphical way, but it's awful for more "complicated" stuff.

Well it is supposed to be easy to use and a starting point, that's why it's popular.

[igormp]

35 minutes ago, tim0901 said:

Lots of scientific computing relies on ML these days. ATLAS at CERN - for example - uses it to analyse the exabytes of data they create each day in the trigger system, as well as to explore the big data sets stored in the Worldwide LHC Computing Grid. Machine learning is an absolute requirement to be able to cope with the huge amounts of data outputted by these experiments and is only becoming more important as time goes on.

Maybe I phrased it badly. By scientific workloads I meant stuff such as CFD simulations and the like, where you do need all of the precision available, whereas for ML is the exactly opposite.

 

38 minutes ago, tim0901 said:

Of course, lots of these ML algoritms are now being run on FPGAs, rather than traditional compute hardware, as they are much more efficient than CPUs or GPUs. One test by CERN found that an Arria 10 PAC FPGA is roughly as fast as a GTX 1080 when running a ML workload, but only uses 1/5 of the power.

Or even dedicated hardware, since you can get higher clock speeds than with a FPGA, such as the ones from habana labs or Google's TPUs.

[RejZoR]

I think Milan-X is a proof that vertical cache "3D V-Cache" planned for upcoming desktop CPU's will be a drop-in upgrade after BIOS update too. New CPU's with 3D V-Cache should be AM4 compatible and will most likely be a Zen3+ design and not Zen4. Sort of refresh of existing Zen3 with more stacked cache. Which, because data also goes vertically through the cache stack should have lower latency just like Milan-X.

 

Would be kind of hilarious if this alone and few minor tweaks could negate entire Alder Lake advantages...

[pas008]

16 minutes ago, RejZoR said:

Would be kind of hilarious if this alone and few minor tweaks could negate entire Alder Lake advantages...

why hilarious?

 

would be right, proper competition gives us better products and better prices

[CephDigital]

Well this is neat.

Best bet is that they're going to announce the consumer versions at CES, definately Zen 3D but no idea about new GPUs. Maybe a refresh? Heard some rumors about a GPU refresh.

[RejZoR]

1 hour ago, pas008 said:

why hilarious?

 

would be right, proper competition gives us better products and better prices

Hilarious because everyone is talking how Intel dropped most resources and all that and all they could muster is to beat a year old CPU from AMD. And AMD drops just a cache refresh some 4 months later and takes all that away. It must really suck to put so much effort into something just to not be really competitive for more than few months...

[Quackers101]

as for the other topics put here, so long they don't rush things with something that is just going to backfire and break on them.

Were some cool ideas or stuff they want to put out to get an "advantage" before it's even ready or going to have more failures.

 

but man, some of their marketing is way more fun than intel.

[pas008]

57 minutes ago, RejZoR said:

Hilarious because everyone is talking how Intel dropped most resources and all that and all they could muster is to beat a year old CPU from AMD. And AMD drops just a cache refresh some 4 months later and takes all that away. It must really suck to put so much effort into something just to not be really competitive for more than few months...

not sure how long you have been alive or even paid attention but tech companies only do enough to milk it(usually)

 

amd hasnt released anything yet to beat it for starters

 

I find it entertaining and exciting that 2 different approaches are getting things done but they are just enough to out do each other

intel hasnt really showed faveros emib chiplet yet

amd hasnt showed up big little yet

both hasnt showed us arm yet

 

not sure how you can sit there an act like either company is your friend or the other doesnt have the potential

watch the next few gens will be enough for them to just out do each other

[GDRRiley]

Fronteir supercomputer is running MI200s+ EYPC

now what flavors I can't say

I wish we got Milan-X to replace Milan because our cabents keep getting delayed

 

I doubt zen4c will come to OEMs, seems likely a cloud only platform. wonder if it will get more DDR5 lanes.