AMD introduces new data center GPU. AMD Instict MI100

[AndreiArgeanu]

Summary

 

 AMD has just released their new data center GPU's named Instinct. They claim to be the first to surpass the 10 TF FP64 barrier on a data center GPU. These GPU's also seem to have some kind of equivalent to nvidia's NVLink. The graphs they include also show the card to be faster than AMPERE at a lower power consumption (300W AMD - 400W NVIDIA).

 

Quotes

Quote

World’s Fastest HPC GPU¹

Delivering up to 11.5 TFLOPs of double precision (FP64) theoretical peak performance, the AMD Instinct™ MI100 accelerator delivers leadership performance for HPC applications and a substantial up-lift in performance over previous gen AMD accelerators. The MI100 delivers up to a 74% generational double precision performance boost for HPC applications.¹³

AMD Instinct™ MI100 accelerator is the world’s fastest HPC GPU, engineered from the ground up for the new era of computing.¹ Powered by the AMD CDNA architecture, the MI100 accelerators deliver a giant leap in compute and interconnect performance, offering a nearly 3.5x the boost for HPC (FP32 matrix) and a nearly 7x boost for AI (FP16) throughput compared to prior generation AMD accelerators.²

 

 

 

My thoughts

Slowly yet surely AMD is starting to take the enterprise market as well with their Epyc, and now Instinct accelerator cards. I'm happy to see them succeed. Of course these are enterprise cards and are less relevant to an enthusiast, but it's still great to know. I'm also a bit curious about the connection the cards use to communicate with one another, it looks quite different than anything else I've seen before it. It's also a little interesting that they basically released the server gpus not long after the Intel Server GPU announcement. Oh yeah and the video AMD made about the card is pretty cool.

 

Sources

 AMD

[LAwLz]

That "superior performance for AI & Machine Learning" graph looks very sketchy and misleading.

 

For those interested, 19.5 TFLOPs of compute is on the CUDA cores. If they do it on the Tensor cores the A100 gets 156 TFLOPS of performance, which is more than 3 times as high as the MI100.

So it seems like AMD are crippling Nvidia's card in one of their comparisons.

[X-System]

1 hour ago, LAwLz said:

-snip-

So it seems like AMD are crippling Nvidia's card in one of their comparisons.

It's called marketing as usual.

[thechinchinsong]

At this point, it doesn't matter how much FP64/32 compute AMD can muster up anymore. The market for lots of ML and AI is already used to using CUDA and the various ways Nvidia have come up with to accelerate workloads through their hardware/software solutions. It's a decent attempt if you only limit the comparison to FP64/32 but in any other workload type, Nvidia has them beat and that's exactly what the market in ML and AI are moving toward anyways.

[X-System]

I consulted the documentation of nVIDIA A100 : https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/a100/pdf/nvidia-a100-datasheet.pdf

 

156 TFLOPS, it's for FP32 standard but with the structural sparsity enabled, it runs 312 TFLOPS. It means about 7 times faster than AMD Instinct MI100 with their "FP32 Matrix".

[bcredeur97]

4 hours ago, LAwLz said:

That "superior performance for AI & Machine Learning" graph looks very sketchy and misleading.

 

For those interested, 19.5 TFLOPs of compute is on the CUDA cores. If they do it on the Tensor cores the A100 gets 156 TFLOPS of performance, which is more than 3 times as high as the MI100.

So it seems like AMD are crippling Nvidia's card in one of their comparisons.

but aren't tensor cores a special use case only? (just asking, i don't know anything about them tbh)

[leadeater]

4 hours ago, LAwLz said:

For those interested, 19.5 TFLOPs of compute is on the CUDA cores. If they do it on the Tensor cores the A100 gets 156 TFLOPS of performance, which is more than 3 times as high as the MI100.

But you don't/can't do FP32 on Tensor cores, very different thing. TensorFloat32 is not FP32. General compute may be faster but the equivalent accelerated paths are faster on Ampere.

[tim0901]

Sounds like they're a bit better if you're using FP32, but not if you're using TF32 or FP16 (which tensor cores also accelerate) which are commonly used in mixed-precision learning.

 

I think the bigger problem will be their lack of CUDA. Every major deep learning framework supports CUDA, but many have limited support for OpenCL. AMD's card could be 50% faster in FP32, but that wouldn't matter as it can't run the code that people want it to.

[justpoet]

I'm glad to see them attempting to compete in this space.  Even if the card were objectively better (as I'm sure it will be for at least some calculation types), the big elephant in the room isn't the card, but CUDA.  They need an answer to that, which is easy to widely adopt, if they want to actually compete heavily in the compute space.

[AlexGoesHigh]

12 hours ago, thechinchinsong said:

At this point, it doesn't matter how much FP64/32 compute AMD can muster up anymore. The market for lots of ML and AI is already used to using CUDA and the various ways Nvidia have come up with to accelerate workloads through their hardware/software solutions. It's a decent attempt if you only limit the comparison to FP64/32 but in any other workload type, Nvidia has them beat and that's exactly what the market in ML and AI are moving toward anyways.

 

8 hours ago, tim0901 said:

Sounds like they're a bit better if you're using FP32, but not if you're using TF32 or FP16 (which tensor cores also accelerate) which are commonly used in mixed-precision learning.

 

I think the bigger problem will be their lack of CUDA. Every major deep learning framework supports CUDA, but many have limited support for OpenCL. AMD's card could be 50% faster in FP32, but that wouldn't matter as it can't run the code that people want it to.

 

3 hours ago, justpoet said:

I'm glad to see them attempting to compete in this space.  Even if the card were objectively better (as I'm sure it will be for at least some calculation types), the big elephant in the room isn't the card, but CUDA.  They need an answer to that, which is easy to widely adopt, if they want to actually compete heavily in the compute space.

They have a CUDA competitor/equivalent, it's called ROCm and they are launching v4.0 with this GPU and one part of it is to ease the transition/port from CUDA.

 

 

More details from the phoronix article: https://www.phoronix.com/scan.php?page=article&item=amd-mi100-rocm4&num=1

 

[Doobeedoo]

I really like that faceplate looks so sleek. Though yeah the separate design for these CDNA based ones is a good move. 

[tim0901]

9 hours ago, AlexGoesHigh said:

 

 

They have a CUDA competitor/equivalent, it's called ROCm and they are launching v4.0 with this GPU and one part of it is to ease the transition/port from CUDA.

 

 

 

More details from the phoronix article: https://www.phoronix.com/scan.php?page=article&item=amd-mi100-rocm4&num=1

 

ROCm isn't a perfect solution though.

 

1. It doesn't support consumer GPUs. Only CDNA compute GPUs are supported (not RDNA or RDNA2), whilst CUDA can be used on anything. Lots of universities will buy Titans for machine learning applications, rather than spending 10x more on server-grade components. Same is true for startups, or even just individuals trying to get into machine learning. None of them will be buying a multi-thousand-dollar CDNA card - they have no choice but to go RTX just based on price.

 

2. It does still require work on the side of the developers to add support for AMD gpus, no matter how easy AMD makes that transition (and then they need to optimise their code for the new platform, which will take significantly longer than mere implementation). It does look promising and I hope it can shake things up, but previous versions have been around for a while now and they haven't done much to change things, so I can't see this doing much either. Just like Intel in the CPU space, Nvidia has a stronghold on the server compute market that isn't going to change with a single generation, their reputation there is far higher than AMD's.

[igormp]

The only market that I can see this card thriving is FP64-heavy, OpenCL-based computations/simulations in the HPC scenario, a really small niche.

23 hours ago, bcredeur97 said:

but aren't tensor cores a special use case only? (just asking, i don't know anything about them tbh)

A "special use case" that's widely used. Most ML stuff relies on FMA of low precision matrices, which is exactly what the tensor cores do.

 

11 hours ago, AlexGoesHigh said:

They have a CUDA competitor/equivalent, it's called ROCm and they are launching v4.0 with this GPU and one part of it is to ease the transition/port from CUDA.

Adding to the excellent points made by @tim0901, ROCm is also a pain to get everything working, and performance is really subpar. Even my 2060 Super can beat a Radeon VII is all under ROCm vs CUDA.

 

1 hour ago, tim0901 said:

It doesn't support consumer GPUs

It does work with pre-RDNA cards (like the Polaris cards), but it isn't worth it.

[Wheresmehammer]

Lisa Su is laying a smaketh teh downeth and you know it will be the rock bottom for Nvidia... 

 

-Yours faithfully WWE & The Rock.

[Fnige]

6 hours ago, tim0901 said:

1. It doesn't support consumer GPUs. Only CDNA compute GPUs are supported (not RDNA or RDNA2), whilst CUDA can be used on anything. Lots of universities will buy Titans for machine learning applications, rather than spending 10x more on server-grade components. Same is true for startups, or even just individuals trying to get into machine learning. None of them will be buying a multi-thousand-dollar CDNA card - they have no choice but to go RTX just based on price.

 

i wonder if AMD will make cheaper CDNA GPUs for that kind of stuff

[igormp]

6 minutes ago, SlimyPython said:

i wonder if AMD will make cheaper CDNA GPUs for that kind of stuff

CDNA is just a rebranding of GCN, which they gave up on the consumer market.

[GDRRiley]

3 minutes ago, igormp said:

CDNA is just a rebranding of GCN, which they gave up on the consumer market.

https://www.anandtech.com/show/15593/amd-unveils-cdna-gpu-architecture-a-dedicated-gpu-architecture-for-data-centers

as far as I know no it isn't

 

11 minutes ago, SlimyPython said:

i wonder if AMD will make cheaper CDNA GPUs for that kind of stuff

they were already cheaper but yeah they'll have at least 2-3 more . the last gen had mi50,60,25

[tim0901]

20 minutes ago, GDRRiley said:

https://www.anandtech.com/show/15593/amd-unveils-cdna-gpu-architecture-a-dedicated-gpu-architecture-for-data-centers

as far as I know no it isn't

 

they were already cheaper but yeah they'll have at least 2-3 more . the last gen had mi50,60,25

These cards were pretty much only available in bulk (100+ card orders) via an AMD representative, and were hardly "cheap". They didn't really have a sticker price, instead like many products like this their price depended on who was inquiring. Small, underfunded university? Maybe a few grand each. Government organisation? Try tripling that as a minimum. An RTX TITAN or a midrange Quadro was (and likely will remain) pretty competitively priced in comparison for startups and universities alike.

 

Unfortunately the Radeon Pro lineup still uses RDNA, rather than CDNA, and that doesn't look to be changing anytime soon. You have to go up two product categories to get ROCm support compared to Nvidia's offerings.

[GDRRiley]

13 minutes ago, tim0901 said:

Unfortunately the Radeon Pro lineup still uses RDNA, rather than CDNA, and that doesn't look to be changing anytime soon. You have to go up two product categories to get ROCm support compared to Nvidia's offerings.

right because the pro line is focused on workloads that need RDNAs gaming performance with pro drivers.

[tim0901]

14 minutes ago, GDRRiley said:

right because the pro line is focused on workloads that need RDNAs gaming performance with pro drivers.

All I'm saying is that AMD are massively limiting their target audience by not giving those cards ROCm support - they would be far more appealing to many people if they did.

[igormp]

2 hours ago, GDRRiley said:

as far as I know no it isn't

They clearly state it on their whitepaper:

Quote

While inspired by the prior-generation GCN architecture, each CU is rearchitected and enhanced with a Matrix Core Engine

...

The CUs are derived from the earlier GCN architecture and execute wavefronts that contain 64 work-items

...

The AMD CDNA architecture builds on GCN’s foundation of scalars and vectors and adds matrices as a first class citizen

It's based on GCN with some new feature to try and reach feature parity with nvidia's tensor cores.

 

1 hour ago, tim0901 said:

by not giving those cards ROCm support

Support is coming soon: 

 

1 hour ago, tim0901 said:

they would be far more appealing to many people if they did.

Only if all you want to do is toy with ROCm and get stuff to work, otherwise you can't really get much work done, and the performance you'd get would be way worse than an equivalent nvidia (so you'd be effectively wasting money to get the same thing done).

 

[tim0901]

10 minutes ago, igormp said:

Support is coming soon: 

Support for RDNA has been "coming soon" since its launch over a year ago. I'll believe it when I see it.