This is what an exascale APU might look like

[kiska3]

Source: https://www.overclock3d.net/news/cpu_mainboard/amd_reveals_a_exascale_mega_apu_in_a_new_academic_paper/1

I am going to try to find this academic paper

 

Quote

One of the largest issues comes when manufacturing large CPU/GPU dies, with yields decreasing and costs rising as you create larger products. Imagine a silicon wafer and imagine that a single wafer has a certain number of defects, each wafer creates a certain number of chips, which means that only a small number of chips will be affected in the whole batch. When creating products with large die sized the number of chips per silicon wafer decreases, which means that defects can destroy a larger proportion of the products in a single silicon wafer. 

According to this paper, AMD wants to get around this "large die issue" by making their Exascale APUs using a large number of smaller dies, which are connected via a silicon interposer. This is similar to how AMD GPUs connect to HBM memory and can, in theory, be used to connect two or more GPU, or in this case CPU and GPU dies, to create what is effectively a larger final chip using several smaller parts. 

 

In the image below you can see that this APU uses eight different CPU dies/chiplets and eight different GPU dies/chiplets to create an exascale APU that can effectively act like a single unit. If these CPU chiplets use AMD's Ryzen CPU architecture they will have a minimum of 4 CPU cores, giving this hypothetical APU a total of 32 CPU cores and 64 threads. 

This new APU type will also use onboard memory, using a next-generation memory type that can be stacked directly onto a GPU die, rather than be stacked beside a GPU like HBM. Combine this with an external bank of memory (perhaps DDR4) and AMD's new GPU memory architecture and you will have a single APU that can work with a seemingly endless amount of memory and easily compute using both CPU and GPU resources using HSA (Heterogeneous System Architecture).     

Right now this new "Mega APU" is currently in early design stages, with no planned release date. It is clear that this design uses a new GPU design that is beyond Vega, using a next-generation memory standard which offers advantages over both GDDR and HBM. 

So we have a new research paper from AMD, hrm, I could see researchers using this chip

[The Benjamins]

Wait so is that 8 GPU dies and 4 CPU dies in one package. That is one hot package.

[kiska3]

5 minutes ago, The Benjamins said:

Wait so is that 8 GPU dies and 4 CPU dies in one package. That is one hot package.

Its 4 CPU + 4 GPU in one package, well according to the article.

I still haven't found this academic paper yet

 

EDIT: Whoops

[Streetguru]

10 minutes ago, kiska3 said:

 

Is Adored ever wrong?
 

 

4 minutes ago, The Benjamins said:

Wait so is that 8 GPU dies and 4 CPU dies in one package. That is one hot package.

Something that big is probably going into a server

[The Benjamins]

2 minutes ago, kiska3 said:

Its 4 CPU + 4 GPU in one package, well according to the article.

I still haven't found this academic paper yet

 

the diagram looks like 8 separate chips, but still even 4 is a lot of GPU

 

2 minutes ago, Streetguru said:

Is Adored ever wrong?
 

video

Something that big is probably going into a server

Naaa I want to put it in a mITX SFF system. :^)

[Mira Yurizaki]

While I like this idea, I'm concerned about heat, since heat transfer is a function of surface area.

[RagnarokDel]

32 minutes ago, The Benjamins said:

Wait so is that 8 GPU dies and 4 CPU dies in one package. That is one hot package.

It's not any hotter than a normal GPU. 8x 1/8 power consumption =1.001 the power consumption(there's always a small loss) but at let's say 1/3 the cost of production.

[Stefan1024]

We had high performance multi chip CPUs back in the days, surely one has go get the idea of combining CPU, GPU and cache / RAM.

 

I'm pretty sure all big chip manufactureres have plans like this somewhere in the basement. And it's a sign that we don't have one by now.

 

However with the recent development of good interposer it's more doable.

[Mira Yurizaki]

9 minutes ago, Stefan1024 said:

We had high performance multi chip CPUs back in the days, surely one has go get the idea of combining CPU, GPU and cache / RAM.

Someone did. It takes form of various names like the Apple A series, Qualcomm Snapdragon, Samsung Exynos, NVIDIA Tegra...

[captain cactus]

This is basically AMD's Navi arch but for APUs as a whole.

 

Scalability to bring the cost down by a lot. I like that.

[Okjoek]

http://www.computermachines.org/joe/publications/pdfs/hpca2017_exascale_apu.pdf

 

 

hpca2017_exascale_apu.pdf

 

@kiska3

[Coaxialgamer]

1 hour ago, The Benjamins said:

Wait so is that 8 GPU dies and 4 CPU dies in one package. That is one hot package.

Well we can assume that :

-this is going to be for servers

-the package will be freaking huge . Think lga 3647 or bigger

 

Plus , considering the heat from the gpu dies will have to go through the hbm stacks , i'm fairly sure they'll be lower powered , lower clock speed chips . Otherwise, the stacks can't dissipate the heat ( which is actually one of the big hurdles to overcome for 3d and 2.5D chips to become viable in processors )

[The Benjamins]

13 minutes ago, Okjoek said:

http://www.computermachines.org/joe/publications/pdfs/hpca2017_exascale_apu.pdf

 

 

hpca2017_exascale_apu.pdf

thanks that is a cool paper, That is one crazy APU.

[DocSwag]

I guessed a while back that AMD would eventually go to manufacturing different parts of processing units in different dies and using interposers to connect them; it looks like I was right!

 

After all, yield and die size is not linear, so it 6 100 mm^2 dies are more likely to be good than 1 600 mm^2 die.

 

I wouldn't be surprised if in the next ten years we may see them start to manufacture parts of the processor such as memory controller, cache, and cores separately. This is really cool! I'm also curious about the memory; could this be the same "next-gen memory" AMD was talking about with Navi?

[leadeater]

2 hours ago, The Benjamins said:

Wait so is that 8 GPU dies and 4 CPU dies in one package. That is one hot package.

Looks like 8 GPU + 8 CPU + 6 interposers to me.

 

2 hours ago, kiska3 said:

In the image below you can see that this APU uses eight different CPU dies/chiplets and eight different GPU dies/chiplets to create an exascale APU that can effectively act like a single unit.

 

16 minutes ago, DocSwag said:

I guessed a while back that AMD would eventually go to manufacturing different parts of processing units in different dies and using interposers to connect them; it looks like I was right!

Yea I was expecting the same thing, first with their GPUs but I didn't expect AMD to jump straight to this.

[DocSwag]

1 minute ago, leadeater said:

Looks like 8 GPU + 8 CPU + 6 interposers to me.

 

 

Yea I was expecting the same thing, first with their GPUs but I didn't expect AMD to jump straight to this.

Yeah I woulda thought they would start with breaking up the parts of the GPU, perhaps by breaking up the IMC, CUs, ROPs, TMUs, etc. I did not think they would start with APUs or CPUs. Still though, I'm not complaining, since I personally find APUs more interesting than CPUs or GPUs, probably since they contain both CPUs and GPUs  

[leadeater]

3 minutes ago, DocSwag said:

Yeah I woulda thought they would start with breaking up the parts of the GPU, perhaps by breaking up the IMC, CUs, ROPs, TMUs, etc. I did not think they would start with APUs or CPUs. Still though, I'm not complaining, since I personally find APUs more interesting than CPUs or GPUs, probably since they contain both CPUs and GPUs  

I was thinking they would do it first with GPUs considering the number of times AMD/ATI have tried to dual GPU cards. If they could design it to act as a single GPU it would work out much better.

 

Also Naples kinda made me think they were sticking to external GPUs for a while with the massive PCIe lanes and reference server system supporting 6 GPUs along with a nice amount of NVMe SSDs.

 

A 4 or 8 socket server with the above APU would start to get very interesting, even if each single chip isn't that powerful alone it would easily start to add up.

[DocSwag]

1 minute ago, leadeater said:

I was thinking they would do it first with GPUs considering the number of times AMD/ATI have tried to dual GPU cards. If they could design it to act as a single GPU it would work out much better.

 

Also Naples kinda made me think they were sticking to external GPUs for a while with the massive PCIe lanes and reference server system supporting 6 GPUs along with a nice amount of NVMe SSDs.

 

A 4 or 8 socket server with the above APU would start to get very interesting, even if each single chip isn't that powerful alone it would easily start to add up.

This APU probably won't be released for at least another year or two. I'd garner a guess and say that the GPU used within this APU is based on Navi. They're using a different memory architecture that can be stacked directly on the CPU and GPU dies without an interposer and AMD stated in their last Capsaicin event that Navi will use "Next-gen memory." Perhaps we'll see this monster in 2019.

[The Benjamins]

2 minutes ago, leadeater said:

I was thinking they would do it first with GPUs considering the number of times AMD/ATI have tried to dual GPU cards. If they could design it to act as a single GPU it would work out much better.

 

Also Naples kinda made me think they were sticking to external GPUs for a while with the massive PCIe lanes and reference server system supporting 6 GPUs along with a nice amount of NVMe SSDs.

 

A 4 or 8 socket server with the above APU would start to get very interesting, even if each single chip isn't that powerful alone it would easily start to add up.

The dock targets them to be 16 Tflops per package with a goal of 10 Tflops, which gives room for thermal/power adjustments. And I could see this being in those 4 blade 1U chassis for VM work.

[Jahramika]

2 hours ago, Stefan1024 said:

We had high performance multi chip CPUs back in the days, surely one has go get the idea of combining CPU, GPU and cache / RAM.

 

I'm pretty sure all big chip manufactureres have plans like this somewhere in the basement. And it's a sign that we don't have one by now.

 

However with the recent development of good interposer it's more doable.

Not one like this matter fact 

 

2 minutes ago, DocSwag said:

This APU probably won't be released for at least another year or two. I'd garner a guess and say that the GPU used within this APU is based on Navi. They're using a different memory architecture that can be stacked directly on the CPU and GPU dies without an interposer and AMD stated in their last Capsaicin event that Navi will use "Next-gen memory." Perhaps we'll see this monster in 2019.

More than likely their NExt Gen Memory a modified HBM system tied together to the CPU/GPU memory controllers

[Tomsen]

33 minutes ago, DocSwag said:

I wouldn't be surprised if in the next ten years we may see them start to manufacture parts of the processor such as memory controller, cache, and cores separately. This is really cool! I'm also curious about the memory; could this be the same "next-gen memory" AMD was talking about with Navi?

I doubt that tho. Interposers aren't all benefits, they introduce additional latency. For low level cache (l1, l2 and L3) you would want to have it on the same die.

[leadeater]

1 minute ago, The Benjamins said:

The dock targets them to be 16 Tflops per package with a goal of 10 Tflops, which gives room for thermal/power adjustments. And I could see this being in those 4 blade 1U chassis for VM work.

Most VM servers don't use GPUs, usually only the case when virtualizing desktops (VDI).

 

But yea easily see this going in to 2U hybrid blades and even the bigger blade systems, so long as they can get the heat away. On of the big reasons hybrid blades came about was needing GPUs and fast local storage, this APU solves one and NVMe M.2 solves the other . Really hasn't been much innovation in the blade space for a while.

[captain cactus]

18 minutes ago, DocSwag said:

Yeah I woulda thought they would start with breaking up the parts of the GPU, perhaps by breaking up the IMC, CUs, ROPs, TMUs, etc. I did not think they would start with APUs or CPUs. Still though, I'm not complaining, since I personally find APUs more interesting than CPUs or GPUs, probably since they contain both CPUs and GPUs  

Oh fuck I didn't even think of making the IMC, ROPs and TMUs as separate wafers as well. Just the CUs, sure, make them in packets of like 8 CUs (512 SPs) but the memory controller and ROPs and TMUs as well? Sure, why not (other than the fact that an interposer does add some more latency but that's not something we can't solve right)?

[DocSwag]

7 minutes ago, Tomsen said:

I doubt that tho. Interposers aren't all benefits, they introduce additional latency. For low level cache (l1, l2 and L3) you would want to have it on the same die.

True, though potentially in the future if the latency could be reduced, perhaps by bringing the dies closer together somehow or something, cache could be separated.

7 minutes ago, lots of unexplainable lag said:

Oh fuck I didn't even think of making the IMC, ROPs and TMUs as separate wafers as well. Just the CUs, sure, make them in packets of like 8 CUs (512 SPs) but the memory controller and ROPs and TMUs as well? Sure, why not (other than the fact that an interposer does add some more latency but that's not something we can't solve right)?

Yeah, something like that probably wouldn't be feasible for a long time. I wouldn't expect seeing something like that for many years, but this seems like a good first step towards achieving that.

[Misanthrope]

3 hours ago, M.Yurizaki said:

While I like this idea, I'm concerned about heat, since heat transfer is a function of surface area.

Also, using an interposer probably means introducing latency too.