Supercomputer v2

[Goku1814]

If the most powerful work CPU is the ThreadRipper 3990X why don't supercomputers switch from Xeno CPUs to 3990X CPU? Wouldn't they be a lot more powerful than they currently are. Also a 3990X cost a lot less than the top of the line Xeon CPU. So therotically super computers could be alot more cheaper but be much more powerful.

[RollinLower]

well, you cannot have multiple 3990X's on the same motherboard, which means you need much more space for the same amount of compute cores.

 

also, there are much more powerfull CPU's than the 3990X. think AMD EPYC

[Guest]

1 minute ago, Goku1814 said:

If the most powerful work CPU is the ThreadRipper 3990X why don't supercomputers switch from Xeon CPUs to 3990X CPU? Wouldn't they be a lot more powerful than they currently are. Also a 3990X cost a lot less than the top of the line Xeon CPU. So therotically super computers could be alot more cheaper but be much more powerful.

Because most super computers do big compute on Tesla Cards. Rows and Rows of Tesla cards. Endless rows of pure GPU epeen.

 

[Guest]

Because most super computers do big compute on Tesla Cards. Rows and Rows of Tesla cards. Endless rows of pure GPU epeen.

 

 

how many sections did you post this in dude??

[GDRRiley]

2 minutes ago, ShrimpBrime said:

Because most super computers do big compute on Tesla Cards. Rows and Rows of Tesla cards. Endless rows of pure GPU epeen.

 

it maters which ones. Many of them use Xeon Phi which is a continuation of larabee

But they are also moving to more GPU compute overall

 

6 minutes ago, Goku1814 said:

If the most powerful work CPU is the ThreadRipper 3990X why don't supercomputers switch from Xeon CPUs to 3990X CPU? Wouldn't they be a lot more powerful than they currently are. Also a 3990X cost a lot less than the top of the line Xeon CPU. So therotically super computers could be alot more cheaper but be much more powerful.

that is what the new epyc chips are doing. the server version of a 3990x can be put 2 to a motherboard.

 

[3rrant]

8 minutes ago, Goku1814 said:

If the most powerful work CPU is the ThreadRipper 3990X why don't supercomputers switch from Xeon CPUs to 3990X CPU? Wouldn't they be a lot more powerful than they currently are. Also a 3990X cost a lot less than the top of the line Xeon CPU. So therotically super computers could be alot more cheaper but be much more powerful.

• Lack of multi-socket support. You can have up to 4 EPYC CPUs in a system or just one TR.
• Lack of 8 channel memory. HEDT platform usually limit it to 4.
• Lack of huge memory size support. EPYC supports 2TB of RAM, TR doesn't.
• Lack of PCI-E lanes: server parts usually have access to more lanes. (128 vs 72 in this case I think)
• Lack of features: server parts generally have some features exclusive to them. Might be an instruction set or hardware assisted virtualization or ECC memory etc. Depends on the platform.
• Validity. Server parts are generally validated for a much more prolonged use and 24/7 operation. Consumer and HEDT parts are not as they are not supposed to be mission critical.

Raw performance is only one part of the equation.

[Goku1814]

4 minutes ago, RollinLower said:

well, you cannot have multiple 3990X's on the same motherboard, which means you need much more space for the same amount of compute cores.

 

also, there are much more powerfull CPU's than the 3990X. think AMD EPYC

Sorry that was the CPU I was talking about. Deep. Wouldn't supercomputers be a lot faster if they used AMD EPYC and cheaper than the best XEON supercomputers?

[Dredgy]

SuperComputers are tailor made - cost isn’t much of a consideration.

Crays next supercomputer will use AMD EPYC and will likely be the fastest computer in history and cost about $600000000. 

https://en.wikipedia.org/wiki/Frontier_(supercomputer)

[Guest]

2 minutes ago, GDRRiley said:

it maters which ones. Many of them use Xeon Phi which is a continuation of larabee

But they are also moving to more GPU compute overall

 

that is what the new epyc chips are doing. the server version of a 3990x can be put 2 to a motherboard.

 

Intense 1.5ghz insane core count. Yes I kinda forgot about those.

 

But are they really considered........ super? 

[3rrant]

Just now, Goku1814 said:

Sorry that was the CPU I was talking about. Deep. Wouldn't supercomputers be a lot faster if they used AMD EPYC and cheaper than the best XEON supercomputers?

EPYC vs Intel is a different discussion. Supercomputers are not general-usage computers. They run software that is designed for them, which means the software is customized for it and generally it is done so by exploiting all the possible performance the supercomputer can give it. Very deep level optimization for Intel is different than AMD. If you have an Intel system now and the owner switchs to AMD, the software developer will need to modify the software to get the best out of it.

 

Also very important is the longevity and trust with the partner. If you are buying thousands of CPUs and GPUs you're much more likely to stick with Intel as you've done for 5-10-20 years than try something new, because you know it will just work. Building trust in the consumer requires time.

 

And finally.. Intel might be more costly but in server space they are not always destroyed horribly by AMD. It's not just about the number of cores.

[Goku1814]

bc i personally feel like if supercomputers started using EPYC they could be more powerful than ever before and be able to much more than possible with XEON supercomputers

[Goku1814]

but tech is always getting better and improving

1 minute ago, 3rrant said:

EPYC vs Intel is a different discussion. Supercomputers are not general-usage computers. They run software that is designed for them, which means the software is customized for it and generally it is done so by exploiting all the possible performance the supercomputer can give it. Very deep level optimization for Intel is different than AMD. If you have an Intel system now and the owner switchs to AMD, the software developer will need to modify the software to get the best out of it.

 

Also very important is the longevity and trust with the partner. If you are buying thousands of CPUs and GPUs you're much more likely to stick with Intel as you've done for 5-10-20 years than try something new, because you know it will just work. Building trust in the consumer requires time.

 

And finally.. Intel might be more costly but in server space they are not always destroyed horribly by AMD. It's not just about the number of cores.

 

[GDRRiley]

1 minute ago, Goku1814 said:

Sorry that was the CPU I was talking about. Deep. Wouldn't supercomputers be a lot faster if they used AMD EPYC and cheaper than the best XEON supercomputers?

look at 3-4 of the new supercomputers planed. archer 2 in the UK, Perlmutter and others

https://www.cray.com/

8 minutes ago, RollinLower said:

also, there are much more powerfull CPU's than the 3990X. think AMD EPYC

not more powerful but different. they are almost the same chip with some changes.

 

1 minute ago, ShrimpBrime said:

Intense 1.5ghz insane core count. Yes I kinda forgot about those.

they actually are only 55-75 cores. which is why epic is so much faster. it isn't an atom core and it runs 50% faster clocks

[Goku1814]

But can it run RTX Minecraft

2 minutes ago, GDRRiley said:

look at 3-4 of the new supercomputers planed. archer 2 in the UK, Perlmutter and others

https://www.cray.com/

not more powerful but different. they are almost the same chip with some changes.

 

they actually are only 55-75 cores. which is why epic is so much faster. it isn't an atom core and it runs 50% faster clocks

 

[GDRRiley]

2 minutes ago, 3rrant said:

Also very important is the longevity and trust with the partner. If you are buying thousands of CPUs and GPUs you're much more likely to stick with Intel as you've done for 5-10-20 years than try something new, because you know it will just work

 

And finally.. Intel might be more costly but in server space they are not always destroyed horribly by AMD. It's not just about the number of cores.

they use to all run AMD 10-15 years ago. they change to whoever has a stable platform that is fastest. they don't get on the first new chip but a 2nd or 3rd gen of that chip they will hop on

 

very funny in massive parallelized workloads they got nothing close. Their xeon phi had its last 3 updates cancled because epyc was so strong and winning so many super computer bids.

[3rrant]

1 minute ago, GDRRiley said:

they use to all run AMD 10-15 years ago. they change to whoever has a stable platform that is fastest. they don't get on the first new chip but a 2nd or 3rd gen of that chip they will hop on

 

very funny in massive parallelized workloads they got nothing close. Their xeon phi had its last 3 updates cancled because epyc was so strong and winning so many super computer bids.

 

They change to whoever is the most trustworthy at the time. 15 years ago AMD did not fuck up extremely hard. But with FX they did. Now they need to build the trust again. Is not about raw performance.

 

4 minutes ago, Goku1814 said:

but tech is always getting better and improving

 

There are supercomputers that will be using EPYC in the next few years. But that's not the point. The cutting edge of technology is not as refined as a 20 years old platform. The key to supercomputing is stability and knowing what your platform can do and how. Software optimization can get you extraordinary results, even compared to new hardware.

 

As a company that is spending hundreds of millions or BILLIONs in building a new supercomputer, you cannot just easily trust a new product that nobody tested already, like EPYC is. Think about what is at stake here, we're not talking about penny. And remember that most supercomputers are used by the military, which is not something you can FK up without consequences.

 

EPYC will come to supercomputers. They are already doing it. The more AMD can prove it's value and trust the more they will expand. Just give it time.

[GDRRiley]

1 minute ago, 3rrant said:

They change to whoever is the most trustworthy at the time. 15 years ago AMD did not fuck up extremely hard. But with FX they did. Now they need to build the trust again. Is not about raw performance.

it isn't about trustworthy.

I toured berkely labs supercomputer and asked if based on roadmaps epyc was going to be the pick no comment.

2 years before they announced it I knew epyc was going to be the next chip in most supercomputers.

Spoiler

 

[3rrant]

1 minute ago, GDRRiley said:

it isn't about trustworthy.

I toured berkely labs supercomputer and asked if based on roadmaps epyc was going to be the pick no comment.

2 years before they announced it I knew epyc was going to be the next chip in most supercomputers.

  Reveal hidden contents

 

So because you toured a single manifacturer who decided not to comment you you think you understood how they decide? Ok.

[GDRRiley]

Just now, 3rrant said:

So because you toured a single manifacturer who decided not to comment you you think you understood how they decide? Ok.

It isn't a manufacture.

I'm saying epyc was looked years ago and they just didn't publicly decide.

[3rrant]

Just now, GDRRiley said:

It isn't a manufacture.

I'm saying epyc was looked years ago and they just didn't publicly decide.

And I did never say EPYC was not looked at, I said that the decision is not just about raw performance. 

[GDRRiley]

1 minute ago, 3rrant said:

And I did never say EPYC was not looked at, I said that the decision is not just about raw performance. 

That is 100% what it is about. They have 1 board and 1 cpu config with plenty of budget to make sure everything  works right 

[3rrant]

2 minutes ago, GDRRiley said:

That is 100% what it is about. They have 1 board and 1 cpu config with plenty of budget to make sure everything  works right 

Ok. Riddle me this then: Why does Aurora next supercomputer, which is also exascale and also coming out in 2021, use Intel? 

[Jumballi]

A super computer is built and will NEVER be upgraded, they just throw them out and replace them

[GDRRiley]

9 minutes ago, 3rrant said:

Ok. Riddle me this then: Why does Aurora next supercomputer, which is also exascale and also coming out in 2021, use Intel? 

Given it isn’t expected to be anywhere near the 2 exascale the amd is I wouldn’t say it is on the same level. It could be that where it is going heavily used avx512 

 

I’ll also add it was supposed to be a 2018 project but intel canceled the CPUs it was using. 

Edited May 24, 2020 by GDRRiley

[3rrant]

Just now, GDRRiley said:

 It could be that where it is going heavily used avx512 

Oh you don't say? Then it's about raw performance or specific use case? And you're not even considering that Aurora is much more military related than your example. Does military is US use AMD or Intel?

That's why your previous comment is nonsense.  Decision making is not about flops.

It's about the whole system, the usage, the software used and the trust of the partner for mission critical development.