Nvidia CEO - Moore's Law is dead; GPUs to replace CPUs

[kagarium]

Nvidia CEO Jensen Huang said at the GTC confrence in Beijing that GPUs will replace CPUs.

http://www.pcgamer.com/nvidia-ceo-says-moores-law-is-dead-and-gpus-will-replace-cpus/

Quote

Huang says Moore's Law is dead is because it can't keep pace with advancements in GPU design. Huang talked about GPUs growing in computational capacity over the years, and how they're the better suited for advancements in artificial intelligence.

Intel denies that Moore's Law is dying, and continues to back it up.

Quote

"In my 34 years in the semiconductor industry, I have witnessed the advertised death of Moore’s Law no less than four times. As we progress from 14 nanometer technology to 10 nanometer and plan for 7 nanometer and 5 nanometer and even beyond, our plans are proof that Moore’s Law is alive and well," Krzanich stated in a blog post outlining Intel's plans. "Intel’s industry leadership of Moore’s Law remains intact, and you will see continued investment in capacity and R&D to ensure so."

It's an interesting thing to consider, GPUs replacing CPUs. We see it happening a lot already in modern gaming consoles.

 

What's everyone else think? Are CPUs failing?

 

-Ken

[Flavio hc 16]

2 minutes ago, kagarium said:

 

 

What's everyone else think? Are CPUs failing?

 

 

up until no one gives a shit about task parallelization, yes. But in reality, you can't stack up as many gpu cores as cpu cores without needing a nuclear power plant, so we are probably hitting the limits cpu wise.

[Windows7ge]

Even if the technology comes around to consumers I'm not sure what people would do with a couple thousand cores processing their daily tasks. Even then there's the issue that all of the software would have to be redesigned in such a way that it can leverage parallel workloads over faster lesser cores. PC video games as a whole are barely breaking the 4 core limit. What are they going to do with thousands?

[Drak3]

Modern GPUs cannot effectively replace 10 year old CPUs for sequential tasks where IPC + clock > core count.

Otherwise we'd all be running 16 core Atom chips instead of quad and dual cores.

[Spenser1337]

Tesla CEO says engines are dying

 

almost as if they believe in the product they produce 

[samcool55]

Did we come full circle? This sounds very similar to what ATI yelled 10 years ago. CPU and GPU would work together at such a close level you wouldn't notice what runs on what.

 

Nvidia, run Linux on a nvidia GPU ONLY first, then we'll talk again.

[silberdrachi]

Yes, GPUs are taking over several sections of computation that they are suited for more than CPUs.

 

No CPUs arent dying because there are many types of software that lots of people use that either cant be parallelized or would have little impact if it was. Which means that the large number of compute cores on GPUs would be sitting idle most times.

 

The answer is that both are required, and for some workloads GPUs are now more of a factor than they were before.

 

 

(I wonder if anything actually has died when some stupid article came out saying "XXXXXXXX IS DEAD")

[NowThatsDamp]

GPU's cannot replace CPU's as they stand today. Just about every application is coded to run efficiently in series and on X86, and well as Windows, Mac and Linux. 

Now, there is the argument to be made that the computational capacity of GPU's is many times that of even the most powerful CPU's, and growing at a far faster pace. 

In order for there to be a world in which GPU's push out CPU's, it wouldn't just require an entirely new OS, programming language and hardware platform... It would require a complete transformation of computer code theory from machine language all the way up. The very way we think about programming would have to be erased and rewritten.

[NowThatsDamp]

6 minutes ago, Spenser1337 said:

Tesla CEO says engines are dying

 

almost as if they believe in the product they produce 

?

[Enderman]

Then it would literally not be called a GPU anymore, it would be a CPU.

The names actually mean something, they aren't just random letters put together...

[Spenser1337]

11 minutes ago, NowThatsDamp said:

?

What is confusing you?

[JoostinOnline]

Nobody can decide if Moore's Law is dead.  It is.  End of story.  I explained it on facebook but I don't want to type it out again.

The conversation goes:

Me

Someone else

Me

Someone Else

Me responding to a name I forgot to block.

[Trixanity]

8 minutes ago, NowThatsDamp said:

?

The CEO of a company is supposed to push their own product, so claiming that your product will supplant another and effectively take over the market isn't surprising. A CEO like Elon Musk is supposed to come out and say electric cars are the future and that they have the product to achieve it. Nvidia is now claiming they can replace CPUs entirely; a bold statement that seems unlikely to happen.

[Spenser1337]

6 minutes ago, Trixanity said:

The CEO of a company is supposed to push their own product, so claiming that your product will supplant another and effectively take over the market isn't surprising. A CEO like Elon Musk is supposed to come out and say electric cars are the future and that they have the product to achieve it. Nvidia is now claiming they can replace CPUs entirely; a bold statement that seems unlikely to happen.

Thank you 

[ScratchCat]

I doubt this greatly. In applications which simply require number crunching such as mining or machine learning GPUs have outperformed CPUs by orders of magnitude. However most applications a user uses are single threaded, not only is it harder to program concurrently but some algorithms cannot be parallelized effectively.

 

There was some research on running the A* pathfinding algorithm on GPUs with the result that it either is more memory hungry than chrome or slower than on a CPU.

http://publications.lib.chalmers.se/records/fulltext/129175.pdf

 

Only in a data center will you commonly find algorithms which will scale to thousands of cores , and those mainly do this as they are running the same algorithm thousands of times separately. All algorithms will have to obey Amdal's Law of diminishing returns with parallel workloads.

[Notional]

I agree. I think x86 is moving fast towards its end. It's simply not a very efficient architecture anymore. Even Intel is starting to remove obsolete parts of the x standard (could be x87 or just legacy stuff, I don't remember). It's simply more efficient to just emulate the code.

 

Problem is really that the only way forward now, are more cores and faster clock speeds. Former is only useful with a lot of multitasking and with software being heavily multithreaded, which is difficult. The latter is very limited by current chip production methods. After all, there's a reason GPU's are taking over a lot of compute work from CPU's these days.

[Trixanity]

4 minutes ago, Notional said:

I agree. I think x86 is moving fast towards its end. It's simply not a very efficient architecture anymore. Even Intel is starting to remove obsolete parts of the x standard (could be x87 or just legacy stuff, I don't remember). It's simply more efficient to just emulate the code.

 

Problem is really that the only way forward now, are more cores and faster clock speeds. Former is only useful with a lot of multitasking and with software being heavily multithreaded, which is difficult. The latter is very limited by current chip production methods. After all, there's a reason GPU's are taking over a lot of compute work from CPU's these days.

I agree that we probably need a radical re-design of modern CPUs and that we need to dump a lot, if not all, of the legacy baggage. Only that way will we achieve better performance (excluding changing materials and just brute-forcing the problem through massive clock speed gains) but it would take an even bigger effort to dump CPUs entirely. Everything would have to be changed top-to-bottom to make GPUs the new main processor. If we re-designed our CPUs to be leaner and faster, we'd only need developers to get their heads out of their asses and utilize the power presented to them and to update their applications to support modern instructions and techniques.

 

It seems to be me that Intel (despite what critics would say) are touching the ceiling of what can be achieved without doing something differently. They may not have hit the ceiling yet but they will soon enough. I do believe (to re-iterate) that processors need to be built differently than they do now to avoid that ceiling. You can only tweak and improve a design so much. 

[WMGroomAK]

What this comes down to is how much can a task be divided up for parallel processing... GPUs will not replace CPUs as some tasks do not scale to splitting up the workload, while other tasks do perform better when you can divide it up.  As well, there may come a time when you have a system that offloads a majority of the tasks to something peripheral, you will still have something that acts as the 'Central' unit that coordinates all of these tasks and acts as a conductor of sorts to determine which GPU gets which tasks in which order.

[Tech_Dreamer]

Intel would soon be like what the nokia company is to mobile industry now.

[Dylanc1500]

13 minutes ago, WMGroomAK said:

What this comes down to is how much can a task be divided up for parallel processing... GPUs will not replace CPUs as some tasks do not scale to splitting up the workload, while other tasks do perform better when you can divide it up.  As well, there may come a time when you have a system that offloads a majority of the tasks to something peripheral, you will still have something that acts as the 'Central' unit that coordinates all of these tasks and acts as a conductor of sorts to determine which GPU gets which tasks in which order.

To elaborate further on what you said. You can't force a serial logical computation to be parallelized. The only way to complete the workload quicker is to process the computation faster and more efficiently, this is why Intel has been pushing more efficient instructions.

 

 Until they can find something other than binary computation, it will not happen outside of very specific cases. There is a reason we moved from parallel I/O to serial. There is a balance to be made.

[Bcat00]

some of these comments completely ignore the fact he's referring to AI.

 

some people can't read

[leadeater]

GPU manufacturer says GPUs will replace CPUs, yep not biased in that at all .

[Juanitology]

GPUs work exceptionally well with massively-parallelized tasks, but I doubt they'll be completely replacing CPUs anytime soon. They work best in conjunction with CPUs.

 

Then again why is anybody surprised that a GPU company CEO says GPUs will kill CPUs?

[GoodBytes]

Context people context. As people points out, it is the context of mass processing that can be paralyzed. GPUs SUCKS at doing logic. An 'if condition' (if this, do that, else do this) is taxing for a GPU. Each SMs or CUDA cores, doesn't have its own set of logic circuit. That is currently impossible to add, else the GPU would be easily as big as an entire wafer of chips. Imagine the mass cooling needed, let alone production cost.

[MadyTehWolfie]

4 hours ago, Trixanity said:

The CEO of a company is supposed to push their own product, so claiming that your product will supplant another and effectively take over the market isn't surprising. A CEO like Elon Musk is supposed to come out and say electric cars are the future and that they have the product to achieve it. Nvidia is now claiming they can replace CPUs entirely; a bold statement that seems unlikely to happen.

It wouldn't on the sole fact it relys on being subsidized by tax payers. The day Elon can make his business model work without the government paying close to half to 80% of his products from him the. I'll believe gas engines are dying.