Nvidia: ‘Pascal’ architecture’s NVLink to enable 8-way multi-GPU

[ahhming]

Nvidia hopes that with the emergence of the code-named “Pascal” GPUs and NVLink bus, it will be considerably easier to build multi-GPU machines.

Today even the top-of-the-range Intel Core i7-5960X processor has only 40 PCI Express 3.0 lanes (up to 40GB/s of bandwidth), thus, can connect up to two graphics cards using PCIe 3.0 x16 or up to four cards using PCIe 3.0 x8 bus. In both cases, maximum bandwidth available for GPU-to-GPU communications will be limited to 16GB/s or 8GB/s (useful bandwidth will be around 12GB/s and 6GB/s) in the best case scenarios since GPUs need to communicate with the CPU too.
 

 

Nvidia will implement support of proprietary NVLink bus into its next-generation “Pascal” GPUs. Each NVLink point-to-point connection will support 20GB/s of bandwidth in both directions simultaneously (16GB/s effective bandwidth in both directions) and each “Pascal” high-end GPU will support at least four of such links. In case a of a system with NVLink, two GPUs would get a total peak bandwidth of 80GB/s (64GB/s effective) per direction between them. Moreover, PCI Express bandwidth would be preserved for CPU-to-GPU communications. In case of four-GPU sub-system, graphics processors would get up to 40GB/s bandwidth to communicate with each other.

 

According to Nvidia, NVLink is projected to deliver up to two times higher performance in many applications simply by replacing the PCIe interconnect for communication among peer GPUs. It should be noted that in an NVLink-enabled system, CPU-initiated transactions such as control and configuration are still directed over a PCIe connection, while any GPU-initiated transactions use NVLink, which allows to preserve the PCIe programming model.
 

Additional bandwidth provided by NVLink could allow one to build a personal computer with up to eight GPUs. However, to make it useful in applications beyond technical computing, Nvidia will have to find a way to efficiently use eight graphics cards for rendering. Since performance scaling beyond two GPUs is generally low, it is unlikely that eight-way multi-GPU technology will actually make it to the market. However, if Nvidia manages to improve efficiency of current multi-GPU technologies in general by replacing SLI [scalable link interface] with NVLink, that could further boost popularity of the company’s graphics cards among gamers.

 

TLDR:
Nvidia's Pascal architectures's NVLink has 8-way multi-gpu support but we wont be seeing it in consumer market as scaling beyond 2 GPUs is generally low . IBM plans to add NVLink to select Power microprocessors for supercomputers and the technology will be extremely useful for high-performance servers . However, we could see  improved multi-gpu scaling if Nvidia replace SLI with NVLink. 

Source:
http://www.kitguru.net/components/graphic-cards/anton-shilov/nvidia-pascal-architectures-nvlink-to-enable-8-way-multi-gpu-capability/

This post has been promoted to an article

[han_han08]

ahhhh

 

edit: -ming..lol joke..
exciting but...

[CoolaxGaming]

Welp, I think the titan x was the last maxwell. The hype of pascal nvidia is creating might suggest that

[kaylexmorgana]

Yes. I need a 8 way titan x with a .1% increase

[TechFan@ic]

Yes, don't get excited though. It's not for desktops but rather for servers.

[Unlikelyjoker]

With the scaling of the cards expect an extra 2fps AVG and -10FPS MIN per card.

[MrRavens]

Lmfao, seeing someone's build with a wall of Nvidia graphics cards.

[Spongy141]

This is so not needed...

EDIT: For gamers.

[Zah]

So with NVLink being a thing, is nvidia taking a page out of AMD's book and probably ditching the SLI bridges?

 

That's kinda sad, pimped out SLI bridges can be fun.

[KungPaoChino]

Based on CEO math. There will be some HUGE performance increase....

[twovaliant2]

Lmfao, seeing someone's build with a wall of Nvidia graphics cards.

sadly and awesome enough...you know someone will build that...

[Kingofpants]

I doubt the need for an 8 way in a single server will be high. 

1- Virtualization Servers Scale well over Multi-Node and Allow for better Performance if All of them aren't running of a single computer.
2- Tasks where You need a single node to have high performance (IE- Ineffecient Rendering or Possible Cad work) Probably don't need the GPUS to communicate to each other at 80 GBPS as much as they need a strong CPU induvidually giving tasks to each others.
3- If you think this will have a performance increase in gaming , Remember that GPUS don't work together on the same frame and there isn't much gain beyond 4 GPUS. Although the Higher bandwidth and DirectX 12 Might allow for them to work together to Draw single Frames Faster.
 

But a computer with 8 GPUs in it will probably look really cool. And maybe some people will find creative uses.

[qwertywarrior]

what are the advantages of higher bandwidth ?

i mean x4 at 3.0 doesnt suffer much

im trying to find cuda specific benchmarks but cant seem to find any

or is nvidia taking advantage of the high bandwidth and doing something with it

[JustCallMeVlad]

This is so not needed...

Ban this dude.

8 way GTX 1080's? BRING IT ON!!!!!!!!!!! Hello HOF

[TheKDub]

Ban this dude.

8 way GTX 1080's? BRING IT ON!!!!!!!!!!! Hello HOF

and if Nvidia figures it out, they may be able to make multi gpu scaling better.

[spartaman64]

so everyone would be switching to linux to use that 8 way sli

[Real_PhillBert]

I wont be running any 8 way GPU configs, but I would love to see some benchmarks of it.

 

The computational power would make your mind hurt.

[ForsakenLive]

The GRID has begun!

 

Can't wait for the day were a massive server full of GPUs can sit in your garage, while it's powering all your devices. We'd be able to manage and share hardware resources in our own private network,

[Thony]

So glad its not for consumers yet ! I really want to upgrade my GPU to 1000 series and if they used the Nvlink then my system would not be compatible.

[ahhming]

what are the advantages of higher bandwidth ?

i mean x4 at 3.0 doesnt suffer much

im trying to find cuda specific benchmarks but cant seem to find any

or is nvidia taking advantage of the high bandwidth and doing something with it

its more of GPU to GPU/CPU communication through NVLink rather than PCIE which is much faster and in theory better scaling for multi-GPU

[NumLock21]

LTT's favorite news source says Nvidia Pascal will use the 2nd gen HBM, which is developed by SK Hynix and AMD.

http://wccftech.com/amd-r9-390x-nvidia-pascal-hbm-stacked-vram-pictured/

More info on Nvidia Pascal

http://wccftech.com/nvidia-pascal-gpu-gtc-2015/

[WereCat]

Now imagine the size of the case that would support 8x 2-slot graphics card I bet you could easily throw 560-700mm radiator in front.

[azurezeed]

if only ceo math was real

[harrynowl]

This is so not needed...

*for gamers

[CommandMan7]

Yes, don't get excited though. It's not for desktops but rather for servers.

That's what they said* about the original titan, but look where the titan X is now.

 

*kind of. Workstations, not servers.