Nvidia expands datacenter GPU and its variants + New Ada workstation card

[williamcll]

 

Nvidia announced their new datacenter card variants, the H100NVL built on the hopper Architecture. Other than the typical PCIe and SXM format, the H100 will also feature an export law restricted model H800 with reduced memory. These cards can also be pared with a dual Grace GPU for maximum performance.

Quotes

Quote

NVIDIA and key partners today announced the availability of new products and services featuring the NVIDIA H100 Tensor Core GPU — the world’s most powerful GPU for AI — to address rapidly growing demand for generative AI training and inference. Oracle Cloud Infrastructure (OCI) announced the limited availability of new OCI Compute bare-metal GPU instances featuring H100 GPUs.

 

Additionally, Amazon Web Services announced its forthcoming EC2 UltraClusters of Amazon EC2 P5 instances, which can scale in size up to 20,000 interconnected H100 GPUs. This follows Microsoft Azure’s private preview announcement last week for its H100 virtual machine, ND H100 v5. Additionally, Meta has now deployed its H100-powered Grand Teton AI supercomputer internally for its AI production and research teams. NVIDIA founder and CEO Jensen Huang announced during his GTC keynote today that NVIDIA DGX™ H100 AI supercomputers are in full production and will be coming soon to enterprises worldwide.

 

The H100, based on the NVIDIA Hopper™ GPU computing architecture with its built-in Transformer Engine, is optimized for developing, training and deploying generative AI, large language models (LLMs) and recommender systems. This technology makes use of the H100’s FP8 precision and offers 9x faster AI training and up to 30x faster AI inference on LLMs versus the prior-generation A100. The H100 began shipping in the fall in individual and select board units from global manufacturers.

Quote

The H100 NVL is an interesting variant on NVIDIA’s H100 PCIe card that, in a sign of the times and NVIDIA’s extensive success in the AI field, is aimed at a singular market: large language model (LLM) deployment. There are a few things that make this card atypical from NVIDIA’s usual server fare – not the least of which is that it’s 2 H100 PCIe boards that come already bridged together – but the big takeaway is the big memory capacity. The combined dual-GPU card offers 188GB of HBM3 memory – 94GB per card – offering more memory per GPU than any other NVIDIA part to date, even within the H100 family.

Quote

Nvidia claims that the H100 delivers up to 9X faster AI training performance and up to 30X speedier inference performance than the previous A100 (Ampere). With a performance of that level, it's easy to comprehend why everyone wants to get their hands on an H100. In addition, Reuters(opens in new tab) reported that Nvidia had modified the H100 to comply with export rules so that the chipmaker could sell the altered H100 as the H800 to China.

 

Reuters contacted an Nvidia spokesperson to inquire about what differentiates the H800 from the H100. However, the Nvidia representative only stated that "our 800-series products are fully compliant with export control regulations." Nvidia already has three of the most prominent Chinese technology companies using the H800: Alibaba Group Holding, Baidu Inc, and Tencent Holdings. While an H800 with half the chip-to-chip transfer rate will undoubtedly be slower than the full-fat H100, it will still not be slow. With companies potentially using thousands of Hopper GPUs, ultimately, we have to wonder if this will mean using more H800s to accomplish the same work as fewer H100s.

Additionally, a new high-end Ada workstation card is announced: the 6000 Ada SFF

Quote

This dual-slot design is a small “half height” card with a single fan. It does not require any external power connectors because the TDP has been set to only 70W. The memory specs are 20GB GDDR6 across a 160-bit interface with a memory clock at 16 Gbps. The specs indicate this is an AD104 based card. According to NVIDIA, this card will launch at $1250.

My thoughts

I think this is the part where Nvidia has finally stopped caring about consumers when you can rake in way more from the enterprise market who in turn run daylight robberies on the typical user. The 4000 Ada SFF seems decent though.

 

Sources

https://nvidianews.nvidia.com/news/nvidia-hopper-gpus-expand-reach-as-demand-for-ai-grows

https://www.anandtech.com/show/18780/nvidia-announces-h100-nvl-max-memory-server-card-for-large-language-models
https://www.tomshardware.com/news/nvidia-gimps-h100-hopper-gpu-to-sell-as-h800-to-china

https://videocardz.com/newz/nvidia-announces-workstation-rtx-4000-sff-ada-desktop-gpu-and-five-mobile-skus
 

[starsmine]

Holy hell I thought I was being fucked with.
Hopper/h100 was announced MARCH 22nd 2022. And was being shipped before the end of last year.
https://www.anandtech.com/show/17327/nvidia-hopper-gpu-architecture-and-h100-accelerator-announced
https://developer.nvidia.com/blog/nvidia-hopper-architecture-in-depth/

Ill come back to this thread later see what was actually announced today. some variant of h100 with more memory from initial glance, but need to get into the weeds.

 

Man, OPs phrasing and timing had me questioning my existence

[williamcll]

9 minutes ago, starsmine said:

Holy hell I thought I was being fucked with.
Hopper/h100 was announced MARCH 22nd 2022. And was being shipped before the end of last year.
https://www.anandtech.com/show/17327/nvidia-hopper-gpu-architecture-and-h100-accelerator-announced
https://developer.nvidia.com/blog/nvidia-hopper-architecture-in-depth/

Ill come back to this thread later see what was actually announced today. some variant of h100 with more memory from initial glance, but need to get into the weeds.

 

God, OPs phrasing and timing had me questioning my existence

edited.

[da na]

I mean, the new cards look nice but I see absolutely no reason to upgrade from my A4500 I just bought...

[leadeater]

There is a spelling mistake in the title, the correction is: Nvidia releases new money printer variants

[Agall]

1 hour ago, williamcll said:

 

Nvidia announced their new datacenter card variants, the H100NVL built on the hopper Architecture. Other than the typical PCIe and SXM format, the H100 will also feature an export law restricted model H800 with reduced memory. These cards can also be pared with a dual Grace GPU for maximum performance.

Quotes

Additionally, a new high-end Ada workstation card is announced: the 6000 Ada SFF

My thoughts

I think this is the part where Nvidia has finally stopped caring about consumers when you can rake in way more from the enterprise market who in turn run daylight robberies on the typical user. The 4000 Ada SFF seems decent though.

 

Sources

https://nvidianews.nvidia.com/news/nvidia-hopper-gpus-expand-reach-as-demand-for-ai-grows

https://www.anandtech.com/show/18780/nvidia-announces-h100-nvl-max-memory-server-card-for-large-language-models
https://www.tomshardware.com/news/nvidia-gimps-h100-hopper-gpu-to-sell-as-h800-to-china

https://videocardz.com/newz/nvidia-announces-workstation-rtx-4000-sff-ada-desktop-gpu-and-five-mobile-skus
 

They sure do like to quadruple up the memory bus for these workstation cards, yet we get screwed on singled up memory bus configurations on 'flagship' cards like the RTX 3080/ti 10/12GB.

[SorryBella]

3 hours ago, williamcll said:

The 4000 Ada SFF seems decent though.

At 1250$. Nvidia decided to commit bank robbery on any sub 6 figure content production firm. Arseholes.

[Arika]

8 hours ago, da na said:

I mean, the new cards look nice but I see absolutely no reason to upgrade from my A4500 I just bought...

All I see is the potential of existing A-series card prices to drop....which means I can get another a4000 or maybe a5000