NVidia's Breakthrough AI Chip Defies Physics

[igormp]

6 hours ago, Quackers101 said:

to a point, right? the use of mixed precision for some workloads.

https://developer.nvidia.com/automatic-mixed-precision

https://docs.nvidia.com/deeplearning/performance/mixed-precision-training/index.html

We actually have models using parameters with less than 2 bits (look into Microsoft's 1.5-something bit LLM), so FP4 is going to be really useful.

If you take an existing model, then you can quantize it to a smaller size with a minor perf loss, or you can train a model from scratch using such small parameters.

 

The docs you linked are old and are meant for automatic use of the tensor cores when they first came out. We're on cuda 12 nowadays and most ml frameworks allow you to do much more than that