Nvidia's newly announced GPUs are marketed with "AI TOPS". If that's the same quantity, the Digits GPU should be comparable to a desktop 5070 or a laptop 5070 Ti. Those come with 6144 and 5888 CUDA cores, which should be equivalent to 48 and 46 Apple's GPU cores.
It is and it isn't. That's why they specifically said sparse FP4, not TOPS. Now Nvidia has used FP4 to market gains in TOPS but aren't in the case of the consumer level hardware as far as I can tell. And this isn't consumer level hardware. To give some context, we can see the data sheets here:
NVIDIA's Blackwell GPU architecture revolutionizes AI with unparalleled performance, scalability and efficiency. Anchored by the Grace Blackwell GB200 superchip and GB200 NVL72, it boasts 30X more performance and 25X more energy efficiency over its predecessor.
resources.nvidia.com
So the B100 has 14 petaflops of sparse FP4 compute and the B200 has 18. They have 60-80 TFLOPs of FP32 compute respectively. GB200 is 2 GPUs and has 40 PFLOPs of FP4 and 180 TFLOPs of FP32. I think the GB10 GPU in DIGITS is the same family of GPUs, but even if it's Blackwell 2 (like the upcoming consumer GPUs), I don't think that changes much.
Based on the above, we would expect the GB10 to have 1/14 the compute capacity of the B100, 1/18 of the B200, and 1/40th of the B200. For all of them, that gives it an FP32 TFLOP of roughly 4.5. That's actually even base M4 GPU level. But it's possible that the relationship between FP4 and FP32 isn't the same in the B10 (i.e. some matrix units are disabled so more FP32 per sparse FP4) or I've done something wrong.
But even if I have, you can even see in the pictures, this is a very small device. It isn't going to be fitting a Max-level GPU. That's probably going to be an M4 Pro-level GPU at best. It is not meant for large compute projects, it IS meant as development and inference tool with huge VRAM pools. Because again for inference, bandwidth and memory capacity matter a lot more than compute.
Lotsa people who seem to think compute is just one undifferentiated pool of sameness, like buying a heater...
Digits is NOT a consumer product. It doesn't ship with a consumer OS. It will be valuable to people who want that sort of workstation, but it will be competing against Linux boxes that used to hold a Xeon, not against Macs.
It shows the inexorable advance of ARM, and it's bad news for x86. But it's kinda irrelevant for Apple.
I'm not even really sure it competes against that to be honest - if I'm doing my math right, DIGITS is pretty small. Although I guess back in the day the smallest Xeons weren't very big either. So that point still holds I suppose. But you're absolutely right about the focus of this device versus the Studio. The Studio is a general purpose consumer/"prosumer" product and DIGITS is definitely not based on hardware and software. DIGITS is in some ways a souped-up
Jetson AGX Orin DevKit (with a potentially smaller, but more advanced GPU, much bigger and better CPU, double the memory pool, actual storage, etc ...).
That said, I would still say that it isn't irrelevant to Apple, because Apple actually provided a threat here. Consumer/prosumer hardware like Apple ships is always a threat to professional hardware because it tends to be cheaper, more familiar, and more flexible. If it can (I making the numbers up) do 70% of the same job for 40% of the cost (and not be a unitasker, capable doing other things) ... suddenly people are buying that commodity hardware instead of the professional one and the cheaper consumer hardware begins to replace the more expensive professional hardware. Now Apple would probably balk at being called commodity hardware, but there's no question in my mind that DIGITS is a direct response to all these ML people suddenly going "hey you know what's kind of shocking? These Maxes and Pros from Apple are actually really good and cost effective for inference! And it can just be an everyday computer as well!"
Even the M4 competitor Strix Halo* was making that point in their marketing where their chip that goes up to 128GB of RAM could also outpace Nvidia consumer hardware and challenge professional hardware in inference when the model size outstripped the Nvidia's memory capacity.
This device to me is thus a reflection that Nvidia needed a relatively cheap inference box (that was bigger/more capable than the Jetsons) to compete against consumer hardware that was in danger of pulling people out of the Nvidia/CUDA ecosystem. Apple had a nice system for that and had carved itself out a niche market for LLM inference so I suppose one could say "Nvidia threatens Apple's LLM market niche and thus its ability to threaten Nvidia in LLM-space" but I'd prefer to think of it as staving off Apple and consumer hardware in general.
=============
*AMDs marketing is definitely comparing with the M4 Pro and at both CPU and GPU level I'm thinking that's probably indeed where the performance will be at for the Strix Halo both based on leaked and advertised benchmarks for the CPU and node/core core count for the GPU relative to the integrated 890M and the 890M's performance profile. The Apple machine will have several advantages: better CPU cores with better efficiency and ST performance and GPU cores on a better node with better efficiency and better RT (Strix Halo is RDNA 3.5 not 4), but obviously many games will be optimized for Windows PCs and so will be better there. And the M4 Pro caps out at 64GB of RAM while, like DIGITS, the AMD chip goes to 128GB. Bandwidth is the same as the M4 Pro or similar enough depending on what lpDDR5x memory modules they go with (both have a 256bit bus). Obviously pricing is not known yet and it's be fascinating to see how AMD/OEMs prices their RAM upgrades above 32GB of RAM. This will be the first time we'll see any consumer unified SOC hardware do that in direct comparison to Apple (it won't be on package, but still).
Yeah this framing makes more sense. I hope Apple responds with better RAM/storage pricing though. It’s been ridiculous for too long
newsroom.arm.com
