The NPU fulfills a different purpose. In fact, Apple Silicon (since M1) contains at least three different ML accelerators, all optimized for different use cases:
- NPU for energy-efficient ML inference accelerating common applications
- AMX/SME for programmable low-latency scientific and ML workloads on the CPU
- GPU for scalable, programmable ML (think research, development, and large models)
The GPU matrix units do not replace the NPU. The later is designed to support common application use cases and do so with very low power consumption. Removing the NPU would be actually detrimental to the user experience.
By the way, this is also the reason why every user-facing platform contains an NPU nowadays. The GPU is a large truck - you use it to haul containers full of stuff. It’s not a practical tool for your daily commute.
Because they did not have AI models ready, and their hardware was not suitable for scalable ML. But for on-device ML inference Apple has always been the state of the art - and still is.
My guess is that they initially though that scalable ML will remain a specialized niche, so they focused on the application side of things and decided that generic GPU hardware solution is “good enough” for researchers and developers. They obviously underestimated the interest. Adding matrix acceleration to the GPU merely serves to address this weakness.
Last edited:
