How are the "neural accelerators" different than the "neural engine" cores?
The neural accelerators are a feature of each GPU core. Think of them as specialized AI instructions supported by logic that is tied to each GPU core. The neural accelerators were introduced with the A19/M5. This aligns with how AI is mostly handled in non-Apple ecosystems, i.e., as part of the GPU, such as with the WMMA instructions on RDNA4. On M5 and A19-series chips, the neural accelerators feature hardware support for handling FP16 workloads, as well as INT4, INT8, and INT16. On the M6, they gained FP8 support. The neural accelerators naturally scale with the number of GPU cores, taking an M5 Max, e.g., with its up to 40 GPU cores to about 70 TFLOPs of FP16 inference performance.
The Neural Engine is a dedicated AI component that is heavily involved in image processing but can also be used for other AI workloads. It was introduced with the A11 – supposedly what gave the chip its "Bionic" moniker. When the Neural Engine was first introduced, Apple explained it as the implementation of its image processing taken by the iPhone's cameras. At that time, there was no Apple silicon for desktop/laptop yet, nor was there a broader interest in AI workloads as a mainstream topic outside of specialist circles. Since then, the Neural Engine has evolved to be the primary ML facility, up to the recent introduction of the neural accelerators. Its hardware is limited to FP16 support. In terms of its performance, A 16-core Neural engine such as in the M5 can get to some 20 TFLOPs of FP16 inference performance. The M6 just doubled that, by adding a second Neural Engine.
A third facility is the SME2 support by the CPU cores, or formerly the ML accelerators / AMX (Apple Matrix Coprocessor) that the newer SME2 implementation replaces. Think of these as specialized ML instructions supported by logic in the various types of CPU cores. The former AMX was introduced with the A13 Bionic and supported INT8 to INT32 and FP16 to FP64 workloads and carried through to the M3 generation. In the M4 and shortly after the A18, it was replaced by SME2 support after it became a standard extension to the ARMv9-A instruction set, adding INT64 support but losing all support on the efficiency cores. Its compute capacity is in the single-digit TFLOPs of FP32 inference. But its closeness to the CPU makes it useful, e.g. for prepping/compressing weights in preparation of feeding models to the more powerful facilities, much like what AMD plans to do with the new ACE instructions starting with Zen 7.
You could rightfully say that its all somewhat historically evolved, and a clean-sheet design would probably take you to something more unified and homogenous. But CoreAI (or formerly CoreML) helps abstract away from it all. In fact you would not be able to target the Neural Engine, e.g., on a low level at all. And depending on the workload, using either of the three facilities, or a combination of all three may actually be optimal in terms of maximizing performance and/or efficiency.
