What he means is that the Apple Foundation Models for Language (the Apple code that handles all AI language requests, so not just Siri "understanding" but also rewriting text, summarizing, translating, etc) has an average bit size per weight of about 3.7b. We know this because Apple published a paper on the subject. The same is probably true for the Vision model (again there's a united Vision model that handles various different types of vision requests, and I suspect also handles Pose). We have a series of papers on various aspects of the Vision model, but they're a few years old, and I don't believe there's been a recent paper giving the model size since Apple started really pushing model quantization.
His point is that Apple provides a platform (model conversion tools, model compiler, and hardware) that all work together to support models that operate well when compressed far beyond what other platforms currently achieve (see my comment above about the 8bit accuracy). So if you're obsessed with tribalism it's "somewhat unfair" to see Apple being punished for their performance at 32b (which only happens on GPU, not NPU) and even 16b, when Apple is all-in on optimizing for 8b and less.
Another way the benchmark is somewhat sub-optimal is that it does not give energy results, but of course that's the entire point of everyone moving to dedicated NPU hardware.
You could maybe write an alternative benchmark that calls into Apple APIs for some of the tasks above (eg image classification, pose detection, translation, etc). The results might not be *directly* comparable with the benchmark (which uses standard open models, so that more-or-less the same test is being run on all platforms) but the result might be VERY interesting in terms of comparing Apple's "systemwide" performance, not just the HW but also code optimized to the hardware.
This is probably an interesting project for any student out there wanting to get started at the very low-end of AI (just calling some APIs correctly) while also making a name for themself...
like Single p. affect tasks like... half p. tasks like and quantized those. I know it is more complicate like that but hope it can be made more simple for rest of us. 
