I keep coming back to these words when I think about what Apple will do with VR/AR, because to me the whole space doesn't really make sense without a user interface that is just as obviously better than the status quo, as when you saw Steve Jobs unlock the iPhone and scroll through his music for the first time. The only reason it makes sense to me that they've held off for so long is that they're really going to swing for the fences with the UI. Something that makes current VR systems feel like you're an elderly person pecking at a keyboard with two pointer fingers.
You really need to look at Ultraleap's developer documentation:
https://docs.ultraleap.com/xr-guide...tracking#provide-as-much-feedback-as-possible
...and understand that it's Unity & Unreal who are going to build the development environments for VR/AR, not Apple.
ARKit is a dumb pipe to Apple's sensor hardware, but developers don't work in ARKit - it's been over 1000 days since the "made in ARKit" twitter account, which was publicising ARKit stuff in the early days has posted anything. AR/VR devs work in Unity/Unreal, which then deploy to the lowest common denominator of ARCore/ARKit to talk to the hardware. But realisticaly, no one is developing AR for "Apple platforms" they're developing for Unity / Unreal - those are the platforms.
Hand tracking and proprioceptive embodiment is the killer app for VR/AR, although Valve have a very good idea wth their strap-on controllers that combine hand tracking with a button and control-surface based peripheral. There's a need for different physical peripheral tools, for different tasks.
I think the fundamental disconnect Apple-centirc folks have from what the technology is, is that there is no reason to expect for a platform-unified UI paradigm in VR/AR. The UI is furnished within the application, and is almost always a custom thing - in the same way that Games do not all use the exact same UI, aside from generalised conventons for what works well within genres. Indeed, I suspect most developers would be inherently hostile to the idea of a VR/AR platform vendor attempying to impose a standard UI upon them. Realistically, what a platform vendor can furnish in a standardised form, are open/save filesystem contents, which of course would be styled by the application to fit its ui aesthetic.
In terms of gestures, pinch to zoom, grab to stretch, thumb-forefinger twist rotation etc - AR/VR is 3D physical, and gestures only make true proprioceptive sense when they map to physical object manipulation.
And this follows on with how gestures have largely become a lost cause on iOS - what started out as something that was only used for its skeumorphic value of imitating the movement of a specific virtual physical object, has become an entirely abstract commandline of touch, where arbitrary gestures have to be rote memorised as connected to arbitrary outcomes.
Again, it you look at what Ultraleap can do, they have full finger tracking and articulation - an AR keyboard can be mapped onto any flat surface, and povide exactly the experience of typing on a touchscreen. Likewise a VR tagged physical keyboard can function equally well (except the key caps can be anything).
In practical terms, I forsee text entry for things like open/save dialogs to be far more likely to be implimented with cellphone style thumb keyboards, than virtual two hand keyboards, from a sheer ergonomic perspective - holding one arm out to type with one thumb is MUCH easier than holding both hands out in the air to attempt to pose over a full keyboard - and VR is largely a stand-in-the-middle-of-the-room activity. There's no reason to use VR seated at a desk, when you could just as easily use a 3d-glasses based 3d monitor.
AR/VR is not about putting a monitor-based computing paradigm onto your face, it's about recreating a physical, pre-computer-abstracted workshop way of working, but with the physical work-devices virtualised. That the thing - physical devices are almost always better, more satisfying things to work with than flat computer onscreen versions. A real airbrush, is in every way a better tool for doing airbrush art, than a wacom tablet and an airbrush tool. I would bet you any 3D animator would prefer to use a Dinosaur Input Device, to do physical stopframe, than they would use an entirely on-screen 3d package.
VR/AR is about creating a place, and manipulatable objects within that place. The entire history of computer UI has been about the destruction of place, and the destruction of objects, and their replacement with abstracted proxies of those places, and objects as flat images.
I'm not sure Apple, who to be fair, has been on a long drawn out collapse in ability to field good UI since 2013ish (most notably having to scrap the Apple Watch UI paradigm and start again from scratch after releasing the original version) has the ability to recognise what AR/VR require, beyond a simplistic "translucent notification HUD, with an appstore attached" - ie an iphone you wear on your face.
