Apple Vision Pro Supports Virtual Typing, Navigation Through Hand Gestures and Eye Movements

[MacRumors]

Rumors about Apple's Vision Pro headset have been circulating for years, but something that was never quite clear was how it would be controlled. There were some early rumors of control devices, which eventually faded away as leaks focused more on gesture control, and as it turns out, that's the way Apple went.

The Apple Vision Pro uses hand gestures, eye movements, and spoken commands for navigation. You can choose something on the display by looking at it, for example, and then select it by tapping your fingers together. Scrolling is done with a quick flick, and only small movements are required. You can look at the microphone button in a search field and then start speaking in order to dictate text, and Siri voice commands can be used to open and close apps, play songs, and more.

Reviewers have said that the Vision Pro navigation experience takes time to get used to, so there will be an adjustment period. Most other headsets on the market use some kind of handheld control mechanism, so the gesture-based control system will be unfamiliar to almost everyone.

Typing can be done with a connected iPhone or Bluetooth keyboard, but there's also a virtual keyboard to type on, and dictation can be used as an alternative as well.

The interface won't be easy to get used to for many, but on the plus side, the app layout and navigation will be immediately recognizable to those who have used an iPhone or an iPad. Apps are arranged in a "Home View" that's similar to the Home Screen so not everything will be unfamiliar.


Article Link: Apple Vision Pro Supports Virtual Typing, Navigation Through Hand Gestures and Eye Movements

 

[doboy]

That keyboard looks cramped 😁

 

[3530025]

With bluetooth keyboard, I can see it being replacement for multi-screen setup on the go.

But the price needs to come down, otherwise I'm not sure how large is the group of people spending 3500 USD for multi-screen setup on the go with OS similar to iPad (iPadOS seems pretty limited compared to macOS experience).

 

[bobmans]

Looks cool, they should connect to the Apple watch to provide haptics when typing / doing actions.

 

[TheYayAreaLiving 🎗️]

This will be the ultimate test!

 

[al256]

I can finally show off my touch-typing skills in the nerdiest way possible.

 

[TheYayAreaLiving 🎗️]

I wonder if the virtual keyboard contains Haptic Feedback. If it does that would be amazing. Otherwise, it's going to feel like you are typing in the air.

 

[Love-hate 🍏 relationship]

I wonder if the virtual keyboard contains Haptic Feedback. If it does that would be amazing. Otherwise, it's going to feel like you are typing in the air.

I'm afraid that's impossible

Yeah typing is gonna feel weird haha

 

[howywood]

I so can't wait for the VisionPro P0rnHub App

 

[TheYayAreaLiving 🎗️]

☝🏻 Mark up! See it in action here. Seems pretty accurate. 🎯

https://twitter.com/x/status/1666147490893115392

 

[seek3r]

With bluetooth keyboard, I can see it being replacement for multi-screen setup on the go.

But the price needs to come down, otherwise I'm not sure how large is the group of people spending 3500 USD for multi-screen setup on the go with OS similar to iPad (iPadOS seems pretty limited compared to macOS experience).

I mean, there's the argument to be made that it's also basically the whole laptop, not just the display. The thing does have an M2 chip, I wonder how much of a full computer replacement it can be. If I can run basically a skinned MacOS, so I have root, the ability to install things through homebrew, my editors and IDEs, docker, full fledged office/teams, and iterm2 etc - tldr if I could travel with *just* this to a conference, at least one I don't expect to work at, just my hotel room, which is most that arent ones I'm repping the company at a booth, that would be worth the money.

 

[3530025]

It certainly has those laser keyboard vibes.

Let's hope the experience is better than that.

 

[szw-mapple fan]

I wonder if the virtual keyboard contains Haptic Feedback. If it does that would be amazing. Otherwise, it's going to feel like you are typing in the air.

I doubt that’s possible unless Apple makes some kind of haptics glove.

 

[szw-mapple fan]

It certainly has those laser keyboard vibes.

Let's hope the experience is better than that.

I’m guessing it’ll both be better and worse. Better because it doesn’t need a flat surface and Apple’s tracking is probably much more accurate. Worse because you don’t have anything to push back at your fingers and typing in midair will get tiring real fast.

 

[3530025]

I mean, there's the argument to be made that it's also basically the whole laptop, not just the display. The thing does have an M2 chip, I wonder how much of a full computer replacement it can be. If I could travel with *just* this to a conference, at least one I don't expect to work at, just my hotel room, which is most that arent ones I'm repping the company at a booth, that would be worth the money.

Definitely.

I mean the software is important part of it.
And also whether you would need to pack keyboard and other accessories with it. If it is not running macOS, then you're not that far from iPad Pro which is WAY cheaper, has a lot better battery life and is great to type on with magic keyboard.

 

[roar08]

Giant, multiple, massive screens!

Tiiiiiiny little keyboard...I wonder if they're virtual butterfly keys.

 

[seek3r]

I'm afraid that's impossible

Yeah typing is gonna feel weird haha

not impossible, just requires extra hardware, Nintendo even had the power glove for this use in *1989*, I think Apple would prefer to avoid making such things though - maybe we'll get third party hardware.

EDIT: not sure the power glove actually had haptic feedback come to think of it, but other simillar hardware that's followed since have

 

[jjudson]

And this is when man made the decision to live and exist on the couch, eventually losing his legs, developing short arms, and giant bug eyes.

Archeologists will gaze in wonder at the new evolution that ultimately destroyed the homo sapien species…

 

[seek3r]

This will be the ultimate test!

View attachment 2214058

No lies detected

 

[seek3r]

And this is when man made the decision to live and exist on the couch, eventually losing his legs, developing short arms, and giant bug eyes.

Archeologists will gaze in wonder at the new evolution that ultimately destroyed the homo sapien species…

I, for one, welcome our new vestigial limbed overlords

 

[jmovie]

“The interface won't be easy to get used to for many” - says who? Most reviewers who have actually had a trial with the device say they got the hang of using it in short order…

 

[jmgregory1]

This will be the ultimate test!

View attachment 2214058

Can you imagine working on a spreadsheet using voice commands? They will need to change how you can interact with apps if this system is going to be for doing actual work and not just consuming or viewing.

 

[BobSc]

Otherwise, it's going to feel like you are typing in the air.

Oh vay. You are purely speculating. You haven't even tried the device. It's controls will take time to get used to, but once done, I suspect it will become very natural. Why are you, and so many others, so very negative about this incredible new device? You previously stated that you will start saving your money for this. Did you change your mind?

 

[AppleIsEveryWhere]

Whats is interesting with Vision Pro is that you don't have any other devieces in your hands to be able to controll apps, Meta and Sony users have no choice, you gotta use those controlls to be able to use your hands

This change everything again like iPhone back in 2007!

 

[Brad7]

☝🏻 Mark up! See it in action here. Seems pretty accurate. 🎯

https://twitter.com/x/status/1666147490893115392

Wow nice. Makes me think, we went from writing to typing, and now back to writing. I knew my cursive wouldn’t go to waste! 😂