Feature Wish List - High Quality Virtual Keyboard and Virtual Mouse

[david.olstein]

Unless I missed something in the latest update, the Apple Vision Pro virtual keyboard continues to a complete joke. I want a virtual keyboard that I can use just as easily as real keyboard. You would think that with all the cameras built in to the AVP, it would be possible to detect finger movements with sufficient accuracy to enable you to touch type on a virtual keyboard. Ideally, the virtual keyboard would pop up any time I hold my hands out in a typing position, with the user having the ability to "grab" the keyboard at the edges and move it around and adjust its size and positioning.

I also want a virtual mouse. Or at least some kind of ability to use hand gentures rather than eye movements to navigate the AVP environment and virtual computer display. I just find the eye tracking mode of navigation fatiguing and often frustratingly inaccurate.

Why do I want these features? Simple. I want to be able to use my AVP as a virtual display for my mac. But I don't want to be tethered to a desk because of the need for a physical keyboard and trackpad. I'd like to have a portable computer system consisting of just the AVP and the new M4 Mac Mini that I can take and use anywhere, while leaving the keyboard and mouse at home.

Is this really too much to ask for?

 

[david.olstein]

132 views and not a single reply???

 

[axantas]

What is wrong with the existing virtual keyboard, that you can move anywhere you want? The keyboard acts like on an iPad: it pos up, if there is something to type. What do you expect from your hand gesture, if there is nothing to type?

 

[terminator-jq]

An alternative to eye tracking would be welcome and it would also open the door to allowing future Visio devices that can be used while wearing glasses since there are some eye corrections that they don’t have lenses for. IIRC the eye tracking was the main reason Vision Pro can’t be used with glasses (and thicker contacts).

For the virtual keyboard, I don’t think there’s anything to worry about. The Vision Pro is still early. 2.0 felt like a true 1.0 so I have no doubt version 3.0 will bring some major updates and fixes to several areas. Especially since Apple would have had a years worth of user data by that time compared to just a few months worth from 1.0 to 2.0.

 

[david.olstein]

What is wrong with the existing virtual keyboard, that you can move anywhere you want? The keyboard acts like on an iPad: it pos up, if there is something to type. What do you expect from your hand gesture, if there is nothing to type?

Have you actually used the existing virtual keyboard? It's like typing with just your pointer fingers. It's nowhere near fast enough for my purposes. And typing with the eye focus/finger gesture is just as painful.

 

[Amazing Iceman]

Have you actually used the existing virtual keyboard? It's like typing with just your pointer fingers. It's nowhere near fast enough for my purposes. And typing with the eye focus/finger gesture is just as painful.

This is why I would either dictate or use a physical keyboard if I have to time a lot.
With a physical keyboard I can focus on what I’m doing and not on the virtual keyboard.

 

[axantas]

Have you actually used the existing virtual keyboard? It's like typing with just your pointer fingers. It's nowhere near fast enough for my purposes. And typing with the eye focus/finger gesture is just as painful

Using it myself, I see that. But what would it have to to look to meet your expectations? As we are not millions of users, Apple might listening to an interesting suggestion.

For longer texts, I started to use dictation, which is a really good alternative in the VisionPro or use a physical keyboard, just as Amazing Iceman writes.

 

[david.olstein]

Using it myself, I see that. But what would it have to to look to meet your expectations? As we are not millions of users, Apple might listening to an interesting suggestion.

For longer texts, I started to use dictation, which is a really good alternative in the VisionPro or use a physical keyboard, just as Amazing Iceman writes.

The virtual keyboard that I'd like to see would more closely resemble a physical keyboard - surely the AVP has the graphics capabilities to do that - and would allow you to type with two hands, and press more than one key at the same time. Being able to adjust the size and positioning of the keyboard in space to fit your hand size and preferred typing angle is needed -- instead of being directly in front of you, which is not how anyone types, the keyboard would be positioned below you, but still within direct line of sight with the AVP's built in cameras. Again, the idea would be to replicate the experience of typing as closely as possible, which the current virtual keyboard does not do. Haptic feedback would be great, those probably impossible without an additional accessory, like haptic gloves.

 

[axantas]

The virtual keyboard that I'd like to see would more closely resemble a physical keyboard - surely the AVP has the graphics capabilities to do that - and would allow you to type with two hands, and press more than one key at the same time. Being able to adjust the size and positioning of the keyboard in space to fit your hand size and preferred typing angle is needed -- instead of being directly in front of you, which is not how anyone types, the keyboard would be positioned below you, but still within direct line of sight with the AVP's built in cameras. Again, the idea would be to replicate the experience of typing as closely as possible, which the current virtual keyboard does not do. Haptic feedback would be great, those probably impossible without an additional accessory, like haptic gloves.

...sounds like a perfect suggestion for the feedback to Apple. You do have access to the feedback-app on the AVP?

 

[Amazing Iceman]

The virtual keyboard that I'd like to see would more closely resemble a physical keyboard - surely the AVP has the graphics capabilities to do that - and would allow you to type with two hands, and press more than one key at the same time. Being able to adjust the size and positioning of the keyboard in space to fit your hand size and preferred typing angle is needed -- instead of being directly in front of you, which is not how anyone types, the keyboard would be positioned below you, but still within direct line of sight with the AVP's built in cameras. Again, the idea would be to replicate the experience of typing as closely as possible, which the current virtual keyboard does not do. Haptic feedback would be great, those probably impossible without an additional accessory, like haptic gloves.

This should work, or similar:

TapXR - Tap

Wearable Controller, Keyboard & Mouse

www.tapwithus.com

 

[schappim]

Unless I missed something in the latest update, the Apple Vision Pro virtual keyboard continues to a complete joke. I want a virtual keyboard that I can use just as easily as real keyboard. You would think that with all the cameras built in to the AVP, it would be possible to detect finger movements with sufficient accuracy to enable you to touch type on a virtual keyboard. Ideally, the virtual keyboard would pop up any time I hold my hands out in a typing position, with the user having the ability to "grab" the keyboard at the edges and move it around and adjust its size and positioning.

I also want a virtual mouse. Or at least some kind of ability to use hand gentures rather than eye movements to navigate the AVP environment and virtual computer display. I just find the eye tracking mode of navigation fatiguing and often frustratingly inaccurate.

Why do I want these features? Simple. I want to be able to use my AVP as a virtual display for my mac. But I don't want to be tethered to a desk because of the need for a physical keyboard and trackpad. I'd like to have a portable computer system consisting of just the AVP and the new M4 Mac Mini that I can take and use anywhere, while leaving the keyboard and mouse at home.

Is this really too much to ask for?

Lighter and 5g

 

[JPack]

Sounds great in theory, but in reality I doubt most users would want to hold out their arms like a zombie for more than a few minutes before fatigue sets in.

I suspect Apple already knows this and didn’t bother developing such a feature.

 

[david.olstein]

Sounds great in theory, but in reality I doubt most users would want to hold out their arms like a zombie for more than a few minutes before fatigue sets in.

I suspect Apple already knows this and didn’t bother developing such a feature.

You would be holding your arms in the same position you would be if you had a real keyboard. A lot less awkward than typing on the current virtual keyboard.

 

[2128506]

Guess YMMV, for me virtual keyboard speed is comparable to iPhone, when using look-and-tap. Air-typing would tire your hands really quick.

In any case, proper hardware keyboard and trackpad/mouse will always be better - they just need to improve keyboard/desk recognition to see them from immersion.

 

[david.olstein]

Guess YMMV, for me virtual keyboard speed is comparable to iPhone, when using look-and-tap. Air-typing would tire your hands really quick.

In any case, proper hardware keyboard and trackpad/mouse will always be better - they just need to improve keyboard/desk recognition to see them from immersion.

How would you know that "air typing" would tire your hands really quick?

I get that a virtual keyboard and mouse is unlikley to match the performance of an actual keyboard and mouse. But it ought to be possible to get much closer than you currently can with the current iteration of the virtual keyboard, which is dreadful. If, to get decent performance, I have to use an actual keyboard and mouse, sitting at a desk, then I'm just going to use my Mac. I spent time this past weekend using my AVP as an external display with my Mac. And it was fine for viewing content. But when it comes to tasks like creating documents and responding to emails, it's just easier to use my regular display.

 

[2128506]

How would you know that "air typing" would tire your hands really quick?

I get that a virtual keyboard and mouse is unlikley to match the performance of an actual keyboard and mouse. But it ought to be possible to get much closer than you currently can with the current iteration of the virtual keyboard, which is dreadful. If, to get decent performance, I have to use an actual keyboard and mouse, sitting at a desk, then I'm just going to use my Mac. I spent time this past weekend using my AVP as an external display with my Mac. And it was fine for viewing content. But when it comes to tasks like creating documents and responding to emails, it's just easier to use my regular display.

I've tried to air-type, it's not comfortable after several minutes.

Lapdesk and wireless keyboard/trackpad work for me.