A few random questions about the Vision Pro

[wrcousert]

1. When using FaceTime with a group of Vision Pro users, will each user appear as an avatar in the same room, capable of "sitting" on furniture, and doing things that a real person would do if they were visiting you? If so, could we see traditional board and card games where people can play together, at a table? Or are we limited to seeing avatars in floating windows? Will there be a gaming API specifically for FaceTime?
2. When using the Black Magic DaVinci Resolve app on the Vision Pro, is it possible that it could have a mode where you can use VR on the Fusion page to walk around and place 3D objects, lights and cameras, and visualize it, before rendering it to 2D or 3D video? Will the Vision Pro make huge expensive LED walls obsolete?
3. Are we likely to see a native version of Blender or Unreal Engine 5 (or greater) for the Vision Pro? If so, could it have the level of interactivity described above?
4. Are we likely to see a shopping app that lets you walk around a virtual store, pick up and examine virtual items, toss them in a cart and check out at a register? Maybe something like the virtual store in Ready Player One?
5. Any chance we could get the full version of Adobe Creative Cloud, optimized for VR/AR, or just limited, scaled down apps like we have for the iPad today?
6. Can I cast the device to an Apple TV or other similar device and use it for traditional 2D apps?
7. Will there be a cellular version?
8. Once the battery issue is resolved, are we likely to see people walk around town wearing these, for the AR apps that are likely to come, or will this pretty much be limited to home or office use only?
9. Will Swift Playgrounds be upgraded to let people design and test VR/AR apps? Is it likely that the iPad version of Swift Playgrounds will support the Vision Pro?
10. I know the device doesn't come with hand controllers. Would Apple object to third party companies releasing their own? Is it possible that Apple itself may release them later? They're partnering with Unity for VR games. I just can't see how they could do this without hand controllers.
11. What options might there be for locomotion in VR mode? I'm pretty sure we aren't getting omnidirectional treadmills anytime soon. Could we use some sort of hand gestures for locomotion? Is something like an iChair, that has foot controls for locomotion?
12. In the long run, is Apple likely to be the company that delivers something like The OASIS from Ready Player One? How close will the Vision Pro bring us to The OASIS?

 

[subjonas]

1. When using FaceTime with a group of Vision Pro users, will each user appear as an avatar in the same room, capable of "sitting" on furniture, and doing things that a real person would do if they were visiting you? If so, could we see traditional board and card games where people can play together, at a table? Or are we limited to seeing avatars in floating windows? Will there be a gaming API specifically for FaceTime?
2. When using the Black Magic DaVinci Resolve app on the Vision Pro, is it possible that it could have a mode where you can use VR on the Fusion page to walk around and place 3D objects, lights and cameras, and visualize it, before rendering it to 2D or 3D video? Will the Vision Pro make huge expensive LED walls obsolete?
3. Are we likely to see a native version of Blender or Unreal Engine 5 (or greater) for the Vision Pro? If so, could it have the level of interactivity described above?
4. Are we likely to see a shopping app that lets you walk around a virtual store, pick up and examine virtual items, toss them in a cart and check out at a register? Maybe something like the virtual store in Ready Player One?
5. Any chance we could get the full version of Adobe Creative Cloud, optimized for VR/AR, or just limited, scaled down apps like we have for the iPad today?
6. Can I cast the device to an Apple TV or other similar device and use it for traditional 2D apps?
7. Will there be a cellular version?
8. Once the battery issue is resolved, are we likely to see people walk around town wearing these, for the AR apps that are likely to come, or will this pretty much be limited to home or office use only?
9. Will Swift Playgrounds be upgraded to let people design and test VR/AR apps? Is it likely that the iPad version of Swift Playgrounds will support the Vision Pro?
10. I know the device doesn't come with hand controllers. Would Apple object to third party companies releasing their own? Is it possible that Apple itself may release them later? They're partnering with Unity for VR games. I just can't see how they could do this without hand controllers.
11. What options might there be for locomotion in VR mode? I'm pretty sure we aren't getting omnidirectional treadmills anytime soon. Could we use some sort of hand gestures for locomotion? Is something like an iChair, that has foot controls for locomotion?
12. In the long run, is Apple likely to be the company that delivers something like The OASIS from Ready Player One? How close will the Vision Pro bring us to The OASIS?

1. Apple execs said there will be non-windowed “personas”. I assume they will float wherever you place them in your space, and they will probably automatically face you. I don’t see how they could interact with your environment (eg. sit in a chair) though.

6. I think I heard somewhere that one can cast or Airplay their VP view to a screen (I forget which device). But one might have to be wearing the VP in order to do so. The VP may not be functional when you take it off. But if it is, you could probably use a Bluetooth keyboard to interact with a VP app while looking at the screen, but I don’t know how you could do any pointing. So this probably would not really work. Out of curiosity, if it worked, what would the use case be?

8. I think there will be very few people wearing these around town—or I should say, I think Apple hopes people won’t wear it around. Otherwise, I think they would have given it a bigger battery. But more so I think Apple hopes they won’t because the VP is a mixed reality headset, not solely augmented reality, which leaves the door too open for users to put themselves in dangerous situations trying to use this device wherever/whenever. It seems evident to me by looking at its design choices and the functions presented so far, that Apple mostly intended it to be used in controlled indoor environments (or in transportation as a passenger if safe to do so). I’m sure we’ll see some people using it in all sorts of public places, but not a lot, and I believe that’s by design. Also there’s the social aspect of cameras in public…

10. It does support controllers because there were people using controllers to play games in the marketing video.

 

[matrix07]

is Apple likely to be the company that delivers something like The OASIS from Ready Player One?

If Apple won’t do it someone else eventually will.

 

[klasma]

10. It does support controllers because there were people using controllers to play games in the marketing video.

Those were regular video game controllers, not 3D hand controllers as used with VR headsets. The latter typically require some level of integration with the headset hardware and software, in order to track the spacial position and orientation relative to the headset.

 

[subjonas]

Those were regular video game controllers, not 3D hand controllers as used with VR headsets. The latter typically require some level of integration with the headset hardware and software, in order to track the spacial position and orientation relative to the headset.

I just can't see how they could do this without hand controllers.

Ah I missed the VR part. But I also forgot/didn’t realize VR (or spatial?) controllers require a different type of support from 2D controllers. Makes sense.

I’m curious though, which types of game functions would you say require controllers? I think I’ve only played two VR games—the Spiderman swinging one and some shooting game, both a long time ago so my memory is hazy—but it seems like I should be able to mostly play them just with the VP cameras tracking my hand position and gestures. (Although, I don’t know—and not sure if anyone knows—how many/which hand gestures the VP can recognize.) The only function I don’t know off the top of my head that hand tracking could have solved is locomotion. Might need a d-pad/thumbstick to move from place to place in the world. But if that’s it, would that require a full on VR controller? If the headset is handling all the hand tracking, wouldn’t one just need a regular 2D game controller for locomotion and any other functions that can be accomplished by a button or thumbstick?

Edit- is precision the concern? Is a VR controller more precise than hand tracking, like a mouse is more precise than touch?

 

[bkkcanuck8]

Ah I missed the VR part. But I also forgot/didn’t realize VR (or spatial?) controllers require a different type of support from 2D controllers. Makes sense.

I’m curious though, which types of game functions would you say require controllers? I think I’ve only played two VR games—the Spiderman swinging one and some shooting game, both a long time ago so my memory is hazy—but it seems like I should be able to mostly play them just with the VP cameras tracking my hand position and gestures. (Although, I don’t know—and not sure if anyone knows—how many/which hand gestures the VP can recognize.) The only function I don’t know off the top of my head that hand tracking could have solved is locomotion. Might need a d-pad/thumbstick to move from place to place in the world. But if that’s it, would that require a full on VR controller? If the headset is handling all the hand tracking, wouldn’t one just need a regular 2D game controller for locomotion and any other functions that can be accomplished by a button or thumbstick?

Edit- is precision the concern? Is a VR controller more precise than hand tracking, like a mouse is more precise than touch?

According to the reviewers that I have heard, the eye tracking is very accurate and very good. That said, if you are doing shooting games or other games and specialized surrogate for the virtual representation would be better (though given that they tend to develop really cheap surrogates -- I am not sure the more complicated ones would be worthwhile - i.e. a compound bow). However, the general controllers (hand loop thing) seem archaic to me now.

 

[subjonas]

According to the reviewers that I have heard, the eye tracking is very accurate and very good. That said, if you are doing shooting games or other games and specialized surrogate for the virtual representation would be better (though given that they tend to develop really cheap surrogates -- I am not sure the more complicated ones would be worthwhile - i.e. a compound bow). However, the general controllers (hand loop thing) seem archaic to me now.

Yeah having something to hold would feel nicer, but again, not sure it needs to be a full on VR controller if the headset cameras are doing the tracking.
For games I would think it would need to use more hand tracking than eye tracking. Though I’m sure both are very good, depending on the game, maybe it would need even more precision. I’m thinking of pro gamers who faint at the idea anything less than pixel precision. Also pro artists.

 

[bkkcanuck8]

Yeah having something to hold would feel nicer, but again, not sure it needs to be a full on VR controller if the headset cameras are doing the tracking.
For games I would think it would need to use more hand tracking than eye tracking. Though I’m sure both are very good, depending on the game, maybe it would need even more precision. I’m thinking of pro gamers who faint at the idea anything less than pixel precision. Also pro artists.

True, but parts of any any surrogate is going to be blocked from the cameras - and that likely would mean it would be problematic being a perfect or reasonably perfect surrogate.

 

[subjonas]

True, but parts of any any surrogate is going to be blocked from the cameras - and that likely would mean it would be problematic being a perfect or reasonably perfect surrogate.

Possibly. In any case it’ll be interesting to see Apple’s approach.

 

[kkee]

1. Apple execs said there will be non-windowed “personas”. I assume they will float wherever you place them in your space, and they will probably automatically face you. I don’t see how they could interact with your environment (eg. sit in a chair) though.

Possibly by the same method as they used for the butterfly that land on your finger if the chair is virtual item. Some people reported that the butterfly flutter by the room before it lands on your fingers (some don't even land) just like a real butterfly. Since both your finger and the butterfly are virtual 'items', it is possible for them to interact in your world.

 

[kkee]

2. When using the Black Magic DaVinci Resolve app on the Vision Pro, is it possible that it could have a mode where you can use VR on the Fusion page to walk around and place 3D objects, lights and cameras, and visualize it, before rendering it to 2D or 3D video? Will the Vision Pro make huge expensive LED walls obsolete?
3. Are we likely to see a native version of Blender or Unreal Engine 5 (or greater) for the Vision Pro? If so, could it have the level of interactivity described above?
4. Are we likely to see a shopping app that lets you walk around a virtual store, pick up and examine virtual items, toss them in a cart and check out at a register? Maybe something like the virtual store in Ready Player One?
5. Any chance we could get the full version of Adobe Creative Cloud, optimized for VR/AR, or just limited, scaled down apps like we have for the iPad today?

2. Depending on how Black Magic DaVinci developers integrate their software into Spatial environment, I believe it is possible to create the scenario you mention. Autodesk already do something similar with Revit VR.
3. So far we haven't gotten confirmation from Apple which 3rd part software will be supported or if the devs of those companies were working on native apps for AVP. But expect more companies to jump in once the headset goes on sales.
4. That is very likely and one of the good example is Ikea virtual shop online with items delivered to your door.
5. Same as no 3.

 

[kkee]

7. Will there be a cellular version?

9. Will Swift Playgrounds be upgraded to let people design and test VR/AR apps? Is it likely that the iPad version of Swift Playgrounds will support the Vision Pro?

11. What options might there be for locomotion in VR mode? I'm pretty sure we aren't getting omnidirectional treadmills anytime soon. Could we use some sort of hand gestures for locomotion? Is something like an iChair, that has foot controls for locomotion?
12. In the long run, is Apple likely to be the company that delivers something like The OASIS from Ready Player One? How close will the Vision Pro bring us to The OASIS?

7. For a variety of reasons, it doesn’t make sense to include 5G at this stage but very likely once it is more portable and widely used. At the moment it is more of a fixed place device with WIFI. Adding 5G will add more into cost, battery life and the first iteration sales number is not high enough to get carrier support.

9. It has already being built. Stay tune. It will be likely built into AVP Dev Kits.

 

[wrcousert]

Thanks! I can't wait to get mine early next year. I'm buying a cheap MacBook Air with 8gb RAM. Hope that's good enough to get me started developing apps.

 

[kkee]

Thanks! I can't wait to get mine early next year. I'm buying a cheap MacBook Air with 8gb RAM. Hope that's good enough to get me started developing apps.

Good luck. The market for the first version being primarily developers and technology enthusiasts and that this product is very difficult to manufacture and Apple will have trouble making this product at scale anyway. That being said, they will likely sell all they can make, they just can't make that many.

 

[subjonas]

Possibly by the same method as they used for the butterfly that land on your finger if the chair is virtual item. Some people reported that the butterfly flutter by the room before it lands on your fingers (some don't even land) just like a real butterfly. Since both your finger and the butterfly are virtual 'items', it is possible for them to interact in your world.

The butterfly can interact with your world because it’s generated and controlled by the computer and only exists in your world. A persona just mirrors the actions of the FaceTime caller, who lives in their own world. How could that person see where a chair in your room is and go over and sit in it? Even if they can somehow see it, they might run into a wall in their room and fall on their bottom when they try to sit in your chair which doesn't exist in their room. The only way I could see everyone interacting in one world is if everyone is in a virtual world as avatars. Either that or if everyone has identical rooms.

So I don’t see how other personas can interact with your world, but they might be able to interact with virtual objects in your world that also exist in their world. So having a few personas over to play an AR table top board game might be possible.

 

[matrix07]

The butterfly can interact with your world because it’s generated and controlled by the computer and only exists in your world. A persona just mirrors the actions of the FaceTime caller, who lives in their own world. How could that person see where a chair in your room is and go over and sit in it? Even if they can somehow see it, they might run into a wall in their room and fall on their bottom when they try to sit in your chair which doesn't exist in their room. The only way I could see everyone interacting in one world is if everyone is in a virtual world as avatars. Either that or if everyone has identical rooms.

So I don’t see how other personas can interact with your world, but they might be able to interact with virtual objects in your world that also exist in their world. So having a few personas over to play an AR table top board game might be possible.

One way to do this is by having group chat in virtual pre-created room and invite everyone in. People who uses 2D device will only see the room as a static wallpaper but people with Vision Pro will be in the room and be able to talk to each other, watching movie or listening to music together like the room is their apartment.
Of course this might be a few years yet.

 

[subjonas]

One way to do this is by having group chat in virtual pre-created room and invite everyone in. People who uses 2D device will only see the room as a static wallpaper but people with Vision Pro will be in the room and be able to talk to each other, watching movie or listening to music together like the room is their apartment.
Of course this might be a few years yet.

Right, I was saying it would have to be a virtual room/environment that everyone would have to meet in. I thought Occulus was actually already doing that but with simpler avatars, but I could be wrong.
I am a little skeptical that Apple or anyone could pull off making it feel even close to hanging out with people in real life. It’ll probably be better than a regular video FaceTime, but I feel like it’ll be hard to forget that the people are computer generated. But I guess we’ll see what happens down the road.