Occlusion

[subjonas]

Anybody know/hear exactly how occlusion works on the VP? I know very little about the technology, but all I know is that while immersed, I’ve seen the VP occlude the wearer’s arms/hands, and I heard from an Apple exec that it occludes other people in the room when they are looking at you.

Other questions I have are, when immersed…

Do we know what else it can occlude? The rest of your body? Anything you’re holding like a drink? Pets? Couch? Coffee table? Your MacBook? A book? A pen?

If it can occlude all those non-human objects, how does it know when to occlude and when not? Or does it always occlude things if it can?

It would be ideal if it could occlude everything, but also if you could choose what to occlude and not occlude. Maybe choosing individual objects to hide might be too much. Maybe it’s all about that immersion dial, though that seemed to be more about how much of your 360 field of view you want covered, as opposed to how much of your immediate proximity do you want covered. But maybe it does both. Does turning the immersion dial all the way up mean you can’t see things next to you? What about things you’re holding like a cup?

Basically, the scenario I’m specifically thinking of is I’m working on my laptop, sitting on my couch, and I want a serene environment, so I crank up the immersion dial of say the scene of Mt. Hood. I’d ideally want the room and everything in it gone except me, my couch, my laptop, a pen and notebook sitting next to me on the couch, and maybe an end table with my drink on it. How likely do you think something like that would be possible?

 

[matrix07]

Basically, the scenario I’m specifically thinking of is I’m working on my laptop, sitting on my couch, and I want a serene environment, so I crank up the immersion dial of say the scene of Mt. Hood. I’d ideally want the room and everything in it gone except me, my couch, my laptop, a pen and notebook sitting next to me on the couch, and maybe an end table with my drink on it. How likely do you think something like that would be possible?

Yes, it will do that but I’m not sure you can choose what to be included. It’s like 100% or 0.

 

[subjonas]

Yes, it will do that but I’m not sure you can choose what to be included. It’s like 100% or 0.

Hm but the Digital Crown is supposed to allow you to choose anywhere between 100 and 0.
Plus it can’t be totally zero occlusion because you can always at least see your hands. So then that at least begs the question of what about objects that you’re holding?

 

[TLewis]

There is a way of moving windows closer and farther away from you (remember the bit in the presentations about the windows having slight shadows?). So, the headset knows the distance of windows, and the lidar knows the distances of physical objects. So, it's not hard to put physical objects in front of or behind windows.

 

[subjonas]

There is a way of moving windows closer and farther away from you (remember the bit in the presentations about the windows having slight shadows?). So, the headset knows the distance of windows, and the lidar knows the distances of physical objects. So, it's not hard to put physical objects in front of or behind windows.

Right, with windows it can probably occlude any object that’s between your eyes and the window. Just confused on how it has no limitations on what it can occlude because I thought I remembered on iPhone (which also has LiDAR), AR occlusion only worked depending on the object. It first worked on furniture, then people were later added, then pets later, or something like that. I just don’t have a clear understanding of how the technology works. Maybe it’s at a point where it doesn’t matter what the object is anymore, it recognizes everything.

But also I’m still wondering how occlusion is managed in the immersion mode rather than with floating windows, because floating windows have a definite position in space so it’s clear what should and shouldn’t be occluded. But I don’t think the immersion background has a defined position in space, or if it does it’s very far away so you should just see your room between it and your eyes (the background is outside your room) which would be useless. So the VP has to have some other way to decide what to occlude (if anything) and what not to, maybe even beyond the immersion dial.

I just looked at the VP film again where the woman is watching a movie in her living room, and I noticed when she turned the immersion dial all the way up (I assume), everything around her was covered up by the background except her and her couch (including couch pillows). Not sure if that’s how it actually works, or if that’s just for the film. But if that’s how it works, then the VP doesn’t occlude purely based on proximity, but also based on what the object is. It knows to keep the whole couch but not the floor underneath it. That’s impressive.

So I guess I can assume that in full immersion mode, it would allow me to see/work on my laptop since it’s right on my lap, and anything in my hands, and anything on my couch. And if I need to see a little more, like a drink on my coffee table, I’d have to turn the dial back a tad.
If that’s how it is, then that sounds good to me, and the technology is pretty amazing.

 

[matrix07]

Hm but the Digital Crown is supposed to allow you to choose anywhere between 100 and 0.
Plus it can’t be totally zero occlusion because you can always at least see your hands. So then that at least begs the question of what about objects that you’re holding?

Yes, I mean 0 excluding your body.

You'll definitely see your hands in all scenarios but I don't think it will include any other objects in your full immersion. Perhaps your pet but that's all.
If you're holding a drink will it include it? I think yes, but if you put it down I don't think at 100% immersion you will see it. For example if you're holding Apple Remote (I think you can use it when you're watching movie) you'll see it but if you put it down you won't so you'll need to put it where you can easily reach.

 

[subjonas]

Yes, I mean 0 excluding your body.

You'll definitely see your hands in all scenarios but I don't think it will include any other objects in your full immersion. Perhaps your pet but that's all.
If you're holding a drink will it include it? I think yes, but if you put it down I don't think at 100% immersion you will see it. For example if you're holding Apple Remote (I think you can use it when you're watching movie) you'll see it but if you put it down you won't so you'll need to put it where you can easily reach.

Yeah that’s possible full immersion is only your body and anything you’re holding. Maybe even only your hands.

Not sure if it was full immersion, but I mentioned in post #5, in the VP film when the woman watching the movie goes into immersion mode, her body and couch stayed visible.

But after looking at it again, I noticed she didn't actually turn the dial. So if that scene is not supposed to be an accurate demonstration of the function, then I’m thinking it’s also possible that they just kept her and the couch visible for the film, but those might really be invisible for the actual experience.

In any case, I think it’s all about that dial. It should give us the granular control we need to find the balance of immersion and visible access to our immediate surroundings that we each want for each situation. I hope and think it will. If so, what an awesome idea that feature.