iPhone 15 Pro Cameras to Support Spatial Video Later This Year, But Key Questions Remain

[kkee]

I am too very much inclined to say that this is the biggest news coming out from this year iPhone event - one which was obviously underappreciated and underfocused in the news and comments alike. People choose to make fun and mock Apple all the time, but when they deliver technology like this, turned to blind eyes and pretend it never happened until other companies like Google came out with their own which everyone would then praise to heaven.

But a few among us really appreciate Apple for making a first step towards spatial world. It's a big step forward to include spatial camera in iPhone 15 Pro, despite being ignored by most people as gimmicky. The foundation has been laid for Vision Pro, and when they arrive, one would not feel regret to already created their own spatial videos months earlier.

 

[Ted_Appleseed]

The 3D video format is almost certainly based on MV-HEVC which is new, and was announced at this year’s WWDC. There’s nothing public for Final Cut Pro yet, but I will certainly be asking the FCP team about that at the FCP Creative Summit this November.

If you've ever wanted to visit Apple in person, this is a great way to do it: fcpcreativesummits.com

Wow, thank you for your information!

 

[Ted_Appleseed]

I had already decided to buy, but this is a great bonus. My worry is that I will want to use it often between now and when the AVP is released, so that I have a library of personal spacial videos to view with my new headset.

But that means I will be choosing to shoot spacial videos at times when I would otherwise shoot regualar videos. I hope that I will be able to extract regular 2D video from the file so I can share them on FaceBook.

Also, will I be able to share the spacial videos on Facebook, so that people using Facebook on the AVP or a MetaQuest 3 headset will be able to view them as intended?

Same and good question. I hope that the whole experience will be seamless.

 

[rajador]

Insert here the Wow video "internet is for VRp0rn", a classic.

 

[bgillander]

I wonder if the VisionPro will let you get up and walk around while viewing spacial video. How will it look if you look at the video from the side rather than from the front? Or just at an angle other than straight on.

I'm thinking of the holograms from Minority Report. When they're viewed from the side they don't model the contours of the faces. There are just two planes (subject and background) Our brains fill in the solidness of the subject from other cues, like the lighting and shadows. And there's a hole in the background, because the camera taking the picture didn't capture what was behind the subject. View attachment 2265468

Or is the image just stereoscopic? We get two angles, one for each eye. And if we try to walk around, the spacial video we're watching will move to stay the same distance from us and facing us head-on. You won't be able to go around to the back of the video any more than you can get to the end of a rainbow. In that case, the iPhone may still create that multi-plane model, masking the foreground subject, then snapping two images at the proper eye separation from the shadow-box they created from the original frame. iOS has been doing that kind of thing for years.

I think there will be utilities eventually to take some 2D movies and make them appear 3D. FaceTime has had a feature that does this to still photos.. That effect will likely range from uncanny to comedic. Remember the melting bridges in Google Maps?

Based on the camera placement, I would expect it to be similar to the VR180 cameras I have used, where you just get a stereo 180 degree view. The Lenovo was 4K and the Vuze was 5.6K, I think, and I haven’t seen any new VR180 cameras for a few years, so hopefully these will push 7 or 8K. Just that is a fairly heavy load, so if Apple does that I will be impressed. The stuff you are talking about still sounds like sci-fi to me, so I expect it is a decade off, at least, but here’s hoping they surprise me and prove me wrong!

As for the other comments about the lenses being too close together, the lenses are a bit closer together than those two VR180 cameras, but seem pretty similar to the old Sony HDR-TD20 3D camcorder, and the 3D on it was fine.

My worst case is that it might just be flat 3D, since I was under the impression that the lenses in those VR180 cameras were both fisheye, specifically to get that 180 degree view. Maybe they can do some software tricks with the Pro, but I don’t expect them to have fisheye lenses. That, and the way they just show the picture like a flat screen projection in the second example picture, does make me worry that this could be just 3D, rather than VR. It is hard to “show” VR in a flat picture, and even a “common” 3D camera would be something I haven’t seen in 10 years, so that would be something, but VR would be a big step up from that, so I hope they do get “true” spatial. If they get better than 7K VR, I will be very happy. If they get to what you describe, I will be blown away (and probably a little creeped out, since it is getting into overwhelming territory for my old mind!)

 

[bgillander]

Insert here the Wow video "internet is for VRp0rn", a classic.

If you like the WOW version, you might want to search for the original Avenue Q production. There are at least a couple of videos on YouTube. I’d forgotten about Avenue Q, which was like Sesame Street gone wrong, so thanks for the reminder!

 

[BuddyTronic]

Don’t the lenses have to be seperated like on the vision pro like the eye distance, how do they achieve the same with the lenses next to each other

with pretty cool computations is how it can be done, leveraging that small distance with math.