Spatial Video Format Details

[4087258]

Hello. Spatial videos are encoding in MV-HEVC which is simply 2 HEVC (h.265) videos playing simultaneously. By the way, when watching on a 2D screen (iPhone), only one stream is played (left eye).

I’m interesting in knowing more about the resolution and the frame rate of the streams.

Would it be 2x 1080p at 90fps?
Would it be 2x 4k at 30fps or 24fps?

Can both M2 and A17 Pro support 2x 4k simultaneous recording at 90fps?


Vision Pro works in 90fps all the time, except when a 24fps movie is playing (it goes to 96fps).

 

[arw]

Interesting question.
For the full immersive/realistic experience I'd expect Apple not to settle for less than 4K@60.
From what I've read, it's not actually two independent video streams but makes use of the technique "2D plus Delta" which is similar to the introduction of mp3 joint stereo (compared to full stereo).
Record one full channel/video (left eye) and only store the differences for the second channel/eye to save huge amounts of space - all within one single frame.
If that now requires more or less computing power than encoding two independent streams, I don't know.
[link]

The MV-HEVC approach to 3D video involves encoding two slightly different views of the same scene (left and right) into a single video frame. This process involves a technique called "2D Plus Delta", where the differences (or delta) between the left and right images are determined and encoded. This allows 2D decoders to use the base 2D view (usually the left eye view), while 3D decoders can calculate and present both views to the corresponding eyes.

 

[4087258]

Interesting question.
For the full immersive/realistic experience I'd expect Apple not to settle for less than 4K@60.
From what I read, it's not actually two independent video streams but makes use of the technique "2D plus Delta" which is similar to the introduction of mp3 joint stereo (compared to full stereo).
Record one full channel/video (left eye) and only store the differences for the second channel/eye to save huge amounts of space - all within one single frame.
If that now requires more or less computing power than encoding two independent streams, I don't know.
[link]

You’re completely right, sir.
I’m also wondering if that makes possible high resolution plus high frame rate MV-HEVC recording on A17 pro.

 

[Jensend]

Interesting question.
For the full immersive/realistic experience I'd expect Apple not to settle for less than 4K@60.
From what I've read, it's not actually two independent video streams but makes use of the technique "2D plus Delta" which is similar to the introduction of mp3 joint stereo (compared to full stereo).
Record one full channel/video (left eye) and only store the differences for the second channel/eye to save huge amounts of space - all within one single frame.
If that now requires more or less computing power than encoding two independent streams, I don't know.
[link]

4K 60 FPS doesn't make sense for a screen that runs at 90 or 96 Hz.
They'll want a divisor of the native refresh rate, so if they go less than the full refresh, 45 or 48 FPS would be best.

 

[arw]

All possible. Perhaps the display is adaptive…?
At least the resolution of 4K is very probable.
Some screenshots from their video:

 

[Jensend]

All possible. Perhaps the display is adaptive…?

All VR headsets used strobed displays. That means that the display only flashes on for about 1/10th of the frame time. If you get much below 90Hz, the strobing is noticeable as flicker. 60Hz strobing gives me a headache.
Also, I don't believe there are any strobed displays that that also support continuously variable refresh rates at the same time.

If displays aren't strobed, you see blur whenever you rotate your head, and that makes for a very uncomfortable experience.

 

[arw]

Thanks for the clarification - I'll leave any future speculations to the more experienced.
Again, not sure what to make of it but regarding possible resolutions, this is the highest 3D encoding profile reference I've found online for Apple devices.

Spoiler

Type Property

mvhevc1440x1440
A preset for MV-HEVC video at 1440 by 1440 pixels.

static let mvhevc1440x1440: AVOutputSettingsPreset

mvhevc1440x1440 | Apple Developer Documentation

A preset for MV-HEVC video at 1440 by 1440 pixels.

developer.apple.com

The other one is 960x960 px.

 

[timshundo]

$10 to anyone who can get spacial videos playing on a Nintendo 3DS screen first.

 

[WalterYoung]

$10 to anyone who can get spacial videos playing on a Nintendo 3DS screen first.

I have what you want.And I think $100 is reasonable.

 

[Timo_Existencia]

What's the FOV of the Spatial videos that the AVP and the iPhone 15 takes?