Apple's Headset Said to Feature 14 Cameras Enabling Lifelike Avatars, Jony Ive Has Remained Involved With Design

[MacRumors]

Earlier this week, The Information's Wayne Ma outlined struggles that Apple has faced during the development of its long-rumored AR/VR headset. Now, in a follow-up report, he has shared several additional details about the wearable device.

Apple headset render created by Ian Zelbo based on The Information reporting​

For starters, one of the headset's marquee features is said to be lifelike avatars with accurate facial expressions captured by 14 cameras:

Other challenges, such as incorporating 14 cameras on the headset, have caused headaches for hardware and algorithm engineers. The cameras include those that will track the user's face to ensure virtual avatars accurately represent their expressions and mouth movements, a marquee feature.

The report adds that Apple's former design chief Jony Ive has remained involved with the headset project as an external consultant to the company:

One person familiar with the matter said Ive's consulting work for Apple since he left includes the headset, adding that he is often brought in to help his former team push through their preferences in areas such as battery, camera placement and ergonomics over those of engineers. Two people said even after Ive left Apple, some employees on the headset project were still required to make the trek from Cupertino to San Francisco, where Ive has a home, to get his approval on changes.

Ive has continued to tweak the headset's design. While earlier prototypes had the battery in the headband, he prefers a design that would tether the headset to a battery the user wears, similar to Magic Leap’s headset design. It couldn't be learned if this approach will make it into the final design.

The initial version of Apple's headset is said to lack a focus on gaming:

Four people who have worked on the project also criticized its lack of focus on gaming, a category of software that appeals to early adopters, which was important to the success of the iPhone and has been a big priority for Meta's VR group. Those people said Rockwell's group almost never mentioned games in internal presentations about possible uses for the headset. Apple isn't developing game controllers for the device and is aiming to use hand tracking or in combination with a clothespin-like finger clip as inputs for the device, multiple people familiar with the project say.

As previously reported, Apple was considering having its headset be tethered to an external base station for some computing tasks, but the headset is now expected to be a fully standalone device. The report claims that the base station was going to use the same chip that was later announced as the M1 Ultra for the Mac Studio.

The headset itself is expected to be powered by two chips, with a streaming codec to reduce latency. The main chip in the headset will be equivalent to the M2 chip that is expected to debut in new MacBook Air and iPad models later this year, the report claims.

The full-length report can be read at The Information with a subscription.

Apple's headset is currently expected to be released in 2023. Bloomberg's Mark Gurman recently reported that Apple's board of directors received a demo of the headset last week, suggesting that the device is nearing completion.

Article Link: Apple's Headset Said to Feature 14 Cameras Enabling Lifelike Avatars, Jony Ive Has Remained Involved With Design

 

[AbhiAchShan]

It's good to see Jony Ive work with them again!

 

[UltimateSyn]

The initial version of Apple's headset is said to lack a focus on gaming​

I figured that gaming would be its primary focus. Wonder what it's actually meant for.

 

[djcerla]

Apple isn't developing game controllers for the device and is aiming to use hand tracking or in combination with a clothespin-like finger clip as inputs for the device

If you need a game controller, you blew it.

 

[contacos]

In case this gets archived like the initial iPod / iPad responses:

Who is this for? I don’t see a use case. B2B market?

 

[TSE]

If you believe in AR/Metaverse/Whatever, buy AAPL. They will be frontrunners.

 

[TinyMito]

Mostly focus on business and collab. I don't see how this is going to fly on the gaming side. I have Oculus CV1, Quest 2, mid-high-end gaming PC to run Oculus store, Steam, and Standalone PC games.

If Apple allows or is backed by OpenVR / OpenXR supports. It would fly but no, lack of native Vulkan support and lock down to Metal only that <1% of the game/app market. Just laughable.

Apple AR/VR = is that mean we can get back to working from home?

 

[LiE_]

Apple's former design chief Jony Ive has remained involved

So it's going to be very thin, unibody and have no power off button ?

 

[TheYayAreaLiving 🎗️]

Looks like this is Jony I've last product.

That is alot of camera's in one device. Hopefully it's not going to give me an eye infection or any of the eye sight problems.

 

[yabeweb]

In case this gets archived like the initial iPod / iPad responses:

Who is this for? I don’t see a use case. B2B market?

With metaverse and such this might be how we go shopping / on the internet in the future.

Opened my store in Multiverse on Quest 2 and it works

 

[macsplusmacs]

Gimme!

 

[TinyMito]

If you need a game controller, you blew it.

xD Quest 2 - Hand Tracking 2.0 is amazing. Been playing with my hands yesterday.

 

[randomgeeza]

Bore off, so not into this AR/VR stuff... it simply isn't there right now...

 

[yossi]

So there are 14 cameras aimed at the face of the person wearing the headset?

 

[TheYayAreaLiving 🎗️]

Bore off, so not into this AR/VR stuff... it simply isn't there right now...

The hardware can be all there. But when it comes to the software that is definitely not there. It'll take another 10 years before AR/VR takes off and that's only "if" it will.

 

[utsava]

So there are 14 cameras aimed at the face of the person wearing the headset?

If this is going to support mixed-reality type functions, I'd suspect some of those cameras are facing outward.

 

[falkon-engine]

With the lack of gaming focus on macOS this doesn't come as a surprise.

 

[macsplusmacs]

With the lack of gaming focus on macOS this doesn't come as a surprise.

Apple needs to velcro a Steam Deck onto it. Problem solved.

 

[gwhizkids]

So it's going to be very thin, unibody and have no power off button ?

And have impeccably rounded corners!

 

[44267547]

‘Life like avatars’……


⬆️ One application in the professional sector I could easily see Apple’s AR/VR device being used for tactical scenarios for law-enforcement room/building clearing.

I really think this product has loads of potential, but I just need to see some ‘hands-on’ demos of what this is capable of, being I’m interested in it, but I’m not quite sold yet.

 

[macsplusmacs]

And have impeccably rounded corners!

Chamfered Corners!

Chamfer and Chamfering Guide [Learn Quick] - Machinist Guides

Learn all about a chamfers including different ways to measure them and how they compare to similar features.

www.machinistguides.com

 

[gwhizkids]

Bore off, so not into this AR/VR stuff... it simply isn't there right now...

You in 2006: "So not into this smartphone stuff...it simply isn't there right now..."

 

[contacos]

So it's going to be very thin, unibody and have no power off button ?

Can you imagine what the case of this thing may look like?

 

[erikkfi]

Get that design team a trusted in-house leader, Tim.

 

[gwhizkids]

‘Life like avatars’……


⬆️ One application in the professional sector I could easily see Apple’s AR/VR device being used for tactical scenarios for law-enforcement room/building clearing.

I really think this product has loads of potential, but I just need to see some ‘hands-on’ demos of what this is capable of, being I’m interested in it, but I’m not quite sold yet.

I don't think you're going to have to wait too much longer for the demo. Could be at WWDC. Or at the iPhone event in September. Either they'll hit it out of the park or people will be scratching their head about the use cases. Or it will evolve, like the Watch did when it re-scoped from fashion to health.