Apple's Mixed Reality Headset to Feature iOS-Like Interface, Advanced Hand Tracking, and Will Work as Second Display for Mac

[subjonas]

Why would I do that?

You can see everything on your phone from within VR. There will be handoff for many apps, or in can just show a jumbo phone screen within VR when you tap a button in your VR environment.

Toggle on video passthrough so you can see your snacks.

You can have a photo widget on your VR desk.

VR whiteboard. (Apple's new Freeform app)

Take off the headset. I have to take off my headphones when putting on a sweater.

I work from home. This would be easier from within VR.

What's that?


Sure, VR/AR won't cover all use cases. But none of my other devices do either. I could make a similar list about iPhones: Screen too small, processor not powerful enough, customization too limited, doesn't run the apps I need, etc.

As I said, the list is really infinite so it doesn’t do a lot of good disputing each example I came up with off the top of my head.
But sure, digital versions of some of those analog things exist, but that’s not really the point. The point is that right now, analog is prevalent and will be for the long haul. Because a) many places are not set up to be completely digital, b) analog is still faster and easier in many circumstances, and c) most of life cannot be digital like snacks and coffee and cardigan sweaters. And people who use their company’s computers still need to be able to look at their phones.

A VR display might be productive for you in your specific case, but I’m just talking about the wider use cases.

Many things can be digital, but most of life is analog. So again, switching modes or taking off goggles to interact with analog each time are time/energy wasters that will add up quickly. And I posit most will not get much value in return by having a VR display instead of AR, compared to what they would lose.

 

[DailySlow]

The AR/VR headset that Apple has been working on for years now is set to launch in 2023, which means rumors about the device are picking up. Bloomberg today shared some information on the headset's interface, the apps it will feature, and how it will work.

Concept render based on purported leaked information by Ian Zelbo​

Apple's mixed reality headset will have an "iOS-like" interface that will include "many functions" available on the iPhone and iPad, with the headset also able to serve as an external display for a connected Mac. Users will be able to see their Mac's display in VR, controlling the Mac with a physical keyboard and trackpad/mouse.

Using the headset will "feel familiar to Apple users," with an interface that is close to identical to the look of an iPhone or an iPad. There will be a Home screen with app icons that can be rearranged, as well as customizable widgets.

Eye and hand-tracking will be a "major selling point" for the headset, with Apple using external cameras that can analyze a user's hands and eyes. According to Bloomberg, the wearer will be able to control the headset by looking at an on-screen item to select it, then using gestures to activate the on-screen item. Unlike other headset options, Apple will not rely on a physical controller.

Likely to be named "Reality Pro," the headset will be able to switch between augmented reality and virtual reality. Augmented reality will overlay virtual objects on the real world, while virtual reality is an entirely virtual environment that shuts out the wearer's surroundings. Augmented reality functions will work through a pass-through mode that will use the exterior cameras on the headset, and swapping between AR and VR will be done with a Digital Crown-like control knob.

Video conferencing will be a focus, with Apple adding FaceTime-based video chatting capabilities and meeting rooms. A user's actual face and full body will be rendered in virtual reality for interactive meetings, with the realistic avatars available for one-to-one chats. FaceTime with multiple participants will use less detailed icons like Memoji.

Apple is working on VR content with media partners like Disney and Dolby, as well as updating Apple TV+ shows and movies to work with the headset. Apple's aim is to provide viewers with the experience of looking at a giant screen in an environment like a desert or outer space.

Apps will include Safari, Photos, Mail, Messages, Apple TV+, Apple Music, Podcasts, and Calendar, and the headset will have a dedicated App Store for third-party content. Siri will be able to be used for text input, as will an iPhone, iPad, or Mac keyboard. Apple is working on an air typing feature, but it is not expected to be ready for launch.

For those who wear glasses, Apple will provide custom lenses that are able to sit within the enclosure, and Apple is expecting users to wear AirPods to get an audio experience on par with the visual experience that the headset provides, though it will have built-in speakers.

As previously rumored, the headset will have an external battery pack to prevent it from overheating on a user's face due to the high-end Mac chips used for the device. The battery is approximately the size of two iPhone 14 Pro Max models stacked on top of one another, and it will power the headset for around two hours. An external battery will allow users to swap one battery and charge another to use the device for a longer period of time.

Other AR/VR rumored headset features include 4K microOLED displays, more than a dozen cameras, iris tracking, facial expression detection, and more, with details available in our roundup. The headset is expected later in 2023, perhaps as soon as the spring, and it will sell for somewhere around $3,000.

To get people interested in the headset, Apple Store locations will have a dedicated area where it can be demonstrated and tested after launch.

Article Link: Apple's Mixed Reality Headset to Feature iOS-Like Interface, Advanced Hand Tracking, and Will Work as Second Display for Mac

This will be historic, without a doubt. Like hieroglyphics, cuneiform, Gutenberg in 1400’s, PostScript, many ideas and innovative modes of communication ideas and genres of art will emerge.

 

[lovehateapple]

The apple store demo area will (probably) be the only place I'll be able to experience these. Hopefully the line isn't too long, 🤞. On second thought, I wonder if apple will offer the same two week return policy on these? 😂😉

 

[newyorksole]

Who is going to wear that thing around their head? No one. You'll have to make them as thick as the thickest available regular glasses manufactured today for them to take off.

See the thing is… everyone is judging this thing based on their own imagination, these rumors and this same old tired render lol.

I 10000% guarantee that when Apple reveals it and markets it… EVERYONE in here will change their tune.

I can’t believe people still doubt Apple.

 

[Mike WA4D]

Well, It sounds cool. It's also unusual, and I can't even begin to speculate how this will sell.

Definitely will be interesting

But I'm getting that ol beta vibe.

It will sell because it’s cheap and an exciting new product.

 

[GlassFingers]

I for one am very excited about this! It could really be something magical. Like the feeling the first time holding an iPad.

 

[klasma]

Apple allow a lot of dating apps on the app store and what happens in those apps Apple has no control over.

Right, but if such an app was a surefire way to obtain desired adult content, and would gain notoriety for that, I’m pretty sure that Apple would shut it down.

I’m also not sure that VR dating apps would become particularly popular, but who knows.

 

[LIVEFRMNYC]

If the UI and software is great, TAKE MY MONEY!!!!

And I’m hoping that it is.

 

[subjonas]

I rather use a normal 2nd display than some AR googles as a 2nd display.

But with AR goggles, you can have 3rd, 4th, 5th, infinite screens and any size you desire* in any arrangement you want without regard to gravity or cables!

*well, limited by hardware power

 

[Chriss_m]

Apple is pretty much the leader in casual games. Those might arguably be better for VR/AR.

I disagree with this. I don’t want to buy a £3000 headset to play Angry Birds, or whatever. I want to play with social games with the rest of the internet.

 

[ghostface147]

See the thing is… everyone is judging this thing based on their own imagination, these rumors and this same old tired render lol.

I 10000% guarantee that when Apple reveals it and markets it… EVERYONE in here will change their tune.

I can’t believe people still doubt Apple.

I doubt this product. The technology isn’t there to make it as seamless as regular prescription glasses.

 

[Thenephi]

The best use case (at this very moment) I can envision that could achieve broad adaptation is partnering with the NFL, NBA, NHL, Premier League, etc., on watching live games from the fan POV. Why pay hundreds of dollars to attend an NFL game when you could pay, say, $5 or $10 to watch the live game from the POV of the 50-yard line, but in the comfort of your living room? You could recharge at halftime and then receive a notification on your Apple Watch or iPhone that it's time to put your headset back on.

Sure, this can also be used for Facetime and video conferencing — but I'm just not sure the corporate world is there yet. Maybe I'm wrong.

I'm guessing someone's already said this, but their 10-year MLS deal is suddenly sounding more interesting.

 

[montuori]

Yeah, iGoggles would be so much better! 😆

iEyes or bust.

 

[Macalway]

Lots of ways to rationalize this purchase.

 

[Jensend]

And I posit most will not get much value in return by having a VR display instead of AR, compared to what they would lose.

But with AR goggles, you can have 3rd, 4th, 5th, infinite screens and any size you desire* in any arrangement you want without regard to gravity or cables!

I feel like we are talking past each other. You clearly see one of the advantages of having a head mounted display (I assume that wasn't sarcastic?). All headsets will give you some ability to view the real world.
I'd simply posit that if your main focus is the digital content, opaque devices are better, and if your primary focus is the analog/real world, transparent devices will probably be a better match for your use case.
UI will be pretty much identical for both devices, but image quality and FOV will be better in the opaque devices for some time to come. But while video passthrough will be fine for seeing a snack on your desk, it probably won't be good enough for you to comfortably read your phone or physical paper.

 

[klasma]

iEyes or bust.

Aye aye, captain!

 

[v0lume4]

People made fun of pretty much all Apple product names when they first came out and now they are just part of our vocabulary. It isn't the best name for search optimization, but I think it sounds fine. Nobody makes fun of the Kindle name, just saying.

Because it is an Apple product, I'm sure there will at least be some friendly making fun of the name. I could see it being compared to the Reality Distortion Field in some way on a late show.

I remember when the name iPad was lampooned. IIRC, I remember people saying it sounded like… erm… a certain feminine product.

 

[Clix Pix]

"Mixed Reality Headset". ?? Two words: NO THANKS!

 

[lovehateapple]

I wouldn't say it's weird; a bit of nostalgia is fun, now-and-again.

I would, however, say that it's highly unlikely.

Future history will be made live from Steve Jobs Theater at Apple Park, Cupertino, CA!

 

[lovehateapple]

I'm sure it will tie in to almost all existing subscription services. I wouldn't be surprised if they launch a new one focused on XR too. I could see attending a live concert or sporting event a service. Especially if Apple manages to make them full light-field experiences (you can move around in them a bit) and not just 360 video. I'd love to see a hologram of a miniature football field on my armrest where I could get a close look at all the plays and rewind or fast-forward while watching the game. In fact size doesn't really matter in XR. You could have a band playing on a stage on your table top or be a towering giant in the stands at a game. Possibly you could have a seat literally in the middle of the field if they can figure out a camera rig or method of digital reconstruction.

I've suggested before that the refs will have to start wearing 360 cameras on their heads, the problem is the cameras will also have to be able to transmit the video signal to be broadcast over the internet.

 

[newyorksole]

I doubt this product. The technology isn’t there to make it as seamless as regular prescription glasses.

yeah? let’s wait and see. I mean… I don’t think this is going to be a perfect and flawless day 1 purchase product, but I believe they’ve worked on it long enough to make it pretty damn seamless.

 

[Abazigal]

Microsoft has laid off entire teams behind Virtual, Mixed Reality, and HoloLens

HoloLens, Virtual Reality, and Mixed Reality are all but dead at Microsoft.

www.windowscentral.com

Interesting that Microsoft is seemingly pulling back from the VR space at the same time when Apple is set to enter it.

 

[jwdsail]

The AR/VR headset that Apple has been working on for years now is set to launch in 2023, which means rumors about the device are picking up. Bloomberg today shared some information on the headset's interface, the apps it will feature, and how it will work.

Concept render based on purported leaked information by Ian Zelbo​

Apple's mixed reality headset will have an "iOS-like" interface that will include "many functions" available on the iPhone and iPad, with the headset also able to serve as an external display for a connected Mac. Users will be able to see their Mac's display in VR, controlling the Mac with a physical keyboard and trackpad/mouse.

Using the headset will "feel familiar to Apple users," with an interface that is close to identical to the look of an iPhone or an iPad. There will be a Home screen with app icons that can be rearranged, as well as customizable widgets.

Eye and hand-tracking will be a "major selling point" for the headset, with Apple using external cameras that can analyze a user's hands and eyes. According to Bloomberg, the wearer will be able to control the headset by looking at an on-screen item to select it, then using gestures to activate the on-screen item. Unlike other headset options, Apple will not rely on a physical controller.

Likely to be named "Reality Pro," the headset will be able to switch between augmented reality and virtual reality. Augmented reality will overlay virtual objects on the real world, while virtual reality is an entirely virtual environment that shuts out the wearer's surroundings. Augmented reality functions will work through a pass-through mode that will use the exterior cameras on the headset, and swapping between AR and VR will be done with a Digital Crown-like control knob.

Video conferencing will be a focus, with Apple adding FaceTime-based video chatting capabilities and meeting rooms. A user's actual face and full body will be rendered in virtual reality for interactive meetings, with the realistic avatars available for one-to-one chats. FaceTime with multiple participants will use less detailed icons like Memoji.

Apple is working on VR content with media partners like Disney and Dolby, as well as updating Apple TV+ shows and movies to work with the headset. Apple's aim is to provide viewers with the experience of looking at a giant screen in an environment like a desert or outer space.

Apps will include Safari, Photos, Mail, Messages, Apple TV+, Apple Music, Podcasts, and Calendar, and the headset will have a dedicated App Store for third-party content. Siri will be able to be used for text input, as will an iPhone, iPad, or Mac keyboard. Apple is working on an air typing feature, but it is not expected to be ready for launch.

For those who wear glasses, Apple will provide custom lenses that are able to sit within the enclosure, and Apple is expecting users to wear AirPods to get an audio experience on par with the visual experience that the headset provides, though it will have built-in speakers.

As previously rumored, the headset will have an external battery pack to prevent it from overheating on a user's face due to the high-end Mac chips used for the device. The battery is approximately the size of two iPhone 14 Pro Max models stacked on top of one another, and it will power the headset for around two hours. An external battery will allow users to swap one battery and charge another to use the device for a longer period of time.

Other AR/VR rumored headset features include 4K microOLED displays, more than a dozen cameras, iris tracking, facial expression detection, and more, with details available in our roundup. The headset is expected later in 2023, perhaps as soon as the spring, and it will sell for somewhere around $3,000.

To get people interested in the headset, Apple Store locations will have a dedicated area where it can be demonstrated and tested after launch.

Article Link: Apple's Mixed Reality Headset to Feature iOS-Like Interface, Advanced Hand Tracking, and Will Work as Second Display for Mac

Man, hopefully the Mac screen is a sign that it will allow use w/ PCs as well?

I mean? $3000 is too high, but they have to be really high themselves if they think I’m paying anything close to that and not going to be able to use it with my sim rig.

 

[JamesHolden]

Right, but if such an app was a surefire way to obtain desired adult content, and would gain notoriety for that, I’m pretty sure that Apple would shut it down.

I don't know about straight dating apps, but the gay "dating" apps are full of porn and adult content. Grindr even lets you post a picture of your butt as your public profile image. I find Apple's rules regarding "adult" content very inconsistent at best.

I’m also not sure that VR dating apps would become particularly popular, but who knows.

Maybe not dating, but virtual sex apps will be hugely popular! You'll be able to have whatever kind of body you want in VR, be as well equipped as you want, etc.

 

[v0lume4]

Microsoft has laid off entire teams behind Virtual, Mixed Reality, and HoloLens

HoloLens, Virtual Reality, and Mixed Reality are all but dead at Microsoft.

www.windowscentral.com

Interesting that Microsoft is seemingly pulling back from the VR space at the same time when Apple is set to enter it.

Interesting indeed… Not to mention the very public financial losses Meta has had recently as they continue to double down on their Metaverse.

Edit - From the end of the article:
“but even Facebook, who rebranded its entire company to Meta in the belief of this technology, is scaling back in this area as well — laying off 11,000 staff back in November.”

That’s new to me. I didn’t realize Facebook had laid off folks in that area too. It increasingly appears that Apple’s placing a big gamble on this one.