xrOS: Everything We Know About the Software Designed for Apple's Mixed Reality Headset

[Love-hate 🍏 relationship]

If that turns out to be the case, I will buy the first gen after the second or third gen comes out.

Well I hope you can . I mean the 14 pros cost 250€ more than the 13 pros… so the obvious choice would be to get a 13 right ? But they aren’t available anywhere . Might also be the case with the headset

Good name you have btw, my brother

 

[anthony13]

As a tool in the professional realm I could see something like AR glasses as very useful for seeing building plans or production layouts overlayed over site visits. But as a consumer device this is just kind of silly to me. We’re all gonna walk around with even more distraction between us and the real world? I sound so old I know but this just seems goofy to me.

 

[cthompson94]

AFAIK iOS is one of the most profitable gaming platforms.

It’s not a platform for AAA games and it’s not targeted at hardcore gamers (in the usual sense), but it makes a **** ton of money for both Apple and publishers.

Correct, I did forget about the mobile market thank you, I guess with the estimated cost of between 1-2k I just cant see this headset taking off even with the mobile games. Even for the PC gaming world the VR headsets and games played are fairly niche.

 

[EdT]

Ultimately I think that the problem is you may be able to achieve visual reality but you can’t really achieve physical reality. If you fall off a cliff in a game it can look completely authentic but your inner ear sense of balance and your visual queues will be out of wack. You either won’t accept it as real or worse, feel queasy because what you see isn’t what your senses, beyond visual, say is happening. And you will need in game “work arounds” to walk or climb or jump because the game can replicate what you should see but it doesn’t replicate physically walking or climbing or jumping. Can gamers get used to this? Yes, they already have to with current generation glasses but some people can get motion sickness because what their body tells them disagrees with their eyes.

 

[lovehateapple]

Is it weird that I would rather be interested in a 16" iPad then AR/VR?

Yes

 

[DeepIn2U]

Apple is developing a mixed reality AR/VR headset that's set to launch in less than a year, if rumors are to be believed. With new hardware comes new software, and Apple is working on a headset operating system that is set to be called realityOS or rOS.

This guide aggregates everything that we know about the realityOS operating system that will run on the upcoming AR/VR headset. We'll update this guide over time as we learn more about realityOS.

rOS

Apple has been working on the software that will run on the AR/VR headset since 2017, and according to Bloomberg's Mark Gurman, Apple will refer to it as realityOS, or rOS. The name will go along with the headset's name, which is expected to be either Reality One or Reality Pro.

realityOS is codenamed "Oak" internally, and references to it have been discovered in App Store logs and Apple source code, confirming the name.

Software Goals

Apple sees the headset as a device for gaming, watching streaming video content, and video conferencing and interfacing with other people, and that will be what Apple focuses on in terms of software. Gurman has described the headset as an "all-encompassing 3D environment" designed for gaming, media consumption, and communication.

FaceTime

FaceTime will be an important communication app for rOS, incorporating existing features like SharePlay and Memojis/Animojis. With SharePlay, headset wearers could watch TV shows and movies together, and use Memoji and Animoji as their virtual avatars. With all of the cameras that the headset is expected to have, it will likely detect facial expressions and work like Animoji and Memoji on the iPhone and iPad.

Messages

Core apps like Messages will be available on the AR/VR headset in addition to FaceTime, though how the Messages app will work isn't known at this time. One rumor has suggested that Apple is creating a whole new version of Messages with chat rooms, support for video clips, a home view, and more.

Maps

The Maps app will be available on the headset, but we don't have additional information on what the Maps app might be capable of. It could perhaps incorporate some of Apple's Look Around and landmark functionality, allowing users to explore cities around the world.

Games

Games are a key feature on the iPhone, iPad, and Apple TV, and the AR/VR headset is also expected to support a range of games. Apple Arcade, for example, could expand to include AR/VR games that are available on the headset.

App Store

The Mac, iPhone, Apple TV, and Apple Watch all have their own dedicated App Stores for downloading apps, and the AR/VR headset is also expected to get its own App Store.

Streaming Video Content

Apple's hiring suggests that Apple is developing a video service for the headset, which will include 3D content that can be played in virtual reality. Apple may also be partnering with third-party services to create content that can be watched in VR.

rOS Launch Date

With the AR/VR headset on track to debut at some point in 2023, likely in the first half of the year, that's also when we expect to see the first version of realityOS. Apple could perhaps introduce realityOS and the headset at the Worldwide Developers Conference in June, but there's also a chance that it could come out before then.

More on Apple's AR/VR Headset

The mixed reality headset that Apple has in the works will have high-resolution displays, more than a dozen cameras, a powerful Mac-level chip, and more, with full details available in our AR/VR headset roundup.

Article Link: RealityOS: Everything We Know About the Software Designed for Apple's Mixed Reality Headset

How is "AR/VR headset on track to debut at some point in 2023" when NOTHING from Apple has ever been stated about it?

It's NOT on track just assumed by rumors.

 

[Zilong]

Wish I could afford this headset

 

[RedWeasel]

Correct, I did forget about the mobile market thank you, I guess with the estimated cost of between 1-2k I just cant see this headset taking off even with the mobile games. Even for the PC gaming world the VR headsets and games played are fairly niche.

Yeah fair point, the cost would need to be much lower to target the same audience that iOS is targeting now. But I would still consider Apple a big player in gaming, even if they are maybe a bit less obvious than say Sony or Nintendo.

 

[EdT]

Yeah fair point, the cost would need to be much lower to target the same audience that iOS is targeting now. But I would still consider Apple a big player in gaming, even if they are maybe a bit less obvious than say Sony or Nintendo.

But Apple is more casual gamers than hard core. They are the play on your phone/iPad with ok graphics and speed not Call of Duty or similar level games that can really shine if you spend $2000 or more on the graphics card alone, and then look at spending hundreds on your input device. Most people don’t want to spend that much. And most Apple Games aren’t that graphically intensive or have add on packs to keep the game going after you finish the initial challenges. They are turn it on, play for 1/2 hour, go get dinner type games not life or death, I must complete this section even if I stay up all night, type.

 

[crsh1976]

Honestly, it sounds like an Apple version of everything not related to gaming we have already seen (seen fail, too) so far, where's the killer app?

Given Apple's ecosystem and how they can leverage it for this sort of device, I imagine it as being a cross between the Apple TV content and games (minus the TV, the goggles are strapped to your face) and the iPhone (Facetime, communications, animojis, etc) - which to me is of no interest, sort of an Apple version of the failed Meta/FB VR experiment.

AR/VR as it stands is for triple-A gaming, everything else makes little to no sense so far - pro applications are just marketing BS so far as well (mostly real estate "VR visits" and tourism).

 

[richinaus]

Honestly, it sounds like an Apple version of everything not related to gaming we have already seen (seen fail, too) so far, where's the killer app?

Given Apple's ecosystem and how they can leverage it for this sort of device, I imagine it as being a cross between the Apple TV content and games (minus the TV, the goggles are strapped to your face) and the iPhone (Facetime, communications, animojis, etc) - which to me is of no interest, sort of an Apple version of the failed Meta/FB VR experiment.

AR/VR as it stands is for triple-A gaming, everything else makes little to no sense so far - pro applications are just marketing BS so far as well (mostly real estate "VR visits" and tourism).

some of us are born with an unlimited imagination and can dream of possibilities from fledgling concepts to what they may become.

Others just see what is in front of them.

 

[PauloSera]

One of the main things I want it to do is replace all my monitors. I want to be able to sit at my desk with only a keyboard and mouse in front of me, no desk space wasted on monitors. I want to put on the goggles and see a huge desktop that spans up and around me (a dome). Then I want the macOS experience, but on this huge virtual desktop. I want to be able to move windows up out of my sight line, but be able to look up and see them again. The desktop would stay pinned to my physical environment, but turning my head would enable me to see around inside the “dome”. But I also want to see “below” the dome to see my physical desktop, my physical keyboard, my physical mouse.

Then I should be able to lean forward and see the window in front of me larger - just like if I leaned in closer to a physical monitor.

And if I display a 3D object it should appear as 3D within my visual space.

Windows should have a bit of depth, but I should be able to move them closer/further away from me. I should be able to overlap them, then be able to lean around to see the window behind the one in front (parallax).

Would I spend $3k for that? Maybe not. But saving desk space, and giving me more desktop, along with other 3D features… maybe I would.

Honestly it sounds like it would be fun for about 15 minutes.

 

[PauloSera]

The $3k price point is just being floated in advance to soften people when the real price ends up being $1k.

 

[crsh1976]

some of us are born with an unlimited imagination and can dream of possibilities from fledgling concepts to what they may become.

Others just see what is in front of them.

That's fine, good for you. That won't sell everything tho, and business doesn't like fancy ideas that don't rake in money.

 

[richinaus]

That's fine, good for you. That won't sell everything tho, and business doesn't like fancy ideas that don't rake in money.

Still don’t get it.

 

[jwdsail]

I think people will say "Reality". "Has Apple updated Reality yet?"

I mean, if Apple could fix reality, like today, that'd be swell.

 

[Unami]

This is quite thin.

I don't believe that there'll be a major focus on gaming. It's hard to imagine something like Half Life:Alyx running on an M2. Also, most VR-games rely on a dedicated controller. Groping around in thin air often won't do the trick or be very immersive. Controllers have a bigger impact on VR gaming than on regular gaming. Sure, there are some games like headmaster that work without one, but most of it had to be written especially with no controller in mind for the apple headset - which won't be an interesting market at it's price. Main competition would be an Occulus Rift for around 400€ or a ps5 with psvr2 (€ 1100). Also, Apple and gaming didn't work that well over the last two to three decades or so (apart from casual gaming on iOS). Sure, there's quite some games in the app store, but apart from ports from other platforms, there is hardly anything that's not in the "casual" category. While most VR games fall in the "easy access/casual" category, there's nothing casual about buying and wearing a headset for gaming. Also, it's quite hard to advertise - until they've tried it, people have usually no clue about or interest in VR. And then there's the problem that some people get motion sick from it - that's decreasing a small target audience even further.

Gaming is a major use case of VR and personally, I'm a fan. But I don't see apple pulling that off.

 

[jwdsail]

PlayStation offers decent “new” AAA games, Apple don’t, and even PlayStation struggles with VR sales.

Apple VR glass will fail…

Yeah, it could?

If Apple supports this device on a variety of platforms, including Windows (for flight sims/racing sims), along with xBox and PS4/5.. in addition to all the Apple - MacOS, iOS, tvOS, rOS features... it'd be an instant buy for many..

If it only works in the Apple ecosystem.. it will be an instant buy for a small number of users in the Apple ecosystem.. and a few holdouts after v2.0 hardware and software is eventually released.. And, possibly be dead in 5 years.

Interoperability would help get the device in more hands faster, and boost native development faster (more developers will be willing to develop for the platform if more people own it/use it)..

But, it's Tim's Apple.. And the matrix he sees in the spreadsheets may show a different story.

So, who knows.

A device that I can connect hardwired or wirelessly to all my gear should be a hit..

A device that only works in Apple-land.. could be a hit.. it could be.. Apple could release a product that wows all of us and makes us think that a $3000 price tag is cheep and the limitations aren't limiting...

I've yet to see Tim wield the Reality Distortion Field effectively enough for that to happen...

But it could? Maybe?

I'm not buying any headset from any company that won't work with all my gear, including my Windows based sim and my Mac, iPhone, and AppleTV, anytime soon? At any price, let alone $3000...

We'll know when we know.

I'm going to pop a lot of popcorn a really enjoy the comment sections when this thing is finally announced regardless?

 

[Unami]

Is Apple working on getting the upcoming new Grand Theft Auto game on Reality at launch?

Nah, GTA VI won't be developed for VR and also probably won't launch before 2025.

 

[jwdsail]

This is quite thin.

I don't believe that there'll be a major focus on gaming. It's hard to imagine something like Half Life:Alyx running on an M2. Also, most VR-games rely on a dedicated controller. Groping around in thin air often won't do the trick or be very immersive. Controllers have a bigger impact on VR gaming than on regular gaming. Sure, there are some games like headmaster that work without one, but most of it had to be written especially with no controller in mind for the apple headset - which won't be an interesting market at it's price. Main competition would be an Occulus Rift for around 400€ or a ps5 with psvr2 (€ 1100). Also, Apple and gaming didn't work that well over the last two to three decades or so (apart from casual gaming on iOS). Sure, there's quite some games in the app store, but apart from ports from other platforms, there is hardly anything that's not in the "casual" category. While most VR games fall in the "easy access/casual" category, there's nothing casual about buying and wearing a headset for gaming. Also, it's quite hard to advertise - until they've tried it, people have usually no clue about or interest in VR. And then there's the problem that some people get motion sick from it - that's decreasing a small target audience even further.

Gaming is a major use case of VR and personally, I'm a fan. But I don't see apple pulling that off.

My most serious attempt to read the tea leaves coming out of the spaceship these past few years is this -

The primary focus of the consumer version of any Apple VR/AR device will be fitness, with a side of entertainment (mostly video, some gaming)

I think that's part of what may be holding up some of the sports streaming/broadcast agreements? Apple wants exclusive partnerships to tie the sports content to the future fitness content, especially future fitness content destined for the consumer version of Apple's VR/AR device.

My best guess.

 

[jwdsail]

Leave it to them to call software “RealityOS” that takes a user completely out of reality.

The killer app will be the Reality Distortion Field...

 

[Wildkraut]

As a tool in the professional realm I could see something like AR glasses as very useful for seeing building plans or production layouts overlayed over site visits. But as a consumer device this is just kind of silly to me. We’re all gonna walk around with even more distraction between us and the real world? I sound so old I know but this just seems goofy to me.

Yeah for professional niche usages it makes sense, but not a restricted Apple VR glass. The Apple platform does not have the software for it, software like Solidworks, CATIA, Pro Engineer, etc.

Well, perhaps they’ll try to sell VR iPhones lol for $1000 🤣

 

[Unami]

My most serious attempt to read the tea leaves coming out of the spaceship these past few years is this -

The primary focus of the consumer version of any Apple VR/AR device will be fitness, with a side of entertainment (mostly video, some gaming)

I think that's part of what may be holding up some of the sports streaming/broadcast agreements? Apple wants exclusive partnerships to tie the sports content to the future fitness content, especially future fitness content destined for the consumer version of Apple's VR/AR device.

My best guess.

Good guess, I completely forgot about fitness. I've played quite a lot of different games on my PSVR, but when it comes to total time, fitness games are probably the ones I played the most. With apple's focus on fitness, that would be a really good fit.

 

[techfreak23]

Games

Games are a key feature on the iPhone, iPad, and Apple TV, and the AR/VR headset is also expected to support a range of games. Apple Arcade, for example, could expand to include AR/VR games that are available on the headset.

I have yet to see anything about controllers for this thing, so it’s really hard to imagine that gaming will be one of the main selling points. Sure, you can get away with hand gestures for a lot of games, but not most, especially shooters. Apple will probably get away with that for the Apple Arcade style games, but those are a joke. Those aren’t anything close to AAA titles. They might get more interest in this thing if they add Steam support out of the box.

This will be competing with the likes of the Quest Pro (which can still be used for gaming) or the HoloLens. It’s not likely going to be heavily marketed to the average consumer. It will be marketed to professionals like they do with the Mac Pro or Mac Studio.

 

[Le0M]

Right?

I mean it’s really bad to wear such a thing for hours anyway. Even a couple of hours and it completely ****s up your face and leaves a deep imprint and the hair around your head gets this strap shape going into it.

Imagine that. You wear this this thing. Then lunch time comes. You go to lunch with these imprints on your face. Everyone feels sorry that you spent $3K to get your face mashed up.

Then go back to work. Again wear this thing. After work you go to the pub or gym or dinner. Everyone looks at these lines on your face and they think ‘He must be one of those VR porn weirdos’.

Well, consider that in a few years for AR we won't be using a headset anymore, but just glasses. That won't ruin our hair or skin. But still, I want a see-through display. That was my point.
I want to see the world in first person.