xrOS: Everything We Know About the Software Designed for Apple's Mixed Reality Headset

[RedWeasel]

Unless it really is for gaming (which Apple doesn't have a good track record catering towards gamers) I don't see how this is going to sell.

AFAIK iOS is one of the most profitable gaming platforms.

It’s not a platform for AAA games and it’s not targeted at hardcore gamers (in the usual sense), but it makes a **** ton of money for both Apple and publishers.

 

[Le0M]

I do think FaceTime calls will play a part in this tech, but I don’t see a problem with it — and I think it could actually make video calls more enjoyable. Right now you’ve got different camera quality, backgrounds, lighting, framing — lots of distractions. I really despise video calls now for these reasons. But I think this is just a tiny piece of the use of this device.

Yeah well, as long as there is no extra distractions, I'm ok with conferencing; I mean, the potential there is good. But I was actually referring to cartoon-like avatars, those so-called animojis. Can't stand them. Just make a 3D print of my face, then we can talk about avatars

 

[enterthemerdaverse]

Remember when being a taxi driver used to be a job you needed to train and get a license for?

Then came navigation apps and all that expertise of knowing every single street in the city became worthless overnight.

You have to train and get tested for your driving license )

Taxi drivers then have to pass vetting and pay for a taxi license.

 

[enterthemerdaverse]

Yeah well, as long as there is no extra distractions, I'm ok with conferencing; I mean, the potential there is good. But I was actually referring to cartoon-like avatars, those so-called animojis. Can't stand them. Just make a 3D print of my face, then we can talk about avatars

Or just use your webcam and forget about avatars.

 

[enterthemerdaverse]

Will they be able to create a *killer app* – that is the question.

(and Animojis sure ain't a part of it).

The only killer app I can think of, and I’ve tried to imagine everything, is an AR version of Battle Chess or Monopoly. Animated board games. But whatever, I can do that on a monitor anyway and don’t need to spend 2-3 grand on ski goggles.

The problem if a lot of the AR use cases people post don’t appear to come from domain expertise point of view.

 

[User 6502]

Hopefully, Apple releases the AR/VR headset safely in 2023. Don't really want to wait till 2024. In addition, I just hope the reality OS is free of bugs.

Apple manages to release extremely buggy software even when it’s about well established and mature products (see iOS / iPadOS ). This reality thing, if it ever sees the light of day, will be an unusable mess for years.

 

[Kottu]

Hopefully, Apple releases the AR/VR headset safely in 2023. Don't really want to wait till 2024. In addition, I just hope the reality OS is free of bugs.

There's gonna be a lot of bugs. The first one will be called Apple price index and gonna bite you when you purchase one.

 

[everlast3434]

Everything we know… not much at all.

 

[enterthemerdaverse]

May I just point out that first you imagine the unknown device's limitations, then you respond to your imagined limitations as if they are real and you didn't just make them up 😂

That post you replied to is not just about the device specifically. It was about physics, which I studied in college along with chemistry. It applies to every device that tries to implement AR. If you want the kind of visual fidelity, sharpness, color depth, and contrast in AR that you have on a monitor then you have to either violate the laws of physics or create a device that has to be used in extreme dark environment. Pitch black. Even then it is extremely difficult to pull off.

 

[asparagus]

I have very little interest in goggles, but would buy something like google glasses in a heartbeat, if it came from Apple.

 

[Neodym]

Really? at the rumored $2k or even $3k they better come up with something more compelling

I would not put too much emphasis on the cost for a first-gen product of a new product line. Interesting will be the development in terms of e.g. size and price in the following generations.

Still saying: a technology in search of a useful consumer application ...

That forum post is from 2019, but imho still valid: Link

That is already existing for training.

Except after training the professionals and experts already know where body parts and mechanical parts are supposed to be. It’s their field of expertise. So they don’t need to keep looking it up on a computer or manual or headset.

It’s all about allowing _untrained_ personnel to replace trained professionals for cost and/or availability reasons.

AR can’t have the image fidelity of a monitor.

With VR glasses you can have a virtual monitor. Not recommended if you need to use a real keyboard or interact with stuff on your desk though.

With AR glasses, the fidelity wouldn’t be there. As some color experts and video editors have mentioned also in older threads, you can’t do color accuracy or true blacks with AR. Ambient light is passing through the AR graphical objects/interface. You can have AR monitors but they would have a slightly ghostly or transparent appearance. I don’t know of any science that can prevent that unless there is a layer on the glasses that can block ambient in real time at a pixel by pixel level.

Wrong approach. AR does _not_ necessarily require see-through glasses. Just takes two (or more) cameras on a set of VR glasses to permanently live-stream reality in front of you and enhance the stream with GUI elements. And suddenly you have the functionality of AR glasses combined with the fidelity of VR glasses. With additional cameras you could even add some kind of rear-view mirror blending into the picture.

Honestly, I think it will have a see-through screen for AR, but it can also turn into a VR headset by blackening the entire screen.

Why blackening it, when you can design it pitch black from the beginning (see my comment above)?!

This is where hype and fantasy meets reality.

Why do you need a headset to find your car when you should be smart enough to know where your car is?

You obviously never parked your car in a foreign town and went to a festival. Even without getting drunk and with otherwise excellent orientation, it may be difficult to get back to the car straight away. Apple Maps already has the function to mark the parking position of your car for a reason.

Why do I need directions to go home or to work or to a shop when you should be smart enough to know where those places are or just use a map app?

When you use a map app anyway, AR glasses would offer more comfort, as you would not need to keep your smartphone in your hand all of the way.

Streets will always have names on them. Streets will always have direction signs and speed limit signs. It’s illegal to remove them.

You may want to leave your bubble and get around the world a bit. There are lots of areas where streets _don’t_ have names (even in Tokyo, to name a familiar big city) and houses don’t have numbers.

Then we have the problem of battery life. The newest untethered Oculus has a 3 hour battery life in best circumstances. So realistically 2.5 hours. But that’s only when the device is new. After a year the battery life is less. After 2 years the battery life is much less.

Until battery tech has improved significantly, one can simply use a tethered version and offload the heavy lifting to e.g. an iPhone. And then there are still power banks …

It’s unhealthy to wear a headset for too long,

In what way? Any studies on that? Or is it just the reflex of “it’s new and we don’t know it, so it must be dangerous”?

but we can’t anyway because the battery will never allow it, and tethered use is annoying,

You can already tether wirelessly with (comparably) low energy consumption.

and then half the people in the world are visually impaired and need glasses, and then there is the cost which most people will scoff at.

Good varifocal glasses can easily set you back up to a grand, so the upsell to $2k glasses with significantly enhanced functionality suddenly looks way smaller than at first glance.

Then after several RealityOS updates the device will run slower just like all system updates do. Newest Ventura and Safari are already jerky and buggy on the fastest M1 Macs. Do we want that kind of bugs and slow downs stuck right to our head movement?

There are many such questions that cut through the hype.

No sane person expects a completely new technology to work perfectly from day one. Development is always a journey and most pioneers are well aware of that.

 

[Kierkegaarden]

Yeah well, as long as there is no extra distractions, I'm ok with conferencing; I mean, the potential there is good. But I was actually referring to cartoon-like avatars, those so-called animojis. Can't stand them. Just make a 3D print of my face, then we can talk about avatars

I can’t stand Animojis either, and have never understood the point — or maybe I’m just way outside the demographic. More customizations on the human faces would be neat though, unless they will try to model an individual face more closely.

 

[wanha]

That post you replied to is not just about the device specifically. It was about physics, which I studied in college along with chemistry. It applies to every device that tries to implement AR. If you want the kind of visual fidelity, sharpness, color depth, and contrast in AR that you have on a monitor then you have to either violate the laws of physics or create a device that has to be used in extreme dark environment. Pitch black. Even then it is extremely difficult to pull off.

Fair enough, and I am somewhat skeptical myself as VR and 3D have never managed to go mainstream outside a few niche applications.

I just think we might want to wait until we see what Apple has produced and what the actual use case is before we make a judgement either way.

I could have done it with less snark, apologies for that.

 

[jmgregory1]

Yeah well, as long as there is no extra distractions, I'm ok with conferencing; I mean, the potential there is good. But I was actually referring to cartoon-like avatars, those so-called animojis. Can't stand them. Just make a 3D print of my face, then we can talk about avatars

Totally agree in regards to FaceTime and how if rOS is going to be popular, it better NOT be generating a stupid animoji representation of your face, though I expect that is exactly what it will be. The thing with a headset version of FaceTime, compared to separate device version (iPhone, iPad or Mac) is that a headset perspective of yourself is only your eyes, vs your head or head and torso which is closer to how we experience face to face communication.

So then rOS has to use computing power to generate a digital version of your head (including getting rid of the headset/glasses you’ll be wearing) instead of using what already exists? That seems like a waste of time and energy.

 

[enterthemerdaverse]

IYou may want to leave your bubble and get around the world a bit.

Before you insult people online, not everyone lives in a “bubble”. I have dual citizenship and grew up in 4 countries. The issue you are having is that if someone doesn’t have the same fantasies as you, it must mean they are in a bubble, or stupid, or lack vision.

There are lots of areas where streets _don’t_ have names (even in Tokyo, to name a familiar big city) and houses don’t have numbers.

If a city doesn’t have street name signs and numbers on houses that doesn’t mean AR is a solution. It means they have either a culture that has chosen privacy, or the regulations are not in place, or it can happen in places that are behind developmentally.

Until battery tech has improved significantly, one can simply use a tethered version and offload the heavy lifting to e.g. an iPhone. And then there are still power banks …

There’s a limit to the battery capacity one can install in a wearable device and improvement is not unlimited. Tethered usage is annoying and uncomfortable. Why would someone pay through the nose for an uncomfortable experience when the monitor and computer they already have has maximised their productivity and it is impossible to work any harder or play games/movies any better?

Maybe you are young, have money flowing from your pockets, like to buy any gadget, have a vision of the future full of science fiction coming to life. Most people aren’t like that.

Wrong approach. AR does _not_ necessarily require see-through glasses. Just takes two (or more) cameras on a set of VR glasses to permanently live-stream reality in front of you and enhance the stream with GUI elements. And suddenly you have the functionality of AR glasses combined with the fidelity of VR glasses. With additional cameras you could even add some kind of rear-view mirror blending into the picture.

You are describing an over complicated Rube Goldberg headset.

Good varifocal glasses can easily set you back up to a grand, so the upsell to $2k glasses with significantly enhanced functionality suddenly looks way smaller than at first glance.

People don’t have money trees growing out of their pockets. Inflation is running high, there is economic and political volatility happening everywhere. People are growing tired of big promises from tech companies who will always under deliver in many aspects.

I can’t even get smooth scrolling on Safari anymore with the latest Ventura update. $5000 dollars on latest Mac hardware and this is the crap we get.

 

[SnappleRumors]

Hopefully, Apple releases the AR/VR headset safely in 2023.

Dateline: January 1, 2025
Cupertino, California

Citing standard COVID and supply chain excuses Apple is rumored to release their AR/VR headset as soon as 2027 with rOS rolling out as a separate deliverable in late 2028. As expected lines are already forming outside Apple Stores across America as anxious Apple fans want to the first to get their hands on the device.

 

[SnappleRumors]

I want to be able to sit at my desk with only a keyboard and mouse in front of me, no desk space wasted on monitors.

I doubt we will see that in our lifetime.

Why would you even leave your bed? Our great grandchildren will be tadpoles with arms.

 

[riverfreak]

As to the Animoji/Memoji debate:

How would cameras mounted in a glasses or goggles frame be able to create the visual framework of your face on which to map the avatar? Wouldn’t an external camera also be required?

 

[hugodrax]

I think it's meant to be used at home or during meetings. It would look so silly for people wearing VR/AR headsets and walking around freely.

i do think that will be the future though, people wearing AR goggles all day everywhere.

 

[Le0M]

I can’t stand Animojis either, and have never understood the point — or maybe I’m just way outside the demographic. More customizations on the human faces would be neat though, unless they will try to model an individual face more closely.

With so many camera in the inner part of the headset, as rumoured, Apple could simply create a software to recreate a live 3D version of our face for other conference participants to see. No need for avatars

 

[Le0M]

I would not put too much emphasis on the cost for a first-gen product of a new product line. Interesting will be the development in terms of e.g. size and price in the following generations.


That forum post is from 2019, but imho still valid: Link


It’s all about allowing _untrained_ personnel to replace trained professionals for cost and/or availability reasons.



Wrong approach. AR does _not_ necessarily require see-through glasses. Just takes two (or more) cameras on a set of VR glasses to permanently live-stream reality in front of you and enhance the stream with GUI elements. And suddenly you have the functionality of AR glasses combined with the fidelity of VR glasses. With additional cameras you could even add some kind of rear-view mirror blending into the picture.


Why blackening it, when you can design it pitch black from the beginning (see my comment above)?!


You obviously never parked your car in a foreign town and went to a festival. Even without getting drunk and with otherwise excellent orientation, it may be difficult to get back to the car straight away. Apple Maps already has the function to mark the parking position of your car for a reason.


When you use a map app anyway, AR glasses would offer more comfort, as you would not need to keep your smartphone in your hand all of the way.


You may want to leave your bubble and get around the world a bit. There are lots of areas where streets _don’t_ have names (even in Tokyo, to name a familiar big city) and houses don’t have numbers.


Until battery tech has improved significantly, one can simply use a tethered version and offload the heavy lifting to e.g. an iPhone. And then there are still power banks …


In what way? Any studies on that? Or is it just the reflex of “it’s new and we don’t know it, so it must be dangerous”?


You can already tether wirelessly with (comparably) low energy consumption.


Good varifocal glasses can easily set you back up to a grand, so the upsell to $2k glasses with significantly enhanced functionality suddenly looks way smaller than at first glance.


No sane person expects a completely new technology to work perfectly from day one. Development is always a journey and most pioneers are well aware of that.

You cannot compare a see-through screen with a regular one. I mean, you can, but it's day and night.
Why would I need a camera and a screen to show me (in a lower resolution) what's already there??
I think it would be plain stupid.
Say we actually end up wearing that thing for 10 hours a day: then for 10 hours a day you won't actually see the world directly with your eyes, as it is, but just a representation of it through cameras. Just the idea gives me goosebumps, and not the good ones. Scary.

 

[enterthemerdaverse]

You cannot compare a see-through screen with a regular one. I mean, you can, but it's day and night.
Why would I need a camera and a screen to show me (in a lower resolution) what's already there??
I think it would be plain stupid.
Say we actually end up wearing that thing for 10 hours a day: then for 10 hours a day you won't actually see the world directly with your eyes, as it is, but just a representation of it through cameras. Just the idea gives me goosebumps, and not the good ones. Scary.

Right?

I mean it’s really bad to wear such a thing for hours anyway. Even a couple of hours and it completely ****s up your face and leaves a deep imprint and the hair around your head gets this strap shape going into it.

Imagine that. You wear this this thing. Then lunch time comes. You go to lunch with these imprints on your face. Everyone feels sorry that you spent $3K to get your face mashed up.

Then go back to work. Again wear this thing. After work you go to the pub or gym or dinner. Everyone looks at these lines on your face and they think ‘He must be one of those VR porn weirdos’.

 

[robertmorris2]

I could see this useful in robotic surgery in the field and hospital based.

 

[enterthemerdaverse]

I could see this useful in robotic surgery in the field and hospital based.

Already exists and the proprietary systems are designed to be super stable, use very little CPU/battery and do one thing in a completely optimized way.

 

[aloysiusfreeman]

realityOS sounds like a cyberpunk concept, I sort of love it