Apple Outlines How Smart Glasses Could Record Jitter-Free, Gaze-Driven Video

[MacRumors]

Multiple reports suggest that Apple is developing an augmented reality headset as well as smart glasses, and a newly granted Apple patent may provide a clue about one of the potential features that could be included in the latter device.

The U.S. Patent and Trademark Office on Tuesday granted Apple a patent describing "systems and methods for gaze-driven recording of video."

Apple's patent notes that head-mounted displays are used to provide virtual reality, augmented reality, and/or mixed reality experiences for users, and envisions video from such a device being recorded for later playback or analysis. Specifically, it envisions such a system in smart glasses, where the use of built-in gaze-tracking sensors could provide an indication of where a person is currently looking, which could direct a built-in camera to record the scene where the user's eyes are trained, instead of simply recording what's in front of the user.

The patent envisions a system with one or more gaze-tracking sensors, one or more image sensors, and a processing apparatus configured to access the gaze data captured. This apparatus would apply a temporal filter to the gaze data to obtain a "smoothed gaze estimate," which would allow it to determine a region of interest based on the sensors. The system would then apply signal processing to the video based on the region of interest to obtain an enhanced recording that could mitigate the impact of rapid-movement eye saccades.

According to the patent, the system would consist of a pair of smart glasses and a separate connected device for power and storage, which could be interpreted to mean an iPhone, iPad or Mac.

The invention harks back to another patent filed last month in which an eye tracking system is used for detecting the position and movements of a user's eyes in smart glasses and other head-mounted displays.

In the latter patent, Apple envisages the eye tracking system being used to analyze the position and movements of the user's eyes, or to detect other information such as pupil dilation. In one example, identifying the point of the user's gaze allows for interaction with content shown on the near-eye display of the head-mounted display, while another application includes the creation of eye image animations that could be used for digital avatars visible to other users in a mixed-reality communal environment.

Of course, Apple files numerous patent applications every week, and many of these inventions do not see the light of day. But they do provide a glimpse into the kinds of technologies Apple is exploring for potential use in its upcoming AR/VR products, and based on the rumored launch roadmap for its mixed-reality headset and smart glasses, the company still has plenty of time to implement them.

The Information and Bloomberg have both said that Apple is working on smart glasses AND an AR/VR headset, with the headset to come out in first followed by the glasses. Apple analyst Ming-Chi Kuo believes the "mixed reality" headset will come out in 2022, with the Apple Glasses to follow in 2025. The headset is AR/VR, while Apple's glasses are augmented reality.

The headset is rumored to be similar to Facebook's Oculus Quest virtual reality headset, but with a sleeker design that uses fabrics and lightweight materials to ensure the headset is comfortable. JP Morgan believes the headset will look similar to other brands of VR headsets, featuring six lenses and an optical LiDAR scanner to map the wearer's environment. The headset will be targeted at the top end of the consumer market and will be more expensive than others on the market.

The glasses, meanwhile, which are reportedly in an early stage of development, are said to resemble high-end sunglasses with thick frames that house the battery and chips. Kuo expects the AR glasses to be marketed as an iPhone accessory and will primarily take a display role offloading computing, networking, and positioning to the iPhone, with the glasses providing a mobile-first "optical see-through AR experience."

Patently Apple was first to report on the patent being granted today. Apple originally applied for the patent in December 2019.

Article Link: Apple Outlines How Smart Glasses Could Record Jitter-Free, Gaze-Driven Video

 

[Salty Pirate]

such a bad idea.........


Guys gonna get in trouble gazing at dat ass

 

[Corsig]

such a bad idea.........


Guys gonna get in trouble gazing at dat ass

Is there another reason to use them?

 

[PlainviewX]

Is there another reason to use them?

Dem tatas?

 

[lazyrighteye]

Regardless of whether these ever see the light of day, nor the implications it would introduce, the phrase “gaze-driven video” is kinda BA.

 

[x-evil-x]

Make it happen apple. I’m tired of holding something something. I want my media consumption to be as lazy as possible

 

[Corsig]

Dem tatas?

I stand corrected

 

[Btaylor_prod]

Privacy and other issues aside [like distracted driving] I’ll now have another thing to charge every night...

 

[jjudson]

A whole new bunch of glassholes coming soon...

 

[BvizioN]

I’ll now have another thing to charge every night...

I can imagine the furstration.

 

[reyesmac]

I wonder if it will work while driving or going high speed. Seems to me they could get bad publicity if used improperly and it results in death. So I am sure there will be some built in detection of stupidity with plenty of warnings so that they cant be held liable when it told you that you were about to fall off a cliff and you do so anyway. I'm sure the TV movie about someone recording their own murder with these glasses will be out before the actual thing happens in real life.]

Also, how long will it be till someone rents out what they see so you can experience life through their eyes? Once again the adult industry will drive migration to this new technology.

 

[ArPe]

Might be better than Go Pro for extreme sports folks

 

[CarpalMac]

Imagine being on a train/park bench/public place, being potentially filmed from multiple angles.

Imagine being in a restaurant and you're not sure whether the person you are talking to is focussing on you or the monkey sat on your head.

Two examples of so, so much horribleness. People walking around with cameras sitting on their face, it'd be like a surveillance planet. Just, yuck.

 

[CJ Dorschel]

Also, how long will it be till someone rents out what they see so you can experience life through their eyes? Once again the adult industry will drive migration to this new technology.

Reminds me of “Brainstorm” (an interesting concept badly executed - would like to see this concept done well with today’s technology).

Brainstorm (1983) ⭐ 6.4 | Sci-Fi, Thriller

1h 46m | PG

www.imdb.com

 

[1258186]

This feature would land Apple in a whole heap of trouble.

 

[Eso]

Watching that video would be awful, considering how frequently and rapidly our eyes change focus points.

 

[CJ Dorschel]

Apple: ”Let’s discontinue the HomePod instead of updating it cause we’re losing money”

also Apple: “Pour more money into the Apple Car and Glasses!”

Over 1000 posts on the HomePod in a day or two and barely anything on the Apple Car and Glasses. Just sayin🤷🏼‍♂️

 

[subjonas]

This is interesting technology and could be valuable for specific use cases, but I don’t want this to be a public norm. If it is, I’ll wear a mask all the time. The only way I’ll welcome AR glasses in mass adoption is if they only use lidar with no way of identifying specific people. Not sure how I feel about eye tracking. Depends on how useful it will be and how safe and private Apple can assure it to be.

Imagine being on a train/park bench/public place, being potentially filmed from multiple angles.

Imagine being in a restaurant and you're not sure whether the person you are talking to is focussing on you or the monkey sat on your head.

Two examples of so, so much horribleness. People walking around with cameras sitting on their face, it'd be like a surveillance planet. Just, yuck.

Totally agree with the horribleness of potentially being filmed 24/7 in public.
But I don’t care too much if people are always able to look at objects in their glasses. If they don’t know how to control themselves, they will reap the consequences.

 

[Unregistered 4U]

This is interesting technology and could be valuable for specific use cases, but I don’t want this to be a public norm. If it is, I’ll wear a mask all the time. The only way I’ll welcome AR glasses in mass adoption is if they only use lidar with no way of identifying specific people. Not sure how I feel about eye tracking. Depends on how useful it will be and how safe and private Apple can assure it to be.



Totally agree with the horribleness of potentially being filmed 24/7 in public.
But I don’t care too much if people are always able to look at objects in their glasses. If they don’t know how to control themselves, they will reap the consequences.

I still think any camera included version will be limited to industrial use and will look different from the consumer ones for obvious reasons. The consumer ones would use LIDAR and your phone’s GPS/compass to determine location direction and what you’re looking at and, if you want to take a photo, you’d take out your phone.

 

[Jordan Klein]

Watching that video would be awful, considering how frequently and rapidly our eyes change focus points.

The next Blair Witch Project

 

[Arsenikdote]

This is interesting technology and could be valuable for specific use cases, but I don’t want this to be a public norm. If it is, I’ll wear a mask all the time. The only way I’ll welcome AR glasses in mass adoption is if they only use lidar with no way of identifying specific people. Not sure how I feel about eye tracking. Depends on how useful it will be and how safe and private Apple can assure it to be.



Totally agree with the horribleness of potentially being filmed 24/7 in public.
But I don’t care too much if people are always able to look at objects in their glasses. If they don’t know how to control themselves, they will reap the consequences.

At first glance your thoughts generate a reaction in me that makes me worry. But then I take a step back and wonder if I am being paranoid. I mean between traffic cameras, security cameras, phones, action cams, etc.... its not like there is any lack of way to film people. In fact the new Insta360 is something none of us would even notice if we were walking down a sidewalk and someone had it on. For generations people who saw some new tech as invasive have predicted all of the harm it would do to everyone. While they might be totally correct, it doesn't stop people from still using it and making it the "new norm". It was said about cell phones, it was said about smartphones and now it is being said about potential glasses. I feel like this would likely be the same thing, people might be worried for good or bad reasons but likely it won't stop mass adoption and it changing the world we live in.

 

[subjonas]

At first glance your thoughts generate a reaction in me that makes me worry. But then I take a step back and wonder if I am being paranoid. I mean between traffic cameras, security cameras, phones, action cams, etc.... its not like there is any lack of way to film people. In fact the new Insta360 is something none of us would even notice if we were walking down a sidewalk and someone had it on. For generations people who saw some new tech as invasive have predicted all of the harm it would do to everyone. While they might be totally correct, it doesn't stop people from still using it and making it the "new norm". It was said about cell phones, it was said about smartphones and now it is being said about potential glasses. I feel like this would likely be the same thing, people might be worried for good or bad reasons but likely it won't stop mass adoption and it changing the world we live in.

Oh I make no predictions about whether cameras on glasses will become the norm. It may or may not happen, I have no idea. And I don’t speak for anyone else. All I’m saying is what I want, which is for that not to happen. I don’t particularly like being recorded on security cams either, but they’re necessary and there’s a huge difference between, depending on where you go, occasionally being caught on various security cameras or dash cams at a distance which will probably never be viewed by anyone (those other cameras you mentioned are usually only used for specific recording purposes, not constant recording just walking down the street every day) -versus- every stranger you encounter and in your vicinity potentially recording you constantly every day, possibly right up to your face, which would be inescapable. Not sure what you mean by “harm” in reference to cell/smart phones, but to me this is just about privacy. People who don’t care about privacy probably won’t care about this, but to me it’s important. It’s why I choose to not be active on social media. Cameras on every face will mean the only choice I’ll have is whether or not to go out in public, or whether or not to wear a mask in public. Not going out in public is not an option which is why I say I’d wear a mask.

 

[macsareveryinteresting]

After about a couple of months of the Apple Glasses selling on Apple Store. Amazon will steal a bunch of their revenue and then do discounts to piss people off.

 

[Arsenikdote]

Oh I make no predictions about whether cameras on glasses will become the norm. It may or may not happen, I have no idea. And I don’t speak for anyone else. All I’m saying is what I want, which is for that not to happen. I don’t particularly like being recorded on security cams either, but they’re necessary and there’s a huge difference between, depending on where you go, occasionally being caught on various security cameras or dash cams at a distance which will probably never be viewed by anyone (those other cameras you mentioned are usually only used for specific recording purposes, not constant recording just walking down the street every day) -versus- every stranger you encounter and in your vicinity potentially recording you constantly every day, possibly right up to your face, which would be inescapable. Not sure what you mean by “harm” in reference to cell/smart phones, but to me this is just about privacy. People who don’t care about privacy probably won’t care about this, but to me it’s important. It’s why I choose to not be active on social media. Cameras on every face will mean the only choice I’ll have is whether or not to go out in public, or whether or not to wear a mask in public. Not going out in public is not an option which is why I say I’d wear a mask.

My main point is there is always "big brother/big corp" concerns about new technology being given to the masses. With cell phones it was they could track you and listen in anywhere. It was only heightened with smartphones. While people are free to make whatever choices they would like, it is interesting to me what people say vs what becomes reality. I knew quite a few people that spoke out against cell phones and yet a decade later every one of them uses cell phones. Not saying this applies to you, just me making an observation about how initial reactions shift over time, not because the underlying concerns are no longer valid, but because the technology is more ubiquitous.

 

[subjonas]

My main point is there is always "big brother/big corp" concerns about new technology being given to the masses. With cell phones it was they could track you and listen in anywhere. It was only heightened with smartphones. While people are free to make whatever choices they would like, it is interesting to me what people say vs what becomes reality. I knew quite a few people that spoke out against cell phones and yet a decade later every one of them uses cell phones. Not saying this applies to you, just me making an observation about how initial reactions shift over time, not because the underlying concerns are no longer valid, but because the technology is more ubiquitous.

I don’t think I mentioned anything alluding to government/corp surveillance. My concern is in general I don’t want recordings of me possibly being in the possession of every person I encounter. (By the way, before cell/smart phones were a thing I didn’t hear the concerns about surveillance you mention. If anything, I’m only hearing those concerns recently.) But your point is really just that it’s possible for a society to eventually accept anything, whether good or bad, probably yeah. I guess the Holocaust is proof of that...