Apple's AR Team Includes Talent From Lucasfilm and Weta Digital, Smart Glasses Still 'A Ways Off'

[MacRumors]

It's been known that Apple has people working on augmented reality initiatives for future devices, which range from the technology's inclusion in a future version of the iPhone to separate "mixed reality" glasses. Today, a Bloomberg report has gathered all of the speculation surrounding Apple and AR together, while also providing some insight into a few lesser-known areas of Apple's AR project.

Apple's augmented reality team is said to combine "the strengths of its hardware and software veterans," along with new additions within the company, according to people familiar with Apple's plans. The team is run by Mike Rockwell, who came from Dolby, and also consists of Yury Petrov (formerly of Oculus), Avi Barzeev (formerly of HoloLens), Cody White (formerly of Amazon's VR project "Lumberyard"), Tomlinson Holman (formerly of Lucasfilm), and more.

A concept image of what AR on future iPhones could look like.​

The total scope of Apple's AR team is rounded out by many camera and optical lens engineers, as well as "people with experience in sourcing the raw materials for the glasses." Apple has even included talent from 3D animation company Weta Digital, which worked on films like Avatar and The Lord of the Rings. This team of individuals, along with AR advocate Tim Cook, see the new technology as a way for Apple "to dominate the next generation of gadgetry and keep people wedded to its ecosystem."

Apple has built a team combining the strengths of its hardware and software veterans with the expertise of talented outsiders, say the people, who requested anonymity to discuss internal strategy. Run by a former Dolby Laboratories executive, the group includes engineers who worked on the Oculus and HoloLens virtual reality headsets sold by Facebook and Microsoft as well as digital-effects wizards from Hollywood. Apple has also acquired several small firms with knowledge of AR hardware, 3D gaming and virtual reality software.

The separate glasses are said to still be "a ways off," but AR integrated iPhone devices could show up much sooner, said the inside sources. Bloomberg compared the AR glasses to the Apple Watch, mentioning that the technology would come with its own OS and likely be tethered to an iPhone to send images and content to the user while consuming a lot of power, forcing Apple to find a battery life solution that would fit in the wearable's small frame. In addition to battery problems, Apple will have to find a way to convince users to wear the glasses in the first place.

Explanations regarding the usefulness of AR on an iPhone remain scarce, but some feature details were also provided by Bloomberg today. The camera-specific abilities include ways for users to change the depth of an entire photo, or the depth of a specific object in the photo. In the past, Apple has cited interest in such technology, filing a patent for a digital camera with a refocusable imaging mode adapter that could be included in an iPhone. Former CEO Steve Jobs even met with a company, Lytro, who created the first light field camera.

Hundreds of engineers are now devoted to the cause, including some on the iPhone camera team who are working on AR-related features for the iPhone, according to one of the people. One of the features Apple is exploring is the ability to take a picture and then change the depth of the photograph or the depth of specific objects in the picture later; another would isolate an object in the image, such as a person's head, and allow it to be tilted 180 degrees. A different feature in development would use augmented reality to place virtual effects and objects on a person, much the way Snapchat works.

Apple is believed to be working on virtual reality technology, in addition to its interest in augmented reality, but with the success of apps like Pokémon Go the rumor cycle has taken to suggest that the company is betting more on AR. It's still unclear when a product including either piece of technology might launch. Recent concept images of the "iPhone 8" have taken a crack at visualizing AR features on an Apple smartphone, baking in "enhanced Siri" abilities and augmented reality directly into the user interface.

Although many companies continue to invest time and money into both AR and VR, data collected by a number of market research firms late last year suggested that sales for such devices were weakening amid consumer apathy, grown out of a lack of interesting content and expensive prices.

Article Link: Apple's AR Team Includes Talent From Lucasfilm and Weta Digital, Smart Glasses Still 'A Ways Off'

 

[Santabean2000]

AR will be massive. Just not any time soon, it seems...

 

[tkukoc]

Just don't see a reason for AR, but to each is own. To me it seems like another big hyped addition that will fall flat with little to no updates. 3D Touch being the current front runner in that category.

 

[themick4u]

This looks enticing enough to hang on and see where they take this....

 

[Lapidus]

Just don't see a reason for AR, but to each is own. To me it seems like another big hyped addition that will fall flat with little to no updates. 3D Touch being the current front runner in that category.

I love 3D touch and miss it all the time when using my iPad. Especially to change cursor / select text using 3D touch on the keyboard.

 

[citysnaps]

Makes a lot of sense...

Especially considering Apple is in full control of their Ax processors (and not reliant on outside vendors), and that they can be specifically tailored to support AR applications on a handheld device. Mobile is where AR is at.

I see a lot of exciting opportunities in the future and huge dividends for Apple.

 

[MrGimper]

Looking forward to Apple being 4th in a 4 horse race in this field, the same as they are with Siri vs the others.

 

[Sasparilla]

This is the kind of thing Apple (at least in the past) is great at. Keep working at a technology and eventually release it when its ready. A pair of glasses linked up to your iPhone, for the processing power, for enhanced situational awareness with privacy - only Apple will give us that.

Looking forward to it when the tech gets there.

 

[drewyboy]

The problem with AR is they don't even know if people will want or accept it. So many things these days get hyped up then crash and burn or fall by the wayside after introduction simply because people don't want it and I'm leaning towards AR being just that. An additional feature that simply isn't needed.

 

[nwcs]

And Lytro also failed as a camera company and re-tooled. I still don't see the actual user problems solved by AR outside of a few narrow applications. The gulf between technology believers and the real world grows a bit wider.

 

[tkukoc]

I love 3D touch and miss it all the time when using my iPad. Especially to change cursor / select text using 3D touch on the keyboard.

Me too, I just mean there's so much more they could allow to enhance it. Sadly they don't really like the options of customization it can do when using xcode so they reject app updates I've done with it. I'm still holding out that they do allow for more enhancements to take place in IOS itself as well. Would be really nice to press on the wifi button to get a list of information or press the battery.. headphones.. alarms.. there's so much they could do it with it.

 

[smallcoffee]

Just don't see a reason for AR, but to each is own. To me it seems like another big hyped addition that will fall flat with little to no updates. 3D Touch being the current front runner in that category.

You don't see any reason for AR? Imagine walking through the streets of Rome, pulling up your iPhone, pointing it at some random ruin, and then getting a full history of that ruin, it's name, and maybe checking it off some "to go see" list.

That's just a quick, contrived example. The possibilities are endless.

 

[NT1440]

Me too, I just mean there's so much more they could allow to enhance it. Sadly they don't really like the options of customization it can do when using xcode so they reject app updates I've done with it. I'm still holding out that they do allow for more enhancements to take place in IOS itself as well. Would be really nice to press on the wifi button to get a list of information or press the battery.. headphones.. alarms.. there's so much they could do it with it.

I think, true to Apple fashion, we'll see expanded features on things like 3D touch at each WWDC going forward. They don't leap ahead like some people want, but Apple tends to expand on their major offerings once a big enough install base is in place.

 

[nwcs]

You don't see any reason for AR? Imagine walking through the streets of Rome, pulling up your iPhone, pointing it at some random ruin, and then getting a full history of that ruin, it's name, and maybe checking it off some "to go see" list.

That's just a quick, contrived example. The possibilities are endless.

That isn't really a user problem. That's more of a parlor trick of convenience. There are some limited applications of AR that are genuinely useful that solves specific user problems but, for the most part, it's a parlor trick with limited practical use.
[doublepost=1490025161][/doublepost]

I think, true to Apple fashion, we'll see expanded features on things like 3D touch at each WWDC going forward. They don't leap ahead like some people want, but Apple tends to expand on their major offerings once a big enough install base is in place.

Thing is, a context menu doesn't really need to be slowly walked. Context menus have been around for a very long time and there's not much reason to have it so limited out of the gate.

 

[NT1440]

That isn't really a user problem. That's more of a parlor trick of convenience. There are some limited applications of AR that are genuinely useful that solves specific user problems but, for the most part, it's a parlor trick with limited practical use.

...It's a platform. Just like when the app store came out and there will thousands of "level" apps taking advantage of the accelerometer there will be "parlor tricks" at first, but if you can't seem to figure out how AR could redefine how we interact with the real world that's a lack of imagination on your part.

How about when I download the VW AR app point it at my engine and learn in real time layovers how to fix X or Y?

Maybe you work in an archive and you want to find an obscure reference material, take out your phone open the app and watch as it guides to the exact location of what you're looking for.

There is limitless potential as far as use cases go for overlaying data directly onto the real world, to say that's a parlor trick is to purposely blind yourself to how big this field is going to be in every aspect of life just a decade from now. Will we see downright stupid uses of it? Of course, look at the gaming industry. But it does detract from the power of that kind of platform.
[doublepost=1490025242][/doublepost]

Thing is, a context menu doesn't really need to be slowly walked. Context menus have been around for a very long time and there's not much reason to have it so limited out of the gate.

Really? No reason at all huh.

Think about how many people even know how to use 3D touch now. One way to kill a feature is to make it overwhelming to use at launch. More than just contextual menus is coming to 3D touch.

 

[citysnaps]

That isn't really a user problem. That's more of a parlor trick of convenience. There are some limited applications of AR that are genuinely useful that solves specific user problems but, for the most part, it's a parlor trick with limited practical use.
[doublepost=1490025161][/doublepost]
Thing is, a context menu doesn't really need to be slowly walked. Context menus have been around for a very long time and there's not much reason to have it so limited out of the gate.

Seeing the range of potential AR applications as "parlor tricks" speaks to one's level of imagination and curiosity.

 

[nwcs]

...It's a platform. Just like when the app store came out and there will thousands of "level" apps taking advantage of the accelerometer there will be "parlor tricks" at first, but if you can't seem to figure out how AR could redefine how we interact with the real world that's a lack of imagination on your part.

How about when I download the VW AR app point it at my engine and learn in real time layovers how to fix X or Y?

Maybe you work in an archive and you want to find an obscure reference material, take out your phone open the app and watch as it guides to the exact location of what you're looking for.

There is limitless potential as far as use cases go for overlaying data directly onto the real world, to say that's a parlor trick is to purposely blind yourself to how big this field is going to be in every aspect of life just a decade from now. Will we see downright stupid uses of it? Of course, look at the gaming industry. But it does detract from the power of that kind of platform.
[doublepost=1490025242][/doublepost]
Really? No reason at all huh.

Think about how many people even know how to use 3D touch now. One way to kill a feature is to make it overwhelming to use at launch. More than just contextual menus is coming to 3D touch.

Again, these are the same user stories that keep being trotted out yet they haven't really addressed a systemic user problem. "limitless potential?" I've been in software development for decades. So many times that term gets pulled out to justify something when in the end the market decides something totally different and unexpected.
[doublepost=1490025676][/doublepost]

Seeing the range of potential AR applications as "parlor tricks" speaks to one's level of imagination and curiosity.

Not really. It speaks to experience and realism. On the one side are the wide-eyed dreamers who see, with their hammer, nails everywhere. On the other side are those realists who say "show me something meaningful, practical, that really solves a problem for me."

 

[ApfelKuchen]

To me, AR means, "integrate useful data with a live scene." Nowhere may this be more compelling than in CarPlay. It's easy to envision CarPlay taking advantage of a vehicle's growing array of cameras and proximity sensors, to deliver enhanced visualizations on today's large dash-mounted LCD displays and tomorrow's heads-up displays.

And while it's not solely related to AR... Though Apple has so far taken a different approach to Street View than Google, once there are fleets of private, camera-equipped vehicles capable of contributing anonymized image data to Apple, automatic delivery of augmented "preview" images of upcoming waypoints, including accurate simulations of current traffic, would seem to be quite possible. Seems awfully compelling to me.

 

[NT1440]

Again, these are the same user stories that keep being trotted out yet they haven't really addressed a systemic user problem. "limitless potential?" I've been in software development for decades. So many times that term gets pulled out to justify something when in the end the market decides something totally different and unexpected.

That's an entirely different argument and you know it.

 

[polterbyte]

This technology is going the way I want it to go: towards natural interaction with the environment. The less I need to deviate my attention from my surroundings, the better. Looking at a screen is fantastic, of course, especially one as non-intrusive as a smartphone's. But ideally I would not need to look elsewhere, having the content as part of the scene.

I, for one, hope they can pull this off. And I expect nothing less than a pair of very slick glasses, or possibly a pair of contacts. (Yep, I'm quite aware I'm dreaming way too high on this last one).

 

[nwcs]

That's an entirely different argument and you know it.

Yes, the market deciding something different and unexpected is a different thing. It wasn't part of my argument but rather an acknowledgement that the future will be different than what we expect. Technologists have predicted the future with a very poor accuracy rate. Thus my comment about how many times I have read the term "limitless potential" and the like to justify some product or idea as inevitable or pervasive. Yet they don't seem to pan out the way they thought if they even pan out at all.

 

[citysnaps]

Not really. It speaks to experience and realism. On the one side are the wide-eyed dreamers who see, with their hammer, nails everywhere. On the other side are those realists who say "show me something meaningful, practical, that really solves a problem for me."

Yup. And if one's imagination and curiosity are stunted, for whatever reason, there will not be much progress. I see lots of exciting and very useful applications ahead.

 

[nwcs]

The future may be like this with AR if it takes off as people suspect. Everyone looking at their phone but one person who actually experiences the event:

[doublepost=1490026182][/doublepost]

Yup. And if one's imagination and curiosity are stunted, for whatever reason, there will not be much progress. I see lots of exciting applications ahead.

OK, one cannot debate about a theory of the future with any real credibility. We'll see what happens.

 

[brianvictor7]

Just don't see a reason for AR, but to each is own. To me it seems like another big hyped addition that will fall flat with little to no updates. 3D Touch being the current front runner in that category.

There is a great deal of utility in having informational overlays on one's picture of the real world. Simply getting visual directions would be nifty. You could also subscribe to deal hunter information. Say you're walking in the mall and your favorite stores automatically display a list of deals above their entrances. The Pokémon Go phenomenon gave us a taste of the gaming possibilities. Also, you're talking about a MUCH bigger effective viewing area for any normal computer use: who wants a 13" laptop when they can have a hi-def, 120+ degree screen just millimeters from their eyeballs? Oh, and think of how much better our posture will be once we can stop staring DOWN at our phones. My neck is thanking me for just thinking about it!

The drawbacks, of course, may not outweigh the benefits for many. No one I know likes wearing glasses. Prescription VR gear may be required for some. Also, there's the whole "Glassholes" phenomenon: few people like it when their picture is being taking without permission (among other privacy concerns). And the problem goes both ways: people will be able to see a reverse image of anything you're viewing. Battery life will be a constant issue I suspect, but maybe wireless charging from a portable battery will come into vogue.

 

[smallcoffee]

That isn't really a user problem. That's more of a parlor trick of convenience. There are some limited applications of AR that are genuinely useful that solves specific user problems but, for the most part, it's a parlor trick with limited practical use.
.

Well I disagree and think you're wrong, so we will see in a few years.