Apple to Integrate Augmented Reality Functionality Into iPhone's Camera App

[MacRumors]

Apple has a team of people working on integrating augmented reality functionality into the iPhone's camera app, reports Business Insider. Employees from several different augmented reality companies Apple has acquired are said to be involved in the effort, including those from Metaio and Flyby Media.

Citing sources familiar with Apple's plans, Business Insider says Apple's goal is for consumers to be able point the phone at a real-world object and have that object be recognized. Apple is already working on similar technology using machine learning techniques and has built basic object recognition capabilities into the Photos app.

Subscribe to the MacRumors YouTube channel for more videos.​

Object recognition in the camera app would potentially allow for the real-time recognition of objects, but it is more complex than what is available in the Photos app because it involves 3D objects. Such a feature could be used for spacial recognition and mapping, among other things.

Facial recognition and manipulation may also be a goal, perhaps allowing the camera app to be used to apply Snapchat-style augmented reality filters to people. Apple is said to be planning to release an augmented reality SDK for developers following the debut of the rumored camera feature, which would allow third-party apps access the AR camera technology too.

Another early feature for Apple's AR integration into the camera app could be to recognize and manipulate people's faces. Apple integrated facial recognition technology into the photos app in the most recent version of its iOS software, and purchased FaceShift, a company with similar technology in 2015.

Recent rumors have suggested Apple's ultimate augmented reality ambition may be a set of smart glasses, which would connect wirelessly to the iPhone and display "images and other information" to the wearer.

A recent hire, John Border, seems to confirm Apple's rumored interest in smart glasses. Border, who describes himself as a "subject matter expert" in head-worn displays, plastic optics manufacturing, and camera systems and image sensors, recently joined Apple after serving as the chief engineer at Osterhout Design Group, a company that developed a set of smart glasses.

Apple's work on smart glasses is said to be in the early stages, and no product is expected to debut before 2018 should the project continue. It is not clear when Apple might introduce AR functionality in the camera app.

Article Link: Apple to Integrate Augmented Reality Functionality Into iPhone's Camera App

 

[FatherJack1980]

"[facial] manipulation may also be a goal, perhaps allowing the camera app to be used to apply Snapchat-style augmented reality filters to people."

Oh goody, yet another social media thingy! Man, can't wait to emoji-fy my face!! *sigh*

....I can just imagine the presentation for this in the next Apple event.

 

[Luscious]

I'll believe it when I see it

 

[Mac Fly (film)]

Car project cancelled (lie). Glasses project full steam ahead (lie). Good luck, Samsung!

 

[jclo]

"[facial] manipulation may also be a goal, perhaps allowing the camera app to be used to apply Snapchat-style augmented reality filters to people."

Oh goody, yet another Snapchat thingy! Man, can't wait to emoji-fy my face!! *sigh*

....I can just imagine the presentation for this in the next Apple event

Seems like it would be a fun addition to FaceTime.

 

[caligomez]

I can only imagine the sharp rise of sidewalk accidents.

 

[5105973]

"manipulate people's faces"

Oh goody, yet another Snapchat thingy! Man, can't wait to emoji-fy my face!! *sigh*

....I can just imagine the presentation for this in the next Apple event

I am not familiar with Snapchat. Are their filters the thing that's been turning people into weird morphs with animal muzzles and ears? I've been seeing that around a lot lately and wondering what the hell?

It sounds like Apple is trying to copy Lenovo/Tango now http://shop.lenovo.com/us/en/tango/...s0&PID=10078&acid=ww:affiliate:bv0as6&irgwc=1 Edit to add I have been corrected in my erroneous assumption by @Zirel

 

[BeyondtheTech]

I'd be happy if they added a QR/barcode scanner into the Camera app. Hate having to launch another app, especially since we don't yet have the ability to quickly launch our own apps from the Control Center.

 

[AngerDanger]

I propose Apple create a feature whereby you aim your camera at a new MacBook and nearby device, and the app overlays all the dongles and adaptors you'd need to connect the two devices. You could even buy them right from the camera app!

 

[jimbobb24]

It's a good start. Phone holding is lame and glasses will be better but starting with phone is still progress.

 

[Zirel]

VR. It's the future! Guys!

 

[ErikGrim]

While object recognition would be of limited use to most users, imagine what it could do for the vision impaired? A set of smart glasses that translated what they were seeing into sound with Voice Over would be a game changer. Kind of like how VO can identify faces in the camera app already, but for a whole library of every day objects. The blind would literally be able to see and navigate the world by sound.

 

[duffman9000]

VR. It's the future! Guys!

So is die-a-beetus!

 

[Zirel]

It sounds like Apple is trying to copy Lenovo/Tango now http://shop.lenovo.com/us/en/tango/...s0&PID=10078&acid=ww:affiliate:bv0as6&irgwc=1

That's rich, because Tango uses Primesense chips to do its thing. Primesense is owned by Apple.

http://www.cultofmac.com/274819/google-swiped-apples-hot-new-primesense-3-d-tech-project-tango/

 

[BeefCake 15]

They should've called it the Reality Distortion Field in memory of Steve

 

[5105973]

That's rich, because Tango uses Primesense chips to do its thing. Primesense is owned by Apple.

http://www.cultofmac.com/274819/google-swiped-apples-hot-new-primesense-3-d-tech-project-tango/

Thanks, I stand corrected! Well actually I'm sitting... But I appreciate the accuracy. I'll go amend my original post.

 

[62tele]

The skeptic in me says that this is just the next piece of feces that Apple is throwing against the wall to see if it sticks. The Watch was not the next iPod. The car went down a dead end.
I hope this doesn't distract them from the next revolutionary breakthrough in emoji development.

 

[rizzo41999]

Tim's being a real sucker for this stuff.

 

[lincolntran]

While object recognition would be of limited use to most users, imagine what it could do for the vision impaired? A set of smart glasses that translated what they were seeing into sound with Voice Over would be a game changer. Kind of like how VO can identify faces in the camera app already, but for a whole library of every day objects. The blind would literally be able to see and navigate the world by sound.

While most people here will just whine at anything new, I'm glad to see your post. I've been waiting to make an app to help blind people navigate but the current iPhone technology isn't possible. With this, my dream might finally be realized.

 

[Stella]

Just as long as it doesn't reach 'Black Mirror' proportions!!

 

[rGiskard]

I'd rather have a larger camera sensor.

 

[willgreene99]

I'll believe it when I see it

So true! How many times have we heard that Apple acquired this firm and that firm, in hopes they can produce something useful and fun? I believe they have a large stockpile of curiousities that we might find useful and fun but it did not pass some design criteria.

The skeptic in me says that this is just the next piece of feces that Apple is throwing against the wall to see if it sticks. The Watch was not the next iPod. The car went down a dead end.
I hope this doesn't distract them from the next revolutionary breakthrough in emoji development.

Ever since Jobs was quoted as having "cracked tv" was I hopeful to be getting my hands on something great! But alas, the years go by and I am losing patience they will ever release it.

It's great that Apple has gotten to a position of doing R&D, and I would love to go shopping in that lab, but its wearisome when that unicorn never materializes.

Apple TV - Not really, yet?
Apple Car - Not really, yet?
ARM MacOS - Not really, yet?
Touchscreen Mac - Not really, yet?
Apple AR - We'll see, but trend is Not really
Etc., etc.,

 

[macduke]

They might integrate the snap chat filters into the app and have it be a mini App Store like they did for iMessages, but integrating general AR into the camera app makes absolutely zero sense as it's an app for taking photos. It makes more sense to use the camera API, remade into an AR specific API, as the backdrop for overlays in AR apps.

It's weird that they're Justin now getting into AR. I remember using AR apps at least as far back as 2010. It wasn't super useful to have random text boxes covering up everything when I could just use a maps app to find something. AR is more useful as a guide. For instance, if I'm looking at some plant it could tell me what it is and how to take care of it or if it looks healthy. Or it could guide me through doing a tracheotomy on someone whose airway was blocked, or highlight controls while showing me how to land an airplane in an emergency. Or if I see some product I could somehow select it and have it find it online for me to purchase or add it to a wish list. Where it gets creepy is scanning people's faces and showing me their stats from social media. People seen IRL who don't have social media profiles attached to their face in AR will be seen as outcasts and weirdos who should be avoided because they're trying to hide something. Gets weird and creepy fast! But the ability to share and expand knowledge is immense.

 

[Rogifan]

The skeptic in me says that this is just the next piece of feces that Apple is throwing against the wall to see if it sticks. The Watch was not the next iPod. The car went down a dead end.
I hope this doesn't distract them from the next revolutionary breakthrough in emoji development.

And if this "rumor" said Apple was shelving this project every comment here would be how Apple is doomed and wtf are they doing in Cupertino other than releasing coffee table books and crippled laptops etc. Fire Tim Cook blah blah blah. I thought augmented reality was one of the next big things. Or is that only the case when companies like Google and Microsoft are working on it?

 

[Tech198]

I'll believe it when I see it

Yep,, i think all of us will..