MacRumors

macrumors bot
Original poster
Apr 12, 2001
53,060
14,811



Apple has a team of people working on integrating augmented reality functionality into the iPhone's camera app, reports Business Insider. Employees from several different augmented reality companies Apple has acquired are said to be involved in the effort, including those from Metaio and Flyby Media.

Citing sources familiar with Apple's plans, Business Insider says Apple's goal is for consumers to be able point the phone at a real-world object and have that object be recognized. Apple is already working on similar technology using machine learning techniques and has built basic object recognition capabilities into the Photos app.


Object recognition in the camera app would potentially allow for the real-time recognition of objects, but it is more complex than what is available in the Photos app because it involves 3D objects. Such a feature could be used for spacial recognition and mapping, among other things.

Facial recognition and manipulation may also be a goal, perhaps allowing the camera app to be used to apply Snapchat-style augmented reality filters to people. Apple is said to be planning to release an augmented reality SDK for developers following the debut of the rumored camera feature, which would allow third-party apps access the AR camera technology too.
Another early feature for Apple's AR integration into the camera app could be to recognize and manipulate people's faces. Apple integrated facial recognition technology into the photos app in the most recent version of its iOS software, and purchased FaceShift, a company with similar technology in 2015.
Recent rumors have suggested Apple's ultimate augmented reality ambition may be a set of smart glasses, which would connect wirelessly to the iPhone and display "images and other information" to the wearer.

A recent hire, John Border, seems to confirm Apple's rumored interest in smart glasses. Border, who describes himself as a "subject matter expert" in head-worn displays, plastic optics manufacturing, and camera systems and image sensors, recently joined Apple after serving as the chief engineer at Osterhout Design Group, a company that developed a set of smart glasses.

Apple's work on smart glasses is said to be in the early stages, and no product is expected to debut before 2018 should the project continue. It is not clear when Apple might introduce AR functionality in the camera app.

Article Link: Apple to Integrate Augmented Reality Functionality Into iPhone's Camera App
 

FatherJack1980

Suspended
Dec 31, 2015
146
347
"[facial] manipulation may also be a goal, perhaps allowing the camera app to be used to apply Snapchat-style augmented reality filters to people."

Oh goody, yet another social media thingy! Man, can't wait to emoji-fy my face!! *sigh*

....I can just imagine the presentation for this in the next Apple event.
 
Last edited:
Comment

jclo

Editor
Staff member
Dec 7, 2012
1,756
3,598
California
"[facial] manipulation may also be a goal, perhaps allowing the camera app to be used to apply Snapchat-style augmented reality filters to people."

Oh goody, yet another Snapchat thingy! Man, can't wait to emoji-fy my face!! *sigh*

....I can just imagine the presentation for this in the next Apple event :oops:

Seems like it would be a fun addition to FaceTime.
 
Comment

GrumpyMom

macrumors G3
Sep 11, 2014
9,538
13,684
"manipulate people's faces"

Oh goody, yet another Snapchat thingy! Man, can't wait to emoji-fy my face!! *sigh*

....I can just imagine the presentation for this in the next Apple event :oops:
I am not familiar with Snapchat. Are their filters the thing that's been turning people into weird morphs with animal muzzles and ears? I've been seeing that around a lot lately and wondering what the hell? :confused:

It sounds like Apple is trying to copy Lenovo/Tango now http://shop.lenovo.com/us/en/tango/...s0&PID=10078&acid=ww:affiliate:bv0as6&irgwc=1 Edit to add I have been corrected in my erroneous assumption by @Zirel
 
Last edited:
Comment

BeyondtheTech

macrumors 68020
Jun 20, 2007
2,126
633
I'd be happy if they added a QR/barcode scanner into the Camera app. Hate having to launch another app, especially since we don't yet have the ability to quickly launch our own apps from the Control Center.
 
  • Like
Reactions: longpath
Comment

jimbobb24

macrumors 68000
Jun 6, 2005
1,701
2,438
It's a good start. Phone holding is lame and glasses will be better but starting with phone is still progress.
 
Comment

Zirel

Suspended
Jul 24, 2015
2,196
3,008
1DobD00.jpg


VR. It's the future! Guys!
 
Comment

ErikGrim

macrumors 603
Jun 20, 2003
5,577
4,259
Brisbane, Australia
While object recognition would be of limited use to most users, imagine what it could do for the vision impaired? A set of smart glasses that translated what they were seeing into sound with Voice Over would be a game changer. Kind of like how VO can identify faces in the camera app already, but for a whole library of every day objects. The blind would literally be able to see and navigate the world by sound.
 
Comment

62tele

macrumors 6502a
Apr 11, 2010
739
674
The skeptic in me says that this is just the next piece of feces that Apple is throwing against the wall to see if it sticks. The Watch was not the next iPod. The car went down a dead end.
I hope this doesn't distract them from the next revolutionary breakthrough in emoji development.
 
Comment

lincolntran

macrumors 6502a
Jan 18, 2010
843
471
While object recognition would be of limited use to most users, imagine what it could do for the vision impaired? A set of smart glasses that translated what they were seeing into sound with Voice Over would be a game changer. Kind of like how VO can identify faces in the camera app already, but for a whole library of every day objects. The blind would literally be able to see and navigate the world by sound.

While most people here will just whine at anything new, I'm glad to see your post. I've been waiting to make an app to help blind people navigate but the current iPhone technology isn't possible. With this, my dream might finally be realized.
 
Comment

willgreene99

macrumors regular
Dec 16, 2010
217
16
DFW
I'll believe it when I see it :)

So true! How many times have we heard that Apple acquired this firm and that firm, in hopes they can produce something useful and fun? I believe they have a large stockpile of curiousities that we might find useful and fun but it did not pass some design criteria.

The skeptic in me says that this is just the next piece of feces that Apple is throwing against the wall to see if it sticks. The Watch was not the next iPod. The car went down a dead end.
I hope this doesn't distract them from the next revolutionary breakthrough in emoji development.

Ever since Jobs was quoted as having "cracked tv" was I hopeful to be getting my hands on something great! But alas, the years go by and I am losing patience they will ever release it.

It's great that Apple has gotten to a position of doing R&D, and I would love to go shopping in that lab, but its wearisome when that unicorn never materializes.

Apple TV - Not really, yet?
Apple Car - Not really, yet?
ARM MacOS - Not really, yet?
Touchscreen Mac - Not really, yet?
Apple AR - We'll see, but trend is Not really
Etc., etc.,
 
  • Like
Reactions: Luscious
Comment

macduke

macrumors G4
Jun 27, 2007
11,534
15,822
Central U.S.
They might integrate the snap chat filters into the app and have it be a mini App Store like they did for iMessages, but integrating general AR into the camera app makes absolutely zero sense as it's an app for taking photos. It makes more sense to use the camera API, remade into an AR specific API, as the backdrop for overlays in AR apps.

It's weird that they're Justin now getting into AR. I remember using AR apps at least as far back as 2010. It wasn't super useful to have random text boxes covering up everything when I could just use a maps app to find something. AR is more useful as a guide. For instance, if I'm looking at some plant it could tell me what it is and how to take care of it or if it looks healthy. Or it could guide me through doing a tracheotomy on someone whose airway was blocked, or highlight controls while showing me how to land an airplane in an emergency. Or if I see some product I could somehow select it and have it find it online for me to purchase or add it to a wish list. Where it gets creepy is scanning people's faces and showing me their stats from social media. People seen IRL who don't have social media profiles attached to their face in AR will be seen as outcasts and weirdos who should be avoided because they're trying to hide something. Gets weird and creepy fast! But the ability to share and expand knowledge is immense.
 
Comment

Rogifan

macrumors Core
Nov 14, 2011
22,547
28,508
The skeptic in me says that this is just the next piece of feces that Apple is throwing against the wall to see if it sticks. The Watch was not the next iPod. The car went down a dead end.
I hope this doesn't distract them from the next revolutionary breakthrough in emoji development.
And if this "rumor" said Apple was shelving this project every comment here would be how Apple is doomed and wtf are they doing in Cupertino other than releasing coffee table books and crippled laptops etc. Fire Tim Cook blah blah blah. I thought augmented reality was one of the next big things. Or is that only the case when companies like Google and Microsoft are working on it?
 
Comment
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.