Apple's AI Team Publishes First Research Paper Focused on Advanced Image Recognition

[MacRumors]

Earlier in December, Apple announced that it would begin allowing its artificial intelligence and machine learning researchers to publish and share their work in papers, slightly pulling back the curtain on the company's famously secretive creation processes. Now, just a few weeks later, the first of those papers has been published, focusing on Apple's work in the intelligent image recognition field.

Titled "Learning from Simulated and Unsupervised Images through Adversarial Training," the paper describes a program that can intelligently decipher and understand digital images in a setting similar to the "Siri Intelligence" and facial recognition features introduced in Photos in iOS 10, but more advanced.

In the research, Apple notes the downsides and upsides of using real images compared with that of "synthetic," or computer images. Annotations must be added to real images, an "expensive and time-consuming task" that requires a human workforce to individually label objects in a picture. On the other hand, computer-generated images help to catalyze this process "because the annotations are automatically available."

Still, fully switching to synthetic images could lead to a dip in the quality of the program in question. This is because "synthetic data is often not realistic enough" and would lead to an end-user experience that only responded well to details present in the computer-generated images, while being unable to generalize well on any real-world objects and pictures it faced.

This leads to the paper's central proposition -- the combination of using both simulated and real images to work together in "adversarial training," creating an advanced AI image program:

In this paper, we propose Simulated+Unsupervised (S+U) learning, where the goal is to improve the realism of synthetic images from a simulator using unlabeled real data. The improved realism enables the training of better machine learning models on large datasets without any data collection or human annotation effort.

We show that this enables generation of highly realistic images, which we demonstrate both qualitatively and with a user study.

The rest of the paper goes into the details of Apple's research on the topic, including experiments that have been run and the math proposed to back up its findings. The paper's research focused solely on single images, but the team at Apple notes towards the end that it hopes to sometime soon "investigate refining videos" as well.

The credits on the paper go to Apple researchers Ashish Shrivastava, Tomas Pfister, Oncel Tuzel, Josh Susskind, Wenda Wang, and Russ Webb. The team's research was first submitted on November 15, but it didn't get published until December 22.

At the AI conference in Barcelona a few weeks ago, Apple head of machine learning Russ Salakhutdinov -- and a few other employees -- discussed topics including health and vital signs, volumetric detection of LiDAR, prediction with structured outputs, image processing and colorization, intelligent assistant and language modeling, and activity recognition. We'll likely see papers on a variety of these topics and more in the near future.

Article Link: Apple's AI Team Publishes First Research Paper Focused on Advanced Image Recognition

 

[samcan]

I'm not sure if it is the competition getting better, but I feel as though Siri is getting dumber by the minute. Context requests are out of the question, it isn't current with sports anymore, and I've found myself being cut off with "sorry I didn't get that" while in a quiet room.

Siri stopped being a useful tool ever since they dropped "raise to speak". I find myself using my Pixel for anything requiring hands free.

 

[djcerla]

I don't know if it's because of Apple's recent acquisition and hiring spree in AI, but I found the (Italian) Siri vastly improved recently.

Using Siri with the Airpods is the best so far, she perfectly gets all sentences, including odd names.

 

[and 1989 others]

Woah, woah, woah. We're not interested in artificial intelligence and autonomous learning, they play second fiddle to the real issues. What we really want to know is how thin it is? And can it be thinner?

 

[AngerDanger]

If anybody wants to play around with AI image recognition, CloudSight (scroll down to the Try it Out area) allows users to upload an image for recognition. It can be pretty cool to see how accurate its tagging is.

This image was described as "grey jar carton and bottle sketch" after uploading.

 

[latweek]

Can Apple make an AI that shows them how to eliminate dongles?

 

[A MacBook lover]

Bunch of sad half witty replies fishing for likes. Try to post some quality discussion next time guys

 

[smulji]

I don't know if it's because of Apple's recent acquisition and hiring spree in AI, but I found the (Italian) Siri vastly improved recently.

Using Siri with the Airpods is the best so far, she perfectly gets all sentences, including odd names.

In light of all the excitement around Airpods, this is an excellent article with former Apple engineer on Siri, Airpods, AI, etc.

https://www.fastcompany.com/3064105...siri-according-to-apple-veteran-bill-atkinson

 

[Math889]

Apple is working to bring new emojis and new updates that add nothing to the OS ,( just the build number changes)

 

[npmacuser5]

AI development to date; Trying to run before learning too walk. I would prefer fewer options. Concentrate on music, or home automation, or pick one or two. Excute the tasks well. The problem for me is knowing what will work and what does not. Thus, I get frustrated and stop using features like Siri.

 

[69Mustang]

Bunch of sad half witty replies fishing for likes. Try to post some quality discussion next time guys

Recognize that your complaint and my response to it contribute nothing to the quality of the discussion either. Maybe we both should take your advice and post something quality and relative to the topic.
[doublepost=1482774037][/doublepost]I mentioned when it was first announced that this (researchers being allowed to publish) is one of the best thing coming out of Apple's AI push. Peer review and recognition is one of the cornerstones of research. I bet the group is happy their work can be recognized and possibly appreciated. Happy for them.

 

[Math889]

I don't use Siri ....... and the last time that I used Siri didn't understand what I said ...

 

[twinlight]

This is used for learning the AI looking at our pictures? Next step video and then they can start understanding our enviroment for AR I guess. I want navigation in my car to understand the road and direct me accordingly to the correct lane or my phone to show me the way in an airport I never visited before.

 

[lotzosushi]

This is cool and all, but is this really *that* useful for most consumers right now? I feel like they should focus their attention on much more dire issues...

Siri: "Sir, would you like me to file your new photos into the 'dick pics' album?" is probably what she'll be doing most.

 

[and 1989 others]

Bunch of sad half witty replies fishing for likes. Try to post some quality discussion next time guys

When there's something of quality to write about, I'll write about it.

Until then, release joke products, receive joke replies.

Quid, pro, quo.

 

[drewyboy]

Bunch of sad half witty replies fishing for likes. Try to post some quality discussion next time guys

Ok, what? You mean like Apple spent more time on describing the new iMessage at WWDC than any other feature for iOS10? Clearly highlighting emoji as the flagship feature for iOS10? Oh, and lets not forget, they were too busy with emoji to realize they have a horrible battery bug in iOS10? I mean, using Apple's built in flashlight app shouldn't drain 10% of my battery for 4 minutes of use should it? Or shutting down when my phone is at 37% just yesterday only to plug it in and it be back at 37% then drain somewhat normal only to shut down again at 12%? And no, the health of my battery is just fine, or is the Apple Store lying to me when they checked. Or how about how they completely compromised the user experience of the new MBP by sacrificing 25% battery capacity to thin it down and make it lighter for a device that sits on a fixed surface for 99% of the users. Or how about Siri has gotten worse as time as gone on, while competitors get better and better each year?

So please, take your pick and lets have some "quality discussion". All I usually see is people offering validated criticism and then the other half defending apple as if it was their child and blaming the user. You're right, Apple can never do any wrong. They are always right and never wrong. Silly me, my messed up iPhone battery life is a new iOS feature, or is it because I'm using an iPhone 5S and as Phil said, I should be upgrading since it's ancient.

Edit: And as far as Photos go, maybe they should actually do something about families because their current "family share" features are a complete joke.

 

[laz232]

Note that this is a conference paper, not a journal paper, and Apple allowing this is needed to attract high calibre researchers. Browsing through it it seems aimed at technologies 5+ years into the future, for UI that are way beyond Siri.

The ironic thing is that Siri remains a POS. Just about good enough for timers, but don't ask her anything mildly complex (not even complicated) like:
Me: "Add a reminder about defrosting the fish. Actually no, remove the reminder and add it to my calendar instead, for 3pm tomorrow afternoon. "
Siri: I didn't find any appointments about "Reminder" for tomorrow afternoon.
Me: Thank you Siri, you suck

 

[nt5672]

Woah, woah, woah. We're not interested in artificial intelligence and autonomous learning, they play second fiddle to the real issues. What we really want to know is how thin it is? And can it be thinner?

Yes, Mr. Ive, maybe the future does belong to technology not overpaid wordsmiths with a single focus on thin, smooth, empty, uncomplicated, and useless computers.

 

[taran420]

When there's something of quality to write about, I'll write about it.

Until then, release joke products, receive joke replies.

Quid, pro, quo.

But what you posted here had nothing to do with the subject. If there was a joke product released, then your "joke" reply would work. If you have nothing to add about the subject at hand, then why even bother posting?
Probably the reason that was stated earlier, for likes.

 

[tampageek]

No. No. No. Bad idea for a company that can't do an update without making machines behave badly. Ahhhh....the computer zombies are coming ... No, really. This is Tim's Apple. There probably won't be any ports to turn it off.

 

[mi7chy]

I'm not sure if it is the competition getting better, but I feel as though Siri is getting dumber by the minute.

It's actually both with competition pushing further ahead and Siri getting worse.

https://www.reddit.com/r/SiriFail/

 

[tampageek]

Can Apple make an AI that shows them how to eliminate dongles?

No. But I think the idea here is to make robots to build iPhones. Then they can say made in USA without hiring any people.

 

[darknyt]

Edit: And as far as Photos go, maybe they should actually do something about families because their current "family share" features are a complete joke.

This^^^^ x 100. Apple's implementation of family sharing makes me think it was designed by a bunch of single guys who have no idea what a family is or how it operates. Photos is a big failure area for the feature and one of several reasons we don't use it.

And I hope apple is reading whatever google puts out on this topic cuz Google Photos still wipes the floor with Photos even after they shamelessly copied part of their app. Photos is literally just a backup method or phone storage. Google photos even on my phone is so much more capable.

 

[A MacBook lover]

Ok, what? You mean like Apple spent more time on describing the new iMessage at WWDC than any other feature for iOS10? Clearly highlighting emoji as the flagship feature for iOS10? Oh, and lets not forget, they were too busy with emoji to realize they have a horrible battery bug in iOS10? I mean, using Apple's built in flashlight app shouldn't drain 10% of my battery for 4 minutes of use should it? Or shutting down when my phone is at 37% just yesterday only to plug it in and it be back at 37% then drain somewhat normal only to shut down again at 12%? And no, the health of my battery is just fine, or is the Apple Store lying to me when they checked. Or how about how they completely compromised the user experience of the new MBP by sacrificing 25% battery capacity to thin it down and make it lighter for a device that sits on a fixed surface for 99% of the users. Or how about Siri has gotten worse as time as gone on, while competitors get better and better each year?

So please, take your pick and lets have some "quality discussion". All I usually see is people offering validated criticism and then the other half defending apple as if it was their child and blaming the user. You're right, Apple can never do any wrong. They are always right and never wrong. Silly me, my messed up iPhone battery life is a new iOS feature, or is it because I'm using an iPhone 5S and as Phil said, I should be upgrading since it's ancient.

Edit: And as far as Photos go, maybe they should actually do something about families because their current "family share" features are a complete joke.

It's unfortuante that your experience is not up par. No one is stopping you from choosing another platform. Perhaps you should try a support forum or heading to a different store rather than venting on macrumors in an article about AI.

Vote with your money. I'm still happy enough with Apple to continue buying from them, and many others as well

 

[TMRJIJ]

Ok, what? You mean like Apple spent more time on describing the new iMessage at WWDC than any other feature for iOS10? Clearly highlighting emoji as the flagship feature for iOS10? Oh, and lets not forget, they were too busy with emoji to realize they have a horrible battery bug in iOS10? I mean, using Apple's built in flashlight app shouldn't drain 10% of my battery for 4 minutes of use should it? Or shutting down when my phone is at 37% just yesterday only to plug it in and it be back at 37% then drain somewhat normal only to shut down again at 12%? And no, the health of my battery is just fine, or is the Apple Store lying to me when they checked. Or how about how they completely compromised the user experience of the new MBP by sacrificing 25% battery capacity to thin it down and make it lighter for a device that sits on a fixed surface for 99% of the users. Or how about Siri has gotten worse as time as gone on, while competitors get better and better each year?

So please, take your pick and lets have some "quality discussion". All I usually see is people offering validated criticism and then the other half defending apple as if it was their child and blaming the user. You're right, Apple can never do any wrong. They are always right and never wrong. Silly me, my messed up iPhone battery life is a new iOS feature, or is it because I'm using an iPhone 5S and as Phil said, I should be upgrading since it's ancient.

Edit: And as far as Photos go, maybe they should actually do something about families because their current "family share" features are a complete joke.

You should go to another Apple Store. Of course battery life decreases as years of updates go on, your particular issue is the exact same as mine. I called Apple Support and they determined it was a battery malfunction. Got it replaced yesterday and already seeing improvements.