Sidebar Sidebar
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

iOS 15: How to Use Visual Lookup in Photos to Identify Landmarks, Plants, and Pets

MacRumors

MacRumors

macrumors bot
Original poster
Apr 12, 2001
67,488
37,766


In iOS 15, Apple made further advancements in on-device machine learning and integrated them into the Photos app to make your iPhone more intelligent at recognizing the contents of pictures.

ios-15-visual-lookup.jpg

In other words, the Photos app can now identify various objects, landmarks, animals, books, plants, works of art, and more in your image library, and then offer information about them that it draws from the web.

This new intelligent feature is called Visual Lookup, and the following steps show you how you can use it to get more clued up on the things you've taken pictures of with your iPhone through the years.

  1. Open the Photos app on your iPhone and select a picture with a clearly defined subject, such as a flower or animal.
  2. Check the info ("i") icon at the bottom of the screen. If it has a little star over it, tap it – this indicates there's a Visual Lookup you can examine.
  3. Tap the little icon in the center of the photo to bring up the Lookup search results.
photos-visual-lookup-ios.jpg

In Visual Lookup, search results consist of Siri Knowledge, similar images found on the web, and other online sources of information. For all the details on the other new Photos features in iOS 15, be sure to check our dedicated roundup.

Article Link: iOS 15: How to Use Visual Lookup in Photos to Identify Landmarks, Plants, and Pets
 
Article Link
https://www.macrumors.com/how-to/use-visual-lookup-photos-ios/
ctg7w6 said:
Funny enough LAST NIGHT I tested googles version of this and it got my dog right. I was amazed.
Click to expand...
Neural Nets are great at identifying things similar to what people label them. If you've got a dog that a lot of people would call a certain breed (and train it close to that), the NN will come to the same conclusion.
 
  • Like
Reactions: eas, Velociround, peanuts_of_pathos and 1 other person
I would like this to work the other way around. Would be nice to be able to search my almost 20,000 photos with words. For instance "Red Car" might help me quickly find that old mustang picture. I suppose it would mean that Apple would use AI to search all your photos and tag them with what it thinks the content is. Now that would be useful.
 
  • Like
  • Haha
Reactions: macintoshmac, eas, Velociround and 2 others
ctg7w6 said:
Funny enough LAST NIGHT I tested googles version of this and it got my dog right. I was amazed.
Click to expand...
How though? Most dogs are not one breed. Even most purebreads aren't one breed. Do you have DNA testing and pedagree papers, or are you saying the phone matched what you have been calling your dog?

MrTemple said:
Neural Nets are great at identifying things similar to what people label them. If you've got a dog that a lot of people would call a certain breed (and train it close to that), the NN will come to the same conclusion.
Click to expand...
Exactly.

1625769660265.png
 
Last edited:
  • Disagree
  • Like
Reactions: peanuts_of_pathos and SMacDuff
4jasontv said:
How though? Most dogs are not one breed. Even most purebreads aren't one breed. Do you have DNA testing and pedagree papers, or are you saying the phone matched what you have been calling your dog?


Exactly.

View attachment 1803746
Click to expand...
Well, I mean, human beings classify animals and call them these names. Obviously there will be issues with heavily mixed dogs. But breeds are not “fake”. Yes they are arbitrary in many ways, but if the neural net is trained with accurate data and you test it on an animal that is not very mixed, it will get it right.
 
  • Like
Reactions: eas and peanuts_of_pathos
Jimmy Goodwood said:
I would like this to work the other way around. Would be nice to be able to search my almost 20,000 photos with words. For instance "Red Car" might help me quickly find that old mustang picture. I suppose it would mean that Apple would use AI to search all your photos and tag them with what it thinks the content is. Now that would be useful.
Click to expand...
You already can... Photos are searchable by keyword, although it's a little basic and can't do the colour bit. I can search for 'car' and get a bunch of photos with a car or cars in it. Divided up into Moments, Categories (car, carton, car park) and Places. Also things like sun, snow, beach, tree, flower, sea, river, dog, fire - and you can use multiple keywords to narrow the search as well (eg 'dog fire' to find pics of the dog by the fire).
 
  • Like
Reactions: insomniac86, peanuts_of_pathos, TheDailyApple and 1 other person
ctg7w6 said:
Well, I mean, human beings classify animals and call them these names. Obviously there will be issues with heavily mixed dogs. But breeds are not “fake”. Yes they are arbitrary in many ways, but if the neural net is trained with accurate data and you test it on an animal that is not very mixed, it will get it right.
Click to expand...
I didn't call breeds fake. As a novelty, this is fine. But it's going to cause major headaches for people that actually rely on breed accuracy. Trained people get breeds wrong. Heck, DNA testing gets breeds wrong. An AI is only going to be as good as the algorithm.
 
  • Disagree
Reactions: SMacDuff
andysw9 said:
You already can... Photos are searchable by keyword, although it's a little basic and can't do the colour bit. I can search for 'car' and get a bunch of photos with a car or cars in it. Divided up into Moments, Categories (car, carton, car park) and Places. Also things like sun, snow, beach, tree, flower, sea, river, dog, fire - and you can use multiple keywords to narrow the search as well (eg 'dog fire' to find pics of the dog by the fire).
Click to expand...
But did you tag all the photos yourself, or were they tagged automatically by Apple when they were uploaded to the photos app? I have to try this
 
  • Like
Reactions: peanuts_of_pathos
andysw9 said:
You already can... Photos are searchable by keyword, although it's a little basic and can't do the colour bit. I can search for 'car' and get a bunch of photos with a car or cars in it. Divided up into Moments, Categories (car, carton, car park) and Places. Also things like sun, snow, beach, tree, flower, sea, river, dog, fire - and you can use multiple keywords to narrow the search as well (eg 'dog fire' to find pics of the dog by the fire).
Click to expand...
That is cool, but you have to add the keywords yourself, manually. That can be very tedious and time-consuming if you have a very large library. I speak as one who has a large photo library and who does use the keywords feature.

The hope here is that that Photos will now analyze your library and automatically identify photos with cars in them.

This visual lookup sounds like it will be a fun feature to play with.
 
  • Disagree
  • Like
Reactions: eas and peanuts_of_pathos
4jasontv said:
I am going to say no. You can not point your camera at a dog and have it tell you what breed it is.

Even if you could do that anyone who cares is going to argue with the results.
Click to expand...
My parents have a Cavapooschon. Sometimes it tells me it’s a King Cavalier and others a Havanese. It’s been accurate with all the flowers I’ve tested.
 
  • Like
Reactions: peanuts_of_pathos
You must log in or register to reply here.
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.
Top Bottom
Back