Toggle sidebar Toggle sidebar
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

Here's What's New With Visual Intelligence in iOS 27

MacRumors

MacRumors

macrumors bot
Original poster


Visual Intelligence is getting a big upgrade in iOS 27. It's now easier to find than it was before, and there are new Visual Intelligence capabilities. Apple also expanded it to the iPad and the Mac, so you can use the same features across devices.

iOS-27-visual-intelligence-feature.jpg

Camera App

Apple moved Visual Intelligence to the Camera app in iOS 27, and it's accessible through a new Siri Mode that lives with the Photo, Video, and other camera mode options. You can swipe over to Siri Mode and take a picture so Siri can see what you see.

ios-27-siri-mode-camera.jpg

Siri will give you details about whatever you're looking at when you swipe down, and you can ask follow-up questions.

Siri Mode replaces Camera Control as the primary method for accessing Visual Intelligence, but you can still get to it by holding down the Camera Control button.

Smart Suggestions

Siri Mode will suggest relevant actions based on what you're taking a photo of. If you take a photo of a plant, Siri will offer to identify it. If you take a photo of a plate of food, Siri can give you the nutritional information.

Siri Integration

Visual Intelligence is integrated with Siri AI, and Siri can answer questions about what you're looking at. Complex questions were previously handed off to ChatGPT, but that's no longer necessary.

ios-27-visual-intelligence-nutrition.jpg

The smarter, more capable version of Siri can do much of what ChatGPT can do, pulling information from the web to answer questions.

New iOS 27 Visual Intelligence Features

  • Nutritional Insights - You can take a picture of a meal or a food item to check its nutritional value. Siri doesn't tell you specific calories in an item, but it can tell you if what you're eating has nutritional value and what's good or bad about it.
  • Bill Splitting (U.S. only) - If you take a photo of a restaurant bill, you can split out what you ate and then send money via Apple Cash. Or calculate what everyone owes and request money.
  • Import Multiple Events - You can import multiple events into the Calendar or Reminders app at one time from a schedule or list.
  • Import Contacts - You can import contacts from a photo of a business card.
  • Wallet Passes - You can turn a photo of a barcode from a membership or pass into a Wallet pass.

Visual Intelligence is much more powerful than before because of Siri's ability to search the web for answers. It can read laundry labels and give you clear instructions, look up items and find reviews on the web, identify odd devices or cables and tell you what they're for, scale recipes, read ingredient lists for you to check for allergens, transcribe messy written notes or lists into text, decipher confusing parking signs, offer plant care advice, solve math problems on a worksheet, and whatever else you can think to ask AI to do.

Other Visual Intelligence Capabilities

Visual Intelligence has a long list of things it can be used for, and these options have been available in earlier versions of iOS.
  • Identify plants, animals, insects, landmarks, art, sculptures, books, and more.
  • Add items to your calendar from a piece of paper like an event poster or flyer.
  • Use Google Image Search to find similar images.
  • Search Etsy, Amazon, Anthropologie, and other apps for an item that you capture with the camera.
  • Provide details about a business in front of you, like hours of operation.
  • Translate, summarize, and read text aloud.

Siri App Integration

All of your Visual Intelligence Siri requests are saved in the Siri app so you can revisit them later. The Siri app can be set to retain conversations for a month, a year, or forever.

Visual Intelligence on Other Platforms

Apple expanded Visual Intelligence to iPadOS 27, macOS Golden Gate, and visionOS 27. On Mac, you can use a Command-Shift-Space keyboard shortcut to access Visual Intelligence and select a part of your display to ask a question about. On iPad, you can access Visual Intelligence by taking a screenshot or swiping up from the lower left corner with Apple Pencil.

On Vision Pro, you can use Visual Intelligence just by looking at something, including physical items around you.

Device Requirements

Visual Intelligence in iOS 27 requires an iPhone 15 Pro or later. For Mac, you need an Apple silicon chip, and on the iPad, you need an Apple silicon chip or the A17 Pro iPad mini.

Article Link: Here's What's New With Visual Intelligence in iOS 27
 
Article Link
https://www.macrumors.com/guide/ios-27-visual-intelligence/
  • Angry
Reactions: Z-4195
Well, Halide Mark III was released a few weeks ago and I've got the 1 year subscription, so far I like it a lot, so no Apple camera app for me and sus no Visual Intelligence, which I am 100% fine with.
 
  • Like
Reactions: bluecoast and fatTribble
I thought Apple isn’t using ChatGPT for Visual Intelligence anymore. Then why are you still using the ChatGPT logo in the camera icon?
 
  • Like
Reactions: AstonSmith
this still feels like a glorified google lens from a couple years ago, just running on device. I wonder how much this feature will catch on, especially with non techy people. it could be useful, though.
 
  • Nutritional Insights - You can take a picture of a meal or a food item to check its nutritional value. ‌Siri‌ doesn't tell you specific calories in an item, but it can tell you if what you're eating has nutritional value and what's good or bad about it.
This sounds like exactly the kind of brain-dead thing that people who use AI think will actually work and help them.
 
  • Like
Reactions: 01cowherd, Parowdy, Schwarzmaler20 and 2 others
walkin said:
  • Nutritional Insights - You can take a picture of a meal or a food item to check its nutritional value. ‌Siri‌ doesn't tell you specific calories in an item, but it can tell you if what you're eating has nutritional value and what's good or bad about it.
This sounds like exactly the kind of brain-dead thing that people who use AI think will actually work and help them.
Click to expand...
And you feel so much better than them, don’t you?
It does work.
 
  • Disagree
Reactions: Parowdy and walkin
Any of this coming to the EU? I still don't fully understand where they make the cut between what will and won't be available in terms of Siri
 
so what action happens when you are on iOS27 and you press physical camera button, now it triggers visual inteligence, is it changing to Siri basically and action button still for something else or how does it work on iOS27? thanks
 
walkin said:
  • Nutritional Insights - You can take a picture of a meal or a food item to check its nutritional value. ‌Siri‌ doesn't tell you specific calories in an item, but it can tell you if what you're eating has nutritional value and what's good or bad about it.
This sounds like exactly the kind of brain-dead thing that people who use AI think will actually work and help them.
Click to expand...
Right? Nutrition is complex. Whilst I accept the at least trying to check the nutritional value of food is better than nothing, it can't be reduced to a basic traffic-light system or simple sliding scale. Everyone's needs are different and whilst there are some broadly good and broadly bad food stuffs for everyone, you need to properly understand what you are consuming and why. A quick -"AI says it's good" is not going to do it.
 
  • Like
Reactions: Schwarzmaler20
Another great feature has been upgraded, but unfortunately, it’s not available in the EU.
 
You must log in or register to reply here.
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.
Top Bottom
Back