Sidebar Sidebar
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

You Can Now Get Visual Intelligence on iPhone 15 Pro – Here's How

MacRumors

MacRumors

macrumors bot
Original poster
Apr 12, 2001
67,504
37,791


On iPhone 16 models, Visual Intelligence lets you use the camera to learn more about places and objects around you. It can also summarize text, read text out loud, translate text, search Google for items, ask ChatGPT, and more. And thanks to the latest iOS 18.4 update from Apple, iPhone 15 Pro models can now get in on the action, too.

Apple-Visual-Intelligence.jpg

Until recently, ‌Visual Intelligence‌ was a feature limited to iPhone 16 models with a Camera Control button, which was necessary to activate the feature. However, Apple in February debuted the iPhone 16e, which lacks Camera Control and yet supports Visual Intelligence. This is because the device ships with a version of iOS that includes Visual Intelligence as an assignable option to the device's Action button.

Apple later confirmed that the same Visual Intelligence customization setting would be coming to iPhone 15 Pro models via a software update. That update is iOS 18.4, and it's available now. If you haven't updated yet, you can do so by opening Settings ➝ General ➝ Software Update.

After your device is up-to-date, you can assign Visual Intelligence to the device's Action button in the following way.
  1. Open Settings on your iPhone 15 Pro.
  2. Tap Action Button.
  3. Swipe to Visual Intelligence.
visual-intelligence-action-button.jpg


Pressing and holding the Action button will now activate Visual Intelligence. Note that you can also activate Visual Intelligence using the new button option in Control Center. Here's how.
  1. Swipe down from the top-right corner of your iPhone's display, then long press on the Control Center.
  2. Tap Add a Control at the bottom.
  3. Use the search bar at the top to search for Visual Intelligence, or swipe up to the "Apple Intelligence" section and choose the button.
  4. Tap the screen to exit the Control Center's edit mode.
visual-intelligence-control-center.jpg


Using Visual Intelligence

The Visual Intelligence interface features a view from the camera, a button to capture a photo, and dedicated "Ask" and "Search" buttons. Ask queries ChatGPT, and Search sends an image to Google Search.

visual-intelligence-buttons.jpg

When using Visual Intelligence you can either snap a photo using the shutter button and then select an option, or you can select an option in live camera view. You cannot use photos that you took previously.

To learn about everything that you can do with Visual Intelligence, be sure to check out our dedicated guide.

Article Link: You Can Now Get Visual Intelligence on iPhone 15 Pro – Here's How
 
Article Link
https://www.macrumors.com/how-to/visual-intelligence-iphone-15-pro/
  • Like
Reactions: AdrianBryce
It’s quite surprising that the iPhone 16 Pro doesn’t allow you to set the action button to Visual Intelligence.
Or maybe it doesn't show on mine, since AI is OFF?
 
diego.caraballo said:
It’s quite surprising that the iPhone 16 Pro doesn’t allow you to set the action button to Visual Intelligence.
Or maybe it doesn't show on mine, since AI is OFF?
Click to expand...
It does, mine is given the option, but you can access it quickly by holding down the slider button for the camera without assigning it and saving the action button for another function
 
  • Like
Reactions: Parowdy, that be me and diego.caraballo
Kind of strange they made sure to get Apple Intelligence on the 16e, which they have had trouble getting onto anything at all, but couldn't get MagSafe on there, which they've managed to do for years.

Strange the things they chose to prioritize as selling points this year.
 
  • Love
  • Like
Reactions: Jovijoker, miq and rmadsen3
CarAnalogy said:
Kind of strange they made sure to get Apple Intelligence on the 16e, which they have had trouble getting onto anything at all, but couldn't get MagSafe on there, which they've managed to do for years.

Strange the things they chose to prioritize as selling points this year.
Click to expand...
One is software and the other is hardware. 🤷‍♂️
 
  • Like
  • Sad
Reactions: Parowdy, miq and frenchcamp49er
bhirt37 said:
Visual intelligence, what a joke. Unless I am missing something, all this feature does is upload an image to either ChatGPT or Google image search, depending on the button. What amazing things you are doing apple. These billions invested in ai are paying off!!
Click to expand...
Agreed. I was so underwhelmed that I instantly unmapped the Action Button. I did see that in some other workflows that the visual context clues would pull up other actions like translate or calendar (assuming this is why there has been an emphasis on the "default apps" setting). But it hardly felt like a game changing feature.
 
  • Like
Reactions: bhirt37 and almostinsane
Howard2k said:
You’re missing something. It’s in-device processing.
Click to expand...
To quote the article above:
MacRumors said:
Using Visual Intelligence
The Visual Intelligence interface features a view from the camera, a button to capture a photo, and dedicated "Ask" and "Search" buttons. Ask queries ChatGPT, and Search sends an image to Google Search.
Click to expand...
Unless I'm missing something, ChatGPT and Google Search aren't processed on-device.
 
  • Like
Reactions: d686546s, miq, demwun and 1 other person
Howard2k said:
You’re missing something. It’s in-device processing.
Click to expand...
If that were the case, wouldn’t it work without internet?

Regardless, this is another sign how Apple has fallen behind the AI game when they have to rely on third party AI models for this feature. When I originally heard about Visual Intelligence I thought it was using their in-house Apple Intelligence model. I don’t see the point of this as it does not buy me any value added features over installing the Google App or ChatGPT App.

More and more I’m beginning to realize Apple is just a “design” company. Great at making things look pretty and consistent, but really bad at doing anything “smart”.
 
  • Like
  • Disagree
  • Love
Reactions: Jovijoker, miq, demwun and 4 others
almostinsane said:
How is it on device if it sends the image to google and the query to Chatgpt?
Click to expand...
AppliedMicro said:
To quote the article above:

Unless I'm missing something, ChatGPT and Google Search aren't processed on-device.
Click to expand...

a.coward said:
If that were the case, wouldn’t it work without internet?

Regardless, this is another sign how Apple has fallen behind the AI game when they have to rely on third party AI models for this feature. When I originally heard about Visual Intelligence I thought it was using their in-house Apple Intelligence model. I don’t see the point of this as it does not buy me any value added features over installing the Google App or ChatGPT App.

More and more I’m beginning to realize Apple is just a “design” company. Great at making things look pretty and consistent, but really bad at doing anything “smart”.
Click to expand...

Apple uses a tired architecture model for the AI suite. 1) There is a smaller LLM used with the on device NPE that performs some tasks. 2) There is an Apple cloud service called Private Cloud Compute, which is the second tier for more advanced functions that the on-device LLM cannot handle, and 3) there is a third tier for more complicated tasks where requests are sent to a third party like ChatGPT.

The on-device NPE does the first pass and obviously would go out to the Internet to get to Apple PCC and ChatGPT.
 
  • Like
Reactions: AdamInKent, tigerintank and mlayer
Apple Intelligence gets some stick (largely for a set of useful or quaint OS features) but Visual Intelligence is brilliant.

Yes it relies on ChatGPT to do much of the heavy lifting but this doesn’t require an account and is no different to using Google as the search engine in Safari.

I’ve used it for all sorts of things from asking what an unknown food in the market might taste like to how to calculate Young’s Modulus of some tensile testing samples at work. On a recent trip to Spain I used it to visually translate nearly everything I came across.

Yes it’s Google Lens for the iPhone. But it’s only a purposeful button hold away at all times and the swishy gradient animations are look slick.
 
  • Like
  • Haha
Reactions: Jovijoker, miq, mlayer and 1 other person
cicalinarrot said:
I'd title this "$1000 phones from a year ago still not outdated, bad luck for $900 phones buyers"
Click to expand...
They've got to draw the line somewhere. 🤣

--

As for this feature overall, well, underwhelming so far. I don't really get how it's different to Google lens which I've had on my phone for years.
 
You must log in or register to reply here.
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.
Top Bottom
Back