Apple Exploring Ways to Run Much Larger AI Models Directly on iPhones

[MacRumors]

Apple has held meetings with PrismML about ways it could use the startup's technology to run much larger AI models directly on iPhones, according to The Information.

The report said PrismML has managed to shrink down Alibaba's open-source large language model Qwen 3.6 to run entirely on an iPhone 17 Pro. The model has 27 billion parameters, which is larger than Apple's on-device AFM 3 Core Advanced model with 20 billion parameters. Apple's model powers iOS 27 enhancements such as Siri AI's more expressive voices and improved systemwide dictation on iPhone 17 Pro and iPhone Air models.

Unlike with AFM 3 Core Advanced, all of Qwen 3.6's parameters can be active at the same time.

"One new on-device Apple model has 20 billion parameters but uses a so-called sparse architecture, in which only 1 billion to 4 billion parameters are active at a time," the report said, in reference to AFM 3 Core Advanced. "In the case of PrismML's on-device model, all 27 billion parameters are active at the same time."

Larger models running directly on iPhones would allow for more Apple Intelligence features to run on device instead of on Apple's Private Cloud Compute servers, which could reduce Apple's costs and further enhance user privacy.

Article Link: Apple Exploring Ways to Run Much Larger AI Models Directly on iPhones

 

[963852741]

Yeah, sure, do something like SETI on phone. Finance prompts with unsolicited 3rd party CPU cycles.

 

[capamac]

These LLMs are power-hungry. The PC folks are running 'em on these 350-watt Nvidia GPUs. To do much on an iPhone you'll have to use the phone plugged in to AC power.

 

[winxmac]

128GB iPhone models are still a thing despite being able to record 4K 60fps and even 4K 120fps but maybe on-device artificial intelligence models will force that to be 256GB minimum or even 512GB

 

[DanteHicks79]

👏 NOBODY 👏 WANTS 👏 THIS 👏 AI 👏 GARBAGE 👏

 

[tech_enthusiast_]

Perhaps we should focus on making the one that hasn't even rolled out yet work first

 

[PBG4 Dude]

128GB iPhone models are still a thing despite being able to record 4K 60fps and even 4K 120fps but maybe on-device artificial intelligence models will force that to be 256GB minimum or even 512GB

128GB models are limited with which video codecs they can use though. You need a minimum of 256GB to record pro res 4K video for example. Also limited to only 30fps 1080p.

About Apple ProRes on iPhone - Apple Support

Learn about Apple ProRes on iPhone. Apple ProRes enables multistream, real-time editing performance while delivering professional image quality.

support.apple.com

 

[turbineseaplane]

Larger models running directly on iPhones would allow for more Apple Intelligence features to run on device instead of on Apple's Private Cloud Compute servers, which could reduce Apple's costs and further enhance user privacy.

Is it more about "user privacy" or more about the "reduce Apple's costs" part?

🤨 🤔

I think they'll want to talk about the privacy part while really doing it for the reduced costs part.

 

[smeagol]

Ultimately, Apple shot themselves in the foot by either being stingy with RAM across all devices for decades, or by upgrading to higher capacity memory prohibitively expensive in the name of profits, saying nonsense like 8GB on an Apple device is like 16GB for everyone else. AI came along and told the truth, 8GB is 8GB.

 

[Taq'aix]

The AI bubble can’t burst soon enough.

 

[neuropsychguy]

This is the future. If we can have current model performance on-device, that will help solve a lot of the energy problems. It's likely years away (if it ever gets there), but it should be one of the goals.

 

[Radeon85]

👏 NOBODY 👏 WANTS 👏 THIS 👏 AI 👏 GARBAGE 👏

Speak for yourself. If nobody wanted it, it wouldn’t exist.

 

[963852741]

Imagine if some actual company had a real AI breakthrough. What ****storm would that create for these frauds.

 

[turbineseaplane]

Speak for yourself. If nobody wanted it, it wouldn’t exist.

AI does not exist right now, in its current form, because of demand for it.

 

[rme]

I think they'll want to talk about the privacy part while really doing it for the reduced costs part.

I'm fine with that! 👍

 

[hagar]

If Apple is smart, it’ll keep these AI features off the non-Pro models. A lot of people would buy a non-Pro iPhone instead of paying extra for AI they don’t want.

 

[rymc02]

Apple and Google have quietly just taken over the consumer market for AI. Regardless of your feelings towards AI on your phone, having all of you content to create content aware actions is the true value add over a standard chatbot like OpenAI. This will only improve over time. Johnny Ive can fart out any device he wants, but without user data, it's still just a chatbot. Outside of some vibe coding, consumer AI will fall to the easiest and most convenient use case, which will be our personal devices.

Now with metering, pro-sumer/enterprise users have quickly found out it's basically cheaper to higher a worker than to use AI and have basically admitted they can't even monitor their ROI on their spend.

Hospitals, banks, law firms, etc., all need in house LLMs due to privacy concerns. in Europe, most are using open-source LLMs.

The only real market left for these AI companies is the research fields. And if you were really worth a trillion dollars bc your product worked, you wouldn't just lease them the LLM and say "Good luck, hope you can make something!". You would tell them you want a percentage of whatever you make. A drug company develops a new drug, OpenAi would get a cut of that drug. That would show the product works and is worth the supposed valuation. The fact that they don't do that kinda lets the cat out of the bag.

And ultimately, it leaves them with a product that will only get more expensive and will probably find itself with less and less users as we move to on device and open source that are good enough in the coming years.

 

[963852741]

Speak for yourself. If nobody wanted it, it wouldn’t exist.

Not the first time people have fallen for this BS.

 

[ChrisA]

👏 NOBODY 👏 WANTS 👏 THIS 👏 AI 👏 GARBAGE 👏

I think statements like the above come from people who don't know what "AI" is. They think AI is a GPTchat or some other generative text app.

But other examples of AI are the map app you use to navigate your car and the spell check that runs as you type. All voice assistants are AI, even the dumb ones like Siri.

Then there is the AI you don't see. The one that looks at credit card transactions and tries to find if someone stole your account or if the transaction looks good.

What about the AI in a car that reads road signs like "no left turn" or just basic stop signs?

Or those short summaries of each email you see before to click to see the full-length version.

I could likely list a dozen more.

And you are wrong; that "nobody" wants it. You don’t, but quite a few people find it useful and even ask for it to be better.

 

[JMalone]

You can’t run full 27 billion parameters on a 12-16GB RAM phone unless you reduce quality a lot to something like 1 bit or reduce active params.

There’s no magic sauce otherwise everyone would be doing it with PrismML.

 

[bluecoast]

Is it more about "user privacy" or more about the "reduce Apple's costs" part?

🤨 🤔

I think they'll want to talk about the privacy part while really doing it for the reduced costs part.

I guess though that it is ok that the two are mutually beneficial?

Apple’s material recycling measures are good but are clearly also about reducing costs for Apple.

Apple clearly always wanted to run AI and ML on devices - that’s what they do, they’re not really a cloud company - and I’m ok with that.

Users get advanced on device ML and Apple gets people to upgrade in order to do this using devices with even more NPU, 512 GB storage being the minimum & 24+ Gb of RAM.

…All of which will push the pro phones nearer to $1500-2000 in today’s ram and ssd starved market.

Hmm. Maybe you have a point, haha!

 

[turbineseaplane]

I think statements like the above come from people who don't know what "AI" is. They think AI is a GPTchat or some other generative text app.

It's not that people don't know what it is, it's that (to your point above) a zillion things have been roped into the same term of "AI".

A whole lot of it is just Machine Learning rebranded and there are excellent use cases and things we can get from that.

 

[neuropsychguy]

👏 NOBODY 👏 WANTS 👏 THIS 👏 AI 👏 GARBAGE 👏

AI on device or AI at all?

Considering ChatGPT has close to 1 billion weekly active users and other models getting heavy use, it seems like at least a few people are using AI. It's a big leap to claim that none of the users really want it.

It's okay to not like it, but don't claim no one likes it or wants it.

 

[JMalone]

👏 NOBODY 👏 WANTS 👏 THIS 👏 AI 👏 GARBAGE 👏

There’s 2.5 billion active users and subscribers across all AI services. That demand has driven up memory and storage costs. If the demand wasn’t there you would be paying 80 bucks for 32GB RAM sticks. But it’s 500 bucks instead.

Don’t be a tech denier otherwise you’ll always be posting cope material. Nobody gave you the right to speak for the rest of humanity and the economy.

 

[JDHiro]

It seems like ASICs and NPUs make a lot more sense on mobile devices - bake the models directly into the hardware to run them fast while sipping power.