One county in Virginia has 37 data centers. I'm sure they didn't consider the implications of all those data centers when it came to their electricity bills. There is short term gain in jobs building the data centers.
Got a tip for us?
Let us know
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
Apple Exploring Ways to Run Much Larger AI Models Directly on iPhones
- Thread starter MacRumors
- Start date
- Sort by reaction score
Just don’t remove the Kill switch, because the first thing I do is turn off AI. It drains the battery and it drains humanity.
Apple has held meetings with PrismML about ways it could use the startup's technology to run much larger AI models directly on iPhones, according to The Information.
The report said PrismML has managed to shrink down Alibaba's open-source large language model Qwen 3.6 to run entirely on an iPhone 17 Pro. The model has 27 billion parameters, which is larger than Apple's on-device AFM 3 Core Advanced model with 20 billion parameters. Apple's model powers iOS 27 enhancements such as Siri AI's more expressive voices and improved systemwide dictation on iPhone 17 Pro and iPhone Air models.
Unlike with AFM 3 Core Advanced, all of Qwen 3.6's parameters can be active at the same time.
"One new on-device Apple model has 20 billion parameters but uses a so-called sparse architecture, in which only 1 billion to 4 billion parameters are active at a time," the report said, in reference to AFM 3 Core Advanced. "In the case of PrismML's on-device model, all 27 billion parameters are active at the same time."
Larger models running directly on iPhones would allow for more Apple Intelligence features to run on device instead of on Apple's Private Cloud Compute servers, which could reduce Apple's costs and further enhance user privacy.
Article Link: Apple Exploring Ways to Run Much Larger AI Models Directly on iPhones
turbineseaplane
macrumors Penryn
Are AI supporters, users and evangelists concerned about the "theft" parts of the tech?
(like model training, or the below 👇, etc)
(like model training, or the below 👇, etc)
inventor7777
macrumors member
Regardless of how you feel about AI, I do think that if you're going to run them for simple tasks, running them on the device is better anyway. It works offline, uses less electricity (compared to spinning up a model at a datacenter to ask for the nearest grocery store), is possibly more private, lower latency for back-and-forth device tool calling APIs, doesn't cut into your mobile data, and doesn't depend on cloud providers.
So in that way, I am happy that the AI is on device. However, non-technical people won't understand why there's so much storage in use, so It would be nice if Apple added a hidden partition somewhere that could store the models needed for various tasks without cutting down your storage.
So in that way, I am happy that the AI is on device. However, non-technical people won't understand why there's so much storage in use, so It would be nice if Apple added a hidden partition somewhere that could store the models needed for various tasks without cutting down your storage.
It was for Macs.
The 8Gb of RAM working as 16Gb of RAM was for workloads typically encountered by most users:
1Gb*5 + 0,5Gb*22 =16 GB
Such workload was great for Apple Silicon Macs due to changes in the memory architecture and macOS.
Where this didn't work was for workload where one application needed a lot of RAM. This worked poorly:
5Gb*1 + 3Gb*1 + 0,5Gb*16 =16 GB
LLMs fall in the latter category.
jimscard
macrumors 6502
Apple already has the tech to do this- they don’t need Prism or a Chinese model.
Apple has held meetings with PrismML about ways it could use the startup's technology to run much larger AI models directly on iPhones, according to The Information.
The report said PrismML has managed to shrink down Alibaba's open-source large language model Qwen 3.6 to run entirely on an iPhone 17 Pro. The model has 27 billion parameters, which is larger than Apple's on-device AFM 3 Core Advanced model with 20 billion parameters. Apple's model powers iOS 27 enhancements such as Siri AI's more expressive voices and improved systemwide dictation on iPhone 17 Pro and iPhone Air models.
Unlike with AFM 3 Core Advanced, all of Qwen 3.6's parameters can be active at the same time.
"One new on-device Apple model has 20 billion parameters but uses a so-called sparse architecture, in which only 1 billion to 4 billion parameters are active at a time," the report said, in reference to AFM 3 Core Advanced. "In the case of PrismML's on-device model, all 27 billion parameters are active at the same time."
Larger models running directly on iPhones would allow for more Apple Intelligence features to run on device instead of on Apple's Private Cloud Compute servers, which could reduce Apple's costs and further enhance user privacy.
Article Link: Apple Exploring Ways to Run Much Larger AI Models Directly on iPhones
But running a dense 27B model on a phone just makes it slow and energy-hungry while offering little benefit. Contrary to common belief, bigger isn’t necessarily better with LLMs for most use cases.
turbineseaplane
macrumors Penryn
Feels on point with the AI discussion these days 😂
If true, this isn't something which is publicly available, so everyone can't do it.
What PrismML has available for the public is an 1-bit 8 billion parameters model.
that's not really how development pipelines work
Tagbert
macrumors 604
Moving the processing from Private Cloud Compute to the device should not meaningfully change the privacy risk but it would change the cost structure significantly.
Dorv
macrumors 6502
Yes, that's what the whole article at the top of this page is, that PrismML did a proof of concept, and Apple is now talking to them about it to learn more.
You're right about the technical limitations, but look at where we were a year ago in this space, and 5 years before that, and 10 years before that. There will continue to be technological breakthroughs that push past those limitations.
fatTribble
macrumors 68030
I don’t use it either, but I’ll admit I’m curious about what the new Siri will be able to do.
If everything ran on-device I’d be much more enthusiastic.
I totally agree on the choice issue. While I understand the point that you don’t have to use it, many of the screenshots I’ve seen show new AI buttons front and center making them hard to ignore. I’d like to see more options like what was done with the Sharesheet. You can toggle features and reorder them.
sos47
macrumors 6502a
cloud based ai cannot do automations on your hardware. local ai can.
am i wrong?
This is good news for multiple reasons, and I really hope Apple gets this working well:
- The privacy advantages are massive; although Apple is probably more serious about privacy than any of the other players in the space (which range from shady to actively abusive), I don't trust their cloud compute much either, so having it on-device would be a huge leap in the right direction.
- On-device means not dumping ungodly amounts of money into AI companies.
- No datacenters necessary. Both in terms of concentrated blight, impact on the entire global electrical and electronic supply chain, and the power use.
- Power use, again. It's going to ratchet up power use of phones, cumulatively, but that's going to be distributed very evenly across the entire geographical area of the users, and the net impact to the grid when those phones are charging--most frequently at night--is going to be both concentrated in off-peak hours and easily absorbed by the existing generation and power infrastructure.
- The privacy advantages are massive; although Apple is probably more serious about privacy than any of the other players in the space (which range from shady to actively abusive), I don't trust their cloud compute much either, so having it on-device would be a huge leap in the right direction.
- On-device means not dumping ungodly amounts of money into AI companies.
- No datacenters necessary. Both in terms of concentrated blight, impact on the entire global electrical and electronic supply chain, and the power use.
- Power use, again. It's going to ratchet up power use of phones, cumulatively, but that's going to be distributed very evenly across the entire geographical area of the users, and the net impact to the grid when those phones are charging--most frequently at night--is going to be both concentrated in off-peak hours and easily absorbed by the existing generation and power infrastructure.
jz0309
macrumors P6
agree, curiosity is there for sure, but, I have a 17PM and my concern is that I won't be able to go back if I don't like 27 because of the "front and center", so as it stands right now, I will hold off on upgrading to 27 until after I get to play with an iPhone at the Apple Store, or, what would even be better, a toggle for Siri AI (like we have in 26 for Apple Intelligence" but I'm not holding my breath for that ;(
Incredible the degree of extreme comments being made here by people who have no FSCKING CLUE what PrismML does, how the tech works, or why this is interesting and significant...
Here's a quick summary:
The point is not ONLY the intelligence density, though obviously that's nice; it's also that binary and ternary models can be executed with much lighter weight MACs (ie you can pack many more of them into the same area, or alternatively run them with substantially lower energy) than the INT8 or FP16 HW.
And Apple likely already has the designs for such binary/ternary optimal HW in house (having acquired it via their purchase of xnor.ai). It's generally believed (though I haven't seen anything absolutely definitive) that they are already using such a 1bit DNN and associated HW on AirPods for various tasks.
Here's a quick summary:
The point is not ONLY the intelligence density, though obviously that's nice; it's also that binary and ternary models can be executed with much lighter weight MACs (ie you can pack many more of them into the same area, or alternatively run them with substantially lower energy) than the INT8 or FP16 HW.
And Apple likely already has the designs for such binary/ternary optimal HW in house (having acquired it via their purchase of xnor.ai). It's generally believed (though I haven't seen anything absolutely definitive) that they are already using such a 1bit DNN and associated HW on AirPods for various tasks.
Gee.
You think maybe that's why Apple is talking to a company whose entire product is about DRAMATICALLY SHRINKING THE SIZE OF LARGE AI MODELS?
??
???