Got a tip for us?
Let us know
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
Apple Exploring Ways to Run Much Larger AI Models Directly on iPhones
- Thread starter MacRumors
- Start date
- Sort by reaction score
turbineseaplane
macrumors Penryn
All potentially good stuff, when/if it works correctly and depending upon what the tradeoffs are to getting it.
A huge part of AI pushback is that there is no discussion, evaluation or even consideration of tradeoffs going on.
It's not ok to force AI data centers into communities that don't want them and against vociferous opposition, to use their water, pollute their air and raise their own costs (while providing basically no ongoing jobs) -- as just one example here.
Enterprise AI is already here and works well for businesses.
Apple's AI version is lacking innovation.
turbineseaplane
macrumors Penryn
Devil is in the details.
The more that Enterprise is being asked to pay the actual costs, the more they are questioning its role in their business.
This story is only recently starting to unfold and is interesting to watch.
StellarVixen
macrumors 68040
Yes please. Power of the AI should be in people’s hands, not given to someone else to distribute to us how they see fit.
jz0309
macrumors P6
This is all ok and I certainly understand that people are using "AI" and it's beneficial for them. Some of us do not use it nor want to have it on their phones especially, and where iOS seems to be going is that it is forced down our throats.
What I would like to see is either the ability in the OS to disable "AI" or to provide HE that is not suitable for "AI".
Choices is a good thing
What I would like to see is either the ability in the OS to disable "AI" or to provide HE that is not suitable for "AI".
Choices is a good thing
Enterprise is used to paying the software costs where AI is the next evolution whether we like it or not.
“Use” isn’t “pay for”. AI is pushed on many people in day to day life and work but very few people actually pay for it or find it that valuable.
sjsharksfan12
macrumors 68030
I feel like it's being forced on us by elite billionaires. It's irrelevant if we want it or not. It's what they want.
They don't care about customer privacy (see the NHS in the UK for example), and western governments *love* Anthropic and OpenAI. They're eager to work with governments and governments can use them to manipulate and spy on populations.
Likely they’re looking forward to something better because “the one that hasn’t even rolled out yet “ doesn’t work all that good.
There are many comments here, and not one about PrismML's new technology.
What they have done is invent a new way to compress a neural network to one bit per parameter. This means each parameter is just a one or a zero. Not only does this save space, it saves a LOT of space. Now Apple's 10B-parameter on-device model will fit in just over 1GB of RAM and hence comfortably into a 6 GB iPhone. (The iPhone 15 has only 6GB of RAM.)
Not only does it save space, but it also runs with less energy because it is very easy to multiply by 1 or by 0. Most of us can do that kind of math in our heads.
How does it work exactly? I don't know yet. I assume it is not so easy as simply normalizing all values to the 0...1 range and thresholding at 0.5. I suspect that replicating the "important" parameters is involved, but I don't know how you would find them.
PrismML says they are not done yet. Of course, a width of 1 is the shortest possible, but maybe they are reducing the number of parameters without doing much harm?
PrismML says the work is based on mathematics. They don't claim AI breakthroughs or better code. This might mean they have some Linear Algebra experts.
Maybe someone here has some better insight?
ApostolisApo
macrumors regular
tell that to the millions of people paying of it. There are many false players, overhyped stuff etc but the same happened with the .com bubble. But look where we are now in regard to Internet 😉
I keep waiting for the "electricity bubble" to burst. I mean, it has gotten out of hand; almost everything I own has some need for power. Whatever happened to basic tools like a hammer and simply building a fire? Can't people walk? We don't need cars or airplanes. I think electricity is a fad, and some day people will get over it.
I think already AI is here to stay. People want cars that self-drive, and people want to use robots in factories and for other kinds of work. They like it that credit card companies can find fraud on their card and not charge their account
ApostolisApo
macrumors regular
wow an actually useful comment instead of copy-paste nonsense
Populus
macrumors 604
Yeah, this is the key. Please Apple, keep walking the local LLM path. On-device AI is the way to go!
On device compute is the future for normal AI usage. Yes there will be data centers too though.
Nvidia RTX Spark is already an example.
Nvidia RTX Spark is already an example.
Last edited:
McKodiak
macrumors 6502
This is not the future. New AI models and the hardware specs they require will always outpace your personal hardware and their limited specs. User privacy on cloud will need to improve but it's the only path forward. Even when a new personal device is released, it will already be unable to run the current models. Your personal device will need to be a gateway device to AI.
Apple should try to contact Romke Jan Bernhard Sloot through a medium. https://en.wikipedia.org/wiki/Sloot_Digital_Coding_System
Kriss_De_Valnor
macrumors regular
turbineseaplane
macrumors Penryn
AI and Electricity as essential utilities have absolutely nothing in common.
That is really not a good comparison to attempt to make.
Astuces iOS
macrumors 68030
BombDialog
macrumors newbie
Apple claims Private Cloud Compute extends the iPhone privacy promise to the cloud: no one (including Apple or Google) can glimpse inside its black box at all.
If so, doesn’t that preclude any privacy impact from shifting compute from PCC to device in the first place?
Put another way, the claim that doing so could enhance user privacy seems to undermine PCC’s ostensible privacy guarantees.
The user benefit would appear to be speed, not privacy.
turbineseaplane
macrumors Penryn
Current phase of the AI companies trying to figure out where to put data centers (since nobody wants them).
One co-sponsor, State Representative Amanda Clinton, a Tulsa Democrat and Cherokee, called the frenzy “the new land run.” Still, she understands the appeal.
“I think Oklahoma is so strained for jobs and economic development that we will roll over too easily and give away the farm,” she said while driving around the perimeter of Project Clydesdale, a $1 billion, 500-acre data center now under construction in Tulsa County.
It's so sad to me to see the misrepresentation of what these data centers are actually bringing to communities.
The "jobs" are just for construction. Once the centers open, the jobs numbers are absolutely tiny.
They want all your resources and somewhere to stick all the externalities.
One co-sponsor, State Representative Amanda Clinton, a Tulsa Democrat and Cherokee, called the frenzy “the new land run.” Still, she understands the appeal.
“I think Oklahoma is so strained for jobs and economic development that we will roll over too easily and give away the farm,” she said while driving around the perimeter of Project Clydesdale, a $1 billion, 500-acre data center now under construction in Tulsa County.
It's so sad to me to see the misrepresentation of what these data centers are actually bringing to communities.
The "jobs" are just for construction. Once the centers open, the jobs numbers are absolutely tiny.
They want all your resources and somewhere to stick all the externalities.
conmee
macrumors 65816
Local inferencing takes more 1) Memory 2) Bandwidth 3) Cores; in that order. Without sufficient memory you simply cannot load larger models. There are a handful of 4-bit quantized models ranging up to ~20B parameters that can fit in 8-12GB but you begin to sacrifice not just accuracy but also run into great frequency of hallucinations. And the more you tune for memory constraints the more compromises a model must make. They get better every day but there is no getting around the need for sufficient memory. There’s also other model factors to consider. Context/KV Cache how much input a model can process at once; RAG how much ancillary information (your documents, photos, other data) the model can incorporate to personalize responses; will the model only process text input or can it also evaluate images, etc); all facets that require memory. If Apple wants to run larger quality models on iPhones/iPads the first and only thing they need to consider is more memory. If they don’t address that variable, more bandwidth and cores won’t make any difference in model quality but will improve responsiveness.
I’ll say that for the types of things Apple wants to do on device, they don’t need super huge models to make Siri and various AI features useful. But larger models with more precision will make the device appear “smarter.” Increased memory bandwidth and GPU/neural cores drive responsiveness. My original M5 Pro was more than adequate for local LLM processing (qwen3.6 36B 8-bit and gemma3 27B 8-bit), but my M5 Max is noticeably faster.
P.S. I just noticed someone mention PrismML above. I too haven’t dug into the details, but suffice to say that it still looks more like proof of concept at the moment. They tackle fine-tuning the algorithm to work with highly quantized models that require less memory first, then figure out how to balance/optimize for precision/performance. The “easy” part is shrinking the model, the more difficult part is what is the limit before accuracy suffers to an unacceptable point. It also appears that their approach requires a lot more processing power to achieve similar results to existing quantized models, meaning at present their solution may overtax existing iPhone hardware and drain the battery at an unacceptable pace.
I’ll say that for the types of things Apple wants to do on device, they don’t need super huge models to make Siri and various AI features useful. But larger models with more precision will make the device appear “smarter.” Increased memory bandwidth and GPU/neural cores drive responsiveness. My original M5 Pro was more than adequate for local LLM processing (qwen3.6 36B 8-bit and gemma3 27B 8-bit), but my M5 Max is noticeably faster.
P.S. I just noticed someone mention PrismML above. I too haven’t dug into the details, but suffice to say that it still looks more like proof of concept at the moment. They tackle fine-tuning the algorithm to work with highly quantized models that require less memory first, then figure out how to balance/optimize for precision/performance. The “easy” part is shrinking the model, the more difficult part is what is the limit before accuracy suffers to an unacceptable point. It also appears that their approach requires a lot more processing power to achieve similar results to existing quantized models, meaning at present their solution may overtax existing iPhone hardware and drain the battery at an unacceptable pace.