Is local AI is the new Personal Computer boom?

[innerproduct]

There is a clear trend where some people like to use local AI instead of cloud based due to many different reasons. To me this trend reminds me of the Personal Computer revolution where earlier there were just mainframes and mini-computers at the hands of government and large corps. The PC took some of that power and made it available to everyone. No big-brother involved.
If you have tested local LLMs you know they are quite far from frontier models like those from OpenAI, Anthropic and Google. But not that far of and it is thinkable that personal AI is viable quite soon. And with recent changes in how well the "cheap" services work like the quality regression on Claude 4.7 one wonders how this will play out. I pay 200-400 $ a month for AI services right now and am seriously considering getting a local beast. Clearly I would have to pay 200K for getting something useful right now but tech moves fast.
To me it seems that Apple would be perfectly suited to introduce machines that cater to this need but I get the feeling they have been taken by surprise and are out of sync. Sure the MS with 512 GB RAM and the TB5 RDMA support shows that they are aware of the development but the dropping of MacPro seem an unusually bad move. Instead there could have been "multi-ultra" machines sold at a premium to this segment. Heck people pay a premium for the MS m3u on the second hand market already.

What do you think? Will the need for local AI drive a new desktop boom or will it not be possible due to the immense pressure from hyperscalers etc? Short term? Mid term?

 

[throAU]

Yes.

AI is the biggest thing to happen in computing in my career, which started in the early 1990s and there are MASSIVE developments in smaller models that can be run locally.

For example, Qwen 3.6 which will now run in a 36-48 GB macbook performs like GPT 4.0 which was state of the art and needed 1TB of memory about 12-18 months ago.

 

[JPack]

No, because the current "PC" is the smartphone.

Most people want to keep conversations across devices and have unlimited contextual memory.

 

[Performa460]

There is a clear trend where some people like to use local AI instead of cloud based due to many different reasons. To me this trend reminds me of the Personal Computer revolution where earlier there were just mainframes and mini-computers at the hands of government and large corps. The PC took some of that power and made it available to everyone. No big-brother involved.
If you have tested local LLMs you know they are quite far from frontier models like those from OpenAI, Anthropic and Google. But not that far of and it is thinkable that personal AI is viable quite soon. And with recent changes in how well the "cheap" services work like the quality regression on Claude 4.7 one wonders how this will play out. I pay 200-400 $ a month for AI services right now and am seriously considering getting a local beast. Clearly I would have to pay 200K for getting something useful right now but tech moves fast.
To me it seems that Apple would be perfectly suited to introduce machines that cater to this need but I get the feeling they have been taken by surprise and are out of sync. Sure the MS with 512 GB RAM and the TB5 RDMA support shows that they are aware of the development but the dropping of MacPro seem an unusually bad move. Instead there could have been "multi-ultra" machines sold at a premium to this segment. Heck people pay a premium for the MS m3u on the second hand market already.

What do you think? Will the need for local AI drive a new desktop boom or will it not be possible due to the immense pressure from hyperscalers etc? Short term? Mid term?

I don't want to think about what they would cost!

That having been said, I wonder how common the need is to run AI locally. I'd say 99% of the people I know wouldn't even know what that means--although many of them have no doubt casually used "AI" on their phones, tablets, and computers.

 

[innerproduct]

Maybe ”need” is not the right phrasing for most. But it should be if anyone cared about freedom , self determination , resilience etc.
Right now such a machine cost too much, but if there were a player that could swallow the upfront costs for a few years in order to own the market…
Cloud and SaaS is so dystopian. My whole being tell me that we all should opt out from services as soon as possible.

 

[I WAS the one]

I am 52 years old, and I started with a PowerMac 6300CD back in 1992. Local LLMs are a major trend now. For instance, there is a developer who is very close to achieving an ideal local setup, allowing you to sync from your computer to your iPhone using local LLMs. The app is called Lehk AI Pro. No, I'm not affiliated with that app; I just mention it because it works really well for keeping things local and synced. Currently, I use a Mac Mini M1 (a 2020 model with 16 GB of RAM), and I run EnConvo AI and Fluent AI at lightning speed! I love these apps and hope they offer an iOS version in the future. The local LLM models available are so powerful that, to be honest, I don't know how big companies will entice users to step out of their comfort zones. To give you an idea, I was a pro subscriber of Perplexity and Claude, but now I do the same things for free. If I need to use paid models, I just invest $20 at OpenRouter to get what I need. I mean, think about it: why pay monthly just to show that you're using a high-end model? That's only beneficial for large companies, but for personal use? Come on... we are in a golden age with these local options.

 

[throAU]

No, because the current "PC" is the smartphone.

Most people want to keep conversations across devices and have unlimited contextual memory.

Give it a few years - what is running in local AI on PCs with 32 GB of VRAM today will be running on-device on a smartphone with more capable models in 2028.

I remember when google maps wowed me on the desktop... a couple of years later (and I. mean literally, it was no more than 2-3 years) we had global colour maps with GPS in our pocket.

 

[splifingate]

If I hadn't purchased my hardware early last Summer, I'd probably be scratching my head on the sidelines 🤷‍♂️

The cost of just the 64GBx2 DIMMS for my Supermicro AM5 Epyc has skyrocketed 400% since then (if you can actually source 'em)! The two Gen5 m.2's reflect a similar increase.

With one AMD R9700 Pro I'm getting 150+ t/s with Qwen3.6-35B-A3B-MTP-GGUF using about 30GB of VRAM. Adding another R9700 this week (price has only gone-up roughly USD100 since last year) for total of 64GB VRAM to use larger FP models.

It can already spit-out complete PHP scenarios in less than a minute; PHP that it took me 15+ years to {mostly} learn in my spare time. If I had no prior understanding, I wouldn't know how/what to vet. And in what we get there are mis-takes, to be sure: this ain't God were talking to 😉

I am more concerned about my personal data not being used to train others' models by keeping things local than I am about tossing $$$ towards o/l tokens.

I'm just shy of USD3K-in on the bespoke Local AI comp, and the addition of the additional GPU will make it comparable to the cost of the Spark.

Decided to not get another Spark (zero val-prop for me), and just wait for the M5/6 Max/Ultra/Über Studio.

Local is the new Frontier Model 🙂

 

[maflynn]

f I hadn't purchased my hardware early last Summer, I'd probably be scratching my head on the sidelines 🤷‍♂️

Same
I picked up a base level M4 Max Studio, and I'm loving it. Great computer, probably the best desktop I've owned.

Had I hemmed and hawed and procrastinated, I pretty much guarantee I'd not be buying anything

As for local AI - nope, I use chatgpt and claude ai. I've played with ollama, on my mac, but my studio doesn't have the ram to get some of the larger AI stuff running.

 

[splifingate]

No, because the current "PC" is the smartphone.

Most people want to keep conversations across devices and have unlimited contextual memory.

To -whit:

 

[splifingate]

I've played with ollama, on my mac

This may appear crass (with no intent implied), but:

Friends Don't Let Friends Use Ollama | Sleeping Robots

Ollama gained traction by being the first easy llama.cpp wrapper, then spent years dodging attribution, misleading users, and pivoting to cloud, all while riding VC money earned on someone else's engine. Here's the full history, and why the alternatives are better.

sleepingrobots.com

 

[splifingate]

To -whit:

To whit-whit:

 

[maflynn]

This may appear crass (with no intent implied), but:

Friends Don't Let Friends Use Ollama | Sleeping Robots

Ollama gained traction by being the first easy llama.cpp wrapper, then spent years dodging attribution, misleading users, and pivoting to cloud, all while riding VC money earned on someone else's engine. Here's the full history, and why the alternatives are better.

sleepingrobots.com

lol that’s ok. No offense taken.