Since you will be using 8GB RAM iPhones in ten years time, that is a definite “no”.
Got a tip for us?
Let us know
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
Mac Studio With M3 Ultra Runs Massive DeepSeek R1 AI Model Locally
- Thread starter MacRumors
- Start date
- Sort by reaction score
Please provide a source for where Deepseek running locally can achieve this. Not being a jerk, but all AI models aren’t created equally and there is a lot of proprietary stuff going on that is not published research at OpenAI, Microsoft, Anthropic, etc.
I don’t think there is one because Deepseek has not demonstrated this capability especially with respect to multimodality. If there is I’d love to read it and welcome you to share. Perhaps someone fine-tuned Deepseek within 2 months to do this, and I’d really like to read that research if it exists but I imagine this was a broad statement and not specific which is important to understand.
My personal use case right now is brainstorming a novel I’m writing. Online AIs tend to be too highly censored to be useful.
I don’t need to use massively large models for this (between 10B - 27B is fine), but I do want a decent sized context window and that takes additional RAM.
It was a broad statement expressing what i believe you can do with current hardware/software applied to a healthcare business
I wouldn’t be surprised if Apple sold accelerator cards for the next gen Mac Pro featuring M3 Ultra chips with 512 GB each… package them as server racks and you have something that could hurt nvidia badly.
Using Ollama I've got the R1 70b running on my Mac Mini M4 Pro (14/20) which has 64G of ram.
It's usable in that the output is about the same speed at which I read at.
Are these useful to run locally?
Not really, not yet anyway. I have a subscription to OpenAI which I use all the time.
What is interesting is that this is the beginning of a new direction in desktop computing.
Apple appears to have been caught napping with AI and still hasn't got its act together with it.
The interesting thing, for me anyway, is how this need or ability to run LLMs on personal computers locally will influence the evolution of Apple Silicon. We probably won't find out for another 2 years though.
Nvdia's making good money on this revolution, it'll also be interesting to see what happens to the RAM market with local AI on personal machines requiring larger amounts of RAM that traditional applications.
It's usable in that the output is about the same speed at which I read at.
Are these useful to run locally?
Not really, not yet anyway. I have a subscription to OpenAI which I use all the time.
What is interesting is that this is the beginning of a new direction in desktop computing.
Apple appears to have been caught napping with AI and still hasn't got its act together with it.
The interesting thing, for me anyway, is how this need or ability to run LLMs on personal computers locally will influence the evolution of Apple Silicon. We probably won't find out for another 2 years though.
Nvdia's making good money on this revolution, it'll also be interesting to see what happens to the RAM market with local AI on personal machines requiring larger amounts of RAM that traditional applications.
Last edited:
How much has top spec VRAM in a Mac increased in the past 11 years? 12GB in 2014 to 512GB today. That is over a 42x increase and now VRAM is significantly more important.
Correct... and also not.
"VRAM" has become an enchanted term (thanks to the likes of Nvidia).
All of the RAM for Apple Silicon Macs are the same: LPDDR, one or more chips, directly connected to the SoC.
And that is why an Apple Silicon device, as currently implemented, will not perform as quickly as the Nvidia systems with their custom HBM implementation. OTOH, Macs are less expensive than Nvidia systems.
Pass. My mother is 85, she's never going to get those terminal commands right.
That's true, but a bit misleading. We're not gonna see another leap like that.
And it isn't technically 512; while the GPU cores can share RAM, the CPU cores (and NPU cores) will be using plenty for themselves that the GPU cores have no use for.
Thanks for the info.
Next year I need to change my desktop, and I'd like to carefully consider the amount of ram I need.
I'm on 128GB now (M1 Ultra, but I'd consider getting 256).
I know it's off topic but this announcement by Nvidia today is pretty insane
My experience is that there’s diminishing returns with Mac when it comes to LLM.
More RAM enables you to load larger LLMs, but it doesn’t necessarily run them quickly. For me, I think the “sweet spot” is 128GB. If I went higher and put in larger LLMs, I’m not sure the time to first token and t/s would make it particularly useful. Even though I can load up 70B models in my Studio, I’m more likely to use something around 30B because it’s more responsive.
And if I run a 30B model and also do something else intensive alongside it (like a Stable Diffusion AI image generation), both slow down - and nothing is using swap space, so there’s no disk-swapping slowing it down.
I’m interested to know what others get with large models and the Ultra (the Ultra has even higher memory bandwidth), but I’d be surprise if large models (400B+) in a 512GB RAM Studio is practical on a daily basis, especially if you want to run anything else alongside it at the same time.
