Do you run LLMs on your Apple Silicon Macs?

[VitoBotta]

I used to play around with LLMs on my M4 Pro mini last year, but I kind of stopped because I only have 64 GB of RAM. That meant I could only run small models, and those just aren't very useful compared to the hosted ones. What about you? Are you actually using LLMs on your Mac for real work? If yes, which models are you using?

 

[maflynn]

Check out the AI forum here at MR, there's a bunch of folks running LLMs on their Macs

 

[Singular1ty]

I've tried a couple of times on my M1 Pro 16GB RAM but got the same results, the best LLM it can run is minimax-2.5 with a "not so low" response time on Ollama, bigger ones, like kimi-k2.5 take forever to answer.

 

[BotchQue]

Not intentionally, but often I ask Siri something and she goes "That's too hard, kin I ask ChatGPT?"