iPad Pro Do you run any local LLMs on your iPad Pro?

[thadoggfather]

It’s kinda wild to see offline qwen3 variants running on this thing pretty well and such small file sizes

I have the m4 1tb/16gb model and I am impressed considering its a tablet

Got a DeepSeek Qwen too and Phi

Just to play around. All in5 models for like lower 20gb

Nowhere near ChatGPT competence and depth and lack of hallucinatory responses and not really a review of those models necessarily but more just running them on an iPad and people’s experiences

Am I the only weirdo? 😛

 

[Digitalguy]

I do, I have the same iPad and use qwen and gemma 4 (plus a couple more to compare sometimes).
They run great and t/s are great too. Sometimes when I push it with high context I have some crashes, but overall it's even faster than on a 16GB RAM M4 Macbook (which makes sense since those are binned).
Anything less than 12GB RAM is barely suitable for local AI, but my iPhone 17 pro does very well too with 12GB.
And both the iPad and iPhone do better than any of my 16GB RAM Windows laptop in terms of local AI.
Having said that for more serious stuff, I use my strix halo laptop with 128GB of unified memory (which is similar to a M4 pro chip in terms of power) or my 20GB vRAM GPU (in a eGPU set-up).

 

[Performa460]

It never actually occurred to me to try and run any local LLMs on my M4 iPad Pro, although I have been playing around with them on my newest MacBook.

 

[thadoggfather]

I do, I have the same iPad and use qwen and gemma 4 (plus a couple more to compare sometimes).
They run great and t/s are great too. Sometimes when I push it with high context I have some crashes, but overall it's even faster than on a 16GB RAM M4 Macbook (which makes sense since those are binned).
Anything less than 12GB RAM is barely suitable for local AI, but my iPhone 17 pro does very well too with 12GB.
And both the iPad and iPhone do better than any of my 16GB RAM Windows laptop in terms of local AI.
Having said that for more serious stuff, I use my strix halo laptop with 128GB of unified memory (which is similar to a M4 pro chip in terms of power) or my 20GB vRAM GPU (in a eGPU set-up).

What about do you use to run them? Which do you prefer between Gemma and Owen?