Anyone tried LLM on Mac mini M4 / M4 pro yet?

[elnews]

My order on M4 Pro will only arrive in December 4th - 11th.. so i couldn't try them yet.

so i wonder if members here who have received the M4 / M4 Pro early .. have tried them and what's the t/s?

Could you provide the info in this format:
Mac Model:
LLM model & size:
Speed: t/s

Thanks.

 

[bobjonesco]

Macbook Pro 14" M4 (10/10) 16GB
Llama3.2:3b (Docker OpenwebUI)
38.9/s

 

[elnews]

Macbook Pro 14" M4 (10/10) 16GB
Llama3.2:3b (Docker OpenwebUI)
38.9/s

Thanks. Looks like the inference speed is not bad for M4.

 

[bobjonesco]

Thanks. Looks like the inference speed is not bad for M4.

I get about 50 on my base MBP M3 Pro for comparison.