Perfomance on M1 and M2 Macbook Pros (14 inch models) on AI, such as Stable Diffusion, Tensorflow, LLama and other AI models

[ARacoony]

I am currently looking at the base models of both of these computers and I would like to know how they perform on certain AI tasks, like the ones mentioned in the title.
How do the base models hold up? Will I notice any difference between M1 and M2 on these task (just for clarity, I don't mean the SSD speeds)? Is upgrading to 32GB worth the steep price?
Keep in mind I'm a student, and the reason I want a mac instead of a laptop with an Nvidia GPU, which yes it will do better, is portability and just not having a laptop that sounds like a jet engine plane. I'm going to move it around a lot, so what the MacBook device offers me is just better.
If you use these programs on your device, I would highly appreciate you tell me your experience with it.

 

[Xiao_Xi]

GitHub - apple/ml-stable-diffusion: Stable Diffusion with Core ML on Apple Silicon

Stable Diffusion with Core ML on Apple Silicon. Contribute to apple/ml-stable-diffusion development by creating an account on GitHub.

github.com

 

[Slartibart]

There are threads here already where you find probably answers, e.g. this one here - which is from a thread on deep learning performance.

Otherwise: what specifically do you want to do?

You ask on performance of »certain AI tasks« which you seem to equal to models - you just want to know how fast e.g. a picture is rendered in DrawThings or Swift Core ML Diffusers? There is already e.g. a thread related to that here too.
Otherwise here is some info on running Facebook’s LLaMa or Alpaca - including performance or RAM usage.

But again: what is the intended application?

nota bene: for just peeking into e.g. PyTorch or creating images using DrawThings a M1 with 16Gb RAM will do.

 

[ARacoony]

There are threads here already where you find probably answers, e.g. this one here - which is from a thread on deep learning performance.

Otherwise: what specifically do you want to do?

You ask on performance of »certain AI tasks« which you seem to equal to models - you just want to know how fast e.g. a picture is rendered in DrawThings or Swift Core ML Diffusers? There is already e.g. a thread related to that here too.
Otherwise here is some info on running Facebook’s LLaMa or Alpaca - including performance or RAM usage.

But again: what is the intended application?

nota bene: for just peeking into e.g. PyTorch or creating images using DrawThings a M1 with 16Gb RAM will do.

At the moment I am pretty new with this.
I have to get my foot in deep with a lot of LLMs. Other tasks might include trying courses on deep learning, voice cloning, video generation.
The main question is for someone who is starting out if the 16GB of RAM will do a great job or if when I start getting more and more experience it will hamper me in the long run.

 

[satcomer]

At the moment I am pretty new with this.
I have to get my foot in deep with a lot of LLMs. Other tasks might include trying courses on deep learning, voice cloning, video generation.
The main question is for someone who is starting out if the 16GB of RAM will do a great job or if when I start getting more and more experience it will hamper me in the long run.

On a M1-2 Mac then 16 is god if you play with a really big name audio rigs or 4-8k video files and have open more then two big programs at the same time! Take me as a network manager I've seen many DJs complain of have email, photoshop and audio daw all running at the same time on older equipment that sounded like a jet engine! I have to break hard to him that multitasking 4 programs at once is NOT good for his computer!

 

[4n0nym0us.us3r]

I would recommend to generally go for a 16 GB upgrade (even if the base model is sufficient for your normal use case).

But I wouldn’t recommend to go for further upgrades just for running stable diffusion or LLM related stuff, especially if you are on a budget.

Those applications need a lot of GPU power that apple just doesn’t provide yet. Even if you go for the highest possible upgrade you’ll get mediocre results at best.

Additionally Apples pricing makes those upgrades prohibitively expensive, especially if are on a budget. For the money you have to pay for this upgrades you’ll probably get a complete PC with at least twice the performance in those tasks.
To put the above mentioned benchmarks into perspective:
On an RTX 4090 you’ll get an effective speed of 44it/s. Using SD 1.5 model to create a 512x512 image takes about 5 seconds … for a batch of 4 images!!
Okay the 4090 is top notch, but smaller cards still provide a much better performance than any apple gpu for this use case.

My advice:
If the base model is sufficient for your other tasks then stick with it and, instead of spending your money on upgrades, get yourself a dedicated stationary PC with nice graphics. You can easily Remote Desktop into it and have it generate the images for you.

You could even do it with an iPad (that’s what I’m doing with my iPad Pro).

Just my two cents.