I'm gunna upgrade so hard. Have been waiting for this one!
This is exactly the sort of well-informed, comment that we could do with more of here. Thank you.lol what
pytorch runs fine
Local models work fairly well despite the drawbacks of M4 (low inference speed mostly).
M5 will hugely improve this: per GPU core neural acceleration, matrix math improvements, etc. Inference should be significantly faster depending on how high they get the memory bandwidth and what they do with the interposer and SoIC / chip stacking, if it happens this generation.
There are metal optimized libraries that work pretty well and there is an apple-funded CUDA -> MLX workflow in progress.
This machine is going to be excellent for personal and professional AI development / fine-tuning, and for ~$12k it’s going to cost what one nvidia card with 1/5 of the GPU memory does.
If you need really big data nvidia stuff you can just connect to the cloud, but for local work nothing will compare to this for a long time that isn’t 10x the cost.
I say all this also owning a 5090 that’s going into a Linux workstation next week for local CUDA work by the way.
Now, fi you were talking about Apple Intelligence… fair point! I have low faith especially given the new rumors that they are not teaming with Anthropic which is an enormous mistake, if true.
Apple’s small models are doing pretty cool stuff but they’re going to take a couple more years to be really relevant relative to the best of what else is out there now, and that is also dependent on them retaining talent. They are doing groundbreaking R&D work with on-device models particularly in RAM constrained areas, but everywhere else they are trailing pretty badly.
Sadly the Mac Pro is not a priority for Apple, and now that they’re all-in on integrated GPUs and memory, I don’t think expandability or modularity are either.They need to release an M5 Extreme chip for the Mac Pro. Such a chip would probably outperform the best current Threadripper chips. The Mac Pro needs more differentiation from the Mac Studio. Support for eGPUs and more expandability or modularity would be good, too, especially upgradable memory.
I have the M3 Ultra with 512GB Unified Memory. It’s amazing for AI/LLMs. Cannot imagine what the M5 will have. A day one upgrade if it has 1TB Unified Memory.Been weighing an M4 Max Studio to replace my M1 Pro MBP as main workhorse. I think maybe I can hang on a little longer if an M5 Ultra is on the horizon.
Or will they present M5 Extreme? Or is it dead completely?......waiting for Russian comrades to make their unboxing video
Also, M5 is based on the A19, it has the same security features exclusive to that design. We know this already based on the recent release. It’s (I think?) the first time they are at parity within weeks of each other.
I think some (very few) people want to use very specific and expensive PCI cards and an old (but still quite powerful) Mac is better than nothing for them.When are they going to stop goofing around and discontinue the Mac Pro? It is still on M2 and still $7k.
If it has that (M5U + 1TB) and it comes in under £4000, it’ll be a day one purchase for me, too. Lol. Sadly, I think I might be a tiny bit out on price. 😂I have the M3 Ultra with 512GB Unified Memory. It’s amazing for AI/LLMs. Cannot imagine what the M5 will have. A day one upgrade if it has 1TB Unified Memory.
I’m taking the liberty of selectively quoting a couple of sentences from your excellent post, I hope that’s okay.This machine is going to be excellent for personal and professional AI development / fine-tuning, and for ~$12k it’s going to cost what one nvidia card with 1/5 of the GPU memory does.
They are doing groundbreaking R&D work with on-device models particularly in RAM constrained areas, but everywhere else they are trailing pretty badly.
I think the Mac Pro will get the m5 ultra, and the Mac Studio will get m5 max + m4 ultra option. but that's just speculation. Unless they are thinking of killing the Mac Pro hehBeen weighing an M4 Max Studio to replace my M1 Pro MBP as main workhorse. I think maybe I can hang on a little longer if an M5 Ultra is on the horizon.