Yes. It can be difficult to conceptualize what exactly the user is getting with each step-up in cores and RAM in Apple's M SOC. The benchmarks help, but, it just seems easier to grasp the discrete GPU gains in PC-land versus the M-class chips.
Thanks for the reference to the RTX 3060 estimation, that's a good approximation. Was interested in that, and also running local LLMs.
For reference, running a base model M2 Max Studio w/ 32GB RAM, I downloaded DSeek R1, 70b at 43GB. My first question in the terminal was to calculate the time to travel, in years, from Earth to Jupiter, assuming a velocity of 60 miles per hour. After a half hour, I had -- literally -- two words of an answer. Two words. So plainly the M2 Max base utterly and completely choked on 70 billion parameters. Had to control-Z that terminal.
But the M2 Max can locally run R1 32b (20GB) at a pretty good speed. It spits out answers to even complex questions with hardly any slowdown. I am so curious to know how an M4 Max at 128GB RAM will handle some of the bigger models. We'll have to wait for the tests. I hope some people here put their machines through the paces of the bigger LLMs.
