Advise needed on getting the M5 Pro

[hidehide]

I’m interested in getting the M5 Pro with 64gb ram for local LLM. My main goal is to get myself hands on experience on local LLM with RAG experience. to build some iOS app and to stay relevant in the latest tech. Reason behind is because my company will be starting to use AI, I’m working in the team that builds RPA and workflow applications.l, we will be part of the main team to help other departments to incorporate AI in their daily work. I would like to expand my knowledge and equip myself better for the fast changing environment.

However, the MacBook Pro is going for a redesign soon in the next 12 months. Would it be unwise to invest so much on a laptop that will become an old design soon in a year?

I think lots of things will have happened if I waited, so I’m more inclined on getting the M5 Pro now. But would like to hear some more opinions.

I have thought of getting a used M1 Pro 16gb MacBook Pro and run cloud LLM at the moment. Then upgrade to the new MacBook Pro when it is released. But heard the price on the new one is going to hike. So that might not be a good choice in terms of financially.

Let me know your thought. Thank in advance.

 

[maerz001]

Can your company wait 12 months with the project just cos you want a new MBP design? Seems unlikely.

I doubt a touchscreen MB will help much in programming.
But when there is money to earn than the cost of a new machine is negligible

 

[Basic75]

What's your goal, getting now the computer you need now, or trying to maximize the time your computer is the latest to the market? Seriously, who cares about a redesign of all things when you have a job to do? And that's ignoring how everything about redesigns, price increases and future performance we pretend to know is just rumor.

 

[ddebacker]

Make sure the 64GB can accommodate the models you plan on using. You'd want 20+ tok/s generation and you'll be likely 'limited' to 34B param models with 64GB of RAM. Unless you're ok with 5-10 tok/s (as a comparison, a ChatGPT experience is 35-50 tok/s to have an immediate/chat like generation).
If your only goal is to run AI models with speed but less params, a PC with a nVidia 4090 RTX might be a better fit. When it comes RAG, it's all about GPU bandwidth and the RTX has roughly 3 times the bandwidth that the M5 PRO has (1000GB/s vs ~310GB/s for the Pro and ~600 for the Max with highest GPU). The downside of the RTX4090 is that it runs fast but only has 24 GB of GPU RAM so you'll only be able to run 13B to max 34B models).

Understand which model you need to run then make your choice.