There is as new wave of people setting up computers locally (both personal and business) for AI inference.
It is striking to me that right before this happens, Apple discontinues the Mac Pro which was perfectly suited for this. We go decades with the Mac Pro having sluggish sales and then the year that it will actually start selling like hotcakes, it's gone!
I guess Nvidia captures the market share with RTX Spark, that they announced a couple of days ago.
For the uninitiated, running a local LLM requires enough VRAM to fully fit it in memory. Models get huge very quickly. Because of the unified memory, this means you can fit much larger models on a mac instead of a traditional machine that has a separate video card (the video card has its own RAM and video cards with a lot of RAM are very, very expensive)
It is striking to me that right before this happens, Apple discontinues the Mac Pro which was perfectly suited for this. We go decades with the Mac Pro having sluggish sales and then the year that it will actually start selling like hotcakes, it's gone!
I guess Nvidia captures the market share with RTX Spark, that they announced a couple of days ago.
For the uninitiated, running a local LLM requires enough VRAM to fully fit it in memory. Models get huge very quickly. Because of the unified memory, this means you can fit much larger models on a mac instead of a traditional machine that has a separate video card (the video card has its own RAM and video cards with a lot of RAM are very, very expensive)
