MP All Models Apple sleeping on the mac pro is comically bad timing

[mattspace]

I do this with an old Mac mini

Yes I did it as well, with a large DAS attached to the mini, but it still shares stuff with SMB, the avoidance of which is the whole point of the suggested product. It was a mediocre solution, compared to local storage on the machine.

 

[Charlie Bonesx]

Yes I did it as well, with a large DAS attached to the mini, but it still shares stuff with SMB, the avoidance of which is the whole point of the suggested product. It was a mediocre solution, compared to local storage on the machine.

so what exactly did this solution prevent you from accomplishing?

 

[mattspace]

so what exactly did this solution prevent you from accomplishing?

As I've said, there are certain apps and workflows, both Apple and third party, that don't work correctly when their storage, or the files being worked upon are not on an APFS volume, ie they're an SMB share.

 

[Omega Mac]

Agree, abandoning the "high-end may have been part of Apple's business strategy and it may have made the most sense overall. I think the OP was arguing it was a mistake. Time will tell. However, let's not pretend their current designs are not without tradeoffs or there were no alternatives. Maybe the best trade-offs given the chosen business strategy that may work out quite well for them but not without trade-off. And nobody likes being the traded-off.

They may have abandoned the old style of high end to circle back with a new style of high end, Apple style, but they had to get the other end and middle which is the mass market share in groove first.

So as you might expect of Apple culture, trying not to signal your prime market moves to the competition and forsake any market advantage, especially in a one more thing scenario (I'm not implying they are perfect or don't strategically leak either).

If the high end is being left to the very end, it may be explained as attenuated within the context of the complete realignment of the entire Apple range with Apple silicon - if you're re-inventing everything in these terms you have a type of leverage few if any have, then it's also logical that the high-end most demanding needs to be left to then end becasue it has to meet expectation while trying to exceed, after you've worked in and worked out a lot of the foibles, then you end with the High-end on a high!

I'm being optimistic.

 

[MoonCakeTropics]

For those that think the studio is an ok substitution for a mac pro, with AI or otherwise, youre wrong.

Beyond the snake pit of cables of the old trashcan mac showing the problem, the unified memory itself is not enough for AI.

Thunderbolt is a huge bottleneck. 100gbe ethernet is now "not a big deal" with 800gbe fiber ethernet available. You cant get 100gbe ethernet through an 80gbe thunderbolt port pinhole. You need PCIe.

Also, in addition to large amounts of RAM (something apple is UNABLE to furnish with studio's being limited to a paltry 128gb of ram for the unforeseeable future), for many AI cases, you need a lot of storage.

Hint, 16TB is paltry. Currently with EDSFF you can get 31, 62, 122, 247TB *single drive* SSDs with insane PCIe5 throughput. We're talking 14GB/sec before you raid them. These are an impossibility via the bandwidth joke that is thunderbolt 5.

These very fast network and storage options are possible on the MacPro M2ultra, but an impossibility on whatever the new studio will be. Sadly the MacPro M2ultra didnt have a 512gb+ ram option (minimum you need to run the really large full LLMs locally), so really, you have no great AI machine that's been made by Apple.

They easily could slap the M5ultra into a MacPro like case like they did with the M2ultra, and let people put in their own fast storage and networking options, but with the enterprise, and pro/enthusiast markets, apple loves to grab defeat from the jaws of victory.

 

[Bug-Creator]

apple loves to grab defeat from the jaws of victory.

Yeah, the victory of selling something that would cost billions in development to a few 1000 costumers who really want to switch (back) from existing server/HPC solutions.

Hint if Apple wanted to be in that market, they would be in that market.

 

[uacd]

But what if AI turns out to be a bubble?
Right now many companies are burning thru investor money to stay afloat. Larger ones can allow more “free” money and data center resources to be burnt, smaller fully rely on loans/stock price.

Local LLM is cool. But these companies will do everything so you don’t want to run it on your computer. That’s a cycle. So for the best experience you will still use smth like server-based Gemini.

There is a chance tho that AI becomes something purely corporate and niche, in case corporations see some perspective in it. For other users it is gonna be like 100$ per month with no free tiers or draconian limits and so no one is gonna use it.

In fact those RTX are either gonna be ultimate dust collectors or Windows will load them with some brand new telemetry scripts to steal even more data, literally data mining but done “locally” under the disguise of “LLM” from Microsoft. They call it Copilot.

See this vid for further thoughts, about laptop NPUs and how those are purely put in for marketing purposes. P.S: yeah I know this video is highly likely made with ChatGPT. But that’s what I am saying: why Apple would ever need their computers to be big and expensive if ChatGPT works in the cloud, wherever you have Internet connection?

However in this unstable world and WWIII risks local LLMs can become one of the “stability” frontiers. But it would take quite some time for those LLMs to mature and hardware to actually become useful.

UPD: my vision is that over time there will be like 3 major companies providing both cloud and paid local LLMs for computers that will be distributed akin to Windows OS. They will be regularly updated and maintained but will require tremendous amounts of power, so this will be aimed at institutions capable of building 20000$ “supercomputers”. It will probably become popular with said private companies, governments, hospitals, universities and such. Regular users will be fed with server-based slop, because even if they will be able to purchase those AI-ready computers, they will highly likely not be able to pay for the electricity bills

 

[257Loner]

There’s no reason why Apple wouldn’t reintroduce a tower Mac in the future.

Note what they said in their original response (to 9-to5 Mac, I believe); they had no plans to release a successor to Mac Pro.

Based on this, and the fact that allegedly Apple was starting to not like the ‘Pro’ moniker for the XDR display, they would release a new tower under a different brand.

We’re already hearing that the touchscreen MacBook and foldable iPhone could be labelled ‘Ultra’. I think it would make perfect sense for this to be the next tower, if indeed they choose to design one.

Apple is clever in its wording, and I believe that if they truly had no desire to introduce another tower in the future, then they would have been more specific.

The 2023 Mac Pro (with the M2 Ultra chip) already had an 'ultra' starting price of $6,999.

1. Can you imagine what a brand-new Mac 'Ultra' would cost?
2. How would Apple differentiate it from the Mac Studio?
3. How would it offer advantageous performance for its customers at a competitive price? It has to have either a unique selling proposition (USP) or offer market competitiveness. Since Macs tend to lack a USP because they're not compatible with all of the software Windows computers are, they therefore need to have market competitiveness. That would mean the next Mac Ultra would need to either offer premium performance for a premium price, or offer comparable performance to the competition at a discount. Since Macs, least of all 'Ultras' and 'Pros', are never sold at a discount, then we're left with offering 'premium performance for a premium price'. That brings us back to Question #2: How would Apple differentiate it from the Mac Studio?

 

[Charlie Bonesx]

How would Apple differentiate it from the Mac Studio?

The answer from those here that don’t understand how Apple silicon works want pcie gpus and slotted ram

 

[257Loner]

(Apparently NVMe over TCP is a thing too...)

Not only is NVMe over TCP a 'thing' these days, but the last datacenter expert I spoke to reassured me that his company's ARM SoC nodes boot via PXE, expose NVMe-oF namespaces over RoCE, connect through QSFP-DD optics to an EVPN/VXLAN fabric, and offload packet processing to FPGA- and DPU-based accelerators while NUMA-aware VMs access ECC RDIMMs over PCIe Gen5 so that applications can process more data with less CPU overhead and less delay! Thrilling!

 

[TechnoMonk]

You can rack mount Mac Studio + Add PCIe expansion via rackmount via Thunderbolt. Frankenstein model, but I've seen companies do it.

Yeah. Everything is possible I guess but not the best solution.

 

[Charlie Bonesx]

As I've said, there are certain apps and workflows, both Apple and third party, that don't work correctly when their storage, or the files being worked upon are not on an APFS volume, ie they're an SMB share.

For example?

 

[leman]

Look at the NEO, apple went into very un-Apple space and game out on top in the low end, and in the end it was their own silicon at this option in time that allowed them do this.

[...]

Continued thinking, if Apple are all-in and so-good at their M-Chip On Memory packages configs, firing out memory blades that are CPU integrated, sure what's the loss, it's there sic lion, economy of scale.

[...]

As they say, where there is a will, there is a way.

To make Neo, Apple didn't need to do anything differently — the technology was already there, and it was affordable. All it took was a business decision and some supply chain/cost optimizations.


However, building truly modular high-end system is first and foremost expensive (in every sense of the world). It can certainly be done — and Apple has filed multiple patents dealing with this, so this is certainly an area they have been working on. The main issue is not whether they can build such a system, but whether it would make financial sense. These systems do exist — Nvidia has been successfully using a similar architecture for their AI servers, but not at consumer-level prices. In addition, architecting solution for datacenter is one thing, but building one for consumer market is something else entirely. A consumer workstation must present the blades as a unique compute device, and that's a tough nut to crack on a limited budget. Apple went a very different route for their own datacenter — their blades are essentially networked computers with a central controller orchestrating work between them. This kind of architecture would be useless for a consumer-oriented workstation.

That’s beyond my pay grade. I don’t see why that level would be required for what I propose but I’m not a hardware guy (as in: those who actually design/build chips and motherboards, etc) so I may be missing something (or many things). However, the core from my understanding would be similar to what desktop setups with iGPUs and dGPUs have been doing for quite some time. The RAM pool and PCIe channels are the constraints. Back in the Northbridge and Southbridge days these things were already handled, if insufficiently for my proposals.

Let's have a look at some numbers — M5 Max memory interface is ~ 600GB/s. Let's say that's the interconnect speed we want between individual compute modules for everything to work seamlessly, so for two modules that's 1.2TB/s (we probably want much more double of that if we care about GPU work — Apple's UltraFusion offers 2.5TB/s, but let's leave that aside for a moment). For 4 potential modules that would be 4 bidirectional links per chip, or 4.8TB/s. If my calculations are correct, that's ~ 150 lanes of PCI-e 7.0 on each modules just for interconnect purpose.

I hope you can see the practical problem here? And by the way, it is entirely possible that my logic is way off the mark here, so please feel free to correct me if I have made any mistakes.

Not to try to cover everything, but, it appears that Apple's GPU architecture scales almost as well as Nvidia, except for Nvidia's current secret sauce, Ray Tracing.

I'd say that M5 at least caught up with Nvidia for ray tracing — M5 Max is currently top 20 on Blender Benchmark and on par with a much larger 4080 RTX. Where Nvidia is ahead is low-precision matrix stuff (for ML).

 

[mattspace]

I'd say that M5 at least caught up with Nvidia for ray tracing — M5 Max is currently top 20 on Blender Benchmark and on par with a much larger 4080 RTX. Where Nvidia is ahead is low-precision matrix stuff (for ML).

Being on-par with the third tier model of the four year old previous generation of a current product that is already closer to the end of its typical generational lifespan, is like claiming to have "won" fifth place.

 

[leman]

Being on-par with the third tier model of the four year old previous generation of a current product that is already closer to the end of its typical generational lifespan, is like claiming to have "won" fifth place.

Keep in mind that this is comparing a mobile GPU and a desktop one. NVIDIA didn’t really improve the baseline performance much since than - it’s mostly chip size and higher clocks.

If you want a more relevant comparison, M5 Max outperforms the mobile 5090 RTX, despite the latter having more shader cores, more RAM bandwidth, and higher clocks.

 

[one more]

Mac Pro has always been a niche product, taking just a tiny fraction of Mac’s global estimated 12% market share, so with AI gobbling RAM, I am not surprised at all.

 

[mercibeau]

I wouldn't say the Mac Pro had any use case for most people: It was too expensive, too big, and too heavy. Also it used over 10 times more power, even at idle it used more wattage than charging a high-end Mx Macbook Pro: it was not economical for everyday use. Also not to forget that many people only have a smartphone on the go and in general daily life, they don't use a desktop or a laptop. Even for the heavy duty professionals like engineers, the Mac Pro is like a huge expensive mainframe, when they only really need a Mac Studio that is orders of magnitudes cheaper and smaller.

 

[Flint Ironstag]

Keep in mind that this is comparing a mobile GPU and a desktop one. NVIDIA didn’t really improve the baseline performance much since than - it’s mostly chip size and higher clocks.

If you want a more relevant comparison, M5 Max outperforms the mobile 5090 RTX, despite the latter having more shader cores, more RAM bandwidth, and higher clocks.

This is the Mac Pro forum. If 1 GPU is providing all this great performance, then give us 4 or more! That's the whole point that's sailed clear over your head. There's more non-MP users in here than owners, I swear.

I wouldn't say the Mac Pro had any use case for most people: It was too expensive, too big, and too heavy. Also it used over 10 times more power, even at idle it used more wattage than charging a high-end Mx Macbook Pro: it was not economical for everyday use. Also not to forget that many people only have a smartphone on the go and in general daily life, they don't use a desktop or a laptop. Even for the heavy duty professionals like engineers, the Mac Pro is like a huge expensive mainframe, when they only really need a Mac Studio that is orders of magnitudes cheaper and smaller.

Too big?? It's the same size as HP Z8, Lenovo Thinkstation, etc. Too heavy? For what?? Used too much power? This is like going on the Ferrari forums and telling them their cars use too much gas.

 

[uacd]

This is the Mac Pro forum. If 1 GPU is providing all this great performance, then give us 4 or more! That's the whole point that's sailed clear over your head. There's more non-MP users in here than owners, I swear.

Too big?? It's the same size as HP Z8, Lenovo Thinkstation, etc. Too heavy? For what?? Used too much power? This is like going on the Ferrari forums and telling them their cars use too much gas.

The main idea is that Apple decided it is counterproductive to manufacture such a machine in order to cut manufacturing costs. They have already tried with Apple Vision Pro and lots of these devices rot in their warehouses. If no one buys it - they are gonna cancel it. Modern people want mobility, and thus go for MacBooks and iPhones mostly, that's why they even created MacBook Neo, which seems to have turned into a hit. Instead of cutting cost on components and manufacturing thousands of niche devices they are probably going to narrow their productions and go back to the roots basically. I believe new iPhone Fold is being made for that exact reason: they are gonna test waters, FAFO and then probably not continue with the device for the next year. Same is going to happen to iPhone Air and probably to main iPhone itself over time

 

[theluggage]

This is the Mac Pro forum. If 1 GPU is providing all this great performance, then give us 4 or more! That's the whole point that's sailed clear over your head.

...and here's the point that seems to be sailing clear over your head:

The M5 Max outperforms the mobile 5090 RTX at least partly because it's integrated onto the M5 Max die and sharing unified RAM with the CPU, NPU and storage controller. You can't do that with discrete GPUs plugged into a PCIe bus. ...and if Apple did want to support 3 more GPUs - even with the PCIe and fragmented VRAM bottleneck - they'd have to make an enormous investment to design a new die just for the Mac Pro which would likely make it impossibly expensive, even by Apple standards - and wouldn't be any better than a generic x86 system with the same GPUs.

Apple will, likely, produce a M4/M5 or M6 Ultra by fusing two Mac cores which will, effectively, add GPU/NPU cores and more CPU cores and more unified RAM bandwidth. That doesn't need a whole new die design, or a PCIe tower case, and which will run fine in a Studio-type form factor (maybe slightly higher if more cooling is required).

(OK, you can open your eyes again and re-post "but Apple should make a new Mac Pro with slots for GPUs"... sigh.)

 

[leifp]

In the case of unified memory, no, it's not a design choice - if the GPU, NPU and CPU aren't on the same die (or some sort of ultra-fusion package) as the memory controller then it isn't unified memory - at least not the sort of "unified memory" that Apple Silicon boasts. Let alone if the GPU/NPU are plugged into a PCIe bus - which is what people have been discussing in this thread.

You want to imagine some sort of fast - but still plug-in - GPU-CPU interconnect that also shares RAM and runs at comparable rates to the "fabric" on a SoC then go ahead and move the goalposts - it's nothing we've seen in a Mac Pro before.

Using DDR5 DIMMS might be a "design choice" but it still probably involves a whole new die and packaging technique just for a Mac Pro that accounts for a tiny fraction of Mac sales. Especially if you also want 2019 MP-like RAM capacities (which, BTW required Intel's M-suffix chips that supported 2TB and added thousands of bucks to the higher-end Xeon options on the MP).

Of course, everything is a design choice... Apple could have made a "design choice" to build the Mac Pro around an AMD Threadripper, or a "design choice" to release a range of plush garden gnomes instead...

Sure, there was a "design choice" but the choice was to use the same Apple Silicon dies for which the development/tooling costs could be shared with the far bigger-selling MBP and Studio & which punched above its weight because of unified memory.

I’m saying “both”

 

[leman]

This is the Mac Pro forum. If 1 GPU is providing all this great performance, then give us 4 or more! That's the whole point that's sailed clear over your head. There's more non-MP users in here than owners, I swear.

Sure thing, you might even get 8x GPUs! Just buy four M5 Ultra Studios (when they come out), connect them together, and write some software that figures out how to coordinate work using the new RDMA feature.

 

[theluggage]

I’m saying “both”

The Mx Ultra only has enough spare PCIe for 1 16-lane GPU card - often lagging a PCIe generation behind. It won't compete with Xeon/AMD CPUs offering 128 lanes (...and probably PCIe6 in the near future).

The laptops and SFF systems making up the vast majority of Mac sales only need PCIe for SSD, USB, ethernet etc.

So, again, the only "design choice" is between the huge expense of designing a whole new die exclusively for the Mac Pro or adding unnecessary complexity (and power consumption) to the laptop-friendly Mx Max die.

 

[BSDnostalgia]

So, again, the only "design choice" is between the huge expense of designing a whole new die exclusively for the Mac Pro or adding unnecessary complexity (and power consumption) to the laptop-friendly Mx Max die.

I agree 100% with you that the discrete GPU-as-GPU is not going to happen (again) for Apple Silicon. As more evidence of that, look no further than what Nvidia is doing with the Spark, including the RTX Spark. Unified memory, and, gluing the GPU into that instead of the separate VRAM.

I disagree, though, with part about I/O bandwidth being fantastically expensive. It wouldn't be. Apple already has all the designs needed. They would just have to sacrifice a little more memory bandwidth for it, just like Nvidia has done. And, I disagree with the implication that there would be no market for it.

That being said, the folks asking for a high-bandwidth PCIe tower are barking up the wrong tree. As I said, Nvidia, which owns a huge piece of that market for high-end gaming, is now working on unified-memory ARM-based configurations. This is validation for what Apple has been doing.

 

[mcnallym]

Sure thing, you might even get 8x GPUs! Just buy four M5 Ultra Studios (when they come out), connect them together, and write some software that figures out how to coordinate work using the new RDMA feature.

Mac model for development based on the AS range where using a SoC so is unified.

In this dream world Apple is going to break there model for a Niche Product that will not only require custom silicon, but also change the development model to support non integrated parts such as additional GPU, additional RAM found in user added memory slots.

On top that they also expect Apple to after a 13 year absence reintroduce support for Nvidia GPU/CUDA. 2013 iMac was last model to have an Nvidia GPU. Mac Pro 5,1 got the Web Driver that added support for newer Nvidia Cards but that was it.

Yet still have people here clinging desperately that someday Apple is going to U-Turn and break its ecosystem choices.

Remember the video where Steve Jobs asked by the chap about Flash, and Steve Jobs basically says that when doing the ecosystem that make choices that some people may not be happy with but fits th Ecosystem.

Making choices for the Ecosystem that people don’t like clearly goes all the way back to Steve’s days. So is not despite some claimants a problem caused by Tim.

Years ago I posted that if you doing CUDA work, needed the multiple GPU then you should be planning the migration off Mac, either to Linux or Windows whichever provided the support for the Apps you require.

Apple apparently started the M1 Extreme development but couldn’t get it to a point where a viable product, whether was down to actual product or just would cost too much. Yet here with the M5 and how far off the M6 are we then not even had an M4 Ultra and will we see an M5 Ultra, yet still expecting some sort of Extreme Chip with massively increased PCI-E lanes that not used in anything other then this Mac Pro Tower.

For people that have these requirements then what is it that requires a Mac Pro Tower as opposed to a Generic Tower with a CPU of some sort with massive Memory Support and massive numbers of PCI-E lanes running Linux/Windows.