I was talking about Macs - and those solutions at NVIDIA are also not standard ethernet, nobody (I mean really, who would???) clusters LLMs - it is far too slow.Nobody like Nvidia?
NVIDIA Spectrum-X Ethernet Networking Platform
The World’s First Ethernet Platform Built for AI.www.nvidia.com
"... NVIDIA ConnectX-9 SuperNICs deliver ultralow-latency, 800Gb/s networking to maximize efficiency in gigascale AI infrastructure. Built for NVIDIA Spectrum-X Ethernet, ConnectX-9 SuperNICs accelerate data movement, optimize RoCE performance, and enable consistent and predictable networking for the most demanding AI workloads. ..."
NVIDIA Launches BlueField-4: The Processor Powering the Operating System of AI Factories
New NVIDIA BlueField DPUs with 800Gb/s throughput, NVIDIA ConnectX-9 SuperNICs and NVIDIA DOCA microservices deliver breakthrough acceleration for AI data storage, networking and security.blogs.nvidia.com
Spark DGX has Connect-X 7 adapter built-in.
" ..Hi @iner, DGX Spark does not support infinband mode and there are no plans to add support in the future. ..."
Infiniband?
I’m curious if anyone has tried to use these with an Infiniband switch?forums.developer.nvidia.com
Yes they have Inifinband solutions for 'midrange' also, but "nobody" would mean every single possible implementation.
Got a tip for us?
Let us know
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
Apple Debuts M5 Ultra as Most Powerful Chip Ever
- Thread starter MacRumors
- Start date
- Sort by reaction score
Not so far measured when? It wasn't until April 2026 that Apple published the recommended cluster configuration settings.
TN3205: Low-latency communication with RDMA over Thunderbolt | Apple Developer Documentation
Learn how to use RDMA over Thunderbolt to enable low-latency communication between clusters of Mac computers.
That was well after they sent out a bunch of demo units to internet influencers to drum up covers of the RDMA feature.
For example, Thunderbolt doesn't support wake-on-LAN so systems can never go to sleep.
IP-over-Thunderbolt and RDMA actually share the same wire. Start sending IP packets and RDMA latency isn't going to be the same.
While Apple says you can build multiple types of topologies, I suspect there is probably only one that has most of the bugs worked out of it. Plug in a bunch of cables in random order and the system 'automatically' configures itself is probably suspect.
they really haven't done vertical just for vertical sake. The is more so driven by miniaturization and "system on a chip" moves than 'vertical. As separate discrete chips collapse they beome one and Apple coordinating becomes a bigger overhead. If there is a vendor that does a better job , at a better cost then Apple tends to leave it along ( e.g., Displays. )
Maximizing perf/wat isn't vertical. It is making something specific that doesn't try to be a sell everything to everybody solution.
Qualcomm did good work for Apple but it become more of a pot-calling-kettle-black situation where Qualcomm wanted a percentage of the price ( sort of hoave RAM and NAND are at the moment) and Apple pushed back.
( I don't think the modem will totally merge because inter die integration means don't have to be momolithic dies to be effieient in the future. And bleeding edge wafer costs are growing ever more expensive.)
One of the most rapid growing businesses Apple has is 'Services'. That has the facade on the outside that it is just Apple ( just Apple collecting money in the store, just provisioning authentication , etc. ) , but there are large portions of "Cloud Services" that run at AWS/Azure/GoogleCloud. The relay/VPN service I'm pretty sure has a major Cloudflare component.
Apple TV the streaming serveice. Apple is the 'front man' for the operation, but a large set of independent operators behind that.
apple took the SSD controler and stuffed it into the SoC a long while back. That didn't move the NAND chips. Apple isn't going to do commodiities. ( e.g. like monitors. Apple does maybe 1-2 Display docking stations , but they don't try to attack the general monitor market. )
Private Cloud Compute is a relatively very narrow workload. ( And Apple is already working on mapping that general service onto Google Cloud. Eventually they probably will map a larger share of that back out, but for now and likely for far into future it is a better scaling option. )
XCode Ckoud is another narrow niche deployment.
Being dependent between one and only one vendor is substantively different than 2-3. Apple didn't try to weave AMD into mix at all ( for good reasons. they were not doing a good job until after Apple has mostly decided to leave x86). Additionally, Apple is contributing part of the reason why the RAM vendors. Apple squeezes all the profits out of RAM and NAND. (charge way over market prices ) and in Scrooge McDuck sent almost none of that back to the RAM/NAND vendors as guaranteed income they could use to invest in more capacity.
Apple got complacent like Intel did. They were a top 3 consumer of RAM/NAND so they dominated their suppliers. to wrangle maximal discounts. Short that looks smart , but long term foolish. Especially if other vendors come around and spend money like drunken sailors ( try to give them vendors profits to increase capacity as oppose to trying to make creating capacity a more risky endeavor. )
Painting yourself into a corner with a butterfly keyboard because worshipping at the "max thinness" politboro is not being 'vertical focused'.
If spend billions on stock buy backs and issuing bonds to duck paying taxes ... they kind of don't. If Apple's 'free' money $20B/year Google paycheck for 'work' that Apple product users actually do disappears then again the finances are tighter than they look now.
For large scale training? Probably not. But the notion that all inference had to happen in the cloud. That is more so driven by trying to artificially inflate valuations ( cash out and get rich) than . Mostly based on wiping out 10 million (or so) peoples jobs and taking all their paycheck money to fund the cloud.
It wasn't a push to thinner clients it was a push to wipe out most of the folks sitting at the client. Take their skills and over time leave them with fewer and fewer skills until have addicts , not clients.
Apple put a number of 'bets' on FibreChannel which didn't work out long term.
While Linux was busy maturing libibverbs from 2005-2010 , Apple was busy ignoring Inifiniband.
2013 Apple's Mac Pro dumped standard PCI-e slots (the most common way to get high end
advanced networking). All the while still ignoring libibverb .
Apple tossed their sever OS variant (2011). It was mostly an app 'add-on' for a while before it become comatose and finally shutting it down completely in 2022.
they did Xgrid for a while and then quit ( in part because quitting sever OS ) .
Xgrid - Wikipedia
" ... RoCE is the only industry-standard, Ethernet-based RDMA solution with broad multi-vendor support, running over standard Layer 2 and Layer 3 Ethernet switches. ..."
200=400GbE is too slow? Or because there are no Mac Product that can even remotely drive a 200-400GbE card . It isn't the interface that is too slow. It is the host.
mode11
macrumors 68000
But what wider point are you making by bringing up the price of computing 40 years ago? That we shouldn't complain about Apple's current prices, because the equivalent of $8000 in 1984 only got you an 8MHz processor? The only meaningful comparison is to prices in recent history. It sucks to spend thousands more this year than you'd have had to for the equivalent* machine last year. Albeit, there's nothing you can do about it either way.
*allowing for the continual increases in processing power.
chucker23n1
macrumors G3
Because RAM is a commodity in a way CPU/GPU/NPU designs aren't. It's just storage.
Apple is interested in designing its own memory controllers, much like their "SSDs" are really just NAND chips but with a custom controller in the SoC, and custom fabric between them rather than a PCIe/NVMe bus. But the actual memory chips and NAND chips are, from Apple's POV, boring.
The dependency on Intel was an achilles heel because
- Intel at the time required Apple to have Intel both design and manufacture the chips
- in both disciplines, Intel was starting to lag behind the competition
- on top of that, Apple had specialized requirements
RAM and NAND aren't like that. Apple uses bog-standard chips. They could use proprietary chips, but it's not clear to me what you propose those would do differently. Then you're just left with: are there enough manufacturers? That's debatable; Apple would like more (but, again, Apple isn't gonna do the manufacturing itself). Are the competing designs good enough? Yes. With the M5 Ultra, Apple's setup lets them to 1.2 TB/s memory bandwidth. For the storage, it isn't clear yet, but it looks like it may go up to 28 GB/s with the highest config.
CWallace
macrumors G5
Designing an all new type of proprietary RAM would be extremely expensive for Apple. They have to hire a design team to create it and then construct a fab to produce it and hire people to staff it. Even if Apple can develop a RAM twice as fast as anything else, if we have to pay 10 times as much for it... Apple then also risks that a production issue would knock out their entire product line until it can be resolved because only they manufacture that RAM.
As @chucker23n1 noted above me, RAM and NAND are commodity products for a reason - commodity products are easy and cheap to manufacture compared to proprietary products. The current prices of both are solely due to simple supply and demand. And more supply is coming, but it takes years. Hence why the OEMs and customers are signing long-term contracts to ensure that the money the OEMs are spending to bring that supply to market will have sufficient demand to cover it.
neotint
macrumors regular
I get the points you and @CWallace are making, solid corrections, good to make me fix and clarify my point. The product strategy side of me keeps seeing the big gap of on device AI bottleneck being Memory, not Logic. Apple is holding back on Memory.
Yes memory in all forms is the most expensive component now due to hyperscalers' demand and contracts, currently no end in sight to memory inflation for years to come. Looking at the market value of Memory makers I see it's astronomical now and beyond Apple's cash. Insane times.
So Apple can continue its architecture design approach, like modems, make SOC and subsystems more custom, faster, more power efficient, get more out the same memory, etc. but they could also treat memory as a pass through cost and cut their margin on it which is a business decision, not a tech issue. Point taken and clarified.
This is going to be problem for *many years* so something has to give. I even wonder how much the memory crisis has made Apple pivot on its SOC/unified memory strategy.
chucker23n1
macrumors G3
In terms of bandwidth, they're basically best-in-class, by running all chips in parallel.
In terms of latency, same — you can't beat having the memory on-package. Unless you put the memory inside the chip, which would severely increase cost.
In terms of capacities offered, sort of, yes. The 2019 Mac Pro allowed 1.5 TiB RAM; the Mac Studio still maxes out at a third of that, seven years later. And you can't even currently get that config. The question becomes whether more than 512 GiB is much of a market. How many people are there who want a single small-form-factor desktop with that much RAM, who can't instead get an array of two or more Mac Studios? I imagine not that many.
Sure, but even if Apple were to have a company manufacture custom RAM, that wouldn't ramp up for years from today.
Well, not yet, anyway. And given that they just discontinued the Mac Pro, which was the closest they had to a plan B in terms of a chassis that would support socketed memory, I think they've abandoned that for years to come.
chucker23n1
macrumors G3
The sequence is generally:
- A-series or base M-series
- Pro and Max
- Ultra
It takes a while for the Ultra to be ready, if it gets released at all (for example, there is no M4 Ultra). Thus, things can overlap, for example:
The M1 Pro came out after the A15 was already out, and the A16 came out between the M2 and M2 Pro.
Howard2k
macrumors 604
CWallace
macrumors G5
Apple have confirmed the M6 will be the only member of the family. There will not be an M6 Pro, M6 Max nor M6 Ultra.
The entire line will return with the M7 family.
chucker23n1
macrumors G3
Yes, but even if there were, they would likely be released later. The one exception is the M3, which came simultaneously with the M3 Pro and M3 Max. (Then, OTOH, the M3 Ultra came much later.)
CWallace
macrumors G5
Like previous Ultras, the M5 Ultra uses two M5 Max. However, the M5 Pro and M5 Max now separate the CPU and GPU onto separate dies on the SOC, which is why the M5 Ultra has four-way connections as opposed to the two-way connections of the M1, M2 and M3 Ultras, which had the CPU and GPU on a single die on the SOC.
chucker23n1
macrumors G3
Yup.
The M1 Pro and M2 Pro were a "chop" of the respective Max: they "cut off" some of the GPU cores and memory controllers. Roughly the bottom third in this image is missing on the Pro:
The M3 Pro and M4 Pro were designs more separate from the respective Max.
The M5 Pro and Max have two separate tiles, attached with a "Fusion" connector. They use the same CPU tile with super cores and performance cores, and both attach to a GPU tile, but the Max's has more chips.
The M5 Ultra, then, is this, but doubled: imagine two of these next to each other, attached with an UltraFusion connector. So you have a 2x2 grid of tiles, where the upper two tiles contain the CPU cores, and the lower two tiles the GPU cores.
So is it four M5 Pros? No. It's two M5 Maxes. Which in terms of CPU means you get twice as many CPU cores as on the Pro and Max, and in terms of GPUs, you get twice as many cores as on the Max (and indeed four times as many as on the Pro).
The M1 Pro and M2 Pro were a "chop" of the respective Max: they "cut off" some of the GPU cores and memory controllers. Roughly the bottom third in this image is missing on the Pro:
The M3 Pro and M4 Pro were designs more separate from the respective Max.
The M5 Pro and Max have two separate tiles, attached with a "Fusion" connector. They use the same CPU tile with super cores and performance cores, and both attach to a GPU tile, but the Max's has more chips.
The M5 Ultra, then, is this, but doubled: imagine two of these next to each other, attached with an UltraFusion connector. So you have a 2x2 grid of tiles, where the upper two tiles contain the CPU cores, and the lower two tiles the GPU cores.
So is it four M5 Pros? No. It's two M5 Maxes. Which in terms of CPU means you get twice as many CPU cores as on the Pro and Max, and in terms of GPUs, you get twice as many cores as on the Max (and indeed four times as many as on the Pro).
I see.
So it’s still 2x max as before to get the ultras (even if they have moved things around)
A max is now 2x m pros then?
In theory then, whatever means they achieve in the future to fit an ultra class Soc into the space (or thereabouts) of a max then glue via UF to another max and viola you now have mExtreme!
So it’s still 2x max as before to get the ultras (even if they have moved things around)
A max is now 2x m pros then?
In theory then, whatever means they achieve in the future to fit an ultra class Soc into the space (or thereabouts) of a max then glue via UF to another max and viola you now have mExtreme!
chucker23n1
macrumors G3
Yes.
No. The M5 Max's GPU is twice the M5 Pro's. But the M5 Max's CPU is identical to the M5 Pro's (other than having faster memory, which the CPU probably won't saturate).
Well, that seems less likely now that they've discontinued the Mac Pro. Not to mention things have only gotten costlier lately.
I found some more info
www.cultofmac.com
M5 vs. M5 Pro vs. M5 Max vs. M5 Ultra: The differences that matter
Our detailed comparison of the M5 chip family shows how each differs from one another, and helps you choose the best option for you.
www.cultofmac.com