Apple Reportedly Building M2 Ultra and M4-Powered AI Servers

[wilhoitm]

No, telling the fact cause Apple Silicon's GPU performance sucks. That's why Mac is only good at 2D based software.

I think you are forgetting about the Apple Vision Pro buddy! No one has anything like it! Microsoft killed theirs and Meta laid off all of their AR/VR people when the Metaverse failed! 🤣

Microsoft has laid off entire teams behind Virtual, Mixed Reality, and HoloLens

HoloLens, Virtual Reality, and Mixed Reality are all but dead at Microsoft.

www.windowscentral.com

 

[dannyyankou]

Would be too limiting

Yes I understand that, but if they're going to have AI servers with the same chip the next generation MacBooks have, in theory all current devices should be able to do tasks that the 2024 iPhones/macbooks can over the cloud. Maybe they will do that, but rumor is those devices will have exclusive features.

 

[Le0M]

Hear me out:
What if Apple made a consumer server that served as the backend for a “local” GenSiri? Something that your iProducts could call back to for AI interactions maintaining privacy.

Probably prohibitively expensive. Nevermind. I’ll go back to hoping for a new AirPort Extreme mesh router.

I like it! I seriously do! Like a siri hub, that fetches new LLM updates and also real time info to be able to share them with other apple devices, while protecting privacy.

 

[gsurf123]

All rumor, but plausible rumors. Definitely shows the advantage of being to make your own processors again. It was the best move made by the company since the release of the iPhone. It is literally the gift that keeps giving. Nothing better than not be dependent on anyone for chips.

 

[Antony Newman]

With the almost exponential Compute requirements for training - who will be in a position to train the largest LLM and Recommendation models in 5 years time.

Who will have 20 Gigawatts of power on tap looking for a use in their global data centres?

Perhaps Apple are moving to add a new ‘services’ to their data centres for:
+) Massive training capability - to rapidly improve their own models in house (using their renewable energy + data centre chippery + data warehouse)
+) Amazon like services that make a tidy income from performing training for the Fortune 500 at competitive rates
+) Huge compute resource to reduce the simulation and layout optimising for Gate All Round transistors (which take an order of magnitude more compute to create optimal 3D layouts)
+) More resources to throw at Apple Maps
+) To eventually (10 years?) replace the default search engine with ….. Siri.

 

[firewood]

I can't imagine Apple's laptop chip comparing favorably to what Nvidia is offering.

You may not have noticed that power and cooling are major limiting factors in both the size and cost of running massive data centers, not raw GPGPU performance. And Apple's silicon competes quite well with Nvidia in the ML performance per Watt metric.

 

[huge_apple_fangirl]

Makes sense. Apple is deploying on their silicon, makes sense to build on it too. But will they sell these servers?

 

[dredlew]

I guess a nice side effect is lower cost + higher yield of their Ultra chips due to the economics of scale. I don't think they have been that high in demand, well, now they are.

That means, lower Mac Pro prices, right? Right? Yeah, no.

 

[Harry Haller]

Im all for this… their SoC are massively powerful, and with their own IP and software…. Never underestimate the power of vertical integration.

Massively powerful?
The M2 Ultra has the CPU performance of an i9 13900K and the GPU performance of an RTX3080.
It’s a very small fraction of Threadripper, Epycs and Nvidia H100s.

Blender - Open Data

Blender Open Data is a platform to collect, display and query the results of hardware and software performance tests - provided by the public.

opendata.blender.org

 

[enb141]

I can't imagine Apple's laptop chip comparing favorably to what Nvidia is offering. Too bad Apple hates Nvidia.

Is not just apple, Windows ARM also doesn't has nvidia video cards nor AMD, just their own proprietary GPU as apple.

 

[enb141]

Massively powerful?
The M2 Ultra has the CPU performance of an i9 13900K and the GPU performance of an RTX3080.
It’s a very small fraction of Threadripper, Epycs and Nvidia H100s.

Blender - Open Data

Blender Open Data is a platform to collect, display and query the results of hardware and software performance tests - provided by the public.

opendata.blender.org

Exactly and getting a PC with an i9 13 gen + Geforce 4070 Ti is way way way way cheaper than getting a Mac Pro.

 

[ChrisA]

Makes sense for them to leverage their own hardware, especially if their AI implementation is designed around their neural engine and GPU designs as opposed to "commodity / off-the-shelf" hardware.

And I do not expect Apple to offer these servers at retail, so this is not the "return of the Xserve".

Any servers they use are for training, With our current technology training requires enormous computational power compared to simply running the model. Many models run on just my M2-Pro but training an LLM on my M2 is just not going to happen.

If Apple is using the M2 or m3 they are likely not building a special version of the chip but rather writing cluster management software that allows large numbers of "normal" computers to work together.


If fact, if you read Apple's recently released open source "openELM" AI model there is a comment by an Apple engineer saying "This only works on Linux as Slurm is is not yet working on MacOS. Slurm is a common cluster manager and job scheduler.

So we know that Apple has been using Linux servers and there has been at least some effort to transition to MacOS.

So me guess is Apple is working on getting Slurm to run on MacOS on Apple Silicon

 

[iPadified]

I wonder if this means that Apple has the tech to get M chips work as a cluster. The Mac Pro would like that tech to put get some more ultra chips into that spacious enclosure. Better local AI training in Mac Pro as spinoff?

 

[iPadified]

Exactly and getting a PC with an i9 13 gen + Geforce 4070 Ti is way way way way cheaper than getting a Mac Pro.

I did not now people used i9 and GTX cards in server, please enlighten me.

 

[ChrisA]

I can't imagine Apple's laptop chip comparing favorably to what Nvidia is offering. Too bad Apple hates Nvidia.

Apple Silicon actually compares very well to Nvidia. The M2 or M3 Pro is about comparable to a mid-range consumer-level Nviais GPU, except for two things, the Mac can be configured with more RAM than a typical consumer GPU and the Mac uses much less electrical power. Power use is a huge deal in a data center because you pay for the paoer to run the computers and the computers turn power into heat and you have to pay for the cooling to remove the heat. It might cost 40 cents per hour to run an Intel/Nvidia server. You might pay $2,000 per year just to keep one server running, then figure you have hundreds or even thousands of them running.

Some Nvia chips can be faster than Apple Silicon but you can simply buy more Apple Silicon to make up the difference. The Apple chips cost less power per computation. It is "power per computation" that matters

BTW, I am able to run most of what I need in My M2-Pro now as both TensorFlow and Pytorch run on AS now. My Mac Mini is about. like an Nvidia 3050, but with more v-ram. Apple might be able to build a data center with thousands of Mac Studios and do well.

 

[ChrisA]

Exactly and getting a PC with an i9 13 gen + Geforce 4070 Ti is way way way way cheaper than getting a Mac Pro.

Why would Apple use the Mac Pro, that is silly. Something like a Mac Studio is better. And no one uses gamer PCs in server rooms. The 4070 is not going to work well, They are more likely running on the Nvidia A100 or H100.

The Mac is much cheaper, especially when Apple does not have to pay full retail. The server-grade GPUs cost between $10K and $20K each.

 

[idmean]

Think of this as a massive snapshot of the internet at a given point in time. Once that snapshot is generated, they can copy it locally to the user

That's a weird definition. An LLM does not necessarily need to be trained on "the Internet" and does not try to be a snapshot. If you want an easy definition, it's a statistical model.

 

[hagar]

For GPT, they need something called a LLM (Large Language Model). Think of this as a massive snapshot of the internet at a given point in time. Once that snapshot is generated, they can copy it locally to the user. So when you ask it something, it checks the local LLM (on device) and gets you the answer.

So what's the point of the AI server?

1. Generating those models takes a LOT of GPU power for a long period of time. This is what all these companies are paying the big bucks for.

2. Those snapshots get stale quickly, so they need to run a new model every so often. It can cost billions to generate a new model so you don't want to do it too often.

3. As they improved their code, they need to regenerate parts or all of those models.

So does that mean if the snapshot is from Jan 1 I won't get any recent data?

Yes.

To fix that, a lot of the GPTs will leverage the internet to get more up-to-date results. That's slower and the results aren't as good. This part will not be handled on-device obviously.

This is why Apple is trying to strike a deal with Gemini and OpenAI. It's also possible that Apple is looking to handle the fresh data on their own, which would require more servers.

This is very incorrect. An LLM is not a “snapshot” of the internet. It’s not a database you query to get “the truth”. You’re confusing LLM’s and AI powered Search.

 

[hagar]

I had the feeling that this "on device" propaganda was not going to be what you would expect. What is called "AI" these days is basically big data rather than real AI, and big data means that datasets take a huge space. Does "on device" mean that you will have your huge datasets on your device? I doubt that's Apple's plan, they'd rather prefer to control the datasets. In the end, "on device" will be just an ad slogan, but I guess the real implementation will be a disappointing cloud-based system.

AI is technically not really big data. If I ask AI to remove the background from an image, it’s not big data. If I ask it what climate change is, it’s not querying a big database. But using an LLM to jumble together a smart sounding answer without “understanding” what it’s saying.

Apple will be able to run algorithms and part of these models on device and use local personal data like photos, contacts and so on… to enrich and personalise the results. But it will not download and run an entire ChatGTP model on device.

 

[enb141]

I did not now people used i9 and GTX cards in server, please enlighten me.

Nobody uses Mac Pros as servers either.

 

[enb141]

Why would Apple use the Mac Pro, that is silly. Something like a Mac Studio is better. And no one uses gamer PCs in server rooms. The 4070 is not going to work well, They are more likely running on the Nvidia A100 or H100.

The Mac is much cheaper, especially when Apple does not have to pay full retail. The server-grade GPUs cost between $10K and $20K each.

If apple uses servers for their own, then price won't be an issue, but if they try to sell those servers (very unlikely but still possible) then those servers will be way wy more expensive than those that run Nvidia's A100 or H100.

Nobody was mentioning that Apple would use a Mac Pro, but that's their best computer so is very likely that those servers would be pretty close in specs to the Mac Pro.

 

[JimmyHook]

The whole narrative went quickly from "new leak shows all kinds of AI features in iOS 18 OMG" to "the real AI features will come to iPhone 17 (2025), almost 3 years(!) after the launch of ChatGPT".

My god, I will be listening to WWDC this year but if they don't play ball quickly and show that they are a development powerhouse, I might leave the whole ecosystem in a year of two. The costs are staggering (1200 bucks for a so so phone) and even for my hobby in AI I had to resort back to Windows (with WSL). I came to this "camp" with the introduction of M1 (and I am typing this on that very MBP which is great for everything not-machine learning). I was about to switch my iPhone 12 Pro to a iPhone 16 Pro, but if they don't come up quickly with things that would actually make my live easier, I will reconsider. Spending 1200 EUR on something that is feature-wise on par with an 2021 Android is just.... unbelievable.

Not to mention their malicious compliance with EU regulation. While typing this, I am more and more thinking "what the hell am I still doing with Apple". I notice it on the job as well; disappointment and apple is becoming the joke.

ChatGPT is hot garbage. I would be upset if Apple did what other sheep corporations did and started throwing half-baked hallucinatory AI at everything. OpenAI is a research project at best, and others public generative AI platforms are just pre-alpha garbage marketed and hyped as the greatest thing since sliced bread.

I would leave Apple if they tried to replicate ChatGPT. If they do release something, which I wouldn’t be upset if they didn’t, it better be superior in every way to the current garbage out there.

 

[JimmyHook]

lol, you must be trolling then. The truth is, even Apple Vision Pro failed as Apple did NOT mention anything about its sale from Q2 sale report instead of talking about something else. No one has anything like it? Guess what? MS already have HoloLens 2 which is very successful for B2B and enterprise uses. They are even making it for military uses!

Also, AVP is a consumer product and yet literally no consumer AR/VR devices ever succeeded and all of them failed. Even Meta is still struggling as people aren't really using well. What makes you think that Apple is doing better than MS and Meta? AVP developers already lost their interested in AVP which has been proven.

Clearly, you know nothing.

Apple never breaks out device sales. Your argument is logically flawed.

Also, Meta and Google are only good at one thing: convincing advertisers that we actually give a crap about the ads they serve based on vacuuming our personal data.

Go play with your Google and Meta spyware. Have fun. While you are at it, enjoy Microsoft’s dumpster fire of an OS

Windows 11 Update Causing Serious Problems For Millions Of PC Users

Hot on the heels of another chaos-causing update, there's another one that's seeing dozens report problems ranging from crashes, blue screens, stuttering and performance

www.forbes.com

 

[leman]

I do not see much point in using M2 Ultra as an ML server. It is just not financially viable. It Apple improves their ML performance (which should be easy for them to do) and gives the system more RAM bandwidth, than it could be interesting.

 

[svish]

Not surprised. Apple Silicon is indeed very powerful