There are people who are massively over invested in NVIDIA, just as there are people who are massively over invested in Apple. The NVIDIA acolytes will deny it with the same ferocity the Apple acolytes deny it.
Got a tip for us?
Let us know
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
Apple to Power AI Features With M2 Ultra Servers
- Thread starter MacRumors
- Start date
- Sort by reaction score
I would imagine they would be trying to have multiple SOC Mx Ultra chips on Mother board/cards to support more memory and processing power. I am glad Apple is trying to use their own Servers, M2 Ultra isn’t near Nvidia, but it’s a good start to improve.
You missed what I said. For now Apple can continue to use hybrid solutions such as tapping AWZ, Google cloud for training until open sourced UXL stack for GPU's is matured by year end or even longer, if needed. For B2C inferencing, a lot of it will be done on device and some on the the M2 Ultra servers. When AI GPUs sporting open source UXL stack are readied by Google, Intel, Qualcomm, ARM, etc. then they can replace their current hybrid solutions with the open source options. No need for Apple to depend directly on Cuda platform. Let the big hyperscalers do it for you as they are all collaborating with each other to replace NVIDIA anyway.
As I said earlier, there is a lot of insider selling at NVIDIA for a reason. They don't know how much longer they can keep up their first mover effect for the AI server sector. Even with the brute force of their GPU revenues, I can assure you they will not be announcing any share buy back or dividend on the scale of what Apple did last week. If they are confident of their future they would. I bet they don't when they report later this month. This is probably because they do not know where they will be in 5 years when Apple clearly does have confidence in their own future revenues.
Last edited:
They heard you!
Apple reportedly nears an OpenAI deal to power upcoming iPhone features with ChatGPT
After also talking to Google about AI tech for iOS 18, Apple has closed in on an agreement with OpenAI.
www.zdnet.com
Everyone keeps saying this, but why would they bother when they've already got 7 friggin' operating systems based on Darwin/XNU/FreeBSD?
No way, is that really how that ended up going down? I assumed they failed because it just didn't work and/or wasn't profitable.
I'll say AAPL has proven to provide more long-term substance. NVDA may yet prove the same, but for now, AI is arguably in a hype cycle.
And some of Huang's assertions make me think he's either pretending to be clueless about LLM capabilities (and non-capabilities) and deliberately exaggerating to trick investors, or he's really chasing something that won't be possible for decades, if ever.
Seriously trying to say my iPhone 15 pro is more powerful than my M1 Pro MacBook Pro 14”?
That’s because of the software stack making specific use of cuda cores. Apple having their own software stack to utilize its GPUs will do the same. Nvidia is about to be last in line between AMD and intel as far as price/performance goes/power usage goes. AMD’s mi300x is already better in many cases at 1/3rd the cost and the mi350x could arguably be the best out there, again cheaper, but it depends on the workload. AI is very broad terms and there’s TONS of specific calculations based on what’s being modeled. Nvidia will just pick the few they do better than. Many of the big key players are signing AMD contracts now and as the AMD software stack matures and is most importantly free and open development it’s expected to see the market share of AMD start taking from Nvidia.
4090 runs out of memory, and doesn’t work with midsized models or higher. For those saying 4090 is gaming GPU, 2X4090 easily matched A100 performance in most inferences. H100 has lot more built in accelerators, but 4x 4090 is decent. Problem is Nvidia wouldn’t put more RAM in 4090, it will eat up higher end GPU sales. For cloud providers, they can stack up 8+ H100 GPUs and milk money. Apple has great opportunity to take the middle market.
The future of ai is local, that means on mobile devices which are the main computers that we use. Nvidia, Google and ibm are developing enterprise hardware.
Therefore apple is running its own race. Mobile use case requires making costs and efficient top priorities.
As a result I think that Apple are already developing an in-house stack from hardware, model, training to inference and evaluation. Long term hybrid approach won't be good optimal.
Nvidia hardware is currently amazing for training but overkill and inefficient for inference compared with groq. Eventually even training will be run locally on a user's own document or content.
It’s about the size of the model. Larger models take up more memory whilst processing, and when computed using the GPU all of that needs to be stored in VRAM.
Apple’s unified architecture has an advantage here because all system RAM is addressable as VRAM - up to 128GB on M1 Ultra and 192GB on M2 Ultra. And then one step further because the same memory can be used by the NPU as well at the same time.
So if you are running a model that exceeds the available VRAM on a 4090 it doesn’t matter how much power it has available because it can’t use it effectively. In such cases a Mx Ultra and even Mx Max chips will be far more performant even if they have less absolute compute power.
To get systems from other vendors with Mx Ultra levels of addressable VRAM costs an order of magnitude more than the amount Apple currently sells the entire system for (Mac Studio for example) to consumers.
From there, take off the 35% margin, and pack the chips into a rack that has a lower production cost per chip and the cost to Apple is really low. It’s a major competitive advantage if they can leverage it effectively. Also, M2 Ultra is produced on a now very mature, high yielding process so the cost per chip is only going down over time.
The point being, even if they have to use more chips to match performance, they can still save an enormous amount of money versus buying from the competition.
We're (or at least should be) talking strictly about the NPU performance here.
The new grail is NPUs capable of 40 TOPS according to some chipmakers that are reaching that level for the next generation of AI-centric chips.
The AI hardware just means the AI functions aren't going to use as much electricity, still need the proper software to be of any use. I find it hard to believe Siri hasn't been improved because apple didn't want to use electricity.
Amazon Fresh kills “Just Walk Out” shopping tech—it never really worked
"AI" checkout was actually powered by 1,000 human video reviewers in India.
https://arstechnica.com/gadgets/202...e-checkout-which-needed-1000-video-reviewers/
one query. It was confirmed that the M1, M2 and M3 series all contain a security flaw within the hardware. With comments about customer security is it wise to utilise flawed hardware?
With regards the latest buzz of AI, it is worth perusing the following article as its nothing new. Even Siri with all its faults can be described as AI, as are the awful newer answerphone systems that seem to confer the title Artificial Ignorance rather than 'intelligence'
The History of Artificial Intelligence - Science in the News
by Rockwell Anyoha Can Machines Think? In the first half of the 20th century, science fiction familiarized the world with the concept of artificially intelligent robots. It began with the “heartless” Tin man from the Wizard of Oz and continued with the humanoid robot that impersonated Maria in...sitn.hms.harvard.edu
The security flaw you describe depends cyrptographic processes being run on the Performance cluster, and the hostile app needs to be run on the same cluster at the same time to have a chance of exploiting the flaw.
Even if Apple runs some cryptographic processes on these ML servers, I can't imagine they'll just run random apps from online on them, so there's very little risk of that issue being exploited.
Extreme was just an unfounded rumor. There are hardware-level things saying, for example, whether an interrupt comes from chip 1 or chip 2, and it doesn't support having more than 2. M1 Ultra is two M1 Max connected together, and there is no way to connect four M1 Max together.
If Extreme was planned and cancelled, it was cancelled several years before the M1 was launched.
Something that does seem weird to me is that M3 Max apparently doesn't have a way to connect two of them together (Ultrafusion). So I guess M3 Ultra is either not gonna happen or it's going to be an entirely different chip.
The yield on older chips like M2 is probably quite high, and the cost much lower than M3/4 chips. Apple can probably just throw a lot of cheap chips at the problem until they learn enough to design hardware better optimized for inference (and training?). Remember, Apple's not competing with Nvidia to sell their AI server hardware. They're just making a cost decision solving their own compute needs.
Long term, I want Apple's commitment to privacy. ChatGPT now has "memory," but I'd rather that information live on my iPhone's Secure Enclave like my health data.
I thought that it's feasible for Apple to release the M4 Ultra for the Mac Studio and/or Mac Pro this summer, but we got now two different rumours saying that the AI servers will have the M2 Ultra for the moment and will update to M4 later, which seems to suggest that maybe we are not going to get any M4 Ultra this summer.
At single-threaded tasks, yes.
So, for example, browsing the web is probably overall faster.
yes, my understanding of any bigger chips, there was no reason to move past design phase as any market fit study showed there just wont be enough sales to justify creating one. apple silicon servers changes that, even if they are purely internal products.
by the very fact of how fast Apple went from M3 to M4 we might assume the M3 was just a stop-gap generation and if Ultra happens again, it will be based on M4.
only question is, how fast TSMC is able to ramp up production/yield to release bigger M4 chips (and just to cool down any expectations, the they need to account for A18 production too)