Got a tip for us?
Let us know
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
iOS 18: AI Server Industry Aiming to Win Business From Apple
- Thread starter MacRumors
- Start date
- Sort by reaction score
This was a news story that was disproven. Bloomberg got bad info from their sources.
Talking about machine learning and esp training, Apple has nothing
Running an LLM locally is one story, but training of a LLM is a completely different story. For this task you need the Nvidia chips.
Beside that fact, Honor is already running Metas LLM locally on a smartphone (a less powerful version).
https://www.trendingtopics.eu/honor-chinese-smartphone-giant-integrates-metas-ai-model-llama-2/
Why is the comparison always to GPT? It seems obvious that Apple would not want or need to replicate GPT on the phone.
With respect to training the models: they only have to be trained once. Even if done on servers (I'm not sold on your supposition that this is a self-evident requirement; doesn't that mostly depend on the data size?), why would it entail a massive, multi-billion dollar investment in AI server hardware on Apple's side?
CUDA is a programming platform /API. A GPU is hardware. Saying that CUDA outperforms Apple GPU is like saying pascal outperforms a 486. It’s a nonsense statement.
And yet there is a point buried in the nonsense, which is that Nvidia’s success is more about the widespread use of CUDA and its optimization for Nvidia GPUs than it is about Nvidia’s actual GPUs.
And that is also why Apple can compete with Nvidia for Apple’s business. Apple has some of the best silicon design people and some of the best compiler and software tools people. Writing highly optimized software for their own hardware and for their own purposes is what Apple is all about.
Are you speculating that Apple's datacenters use these CPU/APUs or is this a known fact? Reason I ask is that my brother was responsible for these.
Seriously? They're called CUDA Cores.
NVIDIA H100
Transform your AI workloads with the NVIDIA H100 Tensor Core GPU, featuring the new Transformer Engine and NVIDIA AI Enterprise.
docs.nvidia.com
You dont specifically need nvidia’s chips, nv just happens to dominate that segment right now, there are and will be more other players.
Now whether Apple wants to get into that or not is a different story, but they definitely have both the expertise and the cash to do so if they wanted. An updated xserve with an M* chip built for performance and massive shared memory and a hugely boosted GPU and NPs could be an interesting competitor in the space. It’s not the kind of area of the market apple usually targets though, the last thing even close would have been xgrid on the old xserves, and that was never more than barely a footnote in any form of HPC or distributed workloads really
terrible example, Apple Silicon doesn’t exist without TSMC and (formally) Samsung.
Apple says a lot of things, but at the end of the day, while they might design everything in-house from their hardware to their software, they don’t actually *produce and run* pretty much anything on their own.
Their displays are produced by Samsung, their cameras are produced by Sony, their modems are produced by QUALCOMM, and so on.
Even the rumored upcoming Apple modems are still going to be mass produced by TSMC.
Edit: and the ultimate irony that Apple uses Google servers for iCloud…
Last edited:
Rumor has it than iOS 18 will be built on AI, so Apple does have some tricks up their sleeve, they have not told us much about it yet, but we will learn more at WWDC next summer. They could run this on their own hardware server platform that is built on their own silicon, and it would give the industry a run for its money. Apple's strong suit is that they develop the hardware, software, and services, and this integration is what makes them so successful.
Why din‘t people understand that running and training LLMs are two different pait of shoes. Apple has no powerful silicon to train their LLMs like the H100, but it can run trained LLMs on its A-silicon, of course.
Have a look:
https://resources.nvidia.com/en-us-tensor-core/nvidia-tensor-core-gpu-datasheet
Seriously? They're called CUDA Cores.
NVIDIA H100
Transform your AI workloads with the NVIDIA H100 Tensor Core GPU, featuring the new Transformer Engine and NVIDIA AI Enterprise.
CUDA - Wikipedia
They do really exceed in efficiency as well.
And the 4090 is a 5nm consumer GPU launched in 2022 while the M3 Max is on 3nm. When Nvidia will move to 3nm it will at least double 4090's performance at similar power. And the 4090 is consumer gaming GPU not the best Nvidia can do.
In order for Apple to get close to Nvidia's level they need do increase the performance in various workloads by 10X in 1 or 2 generations. I don't see how they can do that.
I'm not sure that graphs like this are as informative as they appear.They do really exceed in efficiency as well.
And the 4090 is a 5nm consumer GPU launched in 2022 while the M3 Max is on 3nm. When Nvidia will move to 3nm it will at least double 4090's performance at similar power. And the 4090 is consumer gaming GPU not the best Nvidia can do.
In order for Apple to get close to Nvidia's level they need do increase the performance in various workloads by 10X in 1 or 2 generations. I don't see how they can do that.
When software is changing as fast as AI software changes right now, mainly what you are seeing is which software has currently been ported to/developed on which hardware. For example if we took the Mac Whisper software and spent some time porting it to Mojo how would that change things? If we fixed it to use an Apple Silicon optimized flash-attention, how would that change things?
Insanely-fast-whisper does supposedly run on Apple Silicon, but I can find ZERO benchmarks for it; and that's even assuming it makes optimal use of the most recent libraries (like MLX or Philip Turner's Flash Attention).
nVidia deserve their attention – they have worked hard to keep their SW relevant and improved year after year, along with continuous improvement of their hardware.
BUT like I said, a snapshot like the above of where they appear to be today [and actually that graph is from about 2 months ago] is mainly a snapshot of who has bothered to put together an optimized collection of SW and benchmark; it's in no sense a measure of *intrinsic* abilities and *intrinsic* 10x performance or efficiency differences.