Got a tip for us?
Let us know
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
Apple to Power AI Features With M2 Ultra Servers
- Thread starter MacRumors
- Start date
- Sort by reaction score
they dont even have to. just custom compilation of macOS with Hypervisor framework and they can run any arm64 linux distro.
A chip with no design flaws could protect user privacy
Apple Silicon Security Flaw Discovered in iPhone 12 and M2 MacBook Air
A security vulnerability that could allow attackers to access sensitive data has been discovered in some Apple silicon GPUs. A recent investigation...
www.macrumors.com
Apple Silicon Vulnerability Allows Hackers to Extract Encryption Keys
An unpatchable vulnerability has been discovered in Apple's M-series chips that allows attackers to extract secret encryption keys from Macs...
www.macrumors.com
The flaw — a side channel allowing end-to-end key extractions when Apple chips run implementations of widely used cryptographic protocols — can’t be patched directly because it stems from the microarchitectural design of the silicon itself.
Are there more specifics to this data? Just wondering because I would imagine that not all “TOPS” are equal. For example a Jetson Orin Nano by Nvidia is capable of 40TOPS.
lol
you can desing chip with no known design flaws
you cant design chip without possible flaws.
I don’t think the M2 Ultra will be used for AI tasks, but that it’s just the server’s main CPU. The latest A-chip has a more powerful NPU, so I think Apple is already making big AI co-processors to be used specifically for AI. I think Apple might even offer those co-processors as a PCI-Express card for M4 Mac Pros to directly compete with Nvidia.
They should make their own servers again! The Intel Xserves couldn't compete with any other x86/x86-64 Server. But back when they were on PowerPC, there was at least a novelty. I think the one thing that would make an Apple Silicon Xserve difficult to be viable is the fact that it can only natively boot macOS. But toss on the open source equivalents to the utilities that set Mac OS X Server apart from its client counterparts, and you have a pretty decent alternative to the Windows/Linux duopoly in that space.
no such thing as "server OS" - its just an windows or linux without end-user components like desktop, apps, or media stuff. and its exactly what i meant with "custom compilation of macOS"
As long as Apple is designing its own servers, they should release a new XServe and sell it to the masses.
Apple plans to power some of its upcoming iOS 18 features with data centers that use servers equipped with Apple silicon chips, reports Bloomberg’s Mark Gurman.
M2 Ultra chips will be behind some of the most advanced AI tasks that are planned for this year and beyond. Apple reportedly accelerated its server building plans in response to the popularity of ChatGPT and other AI products, and future servers could use next-generation M4 chips.
While some tasks will be done on-device, more intensive tasks like generating images and summarizing articles will require cloud connectivity. A more powerful version of Siri would also need cloud-based servers. Privacy has long been a concern of Apple’s, but the team working on servers says that Apple chips will inherently protect user privacy.
Gurman previously claimed that all of the coming iOS 18 features would run on-device, but it sounds like some capabilities coming this year will in fact use cloud servers. Apple plans to use its own servers for now, but in the future, it may also rely on servers from other companies.
Article Link: Apple to Power AI Features With M2 Ultra Servers
I don’t have a link but I use an M1 Max and Nvidia 4090. Depends on the usage and data size.
4090 is faster to run some open source models(vision and small LLM models) that can run on 24 GB VRAM.
Usually, it runs out of memory with data sets of more than 7-10 GB.
Nvidia 4090 can have surge of up to 650 W under heavy load. Spikes can be a problem if you do not have a 1600 W or 1350 W Power supply.
M1 Max is slower in GPU execution time than 4090 but can run larger models using 64 GB memory.
In my experience, usually, training on larger data is 60% GPU speed, IO/Memory, batch sizes, and number of workers make up the remaining 40%. For larger datasets, if I am using 4090, I need to feed in small batches with less number of workers because of memory limitations. IO/batch size often bottleneck a 4090. M1 max runs larger batches and more workers, can hold a larger cache in unified memory, and can keep the data/model warm. Here is a snapshot from M1 Max, which handles cache and loading efficiently.
Basically for larger models/Datset its a wash in terms of speed.
Moves from cache to wired memory for GPU processing. Good luck with latency of Large data/models on 4090.
I totally agree. Apple has a fully certified UNIX system in macOS, the plumbing is enterprise-grade. Even if they created a downloadable suite of server tools that could be installed on the consumer version of macOS, along with a proper rack-mountable server hardware product with a RAID array, they could steal the show.
That's a good point but those chips are never going to be anywhere close to even h100 let alone h200 and b200 simply because they were not made specifically for AI. remember that those so called Ultra chips have slower NPU than iPhone 15 Pro Max.
The second most valuable company in the world should not worry about "cost saving" unless they want to cheap out (it's not like they care about cost when they wasted 10 billions dollars for Apple car that went into nothing), like they do on their $800 60hz phones.
It's the next 3D tv, AR/VR, or foldable phone. A dumb trend that no one will care about in a few years.
Man I don't know, each generation of Nvidia GPU gets more and more power efficient. My latest GPU upgrade (a 4070) didn't require a PSU upgrade, but upgrading to a 3080 the previous year would have. For the same performance.
This is an odd statement...
I see the opposite.... that Apple has been using other companys' servers, and will be switching to entirely in-house servers over the next decade. Building their own servers is not some short-term fix.
What operating system will they run? macOS? A specialized build? I vote for the latter.
I see the opposite.... that Apple has been using other companys' servers, and will be switching to entirely in-house servers over the next decade. Building their own servers is not some short-term fix.
What operating system will they run? macOS? A specialized build? I vote for the latter.
True for onesie's. Apple isn't building these in onesie's.
Massive data centers are limited by power and cooling. The lifetime power bill adds up to more than the cost of the chips and servers. When filling a jumbo warehouse sized data center, you can put a ton more M2's per rack than H100's, given the same power and thermal footprint per rack. Plus you get to keep nvidia's profit margin and spend that on even more gigaWatts of power and cooling capacity for the building.
Last edited:
If Apple ever wants to boost their stock price by a significant margin, all they have to do is to announce they will sell these “AI” chips to other hardware vendors.
I thought AI peaked at Home Pod, Alexa and Google Home. Not saying I'm against AI but what qualities can it possibly bring besides increasing OS storage use to process commands/questions offline.
I know the Samsung s24u I have can create unique wallpapers based on my input along with forming and editing messages which I don't care to use.
I know the Samsung s24u I have can create unique wallpapers based on my input along with forming and editing messages which I don't care to use.
AI is the next step in our human evolution! Open your eyes!
The idea is that your prompts would be going to Apple servers instead of OpenAI, Google, M$. If you trust Apple to keep your PII private, then using their servers is inherently more private.