Apple Working on Chips for New Macs and AI Servers

[BNBMS]

The problem is that AI models are basically implemented by researchers, and researchers chose CUDA a long time ago, and they won’t change (an unwise choice if you ask me, because it’s vendor specific, but most researchers are not smart when it comes to real implementation).

Believe it or not, CUDA is just one way to use. There reasons why NPU exists and technically, GPU itself is extremely inefficient for AI which is also why they need a dedicated NPU which is meant for AI. This is why many big tech companies such as Amazon, Microsoft, Tesla, and more are developing their own chips or NPU to replace Nvidia GPU.

Furthermore, GPU or graphic processing unit is literally meant for 3D graphics, NOT AI. That's how it works and that's why we need a dedicated chip for AI which is NPU. They literally are using GPU for something that wasn't meant for.

For now, Nvidia is dominating just because of CUDA ecosystem but doesn't mean it's the only choice.

 

[macduke]

I've barely taxed my M3 Max. Figured by now something would slow it down a bit but haven't encountered it. I worry about Apple slowing down on advancements in these chips because it might affect their bottom line. They are just too good! But maybe it will help grow their Mac user base.

While it has been great for the most part, for some reason my battery health has taken a big dip recently which is strange. For the longest time it was at 100% and then over the past six months it has gone down to 91%. I was out of a job for 5 of those 6 months, so I wonder if that played a part? I have mainly used it docked since then, and quite heavily for a two month period when I learned iOS dev and made my game. So many people say to just use Apple's smart charging algorithms, but I'm now regretting not using an app to limit charging to 80%. It hardly ever drops down below 10% before I put it on the charger either, more typically I put it on between 15-25% to limit wear. But I have AppleCare+ on it, so I guess they will replace it if it gets down to 80%.

Don't really see needing to upgrade this for a very long time because it's so fast, like not until M9 or M10 at the earliest, unless I get my house paid off early and just have money burning a hole in my pocket and want the OLED model.

 

[seek3r]

...but Apple don't make servers, so why would they want to build server chips?

They currently have custom servers for internal use, mostly focused on their AI backends, running an M2 variant. That’s likely the destination for these chips.

 

[seek3r]

My understanding that that since Apple dropped Mac servers several decades ago they rely exclusively on x86 servers running linux and programmed in C++. Apple definitely does not eat their own dog food. And that also explains why there are so many bugs in Apple non-C++ arm software.

Apple Reportedly Building M2 Ultra and M4-Powered AI Servers

Apple has started building its own AI servers that use the M2 Ultra chip, Haitong analyst Jeff Pu reports. Pu provided the analysis based on supply...

www.macrumors.com

 

[zephonic]

Ability to make Mac Pro's chip will directly important for them to create server and super computer just like Nvidia did. If you ever checked Nvidia, they provided a workstation with 4x GPU for a personal user like DGX. Dont forget that Apple also made a server based on Mac Pro.

Then where is Mac Pro's chip? Apple cant even make sever and super computer grade chip unlike Nvidia. Cant and Can is a huge difference.

Doesn't change the fact that Apple Silicon sucks for GPU and AI resources. That's also why they are struggling with their own AI server which only has 50,000 made out of more than 5 years old GPU, Cant even make a powerful desktop chip, and heavily limited by SoC as they need to make a huge chip.

I don't think the issue is Apple can't make a proper workstation or server CPU, I think they don't see the business case for it.

I bet the Mac Pro is one out of every 100,000 Macs sold, or thereabouts. The sales don't justify the expense. If you are Intel and you sell millions of server CPU's annually, that's different.

I think the argument here is that if Apple feels having their own AI hardware is critical to their business model, it will be done, and likely be done well.

But I understand the view that without CUDA it might be a losing proposition for third-party AI developers.

 

[BNBMS]

I don't think the issue is Apple can't make a proper workstation or server CPU, I think they don't see the business case for it.

I bet the Mac Pro is one out of every 100,000 Macs sold, or thereabouts. The sales don't justify the expense. If you are Intel and you sell millions of server CPU's annually, that's different.

I think the argument here is that if Apple feels having their own AI hardware is critical to their business model, it will be done, and likely be done well.

But I understand the view that without CUDA it might be a losing proposition for third-party AI developers.

They do and Mac Pro is directly connected to it. Since Apple hates Nvidia, they cant just rely on 5 years old GPU for their own servers. They will need their own servers eventually and for that, they need abilities to make Mac Pro.

Besides, there are many uses for Mac Pro market and yet, Apple ditched it by themselves. Not a good move.

 

[picpicmac]

If you think that way, tell me if there are any Apple Silicon based super computer to train and study AI for their own company before you even say other things.

How many times does the following have to be repeated?

APPLE IS NOT IN THE BUSINESS OF BUILDING SUPERCOMPUTERS. Whether for AI or anything else.

 

[BNBMS]

How many times does the following have to be repeated?

APPLE IS NOT IN THE BUSINESS OF BUILDING SUPERCOMPUTERS. Whether for AI or anything else.

WRONG. How come they are using Mac Studio for their own AI servers then? They even admitted about using 5 years old GPU for AI development which is another proof why they need it and it also benefits Mac Pro.

Denying is not a way to solve the problem and we still need Mac Pro for many uses.

 

[kwikdeth]

I would instantly throw money at an apple silicon 1U rack server that was silent or near silent. If they can put one in an iPad ...

 

[zephonic]

I’m pretty sure Apple’s back end is entirely Azure or AWS, or maybe Oracle or whatever.

I think they may have some small server farm with Mac Minis or Studios for PR reasons, but no way they’re running their services on those.

 

[tenthousandthings]

I’m pretty sure Apple’s back end is entirely Azure or AWS, or maybe Oracle or whatever.

I think they may have some small server farm with Mac Minis or Studios for PR reasons, but no way they’re running their services on those.

The only thing we can be sure they will be using Apple silicon for is Private Cloud Compute, which requires it. If Apple builds a server, it will be dedicated to that.

 

[deconstruct60]

What a weird story. Of course they are working on new chips. Anything else would be a story.

It isn't 'weird' for Gurman. He writes a weekly rumor newsletter (PowerOn). Leaks don't happen on an entirely daily/weekly basis for an entire year. But he has to ship out content each week. So there is an irregular stream of "the sky is blue" topics covered.

Macrumors has the same problem. Every several weeks or so "new iPHone n+1 coming these are the features you might see. " . Folks who read MacRumors every day are like "same stuff , different day". More sporadic readers might read it as 'new' and possible forget that they read it the next time they stop by.

 

[zephonic]

I asked ChatGPT, and it states:

Apple employs a hybrid cloud infrastructure strategy, combining its own data centers with services from major cloud providers to support platforms like iCloud and Apple Intelligence.


☁️ Cloud Providers Utilized by Apple

• Amazon Web Services (AWS): Apple has been a significant customer of AWS since at least 2011, utilizing services like Amazon S3 for iCloud storage. Reports indicate that Apple spends over $30 million per month on AWS services, making it one of AWS’s largest clients.
• Google Cloud Platform (GCP): In 2016, Apple began leveraging Google Cloud for parts of its iCloud infrastructure. By 2021, Apple was reportedly storing approximately 8 million terabytes of data on Google’s cloud, with annual expenditures reaching around $300 million.
• Microsoft Azure: Initially, Apple used Microsoft Azure alongside AWS for iCloud services. However, by 2018, Apple had phased out its use of Azure in favor of other providers.

🏢 Apple’s Own Data Centers

To reduce dependency on third-party services, Apple has invested in building its own data centers across various locations, including:

• Mesa, Arizona: A facility repurposed from a former sapphire glass plant.
• Maiden, North Carolina: One of Apple’s earliest large-scale data centers.
• Viborg, Denmark: A European data center powered entirely by renewable energy.

These centers handle a significant portion of Apple’s cloud services, enhancing control over data security and operational efficiency.


🤖 AI and Machine Learning Infrastructure


For advanced AI capabilities, Apple has collaborated with Google to utilize its Tensor Processing Units (TPUs) for training AI models, including those powering features like Apple Intelligence. This partnership underscores Apple’s reliance on external cloud infrastructure for computationally intensive tasks.


In summary, Apple’s backend infrastructure is a blend of its proprietary data centers and services from AWS and Google Cloud, with a historical reliance on Microsoft Azure. This multifaceted approach allows Apple to maintain flexibility, scalability, and control over its cloud services.

 

[2128506]

I have yet to get Genmoji to actually give me either a) anything, or b) something that represents what I asked for.
Latest -- "Clock showing time at 3'oclock". What it produced was a clock with 3 hands pointing at 10,12, and 2 o'clock (plus a seconds hand!).

Most other attempts result in nothing. Another useless AI feature on my iPhone 16 PM.

Just wait till they release new iPhone with proper AI chips - would definitely work then!

 

[Axemantitan]

When Apple comes out with the M8 chips, I hope the Apple website says, "G'day M8!"

 

[Serrafemme]

Thank god! We are definitely going to need some beefy chips to run the next update for Image Playground on device…

If I wasn’t so in love with the rest of The Ecosystem I would’ve already moved on…

 

[Howard2k]

I’m pretty sure Apple’s back end is entirely Azure or AWS, or maybe Oracle or whatever.

I think they may have some small server farm with Mac Minis or Studios for PR reasons, but no way they’re running their services on those.

They are running services, but not fully, clearly.

The first tier of Apple AI is on device, the second tier is Private Cloud Compute (Apple Cloud), and the third tier is ChatGPT or other external provider.

 

[etichi]

I DO NOT WANT AI!

But AI wants you or at least your data.

 

[Edgecrusherr]

These are not server CPU’s in the normal sense ie like Xeon. These are specialist chips that will go into AI Servers dedicated for handling AI services in the Cloud.
Don’t expect OSX Server and xServe to be making a return with this.

Yeah, I'm not holding my breath

 

[deconstruct60]

I’m pretty sure Apple’s back end is entirely Azure or AWS, or maybe Oracle or whatever.

I think they may have some small server farm with Mac Minis or Studios for PR reasons, but no way they’re running their services on those.

Small server farm for PR reasons?

" 29 Facilities 12 Markets 821MW Megawatts "

Apple Data Center Locations, News, Photos, and Maps

Apple operates 29 datacenters in 12 regions. Find out where Apple's sites are located. Complete with maps, specs, and Apple news.

baxtel.com

Creeping up on 1,000 MW is a PR 'stunt'? Hardly.

and also for sense of scope:

Apple Inc. Data Center Locations

Click here for a list of Apple Inc. data center locations. Find the right facility for colocation, bare metal and more or call us at +1 833-471-7100.

www.datacenters.com

The Newark, CA ( near HQ ) and Reno (where Apple counts their money. Apple was incorporated in NV to start . Never met a tax loophole they didn't like. ) ones are probably have limited outward facing customer services. Apple is large company and need substantive internal operations/services infrastructure.


Apple data center footprint is dwarfed by hyperscalars like AWS, Azure , and Google (and Facebook) ... but 'small' is also seriously wrong connotation. They have sites with more than several large buildngs on them.

The 'core' of the system isn't farmed out to AWS, Azure, etc. If give a 500-800M people 'free' 5GB of space allocations it is pretty tough do that with storage service have to pay ongoing 'rental' fees for. [ 500M people at 5GB is 2,500 PB (petabytes) of storage space. Have one backup ... twice as much. No direct revenue paying for this.. all out of pocket. ] Apple ID authentications, end-to-end encrypted transmission reports , etc. also pretty hard to do 'free' at rental pricing.

Serving up read only dynamic iTunes traffic, video 'fan out' traffic. , Private Rely , caching for sales site ( product pitch pages) , n etc. is outsourced.
For example

"... Cloudflare is proud to work with Apple to operate portions of Private Relay infrastructure. ..."

iCloud Private Relay: What Cloudflare Customers Need to Know

iCloud Private Relay is an Apple service for browsing more privately and securely. Learn how to ensure the best user experience using iCloud Private Relay.

blog.cloudflare.com

but those are not the inner core of Apple web services.

 

[zephonic]

@deconstruct60

Thanks for sharing those links.

I got two things mixed up, I have known about Apple’s datacenters, my mistake.

But the point I was making was no way are they deploying MacMinis or Studios at that scale, that makes no sense. So let me amend my original statement as follows:

I’m sure they have a datacenter somewhere with Minis/Studios for PR reasons. But I’m equally sure the heavy lifting is done by generic server hardware.

 

[tenthousandthings]

~

I’m sure they have a datacenter somewhere with Minis/Studios for PR reasons. But I’m equally sure the heavy lifting is done by generic server hardware.

If they do, it’s not a very effective PR campaign, because nobody knows about it! On the other hand, we know for a fact that Apple is using their own silicon for Private Cloud Compute (PCC).

This rumor suggests the first PCC servers were built using M2 Ultra, and there is an earlier rumor that points to M3 Ultra (now that we know there is no M4 Ultra) or possibly a different M3 Ultra variant designed for this purpose.

Neither of these would use stock Mac Studios or Mac Pro logic boards — PCC has its own secure OS, they can build whatever they like for that purpose.

 

[Ursadorable]

How come they are using Mac Studio for their own AI servers then?

And why is Apple Intelligence complete garbage? Maybe the two are related.

 

[Howard2k]

WRONG. How come they are using Mac Studio for their own AI servers then? They even admitted about using 5 years old GPU for AI development which is another proof why they need it and it also benefits Mac Pro.

Denying is not a way to solve the problem and we still need Mac Pro for many uses.

They're not, they're using custom hardware.

Blog - Private Cloud Compute: A new frontier for AI privacy in the cloud - Apple Security Research

Secure and private AI processing in the cloud poses a formidable new challenge. To support advanced features of Apple Intelligence with larger foundation models, we created Private Cloud Compute (PCC), a groundbreaking cloud intelligence system designed specifically for private AI processing...

security.apple.com

 

[zephonic]

~

If they do, it’s not a very effective PR campaign, because nobody knows about it! On the other hand, we know for a fact that Apple is using their own silicon for Private Cloud Compute (PCC).

This rumor suggests the first PCC servers were built using M2 Ultra, and there is an earlier rumor that points to M3 Ultra (now that we know there is no M4 Ultra) or possibly a different M3 Ultra variant designed for this purpose.

Neither of these would use stock Mac Studios or Mac Pro logic boards — PCC has its own secure OS, they can build whatever they like for that purpose.

Not to be a d—k, but you say we know for a fact, and then link to two rumors?

Anyway, that is specific to PCC which is a fairly new development. I acknowledged earlier that Apple is perfectly capable of designing their own AI server chips (see post #80).

But some people seem to think they have server farms with Minis/Studios, and I'm calling nonsense.

I do remember seeing some photos of a data center full of MacMinis, but I'm pretty sure none of those power Apple's cloud infrastructure. Hence my assessment those are mostly for PR (just so they can say Minis are a viable option for that use).