Apple Uses Amazon's Custom AI Chips for Search Services

[MacRumors]

Apple uses custom Inferentia and Graviton artificial intelligence chips from Amazon Web Services for search services, Apple machine learning and AI director Benoit Dupin said today at the AWS re:Invent conference (via CNBC).

Dupin said that Amazon's AI chips are "reliable, definite, and able to serve [Apple] customers worldwide." AWS and Amazon have a "strong relationship," and Apple plans to test whether Amazon's Trainium2 chip can be used for pre-training Apple Intelligence and other AI models. Amazon announced rental opportunities for the Trainium2 chip today.

Apple has used AWS for more than 10 years for Siri, Apple Maps, and Apple Music. With Amazon's Inferentia and Graviton chips, Apple has seen a 40 percent efficiency gain, and with Trainium2, Dupin said Apple expects up to a 50 percent improvement in efficiency with pre-training.

Nvidia is the market leader when it comes to GPUs for AI training, but companies like Amazon are aiming to compete with lower-cost options.

Article Link: Apple Uses Amazon's Custom AI Chips for Search Services

 

[JippaLippa]

Great...
One one side Open AI, on the other Amazon.
What's next, Meta?

 

[rp2011]

There has been good buzz for Trainium 2 and the team that designed it.

 

[magicschoolbus]

My amazon shares welcome this news.

 

[jz0309]

good to see that there is more than Nvidia for AI ...

 

[cateye]

Great...
One one side Open AI, on the other Amazon.
What's next, Meta?

There's marketing, and then there's business. Apple does significant business with Google and Microsoft, too. As I'm sure Microsoft does business with Oracle, and Google with Meta, and Amazon with Apple, and on down the line, every possible combination.

The "my favorite company is more pious and pure than your favorite company" is sports ball-style chum for ad impressions on sites like this one. The reality is, they all work together, as it suits them, because that's how business works. Efficiency first.

 

[anakin44011]

So...can I have one of those chips for my MacBook?

...because Spotlight Search has become progressively worse over time. Its speed and top suggestions are almost laughable.

...or is it just me...and the last three M-Series MacBooks I've purchased?

 

[coffeemilktea]

This is sorta off topic, but I thought the "annapurnalabs" logo on that chip was a reference to Annapurna Pictures, and I thought "wow, they really branched out from making overrated films and into hardware manufacturing? 😮" but no, it's an entirely different company.

 

[hoodafoo]

Trainium in Vainium

 

[hieranonymous]

Wait, wasn’t Apple rumored to be using Google’s chips at one point?

And more importantly, is this just a stopgap to Apple using its own silicon down the road?

It feels increasingly plausible that the AI boom caught Apple with its pants all the way down.

 

[neuropsychguy]

Wait, wasn’t Apple rumored to be using Google’s chips at one point?

And more importantly, is this just a stopgap to Apple using its own silicon down the road?

In the linked CNBC article: "Earlier this year, Apple said in a research paper that it had used Google Cloud’s TPU chips to train its iPhone AI service, which it calls Apple Intelligence."

Also in the article: "Unlike leading chatbots like OpenAI’s ChatGPT, Apple’s approach to AI isn’t based on large clusters of Nvidia-based servers in the cloud. Instead, Apple uses an iPhone, iPad or Mac chip to do as much of the processing as possible, and then sends complicated queries to Apple-operated servers using its own M-series chips."

 

[orbital~debris]

AWS and Amazon have a "strong relationship,"

I should hope so 🤣

 

[klmiles61]

I'm sorry may just be me, but I thought AWS stood for amazon web services? If this is not correct please by all means assist me, but if AWS does stand for Amazon web services wouldn't that be a part of Amazon so technically they should have a strong relationship am I not correct?

 

[ProbablyDylan]

Also in the article: "Unlike leading chatbots like OpenAI’s ChatGPT, Apple’s approach to AI isn’t based on large clusters of Nvidia-based servers in the cloud. Instead, Apple uses an iPhone, iPad or Mac chip to do as much of the processing as possible, and then sends complicated queries to Apple-operated servers using its own M-series chips."

That's all for inference, though. Apple needs a lot more power for actually training the models that ultimately run on those M-series servers.

 

[JippaLippa]

There's marketing, and then there's business. Apple does significant business with Google and Microsoft, too. As I'm sure Microsoft does business with Oracle, and Google with Meta, and Amazon with Apple, and on down the line, every possible combination.

The "my favorite company is more pious and pure than your favorite company" is sports ball-style chum for ad impressions on sites like this one. The reality is, they all work together, as it suits them, because that's how business works. Efficiency first.

Humans are tribalistic by nature, hence the console wars and OS wars.
Cringe yet fascinating at the same time.

 

[GermanSuplex]

Apple Intelligence summarized an email by saying it was me who was leaving the US congress in a few weeks.

Half the time I use writing tools, it tells me it’s not meant for that kind of content - even when said content is not political or controversial. It’ll still summarize it, but it takes an extra step.

I know it’s in beta, but it’s been increasingly lackluster. Siri seems as bad as ever. I had to power off and on the phone yesterday because “Siri, Play (insert song here)” would start playing Aerosmith’s “I Don’t Want To Miss A Thing”, regardless of what song, artist, album or playlist I requested. It worked correctly after a reboot, but not better than before iOS 18.1.

Maybe it’s just me, but while it was more basic ten years ago, I swear Siri worked better for fundamental tasks like playing songs and playlists. Pretty underwhelming thus far.

 

[TheOldChevy]

I wonder what makes these AI chips special. Number of CPUs, bus speed, instruction width?

 

[novagamer]

Wish they used AMD so ROCm wouldn’t be such a pile of hot garbage.

Apple Intelligence summarized an email by saying it was me who was leaving the US congress in a few weeks.

…

Maybe it’s just me, but while it was more basic ten years ago, I swear Siri worked better for fundamental tasks like playing songs and playlists. Pretty underwhelming thus far.

It’s not just you, GenAI cannot summarize – it’s a key area they not only struggle with but functionally can not do well.

I invite anyone wants to argue with me to please reply with research papers, I’ve asked before and I’ll ask again in the future… I’m deeply involved in this area and would love someone to figure out how to crack that problem or point me to newer research that even attempts to do so and has measurable success.

Until then, it is puzzling why it was one of the first features they rolled out. It intuitively seems like it can summarize, but it just can not. Statistics and regression aren’t the same thing as critical human thought and novel synthesis.

I don’t hate GemAI, but I wish there was more responsible roll-out of the tooling and more understanding / openness about the limitations.

 

[artifex]

This is sorta off topic, but I thought the "annapurnalabs" logo on that chip was a reference to Annapurna Pictures, and I thought "wow, they really branched out from making overrated films and into hardware manufacturing? 😮" but no, it's an entirely different company.

That reminds me, there's also an Annapurna Interactive, which is the game studio division of Annapurna Pictures, and some of their games have ended up free on Amazon Prime's gaming thing. 😅

 

[2128506]

Let me be the first one:

Apple Silicon is Dead
(news at five)

 

[mdriftmeyer]

Great...
One one side Open AI, on the other Amazon.
What's next, Meta?

It's temporary. It's cheaper leasing time on AWS while they modernize their global data centers.

 

[sherwinzadeh]

Pretty sure that guy violated his NDA. Apple doesn't like partners to reveal what tech they use.

 

[tenthousandthings]

I'm sorry may just be me, but I thought AWS stood for amazon web services? If this is not correct please by all means assist me, but if AWS does stand for Amazon web services wouldn't that be a part of Amazon so technically they should have a strong relationship am I not correct?

I read that as a typo for “AWS and Apple”

 

[cateye]

Pretty sure that guy violated his NDA. Apple doesn't like partners to reveal what tech they use.

He was an invited speaker, appearing there on behalf of Apple.

Think like when Apple trots out someone from Adobe or some game developer or whomever during their product launches to promote their own product and sing the praises of Apple technology—same idea. It's all above-board and likely fully vetted beforehand.

The technology Apple uses is not some huge secret. For example, Apple has rotated among Azure, Google Cloud, and S3 for iCloud storage for years.

 

[erthquake]

Wish they used AMD so ROCm wouldn’t be such a pile of hot garbage.



It’s not just you, GenAI cannot summarize – it’s a key area they not only struggle with but functionally can not do well.

I invite anyone wants to argue with me to please reply with research papers, I’ve asked before and I’ll ask again in the future… I’m deeply involved in this area and would love someone to figure out how to crack that problem or point me to newer research that even attempts to do so and has measurable success.

Until then, it is puzzling why it was one of the first features they rolled out. It intuitively seems like it can summarize, but it just can not. Statistics and regression aren’t the same thing as critical human thought and novel synthesis.

I don’t hate GemAI, but I wish there was more responsible roll-out of the tooling and more understanding / openness about the limitations.

You're going to have to give us a little bit more than, "I'm deeply involved in this area," for us to take you seriously. I've found that generative AI's ability to summarize content is one of its strengths. You don't have to believe me. Just believe Amazon. They apparently think it's good enough to put their retail business at risk using AI to summarize user reviews.

I mean, seriously. Asking us to "reply with research papers" proving GenAI is good at summarization is kinda like asking to provide research saying that water is wet. It's so obvious that nobody's going to research a broad generalization like that.

If you want to provide us research showing specific cases where GenAI doesn't summarize content with 100% accuracy, go for it. Let's pick it apart and debate it. I'm sure they exist. But does something need to be perfectly accurate to be useful? In most cases, no.