Apple Explains How AI-Generated App Store Review Summaries Work in iOS 18.4

[MacRumors]

With the launch of iOS 18.4, Apple introduced a new App Store feature that summarizes multiple user reviews to provide an at-a-glance summary of what people think of an app or a game. In a new blog post on its Machine Learning Research blog, Apple provides some detail on how App Store review summaries work.

Apple is using a multi-step large language model (LLM) system to generate the summaries, with the aim of creating overviews that are inclusive, balanced, and accurately reflect the user's voice. Apple says that it prioritizes "safety, fairness, truthfulness, and helpfulness" in its summaries, while outlining some of the challenges in aggregating App Store reviews.

With new app releases, features, and bug fixes, reviews can change, so Apple's summarizations have to dynamically adapt to stay relevant, while also being able to aggregate both short and long reviews. Some reviews also include off-topic comments or noise, which the LLM needs to filter out.

To begin with, Apple's LLM ignores reviews that have spam, profanity, or fraud. Remaining reviews are then processed through a sequence of LLM-powered modules that extract key insights from each review, aggregating themes that reoccur, balancing positive and negative takes, and then generating a summary that's around 100 to 300 characters in length.

Apple uses specially trained LLMs for each step in the process, ensuring that the summaries are an accurate reflection of user sentiment. During the development of the feature, thousands of summaries were reviewed by human raters to assess factors like helpfulness, composition, and safety.

Apple's full blog post goes into more detail on each step of the summary generation process, and it is worth checking out for those who are interested in the way that Apple is approaching LLMs.

Article Link: Apple Explains How AI-Generated App Store Review Summaries Work in iOS 18.4

 

[Promostyle]

How embarrassing for folks who are still working at Apple waiting for their RSUs to vest to see the company working on bringing ML, but focusing that work on app reviews rather than elevating the user experience in a more meaningful way.

 

[Apple Knowledge Navigator]

“I’m sorry, I can’t find the song 5 Star Reviews by The Pokémon Company”

 

[turbineseaplane]

To begin with, Apple's LLM ignores reviews that have spam, profanity, or fraud.

Why would they not remove reviews like this?

 

[JosephAW]

Can’t wait to see some of the chaos results it’ll produce 🍿

 

[Zapski]

Wasting time and resources for minimal improvement. That’s the AI way

 

[cjsuk]

I can no longer trust these reviews. Good job Apple!

 

[ckoerner]

I don’t want homogenized slop, I want individual experiences and perspectives!

 

[jarman92]

Comment and review summarization is actually the best use-case I’ve seen for AI so far. Amazon and WaPo use it and seems like it works great.

 

[UpsetTheAppleCart]

Must be US only so far.

 

[tennisproha]

I don't see any review summaries. Is this limited to certain devices?

 

[Kottu]

See, Apple Intelligence is here already. Be happy with that.

 

[vantelimus]

How embarrassing for folks who are still working at Apple waiting for their RSUs to vest to see the company working on bringing ML, but focusing that work on app reviews rather than elevating the user experience in a more meaningful way.

Given that Apple stock is up 22% over what it was a year ago even with all the current market instability, I don't think Apple employees should be fretting about their RSUs.

 

[vantelimus]

I don’t want homogenized slop, I want individual experiences and perspectives!

Almost all online reviews are worthless garbage. By definition they are skewed by self-selection bias. Worse, you have no way to judge whether an unknown individuals "horrible app that always seems to be give the wrong information" gives exactly the information you were seeking nor whether their "best app ever" is completely inappropriate for you. Plus, you don't know whether you are reading a review from a shill or an irrationally dissatisfied user.

 

[4tune8chance]

With the launch of iOS 18.4, Apple introduced a new App Store feature that summarizes multiple user reviews to provide an at-a-glance summary of what people think of an app or a game. In a new blog post on its Machine Learning Research blog, Apple provides some detail on how App Store review summaries work.

Apple is using a multi-step large language model (LLM) system to generate the summaries, with the aim of creating overviews that are inclusive, balanced, and accurately reflect the user's voice. Apple says that it prioritizes "safety, fairness, truthfulness, and helpfulness" in its summaries, while outlining some of the challenges in aggregating App Store reviews.

With new app releases, features, and bug fixes, reviews can change, so Apple's summarizations have to dynamically adapt to stay relevant, while also being able to aggregate both short and long reviews. Some reviews also include off-topic comments or noise, which the LLM needs to filter out.

To begin with, Apple's LLM ignores reviews that have spam, profanity, or fraud. Remaining reviews are then processed through a sequence of LLM-powered modules that extract key insights from each review, aggregating themes that reoccur, balancing positive and negative takes, and then generating a summary that's around 100 to 300 characters in length.

Apple uses specially trained LLMs for each step in the process, ensuring that the summaries are an accurate reflection of user sentiment. During the development of the feature, thousands of summaries were reviewed by human raters to assess factors like helpfulness, composition, and safety.

Apple's full blog post goes into more detail on each step of the summary generation process, and it is worth checking out for those who are interested in the way that Apple is approaching LLMs.

Article Link: Apple Explains How AI-Generated App Store Review Summaries Work in iOS 18.4

So Apple is now telling me what I really think?

 

[adh1003]

This is great. After the success of AI notification summaries, we can have confidence that Apple's AI review summaries won't be total and utter ********, no siree!

"Users say that the Wise application is purple, and has passed away. There are 53 people waiting at Wise's front door."

 

[robfoll]

Apple could start by not leading with 3/4/5 even 6 year old reviews of apps. Totally crazy to look at an app and find the featured reviews are many years old. Look at Powerpoint attached, two 3 year old reviews are useless! Most recent reviews are 1 and 2 star as well. So possibly these AI summaries may be useful.

 

[robfoll]

This is great. After the success of AI notification summaries, we can have confidence that Apple's AI review summaries won't be total and utter ********, no siree!

"Users say that the Wise application is purple, and has passed away. There are 53 people waiting at Wise's front door."

Currently on iOS 18.5b3 and OSX 15.5b3 finding Message and Mail summaries fairly accurate and useful

 

[switz]

I have been involved with computers for over 55 years and have NEVER had a need for AI in all that time. A really fast wrong answer is more than useless. The list of what AI could do does not seem relevant for Apple's time and money. AI won't print a document faster or transmit a document across the internet faster. We have had non-AI spelling checkers for years as well as grammar.

Apple is a very rich company with a very restricted product line which is now nearly all based around a single chip for their products. Their business is basically communication technology: computers, music devices and phones. They dropped printers and home wifi equipment years ago.

 

[grovian]

As a developer, will I be able to pay to skew the AI summary in my favor? It's a black box, so no one will be able to figure this out. Also, I can easily see this happening in all stores, and then they'll train users to only read the Gen AI "summary" with some clever UI/UX changes, effectively rendering real user reviews completely meaningless while both developers and store owners make more money. It would take a week to implement this, tops.

 

[Baumi]

So Apple is now telling me what I really think?

No, they’re adding an AI generated summary of people‘s reviews to app store listings, just like Amazon already does for some of its items. Whether or not that is useful, accurate, or a good idea is debatable, but none of it amounts to “telling you what you think”.

 

[svish]

Might not be perfect in the beginning but expecting it to improve over time and this should be very useful.

 

[Octavius8]

Manipulation to show just positive vibes…

 

[Daalseth]

I have been involved with computers for over 55 years and have NEVER had a need for AI in all that time. A really fast wrong answer is more than useless. The list of what AI could do does not seem relevant for Apple's time and money. AI won't print a document faster or transmit a document across the internet faster. We have had non-AI spelling checkers for years as well as grammar.

Agree completely. Apple has stopped looking at the big picture. They no longer have someone like a Jobs that will look at a plan and say all right AHs but what is the benefit to the customer? Apple has fallen for the AI hype, and stopped looking at how it might make the Mac and the iOS platform better. The simple fact is that it doesn’t. They are pouring massive amounts down a rat hole that will not actually benefit the customer.

 

[VulchR]

Can’t wait to see some of the chaos results it’ll produce 🍿

Can't wait for the increased accuracy. Some apps were reviewed highly initially and then quality declined as apps became more complex and/or were neglected (here's looking at you, Scopely), but the early glowing reviews were rated 'most helpful' and therefore the ones on display.