Become a MacRumors Supporter for $25/year with no ads, private forums, and more!

MacRumors

macrumors bot
Original poster
Apr 12, 2001
54,972
17,356


The ongoing Epic Games v. Apple trial continues to provide insight into Apple's App Store processes, with documents today offering up details on the number of apps submitted to the App Store and rejected by Apple's review processes.

app-store-blue-banner.jpg

Apple received an average of five million app submissions per year between 2017 and 2019, and between 33 and 35 percent of apps submitted were rejected by Apple's review team. On average, there were 1.7 million apps rejected per year, though the rejection rate was closer to 40 percent in 2020 based on trial testimony.

app-store-rejections.jpg

Apple's documentation suggests that the App Store sees 100,000 App Store submissions per week, which are dealt with by 500 human experts that Apple has on hand. Prior to getting to a human review, apps are analyzed by Apple-designed tools to check for malware and policy violations.

A testing tool called Mercury runs through static and dynamic analysis processes, with the tool allowing Apple to see inside apps to check for hidden code or abuse, and there are other review tools that Apple has nicknamed "Magellan" and "Columbus." After automated testing, apps receive human oversight.

Dynamic testing includes everything from battery usage to file system access and privacy requests to access device hardware like the camera and microphone, while static analysis checks app size, entitlements, in-app purchases, keywords, descriptions, and more.

In 2015, Apple discussed acquiring SourceDNA, a company that made a tool to allow companies to see the code inside apps. Apple did end up purchasing the company and using its engineers to design a new tool for app oversight.

Interestingly, Apple documents depict the workstation of one of its human reviewers, featuring a desktop with an iMac, MacBook Pro, multiple iOS devices, several displays, game controllers, and more.

apple-human-review.jpg

Apple marketing director Trystan Kosmynka was questioned for most of the morning, and Epic lawyers visited a favorite talking point - App Store mistakes. Kosmynka was grilled about some of the apps that slip through the review process, such as an app about school shooting that he said in an email he was "dumbfounded" had been missed.

On this topic, Kosmynka was asked if the app review process is unnecessary because of the mistakes that are sometimes made, but said that all it means is that Apple has to "continuously be better." He said that Apple works diligently to close loopholes, and that without app review, iOS would be a "free for all" that would be "incredibly dangerous to customers, to kids."

The Epic vs. Apple trial will continue for another two weeks, with the first week set to wrap up today. Apple CEO Tim Cook and other executives are expected to testify during the third week.

Article Link: Apple Rejected Approximately 35% of Apps Submitted to App Store Between 2017 and 2019
 

Realityck

macrumors 68030
Nov 9, 2015
2,634
3,665
Silicon Valley, CA
Apple marketing director Trystan Kosmynka was questioned for most of the morning, and Epic lawyers visited a favorite talking point - App Store mistakes. Kosmynka was grilled about some of the apps that slip through the review process, such as an app about school shooting that he said in an email he was "dumbfounded" had been missed.

On this topic, Kosmynka was asked if the app review process is unnecessary because of the mistakes that are sometimes made, but said that all it means is that Apple has to "continuously be better." He said that Apple works diligently to close loopholes, and that without app review, iOS would be a "free for all" that would be "incredibly dangerous to customers, to kids."
You can have your choice unsafe free for all, or a safe environment for customers.
 

javanate

macrumors regular
May 13, 2005
134
486
Geez, that is 200 apps per person to review per week! Glad that is not my job!
Was just working that out too lol, 40 a day that's quite a lot.

I used to work in a very big industry that had to review about 40 items a day per person, the vast majority of which was prescreened by automated software. It was was about a 500 person team as well, pretty similar. We got spot checked quite a bit and still made mistakes!
 
Last edited:

now i see it

macrumors 604
Jan 2, 2002
7,967
15,731
So according to this, every apple App reviewer has to review roughly 40 apps a day every day, year after year.
No wonder there's so much junk in the App Store. Obviously a lot of it gets rubber stamped with approval without really being scrutinized.
 

neuropsychguy

macrumors 68000
Sep 29, 2008
1,588
3,107
Apple Reviewer's workstation, or MacRumors user's battlestation?
It looks almost like my ideal setup! I only have 4 screens right now (counting my iPhone) but do have 3 computers (not counting my iPhone). Two go to one large 43" screen so I can alternate as needed and one drives two 27" screens.
 

Canada420

macrumors member
Mar 16, 2012
86
98
The specific period of time being reported was selected on purpose. Giving a year over year deltas would show a change or admission of issues should there be one. They lump all the data together so that no story can be told.
 

nm2124

macrumors newbie
Dec 6, 2019
3
4
200 apps a week per reviewer, 40 apps a day, based on an 8 hour work day, that’s an average of 12 minutes to review each app. That’s extremely impressive to have as decent QC as they do.
 
  • Like
Reactions: iModFrenzy

BornAgainMac

macrumors 604
Feb 4, 2004
6,874
4,313
Florida Resident
I am going thru my apps and rejecting them too that get approved by Apple. If an app requires me to sign up, subscribe, shows ads, or interface sucks, links that take me to Safari, or functionality poor then it gets deleted. The Apple Arcade games also increased my expectations on how a game should look and play. Those games are more like the original games that started the app store that you would pay money just up front once.
 

mtneer

macrumors 68040
Sep 15, 2012
3,159
2,688
I expected that number to be higher, given the rich target Apple users paint for scammers and frauds.
 

hagar

macrumors 65816
Jan 19, 2008
1,253
2,716
How bad is the 35% they throw out if you see what does get through? According to that logic the vetting process should even be more strict to keep more garbage out.

This isn’t a smoking gun for Epic but rather a testimony in favour of Apple that it’s good they put up walls around their store.
 

Graphikos

macrumors member
Oct 26, 2007
99
329
Geez, that is 200 apps per person to review per week! Glad that is not my job! (5 apps/hour, 1 app every 12 minutes)

So according to this, every apple App reviewer has to review roughly 40 apps a day every day, year after year.
No wonder there's so much junk in the App Store. Obviously a lot of it gets rubber stamped with approval without really being scrutinized.

200 apps a week per reviewer, 40 apps a day, based on an 8 hour work day, that’s an average of 12 minutes to review each app. That’s extremely impressive to have as decent QC as they do.

As noted, "Prior to getting to a human review, apps are analyzed by Apple-designed tools to check for malware and policy violations."

I imagine the automated tools weed out a lot before they even get to human reviewers lessening the workload. A lot of apps probably just get rejected for simple things that computers can easily check for.
 

Luna Murasaki

macrumors newbie
Jun 24, 2020
18
49
Apple marketing director Trystan Kosmynka was questioned for most of the morning, and Epic lawyers visited a favorite talking point - App Store mistakes. Kosmynka was grilled about some of the apps that slip through the review process, such as an app about school shooting that he said in an email he was "dumbfounded" had been missed.

On this topic, Kosmynka was asked if the app review process is unnecessary because of the mistakes that are sometimes made, but said that all it means is that Apple has to "continuously be better." He said that Apple works diligently to close loopholes, and that without app review, iOS would be a "free for all" that would be "incredibly dangerous to customers, to kids."
My house was robbed because I forgot to lock my front door on Tuesday. So I physically removed all the locks on my house!

How can anyone possibly make an argument this stupid?!
 

xWhiplash

Contributor
Oct 21, 2009
4,747
3,384
So according to this, every apple App reviewer has to review roughly 40 apps a day every day, year after year.
No wonder there's so much junk in the App Store. Obviously a lot of it gets rubber stamped with approval without really being scrutinized.
I guess a solution COULD be to have App Review process take months for 1 single app (depending on complexity of the app). That way iOS would be 100% free (Apple would review the source code and test for months) and it will eliminate the junk as why would I submit a junk app that will be published 1-2 months from now?
 

arlomedia

macrumors member
May 5, 2021
47
41
My development setup looks pretty similar to that, but they only have to deal with macOS and iOS and I'm developing for Android, too!

Anyway, this bit of insight into the review process is interesting. My apps have a lot of free-form user entry fields that sync to a cloud account, but only once or twice have I seen something funny that the reviewers have entered. Apple runs a tight ship and they've succeeded in depersonalizing the process even though there are clearly real people on the other end.
 

Abazigal

Contributor
Jul 18, 2011
15,452
14,804
Singapore
My house was robbed because I forgot to lock my front door on Tuesday. So I physically removed all the locks on my house!

How can anyone possibly make an argument this stupid?!

Exactly. We see the mistakes the App Store makes, but we don’t get any insight into the apps that do get flagged and blocked.

Apple is stuck between a rock and a hard place on this, but I would rather there be a vetting process (however flawed) than none at all.
 
  • Like
Reactions: mhnd

amartinez1660

macrumors 65816
Sep 22, 2014
1,135
1,028
Geez, that is 200 apps per person to review per week! Glad that is not my job! (5 apps/hour, 1 app every 12 minutes)

Apple Reviewer's workstation, or MacRumors user's battlestation?
Haha, oh man, no, I have an iMac with 3 extra screens plus an iPad Air (first one) as the fourth screen yet it looks a whole lot less crowded and cramped. Like the first quoted comment, I’m glad it isn’t my job and I’m glad my work space doesn’t look like it at all.
To be honest, that picture does bring a lot regarding the “human side” behind the review process... it will make me less of a jerk towards the reviewing from now on.
 

Cosmosent

macrumors 68020
Apr 20, 2016
2,117
2,417
La Jolla, CA
FYI, here is a Datapoint outside the norm, v8.35 of my main app has been "In Review" today @ Apple App Review since 11:31 AM; it is now 6:31 PM.

And, this has happened a few times in the past !

NOT every app gets ONLY a few minutes of Review !
 
  • Disagree
Reactions: cyb3rdud3
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.