Apple Has 4 Million Users Beta Testing its Software

[Superhappytree]

Funny how ever since they started the public betas their software has gotten worse, you’d think it would be the opposite. iOS 1-6 really felt like an improvement that got better with every version, whereas iOS 7-11 is an inconsistent mess. Here’s hoping iOS 12 bucks this trend and actually delivers. I’m currently running beta 5 but it still feels very iOS 11.5 to me.

 

[Superhai]

To be honest, I think that, unfortunately, the feedback from beta testers is not a priority for Apple, so it's not really a beta testing in the way it should be.

I report bugs occasionally and I do receive feedback from Apple, either in the sort of check if the issue is fixed in this new version of the beta, or sometimes they want more data.

In other words, those numbers are meaningless unless they correlate with the same amount of reports.

As a beta-tester you are also willingly providing analytics, logs and crash-reports, so it is still more valuable than just the ones reporting bugs.
[doublepost=1533088503][/doublepost]

Funny how ever since they started the public betas their software has gotten worse, you’d think it would be the opposite.

There is probably some true to this, however the reason is that they test certain part of the system in the various betas. They are not bringing out everything newly developed in the first beta and then bet on improvements. The improvements are staged, so that bug reports don't overlap other frameworks and thus make debugging much harder.

 

[nortonandreev]

I report bugs occasionally and I do receive feedback from Apple, either in the sort of check if the issue is fixed in this new version of the beta, or sometimes they want more data.



As a beta-tester you are also willingly providing analytics, logs and crash-reports, so it is still more valuable than just the ones reporting bugs.
[doublepost=1533088503][/doublepost]
There is probably some true to this, however the reason is that they test certain part of the system in the various betas. They are not bringing out everything newly developed in the first beta and then bet on improvements. The improvements are staged, so that bug reports don't overlap other frameworks and thus make debugging much harder.

I used to get responses the past years, but nothing now...
And I agree with “analytics, logs and crash reports” part. Hopefully the data collected is valuable and used.

 

[drumcat]

If only 1/2 of them would report their issues rather than complain on MacRumors about them. Apple doesn't fix them if they don't know about them and they don't browse the forums here to find them.

Please use the Feedback app. It only takes a minute but it's huge for getting problems fixed. DON'T JUST ASSUME SOMEONE ELSE WILL REPORT IT. Apple prioritizes what they fixed based on the volume of feedback. If you choose to not report a problem, that's one less report. Everyone assumes someone else reports a problem and it results in countless less reports so even bigger issues can appear smaller and not take priority.

Help the whole community and report every issue you find. Don't rely on others to do so.

That's sort of silly. First, you don't know that they prioritise based solely on volume. They don't, and no software team would. They usually deal with the severity first, predicated on the frequency of the error.

If they're looking at the volumes of anything, they're looking at the device-reported logs. You know, the extended logging and transmission you do when you sign up for betas? Usually, you're "sharing analytics".

 

[pika2000]

I'm willing to bet close to 80% don't report any bugs at all, and are simply in the program to get access to the latest software... that's the way it usually goes with all betas. In other words, those numbers are meaningless unless they correlate with the same amount of reports.

Yup. One can simply look at this forum. So many threads from those using betas but clearly have no clue what beta means, and all they do is complain instead of reporting their issues to Apple.

 

[zorinlynx]

I'm willing to bet close to 80% don't report any bugs at all, and are simply in the program to get access to the latest software... that's the way it usually goes with all betas. In other words, those numbers are meaningless unless they correlate with the same amount of reports.

Even people who don't report any bugs help out, because their phones are reporting diagnostic information to Apple. When the springboard or apps crash, or the kernel panics, or random things go wrong, iOS will send crash reports. Having millions of people using the betas will mean crash reports even for less common bugs, which Apple can use to fix them.

 

[B4U]

Only 4 million?
Oh, only those who signed up for the program and not including anyone else still beta testing iOS 11 for them...

 

[neutralguy]

Tim Cook is being very conservative here. He should have said 'all". In fact iOS should always have beta tag attached to it like Google apps!

 

[vcboard]

Quality over quantity. Looking at the critical bugs managed to sift through to production releases, it is not considered successful.

 

[ipedro]

The Public Beta is surprisingly stable given that it's still several months away from official release.

 

[Phil Phletcher]

I’ve been on the iOS 12 beta and so far the only issues I’ve been having is decreased battery life (kinda expected) and the podcast app freezing and crashing more than usual. Other than that everything has been smooth sailing. And yes, I did let Apple know about the bugs I found.

 

[KingArthurVI]

I think some of you are missing the point here. Beta testers don't have to submit bug reports or feedback to actually be useful to Apple. Apple has a disclaimer right after you install beta software that says something along the lines of "you consent to automatically submit crash reports and diagnosis, and we may anonymously collect usage data". That alone is huge. Apple can look at how many users actually use the screen time feature for example vs other features and prioritize accordingly. If an Apple app crashes they'll immediately get sent stack traces and can fix it on their end.

 

[ssmed]

There is a big difference between serious beta testers and people who just want to try out the latest SW!

 

[PickUrPoison]

The Public Beta is surprisingly stable given that it's still several months away from official release.

Agreed! And if iOS 11 were the train wreck some still like to claim it is, iOS 12 wouldn’t/couldn’t be as good as it is.

 

[meaning-matters]

Final bug I reported was Weather app showing rain in coming days as light-blue stripes, which on a sunny day are shown on a light-blue background (of the current day), making them virtually invisible.

Apply reply I got: That's intended and bug is closed. I gave up.

We have the same thing with Maps app: Roads are shown in light yellow, so are buildings, making the map virtually useless, especially when used outside during the day. Are they out of their mind?

 

[GadgetBen]

This is why the Betas have to be good and they generally are with Apple.

It's also why you developers need to change your attitude.

Hundreds of thousands are about to ask you again and again: "Is it stable?"

 

[aylk]

Makes sense, iOS 12 makes my 6S feel like a normal phone again.

 

[leon44]

Am I not right to say that even if you don’t report bugs through the feedback app, the usage statistics Apple automatically takes are already useful?

 

[Morgenland]

Pretty smart. Saving them a lot of money.

people have to actually report the bugs instead of just playing with the latest and greatest

Maybe a point system would help to increase feedback?!
10 **qualified** bug feedbacks = 1 new watch strap,
20 feedbacks = 1 new iPhone cover.
win-win idea.....

Then maybe people would waste less time complaining here...

 

[Feenician]

Am I not right to say that even if you don’t report bugs through the feedback app, the usage statistics Apple automatically takes are already useful?

They’re aggregating crash reports for sure. That helps them understand things like memory leak crashes and so on that actually show up in crash reports. It doesn’t help them find or fix, “I couldn’t tap this button”, “the animation when I do X is janky or doesn’t work”, “my Bluetooth didn’t connect to my thingamajig” or a myriad of bugs that don’t show up in crash reports.

 

[honglong1976]

and yet iOS11 was still garbage!

 

[bLackjackj]

Pretty smart. Saving them a lot of money.

It means absolutely nothing. I'm 'participating',..yet I've never done an install! The figure is bogus.

 

[NightFox]

I always thought that the stated purpose of the Developers’ Beta Program was to allow devs to test their test own apps for compatibility with a pre-release (i.e. beta) version of the next iOS, not to actually test the beta itself. Is that no longer it’s official function?

Of course then the public beta program came along, but as others have said, I imagine there’s only a very small amount of useful, original feedback (that hasn’t already come from the Devs’ Program) from that pool.

 

[McScooby]

Wonder how many are *active* users?

 

[44267547]

To be honest, I think that, unfortunately, the feedback from beta testers is not a priority for Apple, so it's not really a beta testing in the way it should be.

How would you come to that conclusion? More specifically, do you have personal experience that you know that Apple doesn’t ‘care about feedback’ from the beta testers? Because the way I see it, unless you have evidence proving that they don’t care, what makes you think that they don’t look into issues that are reported that might take time to investigate?