Register FAQ / Rules Forum Spy Search Today's Posts Mark Forums Read
Go Back   MacRumors Forums > News and Article Discussion > MacRumors.com News Discussion

Reply
 
Thread Tools Search this Thread Display Modes
Old Nov 26, 2012, 05:41 PM   #401
AidenShaw
macrumors G5
 
AidenShaw's Avatar
 
Join Date: Feb 2003
Location: The Peninsula
Quote:
Originally Posted by dv8r View Post
I think they are pretty straightforward - manually verification of the searches provided to the Google API did show more correct results.
I objected to the "naked" links - you could have provided something like:

Quote:
I think this supports my argument:
- linkx - shows results from survey x
- linky - opinion piece by noted commentator
- linkz - comparison of several similar products
instead of

Quote:
- linkx
- linky
- linkz
I'm not suggesting a master's thesis, just a little something to explain why I'd want to click on the links.

Because, without it, I move on and your post is wasted. With some explanation, I might read and reply to support you.
__________________
Edward Snowden - American Hero.
Daniel Ellsberg: "Edward Snowden: Saving Us from the United Stasi of America"
AidenShaw is offline   0 Reply With Quote
Old Nov 27, 2012, 01:31 PM   #402
tbrinkma
macrumors 65816
 
Join Date: Apr 2006
Quote:
Originally Posted by dv8r View Post
Marco's Excel sheet with the results is easily checked towards Google directly, and they do not give the same result (e.g. the China example in the linked posts) - they do correspond more with the results coming out of the new CLGeocoder. Therefore the question is relevant - did he do this test towards the wrong API?

Feel free to believe something else - but that doesn't make it any more true.
That's a valid question. Find some evidence to support that case and you'll have a valid *argument*. Until then, you have supposition.

Feel free to believe something else - but that doesn't make it any more true.

Seriously, though. If you can find evidence that what you're saying is correct, let me know. I'm supporting Apple Maps for three reasons at this point.
1) So far, it's worked flawlessly for me, so it must be ok in my area at least. (I don't expect that to change, so I'll likely to continue happily using it regardless.)

2) So far, the *only* comparative analysis I've seen of the old app and the new app show that there isn't a significant difference in the accuracy of the two. (Right now there's a lot of hyperbole and a bunch of non-specific anecdotes, but not much in the way of actual well-formulated comparisons.)

3) If your 'wrong API call' hypothesis is correct, it doesn't show that the Google results are better than the Apple results. It just shows that the results of the experiment were *completely* invalid from top to bottom, because the same API call was made for both versions. At that point, we're back to where we were *before* that experiment was done. That is, *no* even remotely well-formulated comparisons between the two back-ends. (Although, the test could be rerun easily enough using the *correct* API call in that case.)
__________________
17" MBP (unibody), 2.66GHz i7, 8GB RAM, 750 GB HDD; iPhone 4s 64GB/Black
tbrinkma is offline   0 Reply With Quote
Old Nov 27, 2012, 06:11 PM   #403
xofruitcake
macrumors 6502a
 
Join Date: Mar 2012
Quote:
Originally Posted by tbrinkma View Post
That's a valid question. Find some evidence to support that case and you'll have a valid *argument*. Until then, you have supposition.

Feel free to believe something else - but that doesn't make it any more true.

Seriously, though. If you can find evidence that what you're saying is correct, let me know. I'm supporting Apple Maps for three reasons at this point.
1) So far, it's worked flawlessly for me, so it must be ok in my area at least. (I don't expect that to change, so I'll likely to continue happily using it regardless.)

2) So far, the *only* comparative analysis I've seen of the old app and the new app show that there isn't a significant difference in the accuracy of the two. (Right now there's a lot of hyperbole and a bunch of non-specific anecdotes, but not much in the way of actual well-formulated comparisons.)

3) If your 'wrong API call' hypothesis is correct, it doesn't show that the Google results are better than the Apple results. It just shows that the results of the experiment were *completely* invalid from top to bottom, because the same API call was made for both versions. At that point, we're back to where we were *before* that experiment was done. That is, *no* even remotely well-formulated comparisons between the two back-ends. (Although, the test could be rerun easily enough using the *correct* API call in that case.)
The problem on your assertion is that everyone who use the IOS map are tester and they come to an independent assessment of the product. If 30% of the user population find that they have a problem with the product, the product failed regardless of whether the other 70% of population find it satisfactory or not. On the one hand, you used your own assessment as one of the reason you think the Apple map is o.k. And then you dismiss other's experience by going off to use "expert" analysis opinion. If Apple map does not have problem, why would Tim cook issue the apology letter. It is not funny when the biggest hi tech company in the land has to issue an apology letter. He clearly see that there is a big problem.

I would say move on, IOS map is a problem for Apple. Lucky for Apple that it is not a major problem that affect the sales of any IOS device. Apple will fix the problem over time now that they put the entire corporate resource on it. But it beyond comprehensive to argue that IOS map has no problem.
xofruitcake is offline   0 Reply With Quote
Old Nov 28, 2012, 08:54 AM   #404
tbrinkma
macrumors 65816
 
Join Date: Apr 2006
Quote:
Originally Posted by xofruitcake View Post
The problem on your assertion is that everyone who use the IOS map are tester and they come to an independent assessment of the product. If 30% of the user population find that they have a problem with the product, the product failed regardless of whether the other 70% of population find it satisfactory or not. On the one hand, you used your own assessment as one of the reason you think the Apple map is o.k. And then you dismiss other's experience by going off to use "expert" analysis opinion. If Apple map does not have problem, why would Tim cook issue the apology letter. It is not funny when the biggest hi tech company in the land has to issue an apology letter. He clearly see that there is a big problem.

I would say move on, IOS map is a problem for Apple. Lucky for Apple that it is not a major problem that affect the sales of any IOS device. Apple will fix the problem over time now that they put the entire corporate resource on it. But it beyond comprehensive to argue that IOS map has no problem.
I certainly haven't dismissed anyone's experiences. I've dismissed the claims that the new Maps back end is significantly worse than the Google back end because, despite the hyperbole, the only attempt to actually *compare* the respective accuracy of the two systems showed that at worst, the two systems were on par.

And you should probably avoid pulling numbers out of your *** when you're discussing the accuracy with someone. If you're actually claiming that 30% of people are having significant issues with the new Maps back end, you'll want to provide some evidence to support that, or it'll get filed away with the same priority as any other unsupported claim on an internet forum. The prevalence of a complaint in an online forum rarely correlates to the prevalence of similar complaints in the general population. (The only time I've seen it come within the same order of magnitude was the infamous RROD.)
__________________
17" MBP (unibody), 2.66GHz i7, 8GB RAM, 750 GB HDD; iPhone 4s 64GB/Black
tbrinkma is offline   0 Reply With Quote
Old Nov 28, 2012, 09:16 AM   #405
samcraig
macrumors G5
 
Join Date: Jun 2009
Quote:
Originally Posted by tbrinkma View Post
despite the hyperbole, the only attempt to actually *compare* the respective accuracy of the two systems showed that at worst, the two systems were on par.
I might argue that point. Especially when it comes to international geography, etc.
samcraig is offline   0 Reply With Quote
Old Nov 28, 2012, 10:18 AM   #406
xofruitcake
macrumors 6502a
 
Join Date: Mar 2012
Quote:
Originally Posted by tbrinkma View Post
I certainly haven't dismissed anyone's experiences. I've dismissed the claims that the new Maps back end is significantly worse than the Google back end because, despite the hyperbole, the only attempt to actually *compare* the respective accuracy of the two systems showed that at worst, the two systems were on par.

And you should probably avoid pulling numbers out of your *** when you're discussing the accuracy with someone. If you're actually claiming that 30% of people are having significant issues with the new Maps back end, you'll want to provide some evidence to support that, or it'll get filed away with the same priority as any other unsupported claim on an internet forum. The prevalence of a complaint in an online forum rarely correlates to the prevalence of similar complaints in the general population. (The only time I've seen it come within the same order of magnitude was the infamous RROD.)
Sure, if you don't like 30%, try 20%. If you don't like 20%, try 10%. The point is that Tim Cook, as the CEO of Apple, see enough problem that he need to issue an apology letter. When is the last time you see a CEO of a major tech company do that? And in the middle of several major product launch and 50B revenue quarter (per Apple management projection). We should all be talking about how Iphone 5 thinner and lighter design is industry leading. But instead we are talking about the map problem. Good thing that Apple has royal follower and there are enough alternative map solutions in IOS that map problem does not affect enough sales to matter.

I am a software engineer for 21+ years and I would be ashamed to ship a product that my CEO has to issue a letter to openly apologize for it. Are you o.k. if you are one of the developer in the team and your CEO has to apologize for your work?

I don't see any reason to debate the IOS map quality at this point. Enough users already voice their problem and the company already acknowledge the issue. What Apple will do is fixing the problems. In about a year, the IOS map will be in a good enough shape that no one will talk about the problem anymore.

And I speak of it not as an Android fan, but an Apple share holders. I own a Dell PC, a Samsung Infuse and an Ipad 3. So I am as neutral in term of my tech toy as it come. And I even agree with Apple's decision to kick Google map out of IOS 6. But how they roll the IOS map out and how they positioned the consumer during the map launch clearly showed that they were sandbagged by the software organization especially those who run the IOS 6 beta and Forstall. And Apple as a company is paying for it with their reputation if not the actual sales right now (as they should).
xofruitcake is offline   1 Reply With Quote
Old Nov 28, 2012, 11:33 AM   #407
tbrinkma
macrumors 65816
 
Join Date: Apr 2006
Quote:
Originally Posted by xofruitcake View Post
Sure, if you don't like 30%, try 20%. If you don't like 20%, try 10%.
So your solution for not being able to support a number you posted is to propose two other numbers without any supporting evidence for either of the new values? I'm interested in actual comparisons of Apple's and Google's map back ends. Not hyperbole. Provide some actual, verifiable, reproducible results that don't involve cherry picking and we might get somewhere.

----------

Quote:
Originally Posted by samcraig View Post
I might argue that point. Especially when it comes to international geography, etc.
The comparison I've referenced was done using locations in Canada.

It's also the only comparison I've found which used a significant set of data, rather than cherry picking one way or another by looking for errors in one system and only pointing out the ones where the other system got it right. There's plenty of *that* particular nonsense both ways, and it's not helpful for comparing the accuracy of two systems.
__________________
17" MBP (unibody), 2.66GHz i7, 8GB RAM, 750 GB HDD; iPhone 4s 64GB/Black
tbrinkma is offline   0 Reply With Quote
Old Nov 28, 2012, 12:19 PM   #408
xofruitcake
macrumors 6502a
 
Join Date: Mar 2012
Quote:
Originally Posted by tbrinkma View Post
So your solution for not being able to support a number you posted is to propose two other numbers without any supporting evidence for either of the new values? I'm interested in actual comparisons of Apple's and Google's map back ends. Not hyperbole. Provide some actual, verifiable, reproducible results that don't involve cherry picking and we might get somewhere.[COLOR="#808080"]
heh heh, you must not have your coffee and try to be very slow. My point is that the percentage of user who has a problem for IOS map is not important. The fact that Tim Cook issue the apology letter already validate that there is a problem. The fact that they fired Forstall and Williamson and hiring a lot more programmers to fix the map issue also indicated that it is a major problem. Which part of that you don't understand? Do company normally fire someone if they are doing a good job? or do company normally hire more programmer to sit around if there is not something to work on? If you insist that Apple don't have an IOS map problem, I think I have a bridge to sell you 8-))

The real interesting discussion is not whether IOS map has problem, it is how long it will take Apple to fix it (and they will).
xofruitcake is offline   1 Reply With Quote
Old Nov 28, 2012, 02:30 PM   #409
tbrinkma
macrumors 65816
 
Join Date: Apr 2006
Quote:
Originally Posted by xofruitcake View Post
heh heh, you must not have your coffee and try to be very slow. My point is that the percentage of user who has a problem for IOS map is not important. The fact that Tim Cook issue the apology letter already validate that there is a problem. The fact that they fired Forstall and Williamson and hiring a lot more programmers to fix the map issue also indicated that it is a major problem. Which part of that you don't understand? Do company normally fire someone if they are doing a good job? or do company normally hire more programmer to sit around if there is not something to work on? If you insist that Apple don't have an IOS map problem, I think I have a bridge to sell you 8-))

The real interesting discussion is not whether IOS map has problem, it is how long it will take Apple to fix it (and they will).
I think we may be arguing slightly different points...
  1. Both mapping systems have problems.
  2. Neither is perfect.
  3. That is not in dispute by any rational person.
  4. Unfortunately, there tends to be a shortage of rational people on internet forums.

My own interest is the comparative accuracy of the two systems.
Most of the 'comparisons' to be found are nothing more than someone finding a single example that supports their view point and extrapolating broadly from there. This sort of cherry-picking can be done all day by either 'side', and does nothing to further the discussion or even spread useful information. (Unless you happen to be impacted by one of the cherry-picked examples.)

So far, there has been *ONE* actual attempt to study the comparative accuracy of the two systems. It showed that the two systems had similar levels of accuracy. Both returned roughly the same number of correct matches. In the event of a missed match, Google was more likely to return a wrong result, and Apple was more likely to not return a result at all.

Most of the arguments over the two mapping systems seem to revolve around which of those latter two results is 'better' than the other. To my mind, neither is 'better', because they're both *wrong*.

As an example of this was the #iLost ad fiasco. Searching for "315 E 15th St ny" gave you the corner of a park in Manhattan for Google, and an address in Brooklyn for Apple. The address doesn't actually exist in Manhattan, but E 15th St in Brooklyn more commonly called 'Marlborough Rd'. (In fact, a selection of the numbered streets in that area seem to have been given names, but not all of them have.) Arguing about which was correct inevitably turned into arguing over which version of "the address doesn't exist" was 'more correct'.

Note: As a quick test, I repeated the #iLost lookup in Google Maps and Apple Maps, and discovered that Google Maps still points to the non-existent address at the corner of the park. Apple Maps now asks which of the two locations you actually meant. *THAT*, I'd support as the correct action in this instance, but the publicity surrounding that search and the original results pretty much moots it as an example of one system being better than the other. (Another example of why cherry-picking isn't helpful for comparing accuracy of two systems.)
__________________
17" MBP (unibody), 2.66GHz i7, 8GB RAM, 750 GB HDD; iPhone 4s 64GB/Black
tbrinkma is offline   0 Reply With Quote
Old Nov 28, 2012, 02:57 PM   #410
xA4Hx
macrumors member
 
Join Date: Feb 2007
For iPhones I just use navigon for android I Jude use google. When they fix maps ill use it and sometimes I do. We all know apple will deny it.

Last edited by xA4Hx; Nov 28, 2012 at 03:03 PM.
xA4Hx is offline   0 Reply With Quote
Old Nov 28, 2012, 03:21 PM   #411
macsmurf
macrumors 65816
 
macsmurf's Avatar
 
Join Date: Aug 2007
Quote:
Originally Posted by tbrinkma View Post
That's a valid question. Find some evidence to support that case and you'll have a valid *argument*. Until then, you have supposition.

Feel free to believe something else - but that doesn't make it any more true.

Seriously, though. If you can find evidence that what you're saying is correct, let me know.
He already did. We know that the part of the API he used was reverse geocoding via the CLGeocoder class:

"Determined that Apple Maps uses the CLGeocoder Class by peeking at the iPhone's debug console in Xcode while doing live searches in Apple Maps"
Source: http://www.mtonic.com/applemaps/

We also know that the CLGeocode class results is hopelessly inaccurate in iOS 5 (http://stackoverflow.com/questions/1...sly-inaccurate) compared to the Google maps API reverse geocoding. Note that this post was from before iOS6 came out.

A user on Stackoverflow states that CLGeocode hasn't been using Google as a data source for about a year: http://stackoverflow.com/questions/1...e-geocoder-api

That seems to be the only reasonable explanation for the bad results.

The analysis from Tabini documents that the CLGeocode class returns inaccurate results in iOS 6 as well.

However, when you do a normal search in Maps you search for a town and then get the results. You normally don't input coordinates and then get a list of towns nearby. So in reality these tests shows us nothing about the quality of iOS5 maps vs. iOS6 maps. Reverse geocoding simply isn't a good test for comparing the two applications.

Quote:
I'm supporting Apple Maps for three reasons at this point.
1) So far, it's worked flawlessly for me, so it must be ok in my area at least. (I don't expect that to change, so I'll likely to continue happily using it regardless.)
That's great but it's not much of an argument.

Quote:
2) So far, the *only* comparative analysis I've seen of the old app and the new app show that there isn't a significant difference in the accuracy of the two. (Right now there's a lot of hyperbole and a bunch of non-specific anecdotes, but not much in the way of actual well-formulated comparisons.)
The analysis is flawed on a basic level as you can see above.

Quote:
3) If your 'wrong API call' hypothesis is correct, it doesn't show that the Google results are better than the Apple results. It just shows that the results of the experiment were *completely* invalid from top to bottom, because the same API call was made for both versions. At that point, we're back to where we were *before* that experiment was done. That is, *no* even remotely well-formulated comparisons between the two back-ends. (Although, the test could be rerun easily enough using the *correct* API call in that case.)
I agree completely. The guy from the second Stackoverflow entry do claim that

"I found out of a SVGeocoder Wrapper for google geocoder API. I am testing it now, It indeed brings much for detailed results."

That is not particularly rigorous, though.

What we can conclude from the analysis is this:
1) Reverse geocoding is just as bad in iOS5 compared to iOS6
2) Nobody (or very few) actually use their maps application that way.

What we can conclude from the Stackoverflow posts is this:
1) Reverse geocoding works better using the Google Maps API directly
2) I follows that Apple didn't use the Google Maps API directly for reverse geocoding when these tests were made. Neither in iOS5 nor in iOS6.

Last edited by macsmurf; Nov 28, 2012 at 03:28 PM.
macsmurf is offline   0 Reply With Quote
Old Nov 29, 2012, 04:27 AM   #412
MikeyMike01
macrumors regular
 
Join Date: Apr 2010
Quote:
Originally Posted by xofruitcake View Post
The point is that Tim Cook, as the CEO of Apple, see enough problem that he need to issue an apology letter. When is the last time you see a CEO of a major tech company do that?
Yesterday.

http://www.theverge.com/2012/11/29/3...box-experience
__________________
iPad Mini Retina (32GB, White)
Macbook Air 13" (2013)
Apple TV (3rd Gen)
MikeyMike01 is offline   0 Reply With Quote
Old Nov 29, 2012, 04:32 AM   #413
Solomani
macrumors 65816
 
Solomani's Avatar
 
Join Date: Sep 2012
Quote:
Originally Posted by MikeyMike01 View Post
Well.... in retrospect.... the Japanese have a culture of responsibility and accountability. When a Sony or Nintendo or Nikon executive fails the expectations of his company, he is expected to apologize and then commit ceremonial suicide (sepukku).
Solomani is offline   0 Reply With Quote
Old Nov 29, 2012, 10:14 AM   #414
funkybudda
macrumors newbie
 
Join Date: Jun 2012
Quote:
Originally Posted by tbrinkma View Post
I think we may be arguing slightly different points...
  1. Both mapping systems have problems.
  2. Neither is perfect.
  3. That is not in dispute by any rational person.
  4. Unfortunately, there tends to be a shortage of rational people on internet forums.

My own interest is the comparative accuracy of the two systems.
Most of the 'comparisons' to be found are nothing more than someone finding a single example that supports their view point and extrapolating broadly from there. This sort of cherry-picking can be done all day by either 'side', and does nothing to further the discussion or even spread useful information. (Unless you happen to be impacted by one of the cherry-picked examples.)

So far, there has been *ONE* actual attempt to study the comparative accuracy of the two systems. It showed that the two systems had similar levels of accuracy. Both returned roughly the same number of correct matches. In the event of a missed match, Google was more likely to return a wrong result, and Apple was more likely to not return a result at all.

Most of the arguments over the two mapping systems seem to revolve around which of those latter two results is 'better' than the other. To my mind, neither is 'better', because they're both *wrong*.


As an example of this was the #iLost ad fiasco. Searching for "315 E 15th St ny" gave you the corner of a park in Manhattan for Google, and an address in Brooklyn for Apple. The address doesn't actually exist in Manhattan, but E 15th St in Brooklyn more commonly called 'Marlborough Rd'. (In fact, a selection of the numbered streets in that area seem to have been given names, but not all of them have.) Arguing about which was correct inevitably turned into arguing over which version of "the address doesn't exist" was 'more correct'.

Note: As a quick test, I repeated the #iLost lookup in Google Maps and Apple Maps, and discovered that Google Maps still points to the non-existent address at the corner of the park. Apple Maps now asks which of the two locations you actually meant. *THAT*, I'd support as the correct action in this instance, but the publicity surrounding that search and the original results pretty much moots it as an example of one system being better than the other. (Another example of why cherry-picking isn't helpful for comparing accuracy of two systems.)
is that the best a typical Apple fan boy can come up with? To argue that they are both "wrong" means one is not inferior than the other, but we all know Apple's map is worst than Google's.
funkybudda is offline   0 Reply With Quote
Old Nov 29, 2012, 11:06 AM   #415
MikeyMike01
macrumors regular
 
Join Date: Apr 2010
Quote:
Originally Posted by Solomani View Post
Well.... in retrospect.... the Japanese have a culture of responsibility and accountability. When a Sony or Nintendo or Nikon executive fails the expectations of his company, he is expected to apologize and then commit ceremonial suicide (sepukku).
I just found the timing humorous.
__________________
iPad Mini Retina (32GB, White)
Macbook Air 13" (2013)
Apple TV (3rd Gen)
MikeyMike01 is offline   0 Reply With Quote
Old Nov 29, 2012, 02:42 PM   #416
tbrinkma
macrumors 65816
 
Join Date: Apr 2006
Quote:
Originally Posted by macsmurf View Post
He already did. We know that the part of the API he used was reverse geocoding via the CLGeocoder class:
Good to know, and thank you for explaining how that link was relevant.

He may have found the evidence, but he hadn't presented it yet. Linking to something with no explanation isn't the same as presenting evidence. Had the link had any text which explained its relevance to the discussion at hand, that would have been fine. Unfortunately, it didn't, and I'm not a mind-reader.

Quote:
Originally Posted by macsmurf View Post
"Determined that Apple Maps uses the CLGeocoder Class by peeking at the iPhone's debug console in Xcode while doing live searches in Apple Maps"
Source: http://www.mtonic.com/applemaps/

We also know that the CLGeocode class results is hopelessly inaccurate in iOS 5 (http://stackoverflow.com/questions/1...sly-inaccurate) compared to the Google maps API reverse geocoding. Note that this post was from before iOS6 came out.

A user on Stackoverflow states that CLGeocode hasn't been using Google as a data source for about a year: http://stackoverflow.com/questions/1...e-geocoder-api
So, since iOS 5 Maps and iOS 6 are using this same API call, and apparently *neither* gets data from Google, what accounts for the difference in accuracy?

That's my point. The same API call may well get data from different places, but if neither one is getting the data from Google, the differences still need to be explained somehow.

Quote:
Originally Posted by macsmurf View Post
That seems to be the only reasonable explanation for the bad results.

The analysis from Tabini documents that the CLGeocode class returns inaccurate results in iOS 6 as well.

However, when you do a normal search in Maps you search for a town and then get the results. You normally don't input coordinates and then get a list of towns nearby. So in reality these tests shows us nothing about the quality of iOS5 maps vs. iOS6 maps. Reverse geocoding simply isn't a good test for comparing the two applications.
Or, maybe we're still talking about two different aspects of the results of that study.




Quote:
Originally Posted by macsmurf View Post
The analysis is flawed on a basic level as you can see above.
I don't necessarily think the CLGeocoder results explain away all of the test results, since the iOS 5 APIs (which did still use Google for their data) apparently often returned locations *outside* Canada (sometimes on different continents) for the locations of landmarks *inside* Canada.

As I understand it, the reverse geocoding was used to help determine whether the results from *either* iOS version results were 'close' to the actual location. If I'm wrong, then it being 'known bad' in both versions doesn't explain the differences at all.

If that data was 'bad' in both versions of iOS, then it calls into question the lookups which were judged to be 'close enough' on both versions, but it doesn't change the results for the significant outliers (locations outside Canada, or no result found, respectively).

Quote:
Originally Posted by macsmurf View Post
I agree completely. The guy from the second Stackoverflow entry do claim that

"I found out of a SVGeocoder Wrapper for google geocoder API. I am testing it now, It indeed brings much for detailed results."

That is not particularly rigorous, though.

What we can conclude from the analysis is this:
1) Reverse geocoding is just as bad in iOS5 compared to iOS6
2) Nobody (or very few) actually use their maps application that way.

What we can conclude from the Stackoverflow posts is this:
1) Reverse geocoding works better using the Google Maps API directly
2) I follows that Apple didn't use the Google Maps API directly for reverse geocoding when these tests were made. Neither in iOS5 nor in iOS6.
Even if, for the sake of argument, we accept your position (stated earlier) that the use of a 'bad' reverse geocoding API for *both* versions of the iOS Maps app completely invalidates the results of the study, then we're back to the scenario where the *only* evidence of good/bad for either is cherry-picked anecdotes both ways.

This isn't the same as evidence that one is better than the other. It puts the situation back to the state of no concrete data to refute an argument presented with a lack of concrete data.

What's needed to make a valid comparison is a test that simply compares the results of a Google Maps lookup to the results of an Apple Maps lookup, *and* to a list of known-good coordinates, all mapped to identical search terms. I think we can both agree on that. The trick is to generate a computer-readable list of known-good coordinates along with reasonable search terms for those locations.

A base-line statistical comparison could probably be done with a few thousand randomly chosen locations world-wide, but to make it useful to people, there'd need to be at least a few hundred or thousand in any given country. The trick is to come up with the list of locations *without* referencing either system, because if we used one to put together the master list then we'd be comparing the results of one to itself.

----------

Quote:
Originally Posted by funkybudda View Post
is that the best a typical Apple fan boy can come up with? To argue that they are both "wrong" means one is not inferior than the other, but we all know Apple's map is worst than Google's.
I'm saying that in the scenario where *both* systems pull back an incorrect result, arguing which incorrect result is better is absolutely pointless.

If that makes me a 'fan boy', then so be it.

On the other hand, you seem to be arguing that one system is worse than the other because "we all know" it is. You're arguing from a position of authority, even though you don't actually *have* a position of authority, but you're using the circular 'proof' of your opinion as evidence that your opinion is correct. Forgive me for no taking your argument seriously.
__________________
17" MBP (unibody), 2.66GHz i7, 8GB RAM, 750 GB HDD; iPhone 4s 64GB/Black
tbrinkma is offline   0 Reply With Quote
Old Nov 29, 2012, 05:06 PM   #417
macsmurf
macrumors 65816
 
macsmurf's Avatar
 
Join Date: Aug 2007
Quote:
Originally Posted by tbrinkma View Post
So, since iOS 5 Maps and iOS 6 are using this same API call, and apparently *neither* gets data from Google, what accounts for the difference in accuracy?

That's my point. The same API call may well get data from different places, but if neither one is getting the data from Google, the differences still need to be explained somehow.
There is no way to know that. I would expect that whatever data Apple had would have changed over time (and probably still does). In order to know what is really going on you'd have to ask Apple (and they aren't telling).

Quote:
I don't necessarily think the CLGeocoder results explain away all of the test results, since the iOS 5 APIs (which did still use Google for their data) apparently often returned locations *outside* Canada (sometimes on different continents) for the locations of landmarks *inside* Canada.

As I understand it, the reverse geocoding was used to help determine whether the results from *either* iOS version results were 'close' to the actual location. If I'm wrong, then it being 'known bad' in both versions doesn't explain the differences at all.

If that data was 'bad' in both versions of iOS, then it calls into question the lookups which were judged to be 'close enough' on both versions, but it doesn't change the results for the significant outliers (locations outside Canada, or no result found, respectively).
The process isn't explained very well.

I'm fairly certain about the input being only cities without coordinates from an outside source. He specifically mentions that he screen scraped the city names and the source web page does not include coordinates.

Quote:
Even if, for the sake of argument, we accept your position (stated earlier) that the use of a 'bad' reverse geocoding API for *both* versions of the iOS Maps app completely invalidates the results of the study, then we're back to the scenario where the *only* evidence of good/bad for either is cherry-picked anecdotes both ways.

This isn't the same as evidence that one is better than the other. It puts the situation back to the state of no concrete data to refute an argument presented with a lack of concrete data.
Agreed.

Quote:
What's needed to make a valid comparison is a test that simply compares the results of a Google Maps lookup to the results of an Apple Maps lookup, *and* to a list of known-good coordinates, all mapped to identical search terms. I think we can both agree on that. The trick is to generate a computer-readable list of known-good coordinates along with reasonable search terms for those locations.

A base-line statistical comparison could probably be done with a few thousand randomly chosen locations world-wide, but to make it useful to people, there'd need to be at least a few hundred or thousand in any given country. The trick is to come up with the list of locations *without* referencing either system, because if we used one to put together the master list then we'd be comparing the results of one to itself.
I actually have such a list. Just for fun I pulled some data from Ordnance Survey, the national mapping agency of Great Britain. There is about 800k (!) data points and all of it comes from their own data. Manual checking all the data isn't fun but the few points I've tried are good (I put the coordinate into Google Maps and visually confirmed that they were close to the road name). All in all I think it is reasonable to assume that the data is very accurate; these guys take their maps seriously.

From this data it would be a simple matter to construct a descriptive search query string, run it through iOS5/6 (and the Google geocoder API for that matter), and check the results without using reverse geocoding.

I don't really have the time for it and I would have to figure out how to talk to the iOS APIs (I think I made a "hello, world!" program once in XCode but it would take me a couple of long days to learn ObjC and how to call the API to a level where'd I'd be comfortable with the result). I might do that at some point in the future but if anyone wants the list, feel free to contact me. Ordnance Survey licensed the data under BSD so anyone can legally do pretty much whatever they want with it.

Even if someone did do that test it would only tell us how good the map services are at pinpointing a location. It doesn't tell us whether the route planner will actually lead us to that location. I have a case where iMaps was able to pinpoint To and From correctly but the suggested route led to point X which happened to be a field road ten miles from the destination!

I have personally chosen an Android phone with Google Maps. The main reason is transit information. That is not available in iMaps. In Google Maps it just works (for my country). Accuracy and route planning could be perfect in iMaps but it would still be severely lacking compared to Google. Street view is another feature I use a lot.

On the other hand, I own quite a few Apple devices and consider them superior in other respects. I guess you can't have it all.

Last edited by macsmurf; Nov 29, 2012 at 05:27 PM.
macsmurf is offline   0 Reply With Quote
Old Nov 30, 2012, 08:47 AM   #418
tbrinkma
macrumors 65816
 
Join Date: Apr 2006
Quote:
Originally Posted by macsmurf View Post
There is no way to know that. I would expect that whatever data Apple had would have changed over time (and probably still does). In order to know what is really going on you'd have to ask Apple (and they aren't telling).



The process isn't explained very well.

I'm fairly certain about the input being only cities without coordinates from an outside source. He specifically mentions that he screen scraped the city names and the source web page does not include coordinates.



Agreed.



I actually have such a list. Just for fun I pulled some data from Ordnance Survey, the national mapping agency of Great Britain. There is about 800k (!) data points and all of it comes from their own data. Manual checking all the data isn't fun but the few points I've tried are good (I put the coordinate into Google Maps and visually confirmed that they were close to the road name). All in all I think it is reasonable to assume that the data is very accurate; these guys take their maps seriously.

From this data it would be a simple matter to construct a descriptive search query string, run it through iOS5/6 (and the Google geocoder API for that matter), and check the results without using reverse geocoding.

I don't really have the time for it and I would have to figure out how to talk to the iOS APIs (I think I made a "hello, world!" program once in XCode but it would take me a couple of long days to learn ObjC and how to call the API to a level where'd I'd be comfortable with the result). I might do that at some point in the future but if anyone wants the list, feel free to contact me. Ordnance Survey licensed the data under BSD so anyone can legally do pretty much whatever they want with it.

Even if someone did do that test it would only tell us how good the map services are at pinpointing a location. It doesn't tell us whether the route planner will actually lead us to that location. I have a case where iMaps was able to pinpoint To and From correctly but the suggested route led to point X which happened to be a field road ten miles from the destination!

I have personally chosen an Android phone with Google Maps. The main reason is transit information. That is not available in iMaps. In Google Maps it just works (for my country). Accuracy and route planning could be perfect in iMaps but it would still be severely lacking compared to Google. Street view is another feature I use a lot.

On the other hand, I own quite a few Apple devices and consider them superior in other respects. I guess you can't have it all.
Gotta say, it's nice having an intelligent, rational discussion on these forums. I wish it happened more often.

I'm in the same situation as you with regard to XCode. I keep meaning to sit down and build something, but I have yet to find the time needed to do it. (It's just different enough from Visual Studio, which I'm used to, that I'm having issues making the transition for UI layout and hookup.) It's a real shame, because I've got a couple apps I'd really like to write.

Maybe someone with the time/experience will see the mention of the data in your post and throw something together. (Probably wishful thinking. )

I still remember the 'good old days' when Map Quest was the top on-line trip routing source. Routing has been a mixed bag all along. I'm lucky enough that most of the places I want to go are in or around a major city, so I haven't had any issues since before the iOS 6 change-over. But I haven't had any since, either. I'm not entirely sure how you might do an automated routing comparison between the two systems. (I don't know how much of the information is exposed, vs. how much is neatly wrapped up in the map display controls where you can't access it.) Screen shot comparisons would be a possibility, maybe.
__________________
17" MBP (unibody), 2.66GHz i7, 8GB RAM, 750 GB HDD; iPhone 4s 64GB/Black
tbrinkma is offline   0 Reply With Quote

Reply
MacRumors Forums > News and Article Discussion > MacRumors.com News Discussion

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -5. The time now is 08:32 AM.

Mac Rumors | Mac | iPhone | iPhone Game Reviews | iPhone Apps

Mobile Version | Fixed | Fluid | Fluid HD
Copyright 2002-2013, MacRumors.com, LLC