As someone who has implemented a workout app, you are probably well placed to talk about the details. Are you able to sample data at the same 1 second rate that Apple is writing out in their workouts app? Do you have a theory on why a Garmin watch (and an old one at that) is better able to track a circuit than the AW2/3 hardware? How much do you think it is a hardware issue with raw GPS data accuracy versus Garmin having had many years to hone their GPS data massaging? My old Garmin watch appeared capable of producing tracks that were more accurate than the supposed accuracy of GPS technology would allow, so I assume they did some clever work on guessing location despite relatively noisy data.
Got a tip for us?
Let us know
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
Series 3 GPS Accuracy
- Thread starter wittyphrase
- Start date
- Sort by reaction score
As someone who has implemented a workout app, you are probably well placed to talk about the details. Are you able to sample data at the same 1 second rate that Apple is writing out in their workouts app? Do you have a theory on why a Garmin watch (and an old one at that) is better able to track a circuit than the AW2/3 hardware? How much do you think it is a hardware issue with raw GPS data accuracy versus Garmin having had many years to hone their GPS data massaging? My old Garmin watch appeared capable of producing tracks that were more accurate than the supposed accuracy of GPS technology would allow, so I assume they did some clever work on guessing location despite relatively noisy data.
I store points as I receive position updates from Apple's GPS. These tend to arrive about once a second so I store them at the same frequency. I don't do any sampling to reduce file sizes.
You have clearly done a lot of analysis of workout apps with the Apple Watch. Have you had your phone with you when you did these tests? The iPhone GPS is definitely better than the watch GPS. On my app I see lower signal strength and a grey breadcrumb trail much more often without the iPhone present.
So if you find that the results are better with the iPhone then it is probably the AW hardware that is the issue. If they are still poor with the phone present then I would tend towards thinking that it is the smoothing algorithm that Apple use. This is all just guesswork though - it is difficult to say if it is the hardware or the software that is inferior to your Garmin.
Sounds like apple workouts is using the same API for data retrieval as you are by the looks of the data I've seen from it. It would be nice if they supplied a 'I don't care that much about battery usage' mode that sampled from the GPS several times a second. This would give the software using the data a chance to do better outlier removal, and I wonder if that is something the Garmin does given the Garmin isn't trying to run a fairly powerful CPU, so can afford to be more generous on battery usage.
I've been deliberately testing without the phone so far, largely because I was hoping to use the AW3 as a replacement for my now broken Garmin 305, and don't plan to run with the phone. So far it looks like the AW3 will be ok for easy runs where getting accurate average pace doesn't matter that much, but for races and more time critical training runs, I'm not sure I'll be able to trust it enough based on the GPS maps I'm seeing at the moment. I really love the rest of the watch's functionality so I'd love to be able to have it as the only watch I use. It is possible the accelerometer combined with the GPS might provide distance that is accurate enough, I need to check if Apple is using a pure GPS length distance for its end of run distance, or something that gets input from the other watch sensors. I noticed today that Runtastic provided a different (and more accurate) distance that what a pure point to point measure of its GPX output had, so perhaps they are using something other than the GPS points.
Thanks, I'll have a look at taking the phone along for a test next time. It sounds like the watch API gives you accuracy estimates on the signal, or are you extrapolating those from the signal strength values the device gives you? If the former, what are the smallest and largest accuracy values you've seen on the watch?
Apple do not use a simple GPS length distance. When I compare the distance using GPS points with that reported by Apple's workout API I usually see a slightly shorter distance from the API. This is to be expected because the inaccuracy of the GPS points mean that drawing a line from one to the next would result in a jagged route with an overlong distance. I don't know whether this is just because Apple are just smoothing the GPS points, or if they are also taking into account information from other sensors such as the accelerometer.
The location API gives a horizontal accuracy in metres, which I then translate to a 5 bar signal strength meter. The API uses multiple location inputs so you can get some very high values (> 500m) if it is using the nearest WiFi signal because it hasn't yet got a response from the GPS. The most accurate signals seem to be within 5m.
For my app I consider anything over 500m as a level of 0 (No GPS). Between 151m and 500m is a level of 1. Between 36m and 150m is level 2. Between 11m and 25m is level 3. Between 6m and 10m is level 4, and less than 6m is level 5. This is all pretty arbitrary but it seems to work for me.
When the strength is 4 or 5 then the breadcrumb trail is shown in blue. When it is 3 then the trail is grey. For lower strengths I do not show the trail and I display a warning message at the bottom of the screen.
Thanks, when your app provides distance is that the apple workout API distance, or the GPS distance?
I did some tests today, and it seems that my watch has trouble getting a good satellite lock in the location I'm testing in. I was mostly on strength 3, and very briefly on strength 5. It looks like the map tracks my path very well when on strength 5 (although I'm not 100% sure I recall the correct lap when the strength was showing briefly on 5), but unsurprisingly for an accuracy of 11 to 25m, my path can stray quite some way from the path I'm following. Would it be possible in a future version to encode the signal strength into the gpx file? That would be a great way to compare the GPX coordinates to a satellite image and look at how signal strength and path alignment correlate. I wonder if "mostly 3" is normal for most watch owners. I'll do a test later in the same location with the phone to see how it compares. Having an option to replace the signal bar with a literal metres accuracy output would also be handy for this type of testing, but having the GPX output accuracy estimates is probably more useful as it means I don't need to rely on memory of where the strength was at while I'm moving.
For this test (which I walked, so I'm not sure if that changes the way the watch optimises point reporting), your app was using irregular output intervals according to the GPX file, which implies the API you are using does do some 'smart recording' before the data gets to you, as the duration between points ranged mostly between 2 and 8 seconds, with the average probably 4. There were two outliers at 10 and 16 seconds, which were the 4th and last point respectively. The last one was probably because it took a bit for me to find the stop button (although I think I was faster than 16 seconds in finding it). The apple workouts app on the same track and pace again used regularly 1 second outputs, so it seems like they definitely using a different method to what your app is using (and all the other apps, as the apple app is the only one that is using the regular 1 second recording intervals, every other watch app I've tested is outputting at irregular time intervals, I wonder if apple exposes an API to give you more regular data? It doesn't look like it would necessarily help accuracy, as the regularly intervals are not making the apple watch any more accurate, although it should in theory help any app that wants to do its own GPS error correction/smoothing).
A minor bug report: I use km output option, but on the summary screen after the workout is over, you label the distance as miles (I'm pretty sure the number is correctly in km's, so it is just the label that is wrong).
Anyway, thanks for a great app, the signal bar was particularly helpful in testing. Runkeeper's bar is only 3 units in width, and for the test run today, it was always showing 3, I doubt the lock was any better, and it is probably just more generous than your app on when it decides to show the 'full strength'.
It uses the distance from Apple.
I will look into this. The data is available in the iPhone app, so it would be easy to do, but I just need to find out if there is a standard GPX tag name to use. At first look it seems there isn't, so I may just invent my own extension unless anyone knows of an existing one?
What I may also do is add a new "Accuracy" graph to the iPhone app (alongside, speed, elevation, heart rate and cadence). This would vary the route colour according to the accuracy so you can easily see when the GPS was struggling.
Sorry, I forgot to mention that I don't filter on time but I do filter on distance between points. I ignore any changes less than 5m because it didn't seem worth including them if the accuracy was never better than 5m. This means that you get 1s output intervals when travelling faster than about 11mph / 18kph, but not when travelling slower and definitely not when walking. Sorry, I should have mentioned that before.
Well spotted. That has been fixed in the latest version, which is due out in a week or two.
Glad you like the app!
The hdop field is designed to store the horizontal precision, which looks like it is the closest equivalent in the standard, but it does not take a direct distance range (looks like hdop values probably get translated into the distance ranges reported by the device on a quick read, but didn't really look that closely), might be easier to make up your own extension.
That would be great, I misunderstood your breadcrumb description earlier, and figured this was what you were doing already.
No problem, and that makes sense with the data intervals I was seeing.
I did some testing with the phone with me today, and the accuracy issues I'm having with the watch certainly appear to be related to signal strength issues specific to the watch. When the watch can use the phone GPS I get level 5 accuracy for 4 250m laps, and the resulting track while not quite as perfect as the Garmin results, is very close, and much better than the watch by itself. I tried again with the watch, and it can only get level 5 accuracy for very short periods , and is mostly sitting at 3. Unfortunately it seems that the requirements to have the watch as small as it is leading to some compromises in GPS signal quality. Perhaps the antenna size is an issue, or perhaps the chipset itself just isn't as robust as the one in the phone (assuming they are different - I can't get any data on who makes the GPS receiver on the watch, it seems to be part of the system-on-a-chip setup rather than a standalone part).
If all Apple watches have this signal quality issue, then it would explain why Apple haven't been keen to provide signal strength on their own app. Probably most people care more about total distance than GPS tracks having a perfect path match, and it seems after the watch has learnt stride length patterns, the motion sensors seem to do a pretty good job of providing enough data to give reasonably accurate distances for most users.
The situation is a bit disappointing, as part of the allure of the AW3 having LTE was to be able to run without the phone, and still be in call contact for emergencies, but it seems if I want running tracks with reasonable levels of accuracy, I'll need to bring the phone along. It is still nicer to look at the watch for data than try to twist your arm to look at a phone lodged in an armband, so there is still a point for running usage, and the other features of the watch really are very nice.
I'd be interested to hear what signal levels others get on their watches when not carrying their phones, just to make sure what I'm seeing isn't an issue with my unit (bad antenna for example). I'll probably do some testing in some other locations to see if the watch can more reliably hit a level 5 signal at other locations. What have you been seen on your own watch in standalone mode in terms of average signal levels?
Anyway, thanks again for providing a tool that at the moment looks to be the most convenient available for this type of testing, if you do end up adding more explicit output of the accuracy levels beyond just the 5 levels in a future version, I'd be more than happy to help test it.
Also remember that "signal strength" is one factor. Any GPS chip is listening for as many of the GPS satellites as are above the horizon. Each will have its own signal strength. The more satellites the GPS can hear, the better the location fix.
The Garmin 610 had a nice satellite info display showing which satellites it expected to hear (based on the ephemeris; i.e. the table of orbits of the many GPS satellites), which of them it actually could hear, and the signal strength for each.
So... point being there's a lot more going on than might be easily represented in a single "signal strength" metric.
I would guess the metres accuracy number apple reports to the CoreLocation api users is probably as much about satellite visibility as it is signal strength. The GPX format does have a field to store number of satellites, but from a quick read, I'm not sure if Apple provide that detail to their developers via their usual APIs. When I say output raw signal strength here, I mean whatever cfc is using to determine the metres accuracy the device's GPS is reporting, which may or may not have a direct correlation with Satellite signal strength. I know GPS gets an accuracy boost when going from 3 to 4 satellites, because the 4th can be used for error checking against the others, but I'm unclear how much accuracy benefit it gets from the 5th, 6th etc (other than having backups when one of the currently used satellites disappears from view due to terrain/buildings etc).
In a nutshell, GPS works by the device calculating the time-delay between the signals from the different satellites to then estimate the relative distances between the device and each satellite. It then uses the ephemeris to calculate the location of each satellite, and couples that with the distance-to-each-satellite info to figure out your location. The more satellites it hears, the more data points it has with to refine the accuracy of the location.
So yes, more satellites typically means a better fix, but there are nuances to why that is so. Also note that the relative elevation of each satellite above the horizon plays a factor. Fewer high-in-the-sky satellites might be more accurate at the time than more lower-in-the-sky satellites.
This is also why day to day comparisons in GPS accuracy can be tricky. Ever if all the meteorological and other location characteristics are the same, the satellite positions change. (24 satellites, orbits are ~2 hours, so a great big game of three-card-monty in a way...
)In a nutshell, GPS works by the device calculating the time-delay between the signals from the different satellites to then estimate the relative distances between the device and each satellite. It then uses the ephemeris to calculate the location of each satellite, and couples that with the distance-to-each-satellite info to figure out your location. The more satellites it hears, the more data points it has with to refine the accuracy of the location.
So yes, more satellites typically means a better fix, but there are nuances to why that is so. Also note that the relative elevation of each satellite above the horizon plays a factor. Fewer high-in-the-sky satellites might be more accurate at the time than more lower-in-the-sky satellites.
This is also why day to day comparisons in GPS accuracy can be tricky. Ever if all the meteorological and other location characteristics are the same, the satellite positions change. (24 satellites, orbits are ~2 hours, so a great big game of three-card-monty in a way...
Thanks for the detailed description. Do you think the issues people see with AW accuracy compared to less spaced constrained devices is an issue with inferior antennas due to size constraints? I've seen figures that suggest the newer and smaller Garmin watches often don't perform as well as the older and larger devices, which would suggests compromises are being made to get a package that doesn't look like you are carrying a calculator on your wrist. Same day, same location comparisons certainly suggest the AW is having trouble compared to an iPhone, unfortunately my old Garmin will no longer charge, so I can't do same day comparisons with it, but old run data makes it look like it was able to maintain what looks on the GPX data to be very close to +-1 metre horizontal accuracy (which sounds like it shouldn't be possible with standard GPS, so I guess they are being clever with post-analysis as well as having a high quality receiver).
Wavelength for GPS is around 25cm, so 1/4 wave antenna would be about 6cm. I'm not sure the size differences in the various GPS watches I've had makes much difference, but it could. Garmin switched to a wholly different chipset as of the 620 and I suspect the problems were related to that moreso than a slight difference in antenna size between the 610 and 620 (and later).
As for the phone, do note that it uses both GPS as well as wifi-location services, creating an enhanced positioning. In time perhaps the watch may be able to do similarly. https://en.wikipedia.org/wiki/Wi-Fi_positioning_system
As for the phone, do note that it uses both GPS as well as wifi-location services, creating an enhanced positioning. In time perhaps the watch may be able to do similarly. https://en.wikipedia.org/wiki/Wi-Fi_positioning_system
I have added a new "hacc" field with the accuracy in metres as reported by Apple.
I have also added settings to allow users to control whether each field type (heart rate, cadence, elevation and accuracy) is included in the GPX file.
The trail coloured by accuracy should be easy to add, and should be in the next version.
I definitely see lower accuracy from the Watch without the phone, but it still seems fine for most uses, especially once it gets a lock. I usually see levels of 4 or 5 on the watch, and only see 3 when there are more obstacles such as trees, but this does happen more often on the Watch than the phone. The trail does become slightly more wiggly when the strength is lower, but it is still pretty good, and is fine from a navigational point of view (although I accept that it may affect distance calculations slightly).
Maybe I am being a bit critical with my arbitrary "accuracy to signal level" mapping that unfortunately highlights the difference between the watch and the phone. I am amazed at all the technology they have stuck in the Watch and am not surprised that the GPS isn't quite as good as the phone. To be honest I'd be a bit annoyed if the phone wasn't better given the extra size to play with (in terms of avoiding interference from other components) and the years of experience.
It is always going to be easier for Garmin to build a watch where the GPS is the main focus; size isn't as much of an issue; and the other components can be built around it. Dedicated devices are usually better than general purpose ones, but I still think that the GPS in the Watch does a good enough job for most purposes.
That would be great. I am in a testing phase at the moment but these features are minor enough to add now. I will PM you when they are ready. Thanks.
I was comparing to the Forerunner 305 which has a pretty large enclosure, haven't seen the 610/620 close up, so I'm not sure how they compare. It is a pity the antenna can't be placed in the wrist bands, perhaps if the rumoured 'smart band' idea ever becomes a thing, we might see that happen.
My understanding of the use of the Wifi (and cell towers) via AGPS is that it really only improves the speed of locating the satellites, by giving a rough location estimate which allows much faster satellite lock, and that once lock is achieved, it doesn't really provide much benefit after that (at least not for the type of application that requires 10m accuracy or better). I assumed the AW3 would already be using AGPS, but perhaps that was an overly optimistic assumption.
You are correct in that aGPS essentially gives quicker position lock due to helpers such as giving the device the tower's location and feeding it the ephemeris.
Wifi-positioning is another thing and uses the known-locations of wifi hotspots the device can hear (even if not connected to them) to enhance the positioning, especially if the GPS position is weak such as indoors. Example of one such service: http://www.skyhookwireless.com/
Wifi-positioning is another thing and uses the known-locations of wifi hotspots the device can hear (even if not connected to them) to enhance the positioning, especially if the GPS position is weak such as indoors. Example of one such service: http://www.skyhookwireless.com/
Nice choice of field name
I'll try some different locations and times of day to see if I can get it to sit at 4 or 5 more often. Yes, for many uses, the GPS accuracy as is (even at level "3" is fine). Using the watch for accurate pacing is probably going to be an issue, but perhaps the stride calculation will be accurate enough such that the GPS 'wiggling' doesn't matter, and I haven't tried to assess that yet. Certainly for easy runs where I don't care that much about pace, I think the watch will be fine without the phone.
Indeed, and Garmin does seem to have their own issues with their more general purpose watches in terms of accuracy too (compared to the watches specifically targeting runners instead of general fitness tracking).
Looking forward to the seeing the changes you've mentioned!
What we can’t use the Apple Watch to measure the distance the continental plates move each year. it’s going back.
I have done some analysis of how the watch GPS accuracy in metres relates to real-world performance (which is what matters). I did this previously with the phone GPS and translated accuracy in metres into 5 signal strengths that give a real-world indication of the location accuracy:
5 blue bars: (accuracy < 6m) The best accuracy that the device provides (blue breadcrumb trail)
4 blue bars: (accuracy < 11m) Looks to be following the path/road accurately. (blue breadcrumb trail)
3 grey bars: (accuracy < 35m) Good enough to draw a trail, but a bit wiggly (grey breadcrumb trail).
2 red bars: (accuracy < 150m) Close enough for a rough idea of where you are (red GPS dot, no trail).
1 red bar: (accuracy < 500m) Way out but better than nothing (red GPS dot, no trail).
I have added the ability to draw a trail that is coloured by accuracy, and used this to examine previous workouts where I used just the watch GPS, with the idea of refining the values to handle the watch. As with the analysis of speed/heartrate/elevation/cadence you can swipe your finger across the route profile to see the values at any given point. This allowed me to swipe across the trails and see how the GPS accuracy in metres compared with the wiggliness (!) of the trail at each point.
It seems that the watch GPS returns a different set of discrete values. Whilst the phone returns 5m and 10m at best, the watch returns 8m, 12m, and 16m. These all seemed to give acceptable results using my "follows the trail accurately" criteria, so I have decided to change the 11m value to 17m. I have also changed the 6m value to 9m, to allow the strongest watch signal to give 5 bars without affecting the phone bar strengths.
So my previous signal strengths were probably a bit harsh on the watch GPS. It is definitely not quite as good as the phone GPS, but my choice of bar strength accuracies was based on the discrete values given by the phone and unfairly exaggerated this diffference.
I tested the new values today on a mountain bike ride in the forest, a walk in the woods and several car trips, and the results seem fair. Hopefully the signal strengths on the next version of the app will provide a fairer real-world indication of the watch GPS accuracy.
Apologies if I have gone into too much detail!
5 blue bars: (accuracy < 6m) The best accuracy that the device provides (blue breadcrumb trail)
4 blue bars: (accuracy < 11m) Looks to be following the path/road accurately. (blue breadcrumb trail)
3 grey bars: (accuracy < 35m) Good enough to draw a trail, but a bit wiggly (grey breadcrumb trail).
2 red bars: (accuracy < 150m) Close enough for a rough idea of where you are (red GPS dot, no trail).
1 red bar: (accuracy < 500m) Way out but better than nothing (red GPS dot, no trail).
I have added the ability to draw a trail that is coloured by accuracy, and used this to examine previous workouts where I used just the watch GPS, with the idea of refining the values to handle the watch. As with the analysis of speed/heartrate/elevation/cadence you can swipe your finger across the route profile to see the values at any given point. This allowed me to swipe across the trails and see how the GPS accuracy in metres compared with the wiggliness (!) of the trail at each point.
It seems that the watch GPS returns a different set of discrete values. Whilst the phone returns 5m and 10m at best, the watch returns 8m, 12m, and 16m. These all seemed to give acceptable results using my "follows the trail accurately" criteria, so I have decided to change the 11m value to 17m. I have also changed the 6m value to 9m, to allow the strongest watch signal to give 5 bars without affecting the phone bar strengths.
So my previous signal strengths were probably a bit harsh on the watch GPS. It is definitely not quite as good as the phone GPS, but my choice of bar strength accuracies was based on the discrete values given by the phone and unfairly exaggerated this diffference.
I tested the new values today on a mountain bike ride in the forest, a walk in the woods and several car trips, and the results seem fair. Hopefully the signal strengths on the next version of the app will provide a fairer real-world indication of the watch GPS accuracy.
Apologies if I have gone into too much detail!
The more detail the better! I was a bit confused by this bit:
"Whilst the phone returns 5m and 10m at best, the watch returns 8m, 12m, and 16m."
I've seen transient 5 bar signals on the watch before, which seems to indicate <6m is possible on the watch GPS, was your 8m at best just for a particular journey? Looking forward to being able to try the 'accuracy swiping'.
I thought the same. I was sure that I had seen 5 bars on the watch but couldn't find anything less than 8m accuracy in my workouts. However I hadn't done too many without the phone and most were in forests or tests in cars (useful for dodgy signals), so it could be that I never saw a perfect signal. I'm guessing 4m is possible given the 8,12,16 pattern, but maybe it needs perfect conditions or a nearby WiFi signal or something similar. So I decided to go with 8m to be on the safe side.
The new version of the app will have the ability to tap the signal to see a temporary message at the bottom of the screen showing the accuracy in metres. I thought about an option to show the accuracy in metres all the time instead of strength bars, but felt that it would be slightly misleading because it is a maximum possible error rather than an indication of how inaccurate the signal actually is.
In most cases I find that the signal is a lot more accurate than the maximum error, usually tracking the right side of a road even when the accuracy shows that it is up to 10m out. I can be explicit about this in the temporary message by saying "within 10m" (see screenshot) but there isn't room to display that all the time. Hopefully the bars give a practical indication of the accuracy rather than a slightly arbitrary number, and if you want the number then you can single-tap the signal bars (double-tapping shows/hides them).
Looks fine. I imagine UI design is quite challenging for a screen that small. I'd still be interested in having an option to have the metre area permanently display the metre accuracy, but can live with the tap. If you did end up providing a metres instead of bars option, you could clearly label the option in the settings list as "Show maximum possible error" which would avoid people assuming it was the exact accuracy. With that kind of label on the settings panel, "< 10m" would probably be as good as "Within 10m", and a little more compact, given the user would have had to have read the text of the option to turn it on, assuming it was off by default.
Ok, you convinced me! How does this look? It's not quite as pretty as I would like but it was easy to code using the existing strength meter. It still defaults to the signal bars, but you can change it to show metres in the settings. Tapping it then shows the level at the bottom of the screen instead of the accuracy in metres. This will be in the next version in a week or so.