Series 3 GPS Accuracy

[cfc]

I like it. Probably not many users will choose to turn it on, but for anyone that wants to test accuracy, it will be quite handy I think.

I agree, and I am one of those users, which was one of the reasons that I added it!

Here are screenshots of the new GPS accuracy graphs (which I will also find useful). They both show the same twisty mountain bike trail in a forest. The first one was yesterday with the watch GPS and the second one was last week with the phone GPS. You can see that the watch signal is generally lower (less blue) and has some "greyer areas" in places.

I have selected a similar point on the trail "Red 5" on both screenshots to see the values at that point and you can see that the accuracy is <24m on the watch and <10m on the phone. The profiles below the maps show that this is the worst accuracy on the route and you can see that the trail from the watch fails to match the marked Red 5 cycle trail, whereas the trail with the phone GPS follows it perfectly.

Having said that if you look at Red 8 on the iPhone trail you can see that the phone is not following it correctly, whereas the watch trail follows it much better, despite being greyer there. My guess is that whoever created the cycle trail in OpenStreetMap had a poor GPS signal there as well, and so the watch follows it better! This is only a guess though.

Apologies again for going into too much detail!

 

[runone]

Having said that if you look at Red 8 on the iPhone trail you can see that the phone is not following it correctly, whereas the watch trail follows it much better, despite being greyer there. My guess is that whoever created the cycle trail in OpenStreetMap had a poor GPS signal there as well, and so the watch follows it better! This is only a guess though.

Thanks for the screenshots. Does apple have an API to use their own map imagery? OpenStreetMap is a great resource, but satellite imagery seems to be the best way to verify paths. It seems OpenStreetMap often has coverage of paths the commercial maps don't, but satellite images are better for verifying accuracy because of how the OSM paths are usually made (using the same kind of technology you are trying to test).

Anyway thanks for being so responsive to feedback, I'm really looking forward to seeing the next version.

 

[cfc]

Thanks for the screenshots. Does apple have an API to use their own map imagery? OpenStreetMap is a great resource, but satellite imagery seems to be the best way to verify paths. It seems OpenStreetMap often has coverage of paths the commercial maps don't, but satellite images are better for verifying accuracy because of how the OSM paths are usually made (using the same kind of technology you are trying to test).

Anyway thanks for being so responsive to feedback, I'm really looking forward to seeing the next version.

When you ask about Apple having an API for map imagery do you mean for the Watch? The Maps API on the watch is very limited. It only allows static map images, which is why I wrote my own map renderer for WorkOutDoors that draws everything from scratch (and allows rotation, breadcrumbs, offline caching, GPX routes etc that the Apple Maps app doesn't). I plan to add satellite images in the future but Apple don't provide anything to help with that.

If you mean for the phone as background for the graphs, then yes they do, so I might allow that in the future as an option to replace my OSM maps.

 

[runone]

If you mean for the phone as background for the graphs, then yes they do, so I might allow that in the future as an option to replace my OSM maps.

Yes, more for viewing after the run on the phone. Saves a step of having to export to GPX from the phone to view with a satellite background. Are there any reasonably priced satellite data out there that can be used standalone in places where apple/google/microsoft apis are not an option? I'd imagine the terms of service of using the satellite maps from those three outside of their supported APIs would be quite restrictive.

 

[cfc]

Yes, more for viewing after the run on the phone. Saves a step of having to export to GPX from the phone to view with a satellite background. Are there any reasonably priced satellite data out there that can be used standalone in places where apple/google/microsoft apis are not an option? I'd imagine the terms of service of using the satellite maps from those three outside of their supported APIs would be quite restrictive.

I find the Apple Satellite coverage on the iPhone pretty good. I use it in another app of mine which superimposes OSM POIs and routes on Apple's street or satellite maps. I may incorporate that code into the iPhone app analysis for WorkOutDoors in the future so that you can choose your background.

Although sometimes satellite images don't help. Here's a screenshot of the dodgy Red 8 trail in my POI app with cycle routes selected. The forest is so dense that you cannot see the route from the air, which explains why the watch GPS struggled:

 

[runone]

Although sometimes satellite images don't help. Here's a screenshot of the dodgy Red 8 trail in my POI app with cycle routes selected. The forest is so dense that you cannot see the route from the air, which explains why the watch GPS struggled:

Wow, from the air, that forest looks super dense. Must be a beautiful trail to travel on.

 

[cfc]

Wow, from the air, that forest looks super dense. Must be a beautiful trail to travel on.

That trail is actually a bit dark and gloomy in places, but generally nice and twisty. That particular stretch is pretty straight but with a 5m vertical dip which is shown on the iPhone elevation profile but not the watch profile.

This is not surprising given that I have a series 2 without a barometer, but what is surprising is that the elevation profile from the series 2 isn't as bad as I would expect. Even without a barometer the GPS is giving a reasonable profile. It's not as good as the iPhone profile, and wouldn't be any good for total ascent calculations, but it does give a rough idea of the terrain. It shows the 10m dip beforehand and the missing 5m dip is in dense forest, so it is understandable to miss it when relying on GPS only and no barometric sensor.

 

[runone]

That trail is actually a bit dark and gloomy in places, but generally nice and twisty. That particular stretch is pretty straight but with a 5m vertical dip which is shown on the iPhone elevation profile but not the watch profile.

This is not surprising given that I have a series 2 without a barometer, but what is surprising is that the elevation profile from the series 2 isn't as bad as I would expect. Even without a barometer the GPS is giving a reasonable profile. It's not as good as the iPhone profile, and wouldn't be any good for total ascent calculations, but it does give a rough idea of the terrain. It shows the 10m dip beforehand and the missing 5m dip is in dense forest, so it is understandable to miss it when relying on GPS only and no barometric sensor.

I think I remember reading somewhere that vertical resolution isn't as accurate as horizontal resolution in general for GPS. I think some services can map horizontal to accurate vertical by using what I assume are terrain maps?

 

[cfc]

I think I remember reading somewhere that vertical resolution isn't as accurate as horizontal resolution in general for GPS. I think some services can map horizontal to accurate vertical by using what I assume are terrain maps?

That is true. I guess with all the publicity for the new barometric sensor in the series 3 I had forgotten that the series 2 can at least give rough elevation estimates via GPS.

There are several services for determining elevation along a route, one of which I will probably use in a future version of the app for improved profiles.

 

[domeorletta]

I've had an old Garmin 305 in the past, and it was considerably more accurate than my AW3. I've tried several running apps, but don't have enough data yet to see if one is any better than the others. So far I've tested with the default Apple workout app, Runkeeper, Runtastic, ISmoothRun, and Strava. I test by running around a small running track which is about 250m. It is somewhat oval, with very sharp bends (due to the small size of the track).

Most apps seem to lack a GPS strength signal which is annoying, as I would guess some of the inaccuracy I see is partly due to poor lock at the start of a run. Runkeeper does seem to have signal strength displayed.

It is impossible to guess what the internal sampling rate is of the apps, because the frequency with which they write out points is not necessarily the same as what they are sampling the GPS data at. The Apple workout app is writing out at 1 point per second. The rest are all using variable write rates, of around 3-6 seconds. I suspect they are using what Garmin would call 'smart recording' where they only write out a point when pace or direction has changed, so saving on the total file size. It is possible apple uses a private API to talk to the GPS that allows them to sample more often than the other apps, and the other apps are only seeing the data they are writing, and the official gps APIs are not giving them raw data but already massaged data, which would be a pity, as it would make it harder to provide their own smoothing algorithms.

I think the quality of smoothing algorithms has a lot to do with the differences, and so far the apple output seems less smart about this than some of the third party apps, but I need to do some more tests with the the apple app to make sure I'm getting a proper lock before running.

For now, I'm not really looking at distance, as I think the movement detection used for step counting etc is a confounding factor here, as it can provide distance data also. For now I'm purely looking at how the resulting maps in satellite mode line up with the track I'm running on, which I think is a much better test of pure GPS accuracy than looking at distance, at least in the apple app which is a bit of a black box in terms of where the data is coming from (you can use something like rungap to grab GPX files out of the apple workout app, but I haven't compared the distance of the pure GPS track to what the app reports as the run distance yet).

EDIT: I've just installed cfc's WorkOutDoors app, so I'll add that to the apps I'm testing. Seeing a record of signal strength over the course of the run sounds like a great way to test.

[doublepost=1510943025][/doublepost]I train almost everyday and ran 10km or half marathon race every weekend. Unfortunately, the result is the following:

- if the race is 10km, AW3 reports 10.25
- if the race is 21.1, AWR reports 21.6/7

This is not a matter of buildings or trees around, because it fails everywhere!! I am not a fanatic but we cannot say the measurement is sharp!

All in all, AW3 should not be good for runners.

 

[deeddawg]

I train almost everyday and ran 10km or half marathon race every weekend. Unfortunately, the result is the following:

- if the race is 10km, AW3 reports 10.25
- if the race is 21.1, AWR reports 21.6/7

This is not a matter of buildings or trees around, because it fails everywhere!! I am not a fanatic but we cannot say the measurement is sharp!

Are you hitting every tangent perfectly on those courses? In over a decade of various races with different GPS watches, I've never hit the exact race distance and typically had something 3-5% longer recorded. Nature of the beast since you're usually dodging folks here and there.

Worth a read: http://running.competitor.com/2014/06/training/debating-the-accuracy-of-gps-vs-race-distances_105664

All in all, AW3 should not be good for runners.

All depends on what you're after. Careful not to assume everyone's needs are the same as yours. The AW3 is not on par with the data availability from most mid and upper range Garmins, but it's certainly quite workable for recreational runners. I find my FR235 and AW3 to be within 1% of each other, and I've seen similar differences between different Garmin models as well over the years.

 

[MJ22]

Are you hitting every tangent perfectly on those courses? In over a decade of various races with different GPS watches, I've never hit the exact race distance and typically had something 3-5% longer recorded. Nature of the beast since you're usually dodging folks here and there.

Worth a read: http://running.competitor.com/2014/06/training/debating-the-accuracy-of-gps-vs-race-distances_105664


All depends on what you're after. Careful not to assume everyone's needs are the same as yours. The AW3 is not on par with the data availability from most mid and upper range Garmins, but it's certainly quite workable for recreational runners. I find my FR235 and AW3 to be within 1% of each other, and I've seen similar differences between different Garmin models as well over the years.

Hey, careful who you’re calling a recreational runner! I’m a woman who can run under 20:00 for 5K and under 6:00 for a mile, as well as winning awards at some HM’s with thousands of runners. I actually really love the Workout app and it’s everything I need. I don’t need 13,000 data sources. I need time, pace, and distance. I use a stopwatch timer for track intervals.

On the topic at hand, my AW3 has been very consistent with my old Garmin and with the quarter-mile markings on a path I regularly run on.

And yeah, my races often come out long too. Especially in a race where you may weave around other runners or don’t run the tangent as tightly as possible, as most races are wheel measured to the tightest part of a turn.

 

[domeorletta]

another half marathon yesterday...600 meters more :-(
I have a garmin too and it is much more consistent
it is not a matter of recreational runners, but there is a big difference if you run at 4'5''/km or 3'50''/km
the AW is beautiful, trendy, cool, fancy etc etc, but after spending 400 euros we cannot admit a such error

 

[NickYanakiev]

another half marathon yesterday...600 meters more :-(
I have a garmin too and it is much more consistent
it is not a matter of recreational runners, but there is a big difference if you run at 4'5''/km or 3'50''/km
the AW is beautiful, trendy, cool, fancy etc etc, but after spending 400 euros we cannot admit a such error

That's why I always run with my Stryd footpod paired to the Apple watch. I had a Fenix 5X that I sold due to poor GPS accuracy (had a Fenix 3 and a 3HR before that); Ever since I got a Stryd, I stopped paying attention to accuracy and just focus on running and enjoying myself.

I got the added benefit of good lucks (SB SS Series 3 with milanese loop), music while on the run and cellular connectivity.

 

[MJ22]

another half marathon yesterday...600 meters more :-(
I have a garmin too and it is much more consistent
it is not a matter of recreational runners, but there is a big difference if you run at 4'5''/km or 3'50''/km
the AW is beautiful, trendy, cool, fancy etc etc, but after spending 400 euros we cannot admit a such error

I think you’re a little dramatic. And might need to work on running the tangents better.

 

[ftaok]

another half marathon yesterday...600 meters more :-(
I have a garmin too and it is much more consistent
it is not a matter of recreational runners, but there is a big difference if you run at 4'5''/km or 3'50''/km
the AW is beautiful, trendy, cool, fancy etc etc, but after spending 400 euros we cannot admit a such error

You have an error in your pace numbers.

I'm assuming that the AW stated that you ran 600 meters above the 21.098 km distance. That's an error of 2.8%

Your listed pace(s) 4:05 and 3:50 can't be correct with the above information. If you accept that your real pace was 4:05, then the AW would have stated 3:58. It's still an error, but it's not the 6.1% error that your listed paces would imply.

 

[domeorletta]

You have an error in your pace numbers.

I'm assuming that the AW stated that you ran 600 meters above the 21.098 km distance. That's an error of 2.8%

Your listed pace(s) 4:05 and 3:50 can't be correct with the above information. If you accept that your real pace was 4:05, then the AW would have stated 3:58. It's still an error, but it's not the 6.1% error that your listed paces would imply.

I am with you. The above mentioned was only an example but the difference between 4'05 and 3'58 is several months of training (i'm very dramatic)

 

[MJ22]

As a veteran runner of over 20 years, even assuming the GPS is slightly off, that won’t be the reason you do/don’t hit your goals. Frankly you should not be so reliant on an electronic device. You need to learn how to pace internally and learning how to adjust and push when it’s necessary to. This is the reason I wear a regular old Timex for track intervals and mile repeats. I need to know how to pace without a device telling me what I am doing.

Is it nice to have these tools available? Sure. Is it what makes you a great runner? No way.

 

[QCassidy352]

Personally, I have long trusted DC Rainmaker for my GPS watch reviews. He’s been doing it a long time and tested basically every watch out there, and he has no biases that I can see. Here’s what he said about the Apple Watch series 3 GPS:

Overall though, the Apple Watch GPS accuracy isn’t bad. It’s not spectacular, but it’s also not bad. It’s roughly on-par with other mid-range GPS watches that I’ve tested, each having their pros and cons. The one catch though is the start of a workout. It’s really best to wait a bit outside, away from your phone (if you aren’t running/cycling/whatever with it), to ensure it gets GPS from the very beginning.

Full review here (worth a read): https://www.dcrainmaker.com/2017/11/apple-watch-series3-cellular-fitness-sport-review.html

Also worth nothing that a commenter below asks if Apple has improved the god-awful pace smoothing Ray reported in his Apple Watch 2 review last year, and his reply comment says software updates seem to have helped even before series 3, but he didn’t have concrete examples or video of the improvement.

 

[NickYanakiev]

As mentioned earlier, I moved from Garmin’s Fenix 5X. The unit’s GPS performance was far from stellar (along with my old F3/F3HR);

Combining accurate pace and distance+ power and running without my phone but still with music has made purchasing an AW3 cellular a winner!

Thanks to Stryd I stopped paying attention to pace and ground covered, knowing I can trust the data.

I am yet to look back.

https://www.strava.com/activities/1283696798
https://www.stryd.com/powercenter/run/5168619135696896

 

[ch02ce]

This has been an interesting thread so far! I've got a Series 1 watch and iPhone 8 - what I'm curious about is the elevation data. @cfc - in your experience, how are the z-coordinates derived in your app from Apple? Pure GPS? Or Barometer? Or some mix of both? As I'm interested in slope, wondering how accurate these measures are.

 

[cfc]

Apple make both sources of altitude available via different mechanisms. The GPS altitude is available from what they call the "location manager" and the barometric data (which only gives relative altitude) is available from the "altimeter". Apps can ask the altimeter if relative altitude is available and then decide whether or not to use it. WorkOutDoors uses it if available but resorts to the GPS altitude if not. Hope that helps?

 

[uptownnyc]

I'm late to this thread, but trying to understand the GPS measurements coming from my wife's series 3 GPS watch with what I capture with my iPhone X. She's seen measured race-distances (half-marathon) coming in as being 13.25 miles versus the 13.1 mile course she ran. The time measurement of the watch lined up exactly with the race timers.

We've also frequently seen a mismatch between her watch and my phone when hiking. For instance, I recorded a 1.9 mile hike with AllTrails on the iPhone X and her watch recorded 1.6 miles.

The watch appears to be measuring outside what I'd consider an acceptable accuracy rate.

 

[MJ22]

I'm late to this thread, but trying to understand the GPS measurements coming from my wife's series 3 GPS watch with what I capture with my iPhone X. She's seen measured race-distances (half-marathon) coming in as being 13.25 miles versus the 13.1 mile course she ran. The time measurement of the watch lined up exactly with the race timers.

We've also frequently seen a mismatch between her watch and my phone when hiking. For instance, I recorded a 1.9 mile hike with AllTrails on the iPhone X and her watch recorded 1.6 miles.

The watch appears to be measuring outside what I'd consider an acceptable accuracy rate.

Why do you assume your iPhone X is correctly capturing measurements? All GPS devices are incorrect to some extent. As to the half marathon, did she run the tangents right and exact to every turn? Did she not weave around any runners? Because races are measured by a wheel to the tightest tangent and the straightest line and they’re not certified based on GPS. I have been running for 24 years and never once has a half marathon/marathon (of which I’ve run over a dozen) come out to 13.1 or 26.2 on a GPS watch—because of those factors.

 

[millerj123]

Why do you assume your iPhone X is correctly capturing measurements? All GPS devices are incorrect to some extent. As to the half marathon, did she run the tangents right and exact to every turn? Did she not weave around any runners? Because races are measured by a wheel to the tightest tangent and the straightest line and they’re not certified based on GPS. I have been running for 24 years and never once has a half marathon/marathon (of which I’ve run over a dozen) come out to 13.1 or 26.2 on a GPS watch—because of those factors.

On top of that, this is not high-precision GPS. Currently, GPS is said to be accurate within 16 feet at any given time, with additional variance based on satellite availability, interference, etc. When you think about the scale of what is taking place, 16 feet is pretty amazing. https://www.gps.gov/systems/gps/performance/accuracy/