use Apple Watch as fitness tracker or wait

[exxxviii]

For giggles, here is a data dump of my Garmin and AW distances when I wore and used both devices side by side on the same route. The big chunk is recent data with OS2. The smaller chunk is OS1. The AW had been calibrated and re-calibrated multiple times during these periods.

You can see that the AW is all over the place in distance. It is interesting to look at it this way, because I did not notice that the distances had dropped significantly from the 5.56 mile range down to around 5.30 miles more recently. The distances under 5.20 miles are probably the days that I had the phone with me, but I did not note that in my runs. I have more OS1 run data, but I did not feel like keying it all in.

One of the interesting tells about the AW are the differences between mode, median, and mean. The Garmin is spot on, which reveals a high degree of precision in its output. The AW, by contrast, is wildly out of control. The AW's precision was better in OS1, but it was still inaccurate.


Garmin AW Distance (OS2)
5.07 5.14
5.08 5.30
5.03 5.19
5.04 5.30
5.07 5.28
5.08 5.26
5.03 5.48
5.05 5.28
5.03 5.52
5.06 5.53
5.02 5.53
5.04 5.61
5.05 5.64
5.05 5.54
5.06 5.64
5.06 5.78
5.06 5.65
5.06 5.66
5.07 5.06
5.08 5.11
5.07 5.11
5.09 5.69
5.08 5.72
5.06 5.45
5.08 5.63
5.07 5.69
5.07 5.51
5.06 5.17
5.06 5.10
5.06 5.18
5.07 5.23
5.06 5.30 Mode
5.06 5.42 Mean
5.06 5.48 Median
0.02 0.23 Standard Deviation


Garmin AW Distance (OS1)
5.07 4.97
5.08 5.02
5.06 4.90
5.07 4.93
5.07 5.00
5.04 5.44
5.06 4.94
5.07 4.98
5.07 #N/A Mode
5.07 5.02 Mean
5.07 4.98 Median
0.01 0.17 Standard Deviation

 

[BlueMoon63]

For giggles, here is a data dump of my Garmin and AW distances when I wore and used both devices side by side on the same route. The big chunk is recent data with OS2. The smaller chunk is OS1. The AW had been calibrated and re-calibrated multiple times during these periods.

You can see that the AW is all over the place in distance. It is interesting to look at it this way, because I did not notice that the distances had dropped significantly from the 5.56 mile range down to around 5.30 miles more recently. The distances under 5.20 miles are probably the days that I had the phone with me, but I did not note that in my runs. I have more OS1 run data, but I did not feel like keying it all in.

One of the interesting tells about the AW are the differences between mode, median, and mean. The Garmin is spot on, which reveals a high degree of precision in its output. The AW, by contrast, is wildly out of control. The AW's precision was better in OS1, but it was still inaccurate.


Garmin AW Distance (OS2)
5.07 5.14
5.08 5.30
5.03 5.19
5.04 5.30
5.07 5.28
5.08 5.26
5.03 5.48
5.05 5.28
5.03 5.52
5.06 5.53
5.02 5.53
5.04 5.61
5.05 5.64
5.05 5.54
5.06 5.64
5.06 5.78
5.06 5.65
5.06 5.66
5.07 5.06
5.08 5.11
5.07 5.11
5.09 5.69
5.08 5.72
5.06 5.45
5.08 5.63
5.07 5.69
5.07 5.51
5.06 5.17
5.06 5.10
5.06 5.18
5.07 5.23
5.06 5.30 Mode
5.06 5.42 Mean
5.06 5.48 Median
0.02 0.23 Standard Deviation


Garmin AW Distance (OS1)
5.07 4.97
5.08 5.02
5.06 4.90
5.07 4.93
5.07 5.00
5.04 5.44
5.06 4.94
5.07 4.98
5.07 #N/A Mode
5.07 5.02 Mean
5.07 4.98 Median
0.01 0.17 Standard Deviation

Looks about right to me. Without GPS, how could the Apple Watch be accurate. I would not doubt your Garmin or at least not more than +\- 1% maybe

I assume you ran at least one time with iPhone set to calibrate and when you ran 5 miles the Apple Watch matched the Garmin?

No way the Apple Watch can match a dedicated GPS device. Maybe Apple adds GPS to new bands. Sounds like a great way to make money

 

[zhenya]

Since I saw the tests that Consumer Reports did, I wanted to do my own tests just to be sure. So far I have been testing it against my Polar V800 (a high end GPS fitness watch). I intend to do more testing to try to reproduce any inaccuracies that you claim.

I am pretty well versed on fitness devices as I have owned...
Polar V800
Polar M400
Garmin fenix3
Garmin vivoactive
Fitbit Surge
Fitbit Charge HR
Basis Peak
Jawbone UP24

I have used these devices pretty extensively over the past year so I know what I am doing.

If I cannot reproduce these inaccuracies that you and others claim, the only logical solutions are...
The devices are defective
The devices are not being setup and/or used properly
Problem with the iPhone GPS or some other unknown issue with the iPhone/AW settings, or even a bad GPS signal in certain areas.

As for right now, I am going to have to go with what Consumer Reports says (and other actual tests that I have viewed) over some pissed off people who may or may not know how to use their device properly.

You only need to look to sites that review fitness devices seriously to see where the AW falls down for fitness tracking. The AW sensor is not as accurate, nor does it update as quickly as other good wrist-based devices, let alone a good chest strap. Your opinion that it is every bit as good as a high-end Polar is the one way outside the norm, and would need to be backed up with some solid data. It could be that your standards are different, or that the bit of data that you are basing your opinion on is simply a fluke. The information I have seen from Consumer Reports tests merely stated that 'there were no significant differences'. Well, where are the graphs? What constitutes 'significant'?

This site knows how to review this kind of gear. Here are some graphs from the Garmin 225 review http://www.dcrainmaker.com/2015/08/garmin-forerunner-225-depth-review.html that illustrate how the AW falls short of other wrist based monitors, let alone chest straps. AW is in purple.

So maybe that is 'good enough' for you, but the data doesn't lie - the AW simply isn't as fast or as accurate as competing devices.

I have run extensively with the Garmin 225 for the last year, and with the AW for the last month, and my experiences match the data above. These graphs represent the type of difference I see routinely. The AW simply fails to get a reading during long periods of time, and if I want real time updates I'm left staring at my watch waiting rather than watching where I'm going.

Apple Watch.

Garmin 225.

 

[exxxviii]

Looks about right to me. Without GPS, how could the Apple Watch be accurate. I would not doubt your Garmin or at least not more than +\- 1% maybe

I assume you ran at least one time with iPhone set to calibrate and when you ran 5 miles the Apple Watch matched the Garmin?

The Garmin is probably accurate within 0.2%, depending on your statistics philosophies. The AW is probably accurate to within 4%. 4% is not bad on the surface, but a quarter mile is a pretty big deal to most runners in a 10K.

Yes, I have completed multiple calibrations that far exceed Apple's minimum 20 minutes.

I agree that the AW's accuracy without GPS is limited. However, the problem with the AW is that you cannot override its bad calibration data. My Garmin with its foot pod and Fitbit are both far more accurate than the AW because I can manually update stride factors. Eventually, Apple will fix the calibration problem, and then the device should be as accurate as a FB, Garmin, Nike+, MS, etc. device with a manual stride distance.

Another problem with the AW is its lack of precision. The device should be pretty consistent from run-to-run, even without GPS. But, a .23 mile standard deviation seems to indicate a bigger problem with its internal calculations. I believe Apple will eventually fix this as well.

 

[BlueMoon63]

The Garmin is probably accurate within 0.2%, depending on your statistics philosophies. The AW is probably accurate to within 4%. 4% is not bad on the surface, but a quarter mile is a pretty big deal to most runners in a 10K.

Yes, I have completed multiple calibrations that far exceed Apple's minimum 20 minutes.

I agree that the AW's accuracy without GPS is limited. However, the problem with the AW is that you cannot override its bad calibration data. My Garmin with its foot pod and Fitbit are both far more accurate than the AW because I can manually update stride factors. Eventually, Apple will fix the calibration problem, and then the device should be as accurate as a FB, Garmin, Nike+, MS, etc. device with a manual stride distance.

Another problem with the AW is its lack of precision. The device should be pretty consistent from run-to-run, even without GPS. But, a .23 mile standard deviation seems to indicate a bigger problem with its internal calculations. I believe Apple will eventually fix this as well.

Yea, that is the bad thing. Way different from run to run. I don't doubt you at all either. I will never be a serious runner who would need to know super accurate distance and splits so the watch is great for me. If I had an accurate device I would still want a device that did more and that is why the Apple Watch is a great device for me.

If anyone says they need an accurate fitness device for distance, I would never recommend the Apple Watch. If you one device that does an ok job without GPS and basically ok to good for many other things, the Apple Watch is the best out there.

 

[TxWatch]

If you do the calibration with the apple watch it is extremely accurate. Search Consumer Reports for the testing they did on the apple watch compared to several fitness trackers.

CR did their calibration testing with Watch OS 1 which "may" have worked. (I did not own my Watch before OS 2, so I never tested OS 1) Calibration for Watch OS 2 does not work. I have tried to calibrate mine with OS 2.01 and OS 2.1 more than a dozen times.

I followed the Apple website instructions for calibrating the Watch, including resetting the calibration data which was a new feature introduced in OS 2. The same instructions have been repeated on dozens of review sites to the point that casual readers like yourself, believe it actually works.

Each of my calibration runs were 60+ minutes, 7 miles and 11,000+ steps. I run the exact same route 3 to 4 times a week. My Garmin Forerunner and my iPhone's GPS record the correct mileage within a tenth of a mile of each other on my "known" route. As soon as I leave my iPhone at home, the Watch immediately goes back to the inaccurate distance and calorie measurements. (Calories are based on the distance, so the further the Watch "thinks" you ran, the higher the calorie count)

You have to go through the calibration procedure with the AW for it to be accurate. Once that is done it tested to be the most accurate in Consumer Reports testing when compared to several fitness devices tested. And if it becomes inaccurate for some reason, you just need to do the calibration procedure again. Even when tested right out of the box, it was still only off by 10%, and that is the average that the other fitness devices where any way, which means it's average right out of the box.

I think your clarifications are misinformed.

I am not misinformed because I did the calibration testing myself more than a dozen times. The Watch is off by a factor of 1.21 to 1.39 for my situation. (I am 5' 5" with a short stride length)

Garmin: 7 miles => AW: 8.5 miles (Off by a factor of 1.21) Lowest distance recorded with OS 2.01
Garmin: 7 miles => AW: 8.9 miles (Off by a factor of 1.27) Average distance of both OS versions
Garmin: 7 miles => AW: 9.7 miles (Off by a factor of 1.39) Highest distance with OS 2.1

People who have a shorter than average stride length are off by a higher margin than folks who are taller. The outside temperature seems to be a factor as well. The colder the weather, the shorter the distance the Watch records for the same 7 mile route. As the Watch warms to my wrist, the distance recorded for each loop gets longer and longer, regardless of the starting temperature.

Well the tests speak for themselves, and my personal testing proves it to me.

If people have a defective unit maybe they should return it, and on the same note, if people don't set everything up correctly, maybe they should learn to.

Since your rants are based on what seems to be your personal opinions and other people's posts, and no real testing that I can see, they are pretty much moot to me.

Sorry,

The Watch is not defective and I have been testing complex multi-million dollar networking systems most of my adult life, so I think I know a little about "setting things up".

Since you are new to the forum, I will cut you some slack. Please spend a couple of hours reading this forum as I have, along with others on this thread, and educate yourself to the AW issues. A limited use GPS or a way to manually enter stride-length will go a long way to making the Watch more accurate for running.

Until then, I would not trust the AW for anything more than step counting.

TxWatch

 

[exxxviii]

The colder the weather, the shorter the distance the Watch records for the same 7 mile route.

I just noticed this same association today, when I did a data dump above. My watch has been recording about 0.3 miles shorter on the same 5 mile route, and this began as the weather got colder. I never thought about the relationship to temperature. I just thought it was progressively improving.

CR did their calibration testing with Watch OS 1 which "may" have worked.

Calibration was way better with OS1. I only have a few readings above from OS1, but the rest of my data are similar. The SD and distance accuracy were both much tighter back then.

 

[zhenya]

CR did their calibration testing with Watch OS 1 which "may" have worked. (I did not own my Watch before OS 2, so I never tested OS 1) Calibration for Watch OS 2 does not work. I have tried to calibrate mine with OS 2.01 and OS 2.1 more than a dozen times.

I followed the Apple website instructions for calibrating the Watch, including resetting the calibration data which was a new feature introduced in OS 2. The same instructions have been repeated on dozens of review sites to the point that casual readers like yourself, believe it actually works.

Each of my calibration runs were 60+ minutes, 7 miles and 11,000+ steps. I run the exact same route 3 to 4 times a week. My Garmin Forerunner and my iPhone's GPS record the correct mileage within a tenth of a mile of each other on my "known" route. As soon as I leave my iPhone at home, the Watch immediately goes back to the inaccurate distance and calorie measurements. (Calories are based on the distance, so the further the Watch "thinks" you ran, the higher the calorie count)



I am not misinformed because I did the calibration testing myself more than a dozen times. The Watch is off by a factor of 1.21 to 1.39 for my situation. (I am 5' 5" with a short stride length)

Garmin: 7 miles => AW: 8.5 miles (Off by a factor of 1.21) Lowest distance recorded with OS 2.01
Garmin: 7 miles => AW: 8.9 miles (Off by a factor of 1.27) Average distance of both OS versions
Garmin: 7 miles => AW: 9.7 miles (Off by a factor of 1.39) Highest distance with OS 2.1

People who have a shorter than average stride length are off by a higher margin than folks who are taller. The outside temperature seems to be a factor as well. The colder the weather, the shorter the distance the Watch records for the same 7 mile route. As the Watch warms to my wrist, the distance recorded for each loop gets longer and longer, regardless of the starting temperature.



The Watch is not defective and I have been testing complex multi-million dollar networking systems most of my adult life, so I think I know a little about "setting things up".

Since you are new to the forum, I will cut you some slack. Please spend a couple of hours reading this forum as I have, along with others on this thread, and educate yourself to the AW issues. A limited use GPS or a way to manually enter stride-length will go a long way to making the Watch more accurate for running.

Until then, I would not trust the AW for anything more than step counting.

TxWatch

FWIW, I ran today for the first time with my AW alone without the phone, and my Garmin 225 as the primary tracking device as I wanted to compare HR data in the native workout app vs. the Garmin, as opposed to using the Strava app. Surprisingly, the AW agreed with the Garmin to within 1/10 mile (7.72 for the AW vs. 7.78 for the Garmin. The HR seemed to be quite a bit better as well - although my complaint that you can't see it at a glance stands - it's really only good enough to get an overall average afterwards. And there is no native way to graph HR data in a useful format at all.

I'll also note that this was sub-freezing temperature, and I'm about 5'6" with a short stride length. I would disagree with your assessment that height or temp is likely to have much of anything to do with it. My suspicion is that it, like any other accelerometer based method of tracking distance, is extremely sensitive to how consistent your stride length is. If you are all over the place, changing pace, changing terrain, running then walking, whatever, there is no way that it can give an accurate distance because the only thing it can do is equate a swing of the wrist to a single stride to which it has calibrated an average from the GPS on the phone. If your stride length varies, for whatever reason, the distance will be off, perhaps considerably.

Anyhow, eventually the AW is going to be a killer fitness device. Right now it's kind of frustrating!

 

[mountaindewbass]

I have owned several fitness tracker's: Garmin Vivosmart HR, Garmin Vivoactive, Fitbit One, Body Media arm band, and AW. I would chose AW any day of the week. The best fitness tracker i have ever used is the Body Media Arm Band, its phenomenal but requires a monthly subscription cost. So with all considered i would go with the AW. Its accurate, measures calories, distance, physical activities, and reminds me to move. All that i need. I wish i could track sleep and i wish it could measure stairs both up and down...but besides that. its perfect.

 

[TxWatch]

I just noticed this same association today, when I did a data dump above. My watch has been recording about 0.3 miles shorter on the same 5 mile route, and this began as the weather got colder. I never thought about the relationship to temperature. I just thought it was progressively improving.

I also thought the distance was improving when it first got cold in Texas. (1.2 off is better than 1.3 off...)

However, we have been having 20-30 degree temperature swings in the same week and my Watch always records lower distances when the starting temps are colder. On warmer days, it records longer distances. I also noticed that as I run each loop (I run six 1.17 mile loops) the calculated distance gets longer and longer "per loop" as my wrist heats up the Watch. (i.e. The first loop is always the shortest and the last is always the longest)

It is nice to have someone else corroborate my suspicions of the temperatures affecting the distances. I was hesitant to mention it, but I thought I would throw it out there in case others have noticed the same discrepancy.

I really appreciate hearing from others who have done real life testing, rather than relying on inaccurate "review" websites and forum posting hearsay for their information.

TxWatch
[doublepost=1453751729][/doublepost]

I would disagree with your assessment that height or temp is likely to have much of anything to do with it. My suspicion is that it, like any other accelerometer based method of tracking distance, is extremely sensitive to how consistent your stride length is. If you are all over the place, changing pace, changing terrain, running then walking, whatever, there is no way that it can give an accurate distance because the only thing it can do is equate a swing of the wrist to a single stride to which it has calibrated an average from the GPS on the phone. If your stride length varies, for whatever reason, the distance will be off, perhaps considerably.

Anyhow, eventually the AW is going to be a killer fitness device. Right now it's kind of frustrating!

I agree that it is frustrating!

I have been running outdoors for more than 19 years and I would say my stride length, pace and arm movement are very consistent. I never walk and I always run the same boring 6-loop route which is very flat. I live on a dead-end street, so there is no where else to run without going for more than a mile on a major roadway with no shoulder.

When I run with my phone, the total distance is within a 1/10 of a mile compared to my Garmin, and as stated above, I have done it a dozen times in an attempt to calibrate my Watch. I am not sure why yours is working so well, but I am happy for you. I wish mine was at least closer...

TxWatch

PS. I will also say that I *really* like my Watch for all of the advantages it provides outside of distance tracking. I have been running for so long, I do not need to track my miles or compete with others, etc. I run because I love to run and I am self-motivated. I stopped using my Garmin's many years ago and I only started to use the Watch to collect the $5 a day I get from my insurance company for anything over 3000 steps. It works perfectly for step counting, but the engineer in me could not help but try to make the calibration work.

 

[DdDr2]

I have a Fitbit flex for the past two years. Not really liking the AW as a activity tracker. It doesn't get my steps right at all. Lol.
Anyways, I wear my flex around my ankle ( I use a different band then the actual Fitbit wristband ) and wear my AW in my wrist.
I like my AW for other things but in terms of calculating my steps and sleep, it doesn't trump my Fitbit.

 

[bulldoze]

The thing is, no-one serious about running would trust any of these devices.

Heart rate? you can't even trust that, there are so many different factors that could influence it that it becomes another nonsense slice of data.

Trust verified distance, a decent stop watch, a pen and paper and if you are particularly geeky a spreadsheet to analyse the data in the way that suits you.

 

[zhenya]

The thing is, no-one serious about running would trust any of these devices.

Heart rate? you can't even trust that, there are so many different factors that could influence it that it becomes another nonsense slice of data.

Trust verified distance, a decent stop watch, a pen and paper and if you are particularly geeky a spreadsheet to analyse the data in the way that suits you.

LOL. What? Good watches track good chest monitors very closely. I can guarantee you there are tons of serious runners using that data. You are talking to one right now.

 

[Fthree]

sent back my garmin 630 today, Like it a lot but couldn't justify the cost when i have the fenix 3 in the drawer next to my AW.

 

[BlueMoon63]

LOL. What? Good watches track good chest monitors very closely. I can guarantee you there are tons of serious runners using that data. You are talking to one right now.

I assume he means the Apple watches. That is the one device a serious runner could not count on as a serious device when compared again devices like Garmin/Fenix.

 

[PatrickNSF]

Each of my calibration runs were 60+ minutes, 7 miles and 11,000+ steps. I run the exact same route 3 to 4 times a week. My Garmin Forerunner and my iPhone's GPS record the correct mileage within a tenth of a mile of each other on my "known" route. As soon as I leave my iPhone at home, the Watch immediately goes back to the inaccurate distance and calorie measurements.

Does the AW always use the iPhone's GPS when you do an Outdoor Run with the Workout app? When I was trying to run with my AW last spring under OS1, I didn't believe that to be the case. But I haven't tried the Workout app with the AW enough with OS2 to figure this out. The only times I've run with the AW lately have been trying 3P apps.

 

[Applenoob34]

Does the AW always use the iPhone's GPS when you do an Outdoor Run with the Workout app? When I was trying to run with my AW last spring under OS1, I didn't believe that to be the case. But I haven't tried the Workout app with the AW enough with OS2 to figure this out. The only times I've run with the AW lately have been trying 3P apps.

It's my understanding that if your iPhone is with you, the watch will use the phones gps. If not, it defaults to the accelerometer.

 

[exxxviii]

My run distances are very different when I have my phone with me versus running without. So, the AW is doing something differently, and it probably involves use of the phone's GPS.

 

[Newtons Apple]

The AW uses the phone for calibration but no where does it say it uses the GPS to help,with distances on your watch. A missed opportunity in my opinion. With a work from Apple you could map your run with the watch, for some reason they decided not to do it.

 

[zhenya]

You can map your run with the watch so long as you have your phone with you.

 

[PatrickNSF]

It's my understanding that if your iPhone is with you, the watch will use the phones gps. If not, it defaults to the accelerometer.

That wasn't the case with Apple's Workout app under OS1. If it's true, it may have come as a later update. But my experience (and recollection) is that third party apps used the phones GPS right from the start.

 

[gorkt]

As an owner of an AW1, I'd just wait and use your fitbit. I am not a total fitness data nerd like others on this thread are so as long as it is kind of accurate, I am okay, but the reason why I wouldn't recommend it at this point is that a lot of the running apps are just plain glitchy. I run with my phone so some of the issues with distance accuracy don't really apply to me, but every time my Runkeeper app updates on my phone, it glitches on my watch and I end up having to delete it and reload it on the watch, which is really annoying. It either doesn't record my heart rate, or the running data doesn't stay on the screen etc....

 

[exxxviii]

That wasn't the case with Apple's Workout app under OS1. If it's true, it may have come as a later update. But my experience (and recollection) is that third party apps used the phones GPS right from the start.

This is a complex answer. The Apple workout app does seem to be involved with GPS data to calibrate stride length, and this seems to affect distance calculated by the Workout app when the phone is present. Also, from day 1, apps like Runkeeper would "use" GPS, but that was a loose term. What RK did was simply act as a remote interface to the phone app. So, the watch was not really using phone GPS data with the 3rd party app.

Even today, I think this is still the case. Workout only uses GPS to refine calibration, and RK and the others are just remote iPhone app interfaces. Though the newest fitness apps can act in two modes-- connected to the phone and disconnected. Early on, the apps could only function in a connected mode. But, the use of GPS is essentially the same as before.

 

[Blujelly]

Personally if it was me I'd wait for AW2 I wouldn't even buy a cheap band. before the AW I was using various apps on my iPhone comparing what I was doing on my iPhone and then when I got my AW the stats weren't out massively.

Obviously there is a lot to consider on what my iPhone can measure compared to the AW, but if you're willing to wait I would.

 

[EdwardSmith36]

When it comes to fitness tracker, I only prefer Apple & no other brand I trust. So, I would love to wait for it.