Apple Watch Proves Most Accurate at Measuring Heart Rate in New Fitness Tracker Study

[MacRumors]

In a new study comparing the accuracy of seven different fitness trackers, the Apple Watch was found to have the lowest margin of error when measuring heart rate, beating the Basis Peak, Fitbit Surge, Microsoft Band, Mio Alpha 2, PulseOn, and Samsung Gear S2.

Researchers set out to determine the accuracy of wrist-worn devices at measuring both heart rate and energy expenditure, aka calories burned via physical activity. 60 volunteers participated, including 29 males and 31 females, each of whom wore several fitness trackers and completed activities like cycling, running, and walking.

Data gathered by the fitness devices was compared against a "gold standard" tracking method, which included an electrocardiograph (ECG) for measuring heart rate and clinical grade indirect calorimetry (measuring oxygen and carbon dioxide expelled when breathing) for measuring calories burned. An error rate of 5 percent was determined to be within acceptable limits.

Across all of the modes of activity, the Apple Watch had the lowest median heart rate error at 2 percent (1.2% to 2.8%), while the Samsung Gear S2 had the highest error rate at 6.8 percent (4.6% to 9%). The Apple Watch was also notably more accurate at measuring heart rate during the walking test than competing products.

For the walking task, three of the devices achieved a median error rate below 5%: the Apple Watch, 2.5% (1.1%-3.9%); the PulseOn, 4.9% (1.4%-8.6%); and the Microsoft Band, 5.6% (4.9%-6.3%). The remaining four devices had median error between 6.5% and 8.8%.

When it came to measuring calories, no device, Apple Watch included, managed to accurately determine how many calories were burned through activity. Median error rates across all devices and tasks ranged from 27.4 percent (Fitbit Surge) to 92.6 (PulseOn). Though no device was accurate, the Apple Watch did the best at estimating energy expenditure.

Overall, researchers found that most of the fitness trackers tested were able to measure heart rate with an acceptable error level in a laboratory setting, but calorie estimates are largely inaccurate.

There are three principal findings from the current study. In a diverse group of individuals: (1) most wrist-worn monitoring devices report HR with acceptable error under controlled laboratory conditions of walking, running and cycling; (2) no wrist-worn monitoring devices report EE within an acceptable error range under these conditions; (3) of the devices tested, the Apple Watch had the most favorable error profile while the Samsung Gear S2 had the least favorable error profile.

The full study, conducted by Stanford University and the Swedish School of Sport and Health Services, is available in the Journal of Personalized Medicine.

Article Link: Apple Watch Proves Most Accurate at Measuring Heart Rate in New Fitness Tracker Study

 

[NightStorm]

Why don't these studies ever compare the Apple Watch with more hardcore fitness watches like those from Garmin or Sunto?

 

[jclo]

Why don't these studies ever compare the Apple Watch with more hardcore fitness watches like those from Garmin or Sunto?

I'm not sure why the study didn't include a Garmin product. This was how the trackers were selected:

"Following a comprehensive literature and online search, 45 manufacturers of wrist-worn devices were identified. Criteria for inclusion included: wrist-worn watch or band; continuous measurement of HR; stated battery life >24 h; commercially available direct to consumer at the time of the study; one device per manufacturer. Eight devices met the criteria; Apple Watch; Basis Peak; ePulse2; Fitbit Surge; Microsoft Band; MIO Alpha 2; PulseOn; and Samsung Gear S2."

 

[satchmo]

Why don't these studies ever compare the Apple Watch with more hardcore fitness watches like those from Garmin or Sunto?

Exactly. Garmin is such a giant in the fitness field, that to leave them out seems suspicious.

 

[OldSchoolMacGuy]

Apple puts a lot into R&D and calibration. They know they're the biggest target when it comes to accuracy. Most wouldn't care if FitBit or Samsung aren't overly accurate but you'd see headlines everywhere if Apple were off.

Here's an interesting read on the level technicality they went through to make it incredibly accurate at telling time. It's 4x more accurate than the iPhone actually.

http://mashable.com/2015/12/30/apple-watch-synchronized/#EIku7j18gGq3

 

[macTW]

Even if the Apple Watch isn't as accurate as true fitness bands... having such a technological piece that also is this accurate as a fitness device is a marvel.

Too bad I don't work out enough to benefit from it.

 

[fyun89]

I would assume when you're running (not walking) the error rate will go up more as your watch would slide more

 

[Juicy Box]

This sounds pretty good. I wonder how the AW price compares to the others tested. If these watches/fitness trackers are around the same price, then this is impressive. If there is a big difference in price and the AW is higher, then I guess one would expect this.

I would like all the health features of the AW, but I hate wearing watches.

 

[Mlrollin91]

I would assume when you're running (not walking) the error rate will go up more as your watch would slide more

This 100%. My accuracy drops significantly when running, but when walking, or cycling, its pretty accurate.

 

[mi7chy]

Apple puts a lot into R&D and calibration.

Apple also put a lot of money into Stanford's pocket.

APPLE CHARITABLE MATCHING PROGRAM: Cook decided to put philanthropy on his agenda. Within weeks of taking the CEO position, Cook introduced Apple's charitable matching program, which matches employee contributions up to $10,000. Since the program's inception, it has donated more than $150 million to organizations such as Charity: Water, (Product RED), and Stanford University Hospitals, with Cook himself having donated $50 million a piece to each of the latter two.

 

[madbard]

If you want to look at the datasets, <http://precision.stanford.edu/>

 

[ErikGrim]

Why don't these studies ever compare the Apple Watch with more hardcore fitness watches like those from Garmin or Sunto?

Especially galling since they bothered to include the discontinued Microsoft Band.

 

[phatspider]

Why don't these studies ever compare the Apple Watch with more hardcore fitness watches like those from Garmin or Sunto?

Because it would lose miserably!

 

[daflowerz]

Why don't these studies ever compare the Apple Watch with more hardcore fitness watches like those from Garmin or Sunto?

I love my Garmin 735XT. It seems pretty accurate for heart rate and has an algorithm to calculate Max VO2, all from the wrist. Key is keeping the watch tight on the wrist to minimize extraneous external light. It seems to meet the criteria mentioned in this or another comment. I used to wear a chest monitor and hated it but the 735XT is giving me the same readings off the wrist now.

 

[citysnaps]

Apple also put a lot of money into Stanford's pocket.

APPLE CHARITABLE MATCHING PROGRAM: Cook decided to put philanthropy on his agenda. Within weeks of taking the CEO position, Cook introduced Apple's charitable matching program, which matches employee contributions up to $10,000. Since the program's inception, it has donated more than $150 million to organizations such as Charity: Water, (Product RED), and Stanford University Hospitals, with Cook himself having donated $50 million a piece to each of the latter two.

That's great! And kudos to Apple and Mr. Cook for their philanthropy! The Stanford Medical Center is a worthy cause. Thanks for the heads-up on that.

 

[coolbreeze]

I had to get a Polar strap. My AW series 2 SS was wildly inaccurate. With the strap I have reliable data which is critical for training zones. I hear the Fenix 5 is not very accurate. Wrist sensors just aren't good due to the location. A chest strap is unbeatable.

 

[Stella]

Because it would lose miserably!

Actually, the HR monitors on the Garmin's are pretty good - but this does change between device..

But like all HRM - the accuracy changes between the activity you are doing - running may be more accurate than cycling for an optical based HRM due to vibrations, or swimming may not be possible ( for some ) for example. Also skin tone also effects the accuracy, and frequency of readings ( battery considerations )

How old this is survey? Like others have said some notable products are missing ( Scosche optical has a good reputation for accuracy ) , and products that are no longer sold are included.... which reduces the weight of the results.

 

[kohlson]

This 100%. My accuracy drops significantly when running, but when walking, or cycling, its pretty accurate.

For me, compared to a medical stress test, I know that my 1st get Apple Watch consistently indicates about 30% higher when running. A knowledgeable friend said at my age, this the AW indicated rate were true, my heart would "explode."

 

[cfclay]

They didn't compare the fitbit charge 2? Maybe I missed it, but was this the series 1 or 2 Apple Watch?

 

[Mlrollin91]

For me, compared to a medical stress test, I know that my 1st get Apple Watch consistently indicates about 30% higher when running. A knowledgeable friend said at my age, this the AW indicated rate were true, my heart would "explode."

I wouldn't say it would "explode" lol. Everyone does have a different heart and can handle different things. Yes there are ranges for your age, but those don't have to be set in stone. I'm one of those outliers. I have a heart condition that allows my underlay BPM to be 210+, about 25bpm too high for my age. But my heart doesn't explode, I just have to watch for how long I keep it up at that rate.

But to the point, running, isn't the most accurate. I notice quite a big of errors when running, but only when running. The watch sliding back and forth on your wrist, or even bouncing up and down (depending on how you run) definitely hurts the accuracy.

I have the opposite problem of you, my heart rate is too low while running. I'm curious to know if the watch can keep up with a 210BPM rate, or it errors out because it might see it as not realistic? After over 2 years, I have now just thought about that, thanks for jogging my brain! (Pun intended )

 

[HBOC]

don't you have to have an iPhone for the apple watch to work? DC Rainmaker is a great website that probably does the most in-depth reviews, and i believe they stated that the Fenix 5 is the most accurate wrist based HRM on the market. Suunto is coming out with a wrist based model later this year sometime.

 

[Jetfire]

I would assume when you're running (not walking) the error rate will go up more as your watch would slide more

Yes, That's why I'll wear a HR chest strap, when I run. Using the Sports band and making the watch extra tight can help with accuracy. The other problem is read rate Apple watch. You have to be in Workout mode to get more timely readings.

 

[djcerla]

Tested mine last year vs. a Polar chest band at the gym and readings were very close.

the Samsung Gear S2 had the least favorable error profile.

Samsung being Samsung.

 

[Zirel]

Even if the Apple Watch isn't as accurate as true fitness bands... having such a technological piece that also is this accurate as a fitness device is a marvel.

Too bad I don't work out enough to benefit from it.

Thing is that with the "true" fitness straps it's a lot of cons for what?

• potentially smell bad
• always uncomfortable
• one more thing to put on before training
• one more thing to "wash"
• one more thing to buy batteries for
• require a separate watch (or other receiver)
• hard to get their data to analysis Apps

A mere 0.5 to 1% of better accuracy compared to the Apple Watch? And what are you going to do with that? Not worth it!

The important part here on the right graph, "calories burnt", is that the Apple's circle is the tightest, meaning that while the value might be inaccurate, it's precision is the best.

Meaning that you have a number you can user to compare your workouts to see if you "sweated" enough. That's all I care, for example, at the end of a indoor cycling session, if I did a better job or not compared to the last time.

 

[JPLC]

Where's the Huawei Watch in this test? Last time I remember, most reviews and tests pointed out the Huawei Watch as the most accurate of all smart watches.