First evidence (from Apple) the HR sensor may not be accurate

[maflynn]

Just about very smart watch that I've looked into buying has this criticism. The sensors that try to see your blood flowing are not as good as medical grade equipment. You'll get a fairly good representation of your heart rate but it's not 100% accurate.

As for calories burned, that imo is just a guestimate, some smart watches are better then others. I'm not sure where the apple watch falls in but if I were to guess, it would be one of the better ones.

 

[Julien]

That sounds about right. I still think the guy who photoshopped it put the wrong data in.

Quick! Alerts Phil on twitter.

After starting the thread I have come to believe this also. Now it seems I may be guilty of a 'link bait' thread. Maybe now I'm qualified for a job at BuzzFeed.

It is very surprising to see in the Good Morning America clip the extremes Apple is going to working on in house algorithms for calorie calculations. They could have just license this data for pennies on the dollar to what they are spending. Apple must have confidence the optical HR monitor is very accurate or they are building a database for future generations or it is an expensive PR exercise.

I am anxious to give Watch a test run and see how it fairs at HR accuracy and caloric calculations.

Still seems strange that Apple missed the picture's accuracy/plausibility. They certainly have many departments that must sign off before anything is allowed to be published.

 

[Piggie]

Just about very smart watch that I've looked into buying has this criticism. The sensors that try to see your blood flowing are not as good as medical grade equipment. You'll get a fairly good representation of your heart rate but it's not 100% accurate.

As for calories burned, that imo is just a guestimate, some smart watches are better then others. I'm not sure where the apple watch falls in but if I were to guess, it would be one of the better ones.

Thanks for this posting.
It's exactly how I am seeing it, as I like to see the honest truth about something once you strip back all the glitz and fanfare put on for the audience.

Yes. For me, this is showing you have monitoring systems that are not very accurate, however, they do give you something.

You then use professional monitoring equipment on people, gather a lot of data using this equipment.
See the sort of numbers your cheap and inaccurate method give you, then try and marry the two together.
So that when you cheap device (watch) reports back to the software some data, you can then take a "best guess" what this data means, using the proper data you collected, and come up with some number you hope is probably not that far off being reliable to use for the device wearer.

I suppose that fine, although it's not being "sold" that way to people, and without question many normal people who think Apple are perfect will simply assume their few hundred dollar general purpose watch which has just a few dollars worth of body monitoring equipment inside it, is as accurate and reliable as a piece of medical equipment as something from their doctor or hospital.

And for some people that assumption could be dangerous.

I strongly suspect there will be some VERY HEAVY disclaimer in the Watch terms and conditions (that no one will read) that basically tells people in legal mumbo jumbo, the data from the watch should not be relied upon and it's only intended as a "Guide"

 

[matrix07]

I am anxious to give Watch a test run and see how it fairs at HR accuracy and caloric calculations.

I'm so looking forward to it.

 

[Piggie]

I'm so looking forward to it.

Simple question:

Does, or rather would it worry you, if you knew that it was not giving you accurate results back from the data is collected from you.

And was simply reading some low quality data from you, then looking this up against a database of previously accurately recorded data, and giving you back a best guess based upon this data.

This data you see, may indeed not be far off, but would be looked up against prior data, as opposed to what was actually recorded from you.

Would that make a difference to you?

In a similar way, say I was not a doctor, just someone who took some general readings, used the general data I recorded from you, used the data to look up accurate medical results, and told you a diagnosis.

The fact my diagnosis, was probably quite right, but it based upon a large set of collected data, rather than directly from you.

Would that make any difference?

 

[matrix07]

Does, or rather would it worry you, if you knew that it was not giving you accurate results back from the data is collected from you.

It would bother me greatly if it's less accurate than other smart watches out there. And I will not be the only one, I guess. There will be a sh_t storm in the media and we will get another _gate surely.
I think that's one of the benefit of buying Apple product. It is expected more than anyone else. It's like you got a crowdsource watchdog.

 

[Piggie]

It would bother me greatly if it's less accurate than other smart watches out there. And I will not be the only one, I guess. There will be a sh_t storm in the media and we will get another _gate surely.
I think that's one of the benefit of buying Apple product. It is expected more than anyone else. It's like you got a crowdsource watchdog.

How about this..........

I have no doubt that it will be at least as good as other watches that use the same technique to record the data.

I'm also aware, as we all are that other devices will monitor for far more body data than the Apple watch can.

Also I'm aware there are going to be far more accurate ways of monitoring the body (chest strap?) but general public would not accept that, even though it would probably be more accurate.

It's a compromise device, that will record some level of data, and will use some prior knowledge to display a assumed result I'd say.

Hence all the calorie monitoring Apple say they are doing.
Without, "proper" equipment it's just a best guess based upon prior collected data. And that's what apple say they are doing, or have been doing.

So, for your age, height, weight, sex etc, based upon say xx number of steps, it will look up prior data and give you a best guess value

 

[kdarling]

Wrist based pulse rate readers are well known to be unable to match the actual heart rate, especially above about 120 bpm, because the pulse is smoothed out by the time it reaches the wrist capillaries.

Interestingly, fingertip pulse readers are the opposite and can keep perfect track with an ECG type chest band.

If I were Apple, I'd use the data from all those volunteers and look for a pattern that I could exploit statistically, such as a correlation between the wrist's movements during exercise, and a rising pulse rate that gets less accurate as it rises.

It might provide a way to guesstimate the heart rate better.

 

[Piggie]

Wrist based pulse rate readers are well known to be unable to match the actual heart rate, especially above about 120 bpm, because the pulse is smoothed out by the time it reaches the wrist capillaries.

Interestingly, fingertip pulse readers are the opposite and can keep perfect track with an ECG type chest band.

If I were Apple, I'd use the data from all those volunteers and look for a pattern that I could exploit statistically, such as a correlation between the wrist's movements during exercise, and a rising pulse rate that gets less accurate as it rises.

It might provide a way to guesstimate the heart rate better.

I am sure you are right.
Look, we know this "Wow amazing Apple health labs" has been shown off as PR and marketing for the new watch, otherwise why show it?
What you say it probably exactly right, they are simply gathering data, then whatever the watch "Can detect" they will fit to "best match" against data to try and come up with something useful to show you.

Again, I'm certain there will be BIG medical disclaimers in the terms and conditions hidden away.
Apple are not going to want to be liable if or rather when someone dies or ends up in a serious medical condition due to information the watch was showing them.

 

[8CoreWhore]

What in the world?

Ok, in trying to find out more info about various devices and their degree of accuracy, I watch this review of several being compared to an EKG machine but I am totally perplexed.

The Dr says a chest strap works the same way as an EKG machine, and I agree, but after this "test" I can't help but think something fishy is going on.

She casually jogs to get her heart rate up, but she is not breathing hard nor sweating but her heart rate is 180? How can this be? This whole video, the EKG shows her with a rate that would cause her to collapse if she's not an elite athlete, yet no sweating and no heavy breathing? There is something terribly wrong with this "test."

https://www.youtube.com/watch?v=uWnaqsLSl8c

 

[Piggie]

Ok, in trying to find out more info about various devices and their degree of accuracy, I watch this review of several being compared to an EKG machine but I am totally perplexed.

The Dr says a chest strap works the same way as an EKG machine, and I agree, but after this "test" I can't help but think something fishy is going on.

She casually jogs to get her heart rate up, but she is not breathing hard nor sweating but her heart rate is 180? How can this be? This whole video, the EKG shows her with a rate that would cause her to collapse if she's not an elite athlete, yet no sweating and no heavy breathing? There is something terribly wrong with this "test."

https://www.youtube.com/watch?v=uWnaqsLSl8c

Thank you very much for that video.

Quite funny that the Samsung Galaxy S5 was virtually band spot on accurate.

Let's hope the Apple watch, which is being sold heavily on it's health aspects and of course it's body/heart rate monitoring is as good as Samsung's phone shall we ?

Going to be interesting. I am suspecting everyone here is expecting it to be accurate being all the testing Apple have given it.

 

[Julien]

Wrist based pulse rate readers are well known to be unable to match the actual heart rate, especially above about 120 bpm, because the pulse is smoothed out by the time it reaches the wrist capillaries.

Interestingly, fingertip pulse readers are the opposite and can keep perfect track with an ECG type chest band.

If I were Apple, I'd use the data from all those volunteers and look for a pattern that I could exploit statistically, such as a correlation between the wrist's movements during exercise, and a rising pulse rate that gets less accurate as it rises.

It might provide a way to guesstimate the heart rate better.

Chest straps sensors are the gold standard for the fitness industry and can be very accurate. Fingertip worn sensors are bulky, require pressure and are not well suited for activities like running. I'm sure they could be made smaller but hard arm swinging in running would probably maker them impractical and inaccurate.

The question is how well will the Watch optical sensor compare to HR chest strap sensors. CNet did a fantastic comparison of several optical sensors a few months ago and found their performance poor. The CNet review can be found here.

Has Apple found a way to perfected the optical sensor to near chest strap accuracy? That is the multimillion dollar question (and Apple has spent the money). I'm skeptical knowing how bad optical is but I'm also very hopeful that Apple has pulled another Finger Print sensor type improvement (although they did acquire the Finger Print scanner tech).

If Apple hasn't perfected the optical sensor they should offer a chest strap as an option to serious athletes to make the Watch more acceptable. Of course you can use any BT strap like the Wahoo Tickr with the Watch.

 

[matrix07]

How about this..........

I have no doubt that it will be at least as good as other watches that use the same technique to record the data.

I'm also aware, as we all are that other devices will monitor for far more body data than the Apple watch can.

Also I'm aware there are going to be far more accurate ways of monitoring the body (chest strap?) but general public would not accept that, even though it would probably be more accurate.

If I'm being frank I have to say if Apple Watch is just as accurate as the competitions, it will be a bit disappointing already. I know that we all want a device that's more accurate. Some might not say it but everyone hopes it. It doesn't need to be a perfect accurate like a professional device but have to be accurate enough.. and should be more accurate than the competitions. If Apple achieve this I believe the majority of customers will be satisfied. Less than this... well..

 

[8CoreWhore]

I have to say, Apple's sensors are HUGE compared to the competitors' optical sensors, so I think these sensors and quality software will likely be as good as optical sensing on an arm can be - which then begs the question; What is the most accurate, theoretically, a HRM watch can be? And how much does it vary from person to person?

Let's say it's accurate enough to know you are in your "fat burning zone" which is a range, fine, but would it be accurate enough to look at your resting heart rate in the morning and suggest you take a rest day because you are over-training or under-recovered? If not, and that is important to you, you'd need a chest strap.

For some, monitoring heart rate is part of their medical regimen, and they will need to be more concerned about the claims of these optical readers.

I've decided that if it is not within 95% accurate 95% of the time, I'll get a Polar H7 Bluetooth chest strap/sensor that works with iPhone and many apps, for $52 on Amazon.

 

[01silver4]

I have to say, Apple's sensors are HUGE compared to the competitors' optical sensors, so I think these sensors and quality software will likely be as good as optical sensing on an arm can be - which then begs the question; What is the most accurate, theoretically, a HRM watch can be? And how much does it vary from person to person?

Let's say it's accurate enough to know you are in your "fat burning zone" which is a range, fine, but would it be accurate enough to look at your resting heart rate in the morning and suggest you take a rest day because you are over-training or under-recovered? If not, and that is important to you, you'd need a chest strap.

For some, monitoring heart rate is part of their medical regimen, and they will need to be more concerned about the claims of these optical readers.

I've decided that if it is not within 95% accurate 95% of the time, I'll get a Polar H7 Bluetooth chest strap/sensor that works with iPhone and many apps, for $52 on Amazon.

I am going to compare the AW with my GARMIN chest strap HR monitor, when I am out cycling to see how accurate it is.

 

[WilliamG]

A question for all those who say it is a photoshop publicity shot (which it could very well be). Why would Apple show a pic with info that would be wrong? If Apple is going to show an hour of running activity they should make the calorie count more plausible and in line with the activity.

It seems that Jay Blahnik would have a lot of input in making sure any simulated fitness related info published was as accurate as possible.

Not necessarily. Fitness data is quite private, so Apple may not want to elicit any lawsuits from those involved in helping making the Apple Watch what it is today.

 

[Piggie]

I am going to compare the AW with my GARMIN chest strap HR monitor, when I am out cycling to see how accurate it is.

Thanks.

Most people won't have chest strap monitors, so I'm sure we will all be very keen to hear yours and others results from real world tests.

It would be lovely if all these body monitoring things were 100% accurate.

I will admit to being concerned as to how inaccurate many of them seem to be.

To be brutally honest. I think there should be some consumer sales law put into place ASAP to make sure any body monitoring device is within a certain percentage of accuracy in the same way that say a car speedometer has to be accurate within certain limits.

Don't you agree?

We can't have people buy a device, relying on the data when the data is wildly inaccurate.

 

[01silver4]

Thanks.

Most people won't have chest strap monitors, so I'm sure we will all be very keen to hear yours and others results from real world tests.

It would be lovely if all these body monitoring things were 100% accurate.

I will admit to being concerned as to how inaccurate many of them seem to be.

To be brutally honest. I think there should be some consumer sales law put into place ASAP to make sure any body monitoring device is within a certain percentage of accuracy in the same way that say a car speedometer has to be accurate within certain limits.

Don't you agree?

We can't have people buy a device, relying on the data when the data is wildly inaccurate.

I totally agree with you on this matter for sure. One of the reasons why I went with chest strap monitor, is for accuracy and because u know it's widely use by most pro cycling teams as well as other athletes, for training purposes. So there doctors trainer etc can better guide there training etc. With data they get back from it. I am very curious to see how the AW will perform, seeing that one of the most important part of its features is gear to fitness. And the fact that Apple spent 3years developing it. who knows they could be the first to have a watch that is able to give great accuracy when it comes on to HR etc. But only time will tell, so I am eagerly to put it to the test.

 

[01silver4]

Chest straps sensors are the gold standard for the fitness industry and can be very accurate. Fingertip worn sensors are bulky, require pressure and are not well suited for activities like running. I'm sure they could be made smaller but hard arm swinging in running would probably maker them impractical and inaccurate.

The question is how well will the Watch optical sensor compare to HR chest strap sensors. CNet did a fantastic comparison of several optical sensors a few months ago and found their performance poor. The CNet review can be found here.

Has Apple found a way to perfected the optical sensor to near chest strap accuracy? That is the multimillion dollar question (and Apple has spent the money). I'm skeptical knowing how bad optical is but I'm also very hopeful that Apple has pulled another Finger Print sensor type improvement (although they did acquire the Finger Print scanner tech).

If Apple hasn't perfected the optical sensor they should offer a chest strap as an option to serious athletes to make the Watch more acceptable. Of course you can use any BT strap like the Wahoo Tickr with the Watch.

Well after watching this video, if Samsung can be that accurate. I don't see why Apple woulndnt be able to do the same or better. I will test my AW when I get it, to see how accurate it is compared to my GARMIN chest sensors. So we shall see for sure stay tuned.

 

[Piggie]

Well after watching this video, if Samsung can be that accurate. I don't see why Apple woulndnt be able to do the same or better. I will test my AW when I get it, to see how accurate it is compared to my GARMIN chest sensors. So we shall see for sure stay tuned.

100% agree.

Let's be honest here, probably virtually everyone here dismissed the heart rate sensor on the rear of the Samsung Galaxy S5 as just a sales gimmick not worth bothering with, so it's a nice surprise to find out it's actually very accurate. the best of the devices she tested.

Given that this is, just a nice little extra added to a mobile phone, that's over a year old now, and the Apple watch is so new it's not even out.

AND the heart rate monitoring on the Apple watch is not just a "nice little extra" but a major key feature that's on the list of key points for the watch and all tied into the health aspect.

Also given the fact, Apple was suspected to have stripped away all the other monitoring aspects are they were not 100% leaving just this one.

I'm sure we are all expecting this remaining key feature to be dam good.

Finger crossed it will be excellent, then at least we will know, in the future when Apple start to add in more sensors, they also will be up the same level of accuracy the heart rate one, we hope, is.

 

[Julien]

Well after watching this video, if Samsung can be that accurate. I don't see why Apple woulndnt be able to do the same or better. I will test my AW when I get it, to see how accurate it is compared to my GARMIN chest sensors. So we shall see for sure stay tuned.

Samsung is using a stand nearly still, carefully and precisely compress your finger on (and not above or off in any manor) the sensor, then don't move while it checks method.

If you did this with the Watch it would no doubt be as accurate. The challenge is for Apple to read accurately while you are in the full dynamics of running with the Watch moving about and not compressed to your wrist.

I will also be testing in a variety of HR patterns. I plan on starting with a 10 mile run with an average of 80% to 90% maximal HR. Then try some interval training to near maximal HR (always good to have an reason to push the old ticker to the max). Then some low level cardio in the 40% to 75% (up and down) of maximal HR range.

Todays 11 miler would have been a decent stress test for the Watch.

 

[8CoreWhore]

Well after watching this video, if Samsung can be that accurate. I don't see why Apple woulndnt be able to do the same or better. I will test my AW when I get it, to see how accurate it is compared to my GARMIN chest sensors. So we shall see for sure stay tuned.

BTW, iPhone does the same thing as that Samsung. Use an app called 24/7 by MotionX. You put your index finger over the camera lens and flash - it lights up, you can see your finger turn red from the blood, and the camera views the pulse. The software gives you the rate. I haven't compared it to anything so I don't know how accurate it is, but that's how the Samsung in that video does it. I wonder if it was some shady Samsung tool of a Dr doing that. No way was her heart rate 190 from a casual jog with her not being out of breath or sweating. Total BS.

 

[Julien]

...I wonder if it was some shady Samsung tool of a Dr doing that. No way was her heart rate 190 from a casual jog with her not being out of breath or sweating. Total BS.

It could very well be that the tests were done one day and the video shot the next. It is often done this way in the fitness industry.

 

[01silver4]

BTW, iPhone does the same thing as that Samsung. Use an app called 24/7 by MotionX. You put your index finger over the camera lens and flash - it lights up, you can see your finger turn red from the blood, and the camera views the pulse. The software gives you the rate. I haven't compared it to anything so I don't know how accurate it is, but that's how the Samsung in that video does it. I wonder if it was some shady Samsung tool of a Dr doing that. No way was her heart rate 190 from a casual jog with her not being out of breath or sweating. Total BS.

Agree at 190bpm she should be breathing really heavy. But my focus now is get the AW to put it to the test. Tell u what though I wouldn't be surprised if the AW HR monitor is as accurate as the GARMIN HR monitor. To be honest I am anticipating that the Apple watch will be as accurate. If not apple would have waisted three years of developing this thing.

 

[kdarling]

Well after watching this video, if Samsung can be that accurate. I don't see why Apple woulndnt be able to do the same or better.

Again, the major reason why the Samsung was accurate, is because its sensor is reading from a fingertip.

Wrist based sensors, including ones from Samsung, are nowhere near as accurate.