Series 6 Blood Oxygen / Pulse Ox Has Awful Accuracy

[Glene]

I came across this from Houston Methodist Hospital online:

As a good rule of thumb, a person with COVID-19 monitoring his or her clinical status at home will want to ensure that the SpO2 reading stays consistently at or above 90 to 92%. If the number consistently drops below this threshold, timely medical evaluation is warranted.

 

[Omenamies]

Yep, Twitter is full of morons claiming it's unrealiable:

"First they made unreliable EKG (does not come even 100 miles close to Kardia accuracy), now even more unreliable and untested spO2. Apple just give up and stay out of health unless you do a lot of research and have top scientists."

 

[Rogifan]

The Health app says my average for the past week is 93%. Though when I’ve triggered it on my own with the band higher up on my wrist I get 98%. I’m guessing most people don’t wear their Watch as tight as it needs to be to get a good reading. I know the Watch isn’t a medical grade device but there’s no point to these health sensors if they don’t at least have some level of accuracy. If you have to wear the Watch tighter than you normally would (or than feels comfortable) or higher up on the wrist than normal I’m not sure how useful this sensor is.

 

[zubikov]

The Health app says my average for the past week is 93%. Though when I’ve triggered it on my own with the band higher up on my wrist I get 98%. I’m guessing most people don’t wear their Watch as tight as it needs to be to get a good reading. I know the Watch isn’t a medical grade device but there’s no point to these health sensors if they don’t at least have some level of accuracy. If you have to wear the Watch tighter than you normally would (or than feels comfortable) or higher up on the wrist than normal I’m not sure how useful this sensor is.

That's been my experience also so far. It seems from this thread that a majority of users are getting consistent readouts, and that's great. Perhaps there are other elements at play here outside of our control like skin thickness, color, hair, amount of perspiration, how deep veins are inside or whether they're obstructed, etc.

 

[Rogifan]

That's been my experience also so far. It seems from this thread that a majority of users are getting consistent readouts, and that's great. Perhaps there are other elements at play here outside of our control like skin thickness, color, hair, amount of perspiration, how deep veins are inside or whether they're obstructed, etc.

The Health app says mine dropped to 84% one night between 2-3 am. I don’t believe that at all. Last night it showed 100%, then dropping to 87% an hour later, then going back up to 97%. I’m going to ignore it for now because there’s too much variability. Joanna Stern at the WSJ wrote about this. I don’t remember Jeff Willams saying this had the same FDA clearance as the ECG had so maybe it’s just not ready for prime time yet but Apple needed something to add to this years Watch. And it seems Apple has decided there has to be a new Watch every year.

 

[Glene]

I’d be interested to know how many people that are getting inaccurate readings have the solo loop or the braided solo loop, I don’t think you can get those bands tigut enough to get an accurate reading, like you can with the sport band.

 

[Smoothie]

I’d be interested to know how many people that are getting inaccurate readings have the solo loop or the braided solo loop, I don’t think you can get those bands tigut enough to get an accurate reading, like you can with the sport band.

I have the sport band fitted snugly and the blood oxygen readings were sometimes as much as nine percent lower than a fingertip pulse oximeter used at the same time. Sometimes the readings would match, but the variability was too much for me to find it reliable. I've turned off the blood oxygen sensor.

 

[ssong]

for those using FDA clearance as the golden standard, I’d argue that given the current climate, Apple may have elected to not go for FDA approval as it would require similar clearances in other international markets.

It’s also worth noting that there are two studies being explicitly powered by the new sensor on the Apple Watch

Heart failure detection study

Flu and COVID-19 early detection study


All in all, it would be a useful tool as a general long-term monitoring tool to detect any downward trends or consistent dips. The studies above would normalise the data for any outliers and most likely make use of the trend data they provide, rather than the spot checks that the average users seem to focus on right now.

 

[Apple_Robert]

Just remember that any band that has the word “solo“ in it, may be problematic with blood ox, they simply can’t get tight enough.

The Solo band is more than capable of being tight enough for the SPO2 testing. Not sure why you say it can’t get tight enough.

 

[JBaby]

The Solo band is more than capable of being tight enough for the SPO2 testing. Not sure why you say it can’t get tight enough.

It depends on the range. If your on the side where the band fits looser it may not work. You’d need to go down a size. But that won’t work if you’re already on size 1.

 

[Apple_Robert]

It depends on the range. If your on the side where the band fits looser it may not work. You’d need to go down a size. But that won’t work if you’re already on size 1.

Granted a lot of people have had correct sizing problems with the Solo loop. However, that doesn’t negate the fact that is can get tight enough to produce a blood oxygen test. The Solo loop is akin to the sports loop.

I also think there is a lot of user error with the manual test. My readings has been consistent from day one.

 

[JBaby]

Granted a lot of people have had correct sizing problems with the Solo loop. However, that doesn’t negate the fact that is can get tight enough to produce a blood oxygen test. The Solo loop is akin to the sports loop.

I also think there is a lot of user error with the manual test. My readings has been consistent from day one.

Now I can agree with that. I was just stating a fringe case. I also think people are doing it wrong. People did ECG wrong too. At least the people on Reddit did.

 

[G5isAlive]

I don’t expect a $15 oximeter from Amazon to be more precise and accurate.

Why not? The oximeter from Amazon is single-purpose. The Apple Watch has many many many uses, even tells time. Does your $15 amazon oximeter do even that simple thing? No? Gosh I would expect anything with a screen and chip to tell time!

The oximeter from Amazon is much much much larger (sensor wise). So yep I would expect it to be better. The oximeter from Amazon also has much better placement for consistency (fingertip) without as much movement going around (respective to the fingertip) and also closer to blood (useful for measuring blood). I would expect it to be better.

Apple Watch: Multi-purpose, miniaturized, in a difficult place to measure for a variety of reasons.... yep, I would hope the oximeter from Amazon would be better

 

[G5isAlive]

Tried with a couple of different adjustable straps...huge difference. The tighter the fit, the more consistent were the readings. Hoping the folks at Apple tweak the algorithm over time for more consistent readings.

Well you seemed to sort our your issues expressed in your original post regarding precision. Good. I don't think this is a software issue however. It's more an issue of trying to take a series of measurements in a difficult place, though several layers of skin, hair, to pick up a pulse (it computes the wavelength differences by noting the change between heartbeats, the pulse of blood with each heart beat). It's a lot like trying to hit a moving target while moving in the opposite direct at the same time, with various winds blowing you around. That it works at all impresses me, the fact that I need to average readings to get accuracy doesn't bother or surprise me. As has been pointed out, it's more for observing trends, and if you have a concern, measure again. Average. Even dedicated devices have a fairly broad precision range due to the technical difficulties. And yes, wear it securely to your wrist

 

[jdb8167]

Just remember that any band that has the word “solo“ in it, may be problematic with blood ox, they simply can’t get tight enough.

I found that to get a reliable reading, I’ve had to push my Solo Loop up my forearm a bit. This is something I did not have to do with the Sport Loop or the Sport Bands. It isn’t awful but I didn’t have any failures until I started wearing the Solo Loop which I like a lot and will continue wearing anyway.

 

[davidwes]

The series 6 blood oxygen readout is so inaccurate, it may just do more harm than good. Sudden blood oxygen level drops have devastating and potentially fatal impacts on human health. These levels are also on a scale where a few percent variability could mean the difference between normal function, COPD, heavy smoking and critically low lung function due to COVID.

Apple jammed through this update seemingly without giving false measurements much consideration. I just did four measurements on my wrist within a couple of mins:

#1 - 89% - omg, am I hypoxic? Do I have COPD?
#2 - 98% - whew, I'm in the clear
#3 - 100% - wow, go me!
#4 - 94% - i give up

I have no tattoos, don't smoke, I'm in decent shape and have thankfully never been diagnosed with pulmonary or heart issues before. It's just so unreliable that it defeats the purpose of having such a sensor. Apple will quickly confuse their customers with noise, lose credibility and render the blood oxygen function useless.

Anyone else seeing crazy levels of variability like this??

Can I ask how you know those readings are wrong?

 

[jdb8167]

Can I ask how you know those readings are wrong?

When invoked manually I was getting failures, not inaccurate readings. I’ve also seen as low as 84% overnight but only for a single reading and not every night. That is almost certainly inaccuracies from moving in my sleep or because the watch band wasn’t tight enough.

 

[rillrill]

I returned mine too. Also had annoyances trying to get an actual good reading plus you can buy one of these for like $30. Picked up an s5 cheaper.

 

[cdcastillo]

The series 6 blood oxygen readout is so inaccurate, it may just do more harm than good...
Anyone else seeing crazy levels of variability like this??

Not here. I've compared it to 3 different pulse oximeters, and they are within 1%-2% of what the watch measures. It is absolutely normal to have a fluctuation in saturation during minutes, it is related to physical activity.

 

[Woodcrest64]

I have a Fitbit charge 3 and since I just got my first Apple Watch yesterday I moved the Fitbit to my opposite wrist as I wanted to compare what each provided. My heart rate seems to be the same on both. I do think it depends on how you wear the watch (too loose, too tight, too high on wrist)

Don’t have anything to compare oxygen levels to.

How do you like the Apple Watch vs your Fitbit Charge 3? I ask, as I currently have the Charge 2 and have been holding out on getting the Apple Watch. This is primarily because I'm so use to the 4-5 day battery life of the Fitbit Charge series.

 

[cdcastillo]

When invoked manually I was getting failures, not inaccurate readings. I’ve also seen as low as 84% overnight but only for a single reading and not every night. That is almost certainly inaccuracies from moving in my sleep or because the watch band wasn’t tight enough.

Or you could have dips in oxygen saturation due to a non yet diagnosed sleep problem.

 

[Pink2]

How do you like the Apple Watch vs your Fitbit Charge 3? I ask, as I currently have the Charge 2 and have been holding out on getting the Apple Watch. This is primarily because I'm so use to the 4-5 day battery life of the Fitbit Charge series.

I love it. I think the things that it captures (steps, floors, activity) is slightly more accurate on the watch as the fit is better for me on the watch (the charge band I had didn't keep it in the same place on my wrist and sometimes would count 10 floors for odd activities, I am not a very active person). I use Auto Sleep and Sleep++ for sleep tracking and it is different than what Fitbit shows, but is close and having the ability to actually respond to text messages, answer calls, and all the other features the watch provides that the charge didn't made it worth it to me. It does require to be charged more often, but the watch charges a lot faster and I just do it while watching tv before bed (love the reminders for that). Hope I answered your questions.

 

[canyonblue737]

I'm just not experiencing this at all. My S6 gives me readings between 96-100, 10-20+ times a day, and in just over one week of use ONE time has had an 88 reading in the middle of the night. The rest were again 96-100. I think accuracy has been fantastic for me.

 

[jdb8167]

Or you could have dips in oxygen saturation due to a non yet diagnosed sleep problem.

Except that over 4 nights it only went below 94% twice and the next measurement was back to normal. So I’m going with a couple of inaccurate readings.

 

[Easttime]

Why not? The oximeter from Amazon is single-purpose. The Apple Watch has many many many uses, even tells time. Does your $15 amazon oximeter do even that simple thing? No? Gosh I would expect anything with a screen and chip to tell time!

The oximeter from Amazon is much much much larger (sensor wise). So yep I would expect it to be better. The oximeter from Amazon also has much better placement for consistency (fingertip) without as much movement going around (respective to the fingertip) and also closer to blood (useful for measuring blood). I would expect it to be better.

Apple Watch: Multi-purpose, miniaturized, in a difficult place to measure for a variety of reasons.... yep, I would hope the oximeter from Amazon would be better

Check out the scientific literature. Try a PubMed search.