Heart rate pacing devices - accuracy and responsiveness

[New] Published research on ‘wearables’ confirms questionable accuracy

The watch error rate during activity was, on average, 30% higher than during rest. Differences exist between devices in responding to changes in activity, with variable findings.

“Clinicians should be aware of these biases in HR measurements during exercise when making clinical assessments based on HR data from wearable devices. This could affect exercise interventions that are based off HR feedback; accordingly, clinicians may examine alternative ways of measuring exercise intensity if measurement specificity is critical.”

~~
Findings:

The error of consumer wearables was an average of 7.2 ± 5.4 bpm.

~ During physical activity, it was 10.2 ± 7.5 bpm. Apple Watch 4 had the lowest error (4.4 bpm).
~ At rest, error was highest for the Fitbit Charge 2 and lowest for Apple Watch 4 (7.3 and 2.7 bpm). During physical activity, the Apple Watch 4 had the lowest error (4.6 bpm)

~ The Garmin was most variable in accuracy, and the Apple Watch 4 least (as SD, 9.2 and 3.0, respectively).

~ At rest, missing data was highest for the Fitbit Charge 2 and lowest for the Apple Watch 4 (18.7 and 2.7%).
~ During physical activity missing data was highest for the Fitbit Charge 2 and unchanged for the Apple Watch

~ The rhythmic movement of walking has significantly higher errors in all devices except the Apple Watch 4
~ Typing caused errors in all devices.
~ Walking (and other activity) tended to cause reported HR to be higher than true HR, whereas typing caused the reported HR to be lower than the true HR

The research excluded a possible cause of inaccuracy being the delay between the actual heart beat and the change in blood volume at wrist, as removing lag did not improve the accuracy.

In general, devices with higher cost, a more recent release date, and a larger market had higher accuracy. [However, still not as a high as for chest straps].

~~
Study Info:

Error in wrist-worn devices is usually due to motion artifact, which is typically caused by displacement of the PPG sensor over the skin, changes in skin deformation, blood flow dynamics, and ambient temperature. Motion artifacts may manifest as missing or false beats which result in incorrect HR calculations. Data loss occurs in watches occurs when a device is not making contact with the skin, or when the device removes data that fails internal quality control, (when there is a large motion artifact indicated by high accelerometry sensor values), it removes the data points potentially affected by the artifact).

Each round of the study protocol, included
~ seated rest to measure baseline (4 min)
~ paced deep breathing (1 min)
~ physical activity (walking to increase HR up to 50% of the recommended maximum (5 min)
~ seated rest (washout from physical activity) (~2 min), and
~ a typing task (1 min).

Round 1: Empatica E4 + Apple Watch 4
Round 2: Fitbit Charge 2
Round 3: Garmin Vivosmart 3, Xiaomi Miband, and Biovotion Everion.

The electrocardiogram (ECG) patch was worn during all three rounds.

Skin tone did not correlate with accuracy.

Assessed by mean directional error (MDE, ±standard deviation (SD)) and the mean absolute error (MAE)

https://www.nature.com/articles/s41746-020-0226-6

This study had a very high quality and comprehensive design. Previous research is of varied quality and design, ‘accuracy’ can be defined very loosely (allowed error margins of 5%) or with vague metrics, or minimal sampling (once per minute) is used.

One study (running on treadmill) found the Polar H7 Chest Strap had the highest correlation with the ECG (98). This was followed by the Apple Watch III (96). The Fitbit Iconic, Garmin Vivosmart HR, and Tom Tom Spark 3 all had the same level of agreement with the ECG (89).
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6732081/

~~
For more information on pacing with a heart rate monitor, visit https://www.facebook.com/groups/1511910682235160/

We recommend using a polar chest strap. As the gold standard, it is often used as the comparison measure for device accuracy testing. Research for the device is found here: https://www.ncbi.nlm.nih.gov/m/pubmed/31004219/
Click to expand...
 
Comments
https://www.ncbi.nlm.nih.gov/m/pubmed/31004219/ "A simple chest strap such as the Polar H10 might be recommended as the gold standard for RR interval assessments if intense activities with strong body movements are investigated" A quick skim of this actually puts the H10 in superior position vs the ambulatory ECG people are lent by hospitals.
Click to expand...

This post is <extremely> important. The subjective perception of these issues is minimised by wearers who compare sedentary/supine readings from a wrist device vs an ECG. In contrast, use while active with any degree of motion affects both lag-indeoendent accuracy and responsiveness. In turn, this is often not noticed due to wearers or studies comparing averages that mask spikes, that must be caught when estimating anaerobic threshold level via HR BPM for the purpose of illness management. The analogy would be having car brakes that are worn and insufficiently responsive. Better than no brakes, but working brakes are prudent. A major wearable manufacturer also faced likely litigation for exactly these wrist device limitations - my memory is that the action was halted post plaintiff organisation, but before formal action. This is about taking Workwell etc findings and operationalizing them for management, using an ME-science relevant benchmark, an educated guesstimate of one's anaerobic threshold proxied by HR BPM. Poor wrist device, garbage in, garbage out. Hence chest straps.
Click to expand...
 
I did look into this a few years ago, I do own a H7 chest strap.

I have done 'direct' comparisons between the H7's readings and a few (3) other devices.

My reading, and experience, suggests that whilst these inaccuracies are very real they mainly affect readings taken during intense.exercise.

A difference of 5-10bpm at 80bpm is less significant than an error of 30-40bpm, with a lag of minutes, at 170bpm, especially to someone in active training.
 
I very rarely use it, the last time was probably over a year ago when comparing smart watch HR monitors to decide which to keep (actually more complicated than that but that's the gist).

It probably won't now be used until August/September as I noticed odd readings on my vivoactive 3 (very low HR, staying in the mid 30s for minutes) whilst I was on holiday, so I want to see if they happen again, in the same situation, and confirm that they are actually happening using the chest strap.
 
InfiniteRubix said:
Did you discuss with a medic?
Click to expand...
No, my old GP was useless for anything she couldn't see there and then, and even if she had looked at the logs they are in 10 minute increments and only show it going down into the 40s (it not having stayed in the 30s for longer than 10 minutes).

I also haven't really been able to get out since getting back, I have other more important things to discuss with a new GP as and when I am up to writing/seeing her - it's not a priority, and as I'm not currently wearing the HR watch apart from when outside for all I know it's stopped happening.

I did google it, the conclusion I reached was that provided it's not causing 'symptoms' (how would I tell that lol) then not to worry about it.
 
The problem I've had with optical wrist monitors is dropouts.

But if I want to measure HR, I just use my father's old Garmin ECG type monitor. The ECG straps have their own quirks, specifically they need some moisture for conductivity. If you're an athlete, this is provided free of charge (sweat), and so sedentary users would need to use ECG gel under the strap.
 
Snow Leopard said:
The problem I've had with optical wrist monitors is dropouts.

But if I want to measure HR, I just use my father's old Garmin ECG type monitor. The ECG straps have their own quirks, specifically they need some moisture for conductivity. If you're an athlete, this is provided free of charge (sweat), and so sedentary users would need to use ECG gel under the strap.
Click to expand...

Gel's not needed anymore with the H10, for example. Much of the tech development is not just in the actual electronics, it's in the materials technology. The strap is a conductive silicon blend that just works, which makes it more expensive to replace when it wears.

Can't remember the last time I even wet my finger to wipe under it, let alone gel. I used to at the start, year ago plus, then realised it didn't need it. It basically just works. Unless I'm super greasy!
 
I’ve very occasionally (x3 in 6 weeks) had some 0 readings from my Polar H10, which I’ve generally found excellent. As I’m still here, think it comes from when I twist or some posture that occasionally breaks the contact. Otherwise v helpful especially with an app that has an alarm for high HR.
 
Sounds like twisting with a strap that could be tighter, or the strap has aged and needs replacing. 24*7 use needs replacing every 6 months in my experience

V good docs and support re apps in that FB group BTW
 
You must log in or register to reply here.
Back
Top