Garmin beaten by WHOOP: Study shows 99% accuracy

[caption id="attachment_68219" align="aligncenter" width="300"] Don't do this at home.[/caption] Is WHOOP Accurate? Let's see what science says... Study Concludes WHOOP has the best HR & HRV accuracy Hey, don't bite me. I didn't do the study I'm just reporting it. Study Results: mdpi.com The study attempts to answer the oft-mooted question, "Is WHOOP Accurate?" This research by the Central Queensland University is published in a peer-reviewed journal called SENSORS and it looked at the accuracy of 6 wearables which estimated HR, HRV and sleep (sleep stages and duration). Importantly the study was independently funded by the Australian Institute of Sport and supported by WHOOP. It compared a II-lead ECG / polysomnography with the accuracy of 5 well-known wearables, from WHOOP, Garmin, Oura, Apple, Polar and also a head-worn wearable called Somfit. Findings The study found that WHOOP 3.0 had a higher degree of accuracy than all the other wearables when it came to measuring HR and HRV. WHOOP proved to be over 99% accurate when measuring both HR & HRV. Although readers should take note that the study was based on low levels of HR activity whilst sleeping. Must Read: Detailed WHOOP Review Problems with the study This is a sleep study. Don't confuse it with a study showing WHOOP's accuracy in sports compared to these other devices. (Though actually, WHOOP is more than sufficiently accurate for sports when worn on the bicep as I and other reviewers have found) All 6 devices were worn simultaneously by each participant. Not forgetting the polysomnography / ECG, that's the WHOOP 4.0 band plus 3 watches, a ring and a forehead-based wearable. I don't know about you but I've only got two wrists. So only two of the devices can have been worn properly. Furthermore was there any favouritism in allowing WHOOP to be worn on the bicep where better results WILL be obtained? or were other devices worn on the inner side of the forearm? Old devices were used. Oura 3 has improved hardware so Oura 2 is a bad choice. I also understand Oura 3.0 has a new, verified sleep stage algorithm but it is not yet live. Vantage V lacks the 2nd generation Precision Prime sensor in the Vantage V2. So that is a bad choice. Apple Watch 6 is a good choice and does have the latest generation optical HR sensor. However, the study uses the Sleep Watch app and the authors note "the exact period of sampling used by Sleep Watch was not specified". that's because the authors probably don't understand how Apple Watch works and did not enable AF History in order for the Apple Watch to regularly record HRV. Most likely, the SleepWatch app only had 3 or 4 HRV readings, per participant per night! Oh dear. [The study results were submitted on 29 July 2022 and hence could not have used the latest version of Watch OS which is the only version that produces frequent HRV measurements via AFib History] Even if the SleepWatch app somehow got more frequent HRV estimates it would be using a proprietary algorithm and not Apple's Garmin 245 has the previous generation of optical HR sensors. WHOOP 3.0 is the older generation sensor. More detailed data was provided by WHOOP to the researchers (this is not unusual they have done that for me too, however, it's data that you won't necessarily see in normal use) The study used a sleep lab which includes polysomnography. That is about as state-of-the-art as you need to get. However, only one technician was used to score the results. This might sound like I am being pedantic yet one of the several drawbacks of polysomnography is that different technicians WILL score (identify) sleep stages with differences of up to 20%. But using one technician is reasonable. Conflicts of interest are reported thus "GD Roach., C. Sargent, and DJ Miller are members of a research group at Central Queensland University (i.e., The Sleep Lab) that receives support for research (i.e., funding, equipment) from WHOOP Inc. However, WHOOP was not involved in the design, conduct or reporting of this study." Note these release dates of current/later-gen models: WHOOP 4.0 was released in September 2021, Oura 3.0 in November 2021, Forerunner 255 in June 2022, Polar Vantage V2 in October 2020 and the Apple Watch 7 in September 2021 (Watch 8 in September 2022). So, realistically, at the time when the study was conducted perhaps only the Vantage V2 would have been available to use. Thoughts This is a sleep study that only compares previous-generation sensors & algorithms. We can't say anything about WHOOP 4.0's accuracy based on this study as an older model was used. However, in one sense this study is indicative of WHOOP's accuracy given the right circumstances. Sleep is one of those circumstances. I'm reasonably confident that the newest models of Polar, Garmin, WHOOP and Oura would all yield improved results. And I'm positive that enabling AFib on Watch OS 9 would give massively different HRV/rMSSD results. Furthermore, my own findings are that WHOOP is more than sufficiently accurate for athletes when worn on the biceps. Even so, my own sleep studies with the latest tech (N=1) show that WHOOP still consistently beats Garmin. But I know other researchers/reviewers who get different results with similar tech hence the need for larger sample sizes. [caption id="attachment_74182" align="aligncenter" width="1024"] Correlation of HRV Readings to Polar H10/HRV4Training - day-to-day (Column 1) and baseline (Column 2) correlations. Garmin 955, Oura 2 and WHOOP 4 used.[/caption] As well as reminding readers not to extrapolate accuracy with resting readings to the accuracy of readings from workouts, remember also that no wearable can use heart rate data to correctly measure the strain of a strength-type workout. Heart rate just isn't a good measure of that even if it is captured accurately. However, this study DOES validate the accuracy of WHOOP's measurement of your response to such strength workouts, albeit with the previous generation of sensor. That said, people at WHOOP will be super-happy with the results. The comparisons of WHOOP to a II-lead ECG/polysomnograph give as good a comparison to 'correctness' as you will easily get.