Garmin Running Power :: Garbage In…Garbage Out?

I quite like the Garmin Running Power app. It installs well, is nicely configurable and seems to run awesomely in all its bug-free glory.

But is it showing complete garbage?

RunPow STRYD RunScribe Comparison Review Features
.

I had a little play on Sunday, comparing Garmin Running Power (GRP) with RunScribe and STRYD. Here are the thoughts and results of that day’s runs, where I looked a little at VO and GCT.

Knowing that the Vertical Oscillation (VO) and Ground Contact Time (GCT) are inputs to the power model, my takeaway thought from Sunday was “BUT what if the sources of VO/GCT were either wrong or inconsistent or inconsistent between source devices?” For example, I used the HRM-TRI as a source of GCT and VO but what if, instead, I had used an older, compatible HRM-RUN or an RD-POD? The HRM-RUN might have the same firmware but I doubt the RD-POD would produce the same values for GCT and VO due to its position either on the foot or on a waist belt.

Today I also thought “What if I use STRYD as a source of pace?” and “What if I use a Garmin footpod as a source of pace?” and “What if I use GPS only as a source of pace?” and “What if I use GPS+GLONASS as a source of pace?” and “What if I used the Milestone POD as a source of pace?” and “What if I used a Fenix 5x’s GPS as a source of pace rather than the 935’s GPS?”

Anyway. You can hopefully see where I am going with this. I’ll just cut to the chase and tell you what I did as I don’t quite have access to all that kit.

What I Did

I used a variety of different sources of pace to see how the different accuracies of the pace sources affected the GRP estimate of power around a running track in windless conditions.

After a 15-20 minute warmup (also a GPS warmup) I did one mile at about 5:00/km and then one km at about 4:00/km. I had a break between those intervals to check/swap pairings and sources of pace data. I then changed the PACE SOURCE and repeated the 1mile+1km.

What I expected

Previously I have found Garmin’s pace accuracy is repeatedly not great compared to other brands. Maybe you have found otherwise? The 935 is, however, one of the better Garmins, IMHO.

Contrary to the received wisdom, I find GLONASS does NOT help with running pace accuracy in ‘open sky’ conditions like today on the track. Maybe you have found otherwise?. So I expected a notable difference in the resultant POWER reading when using a footpod as a source of distance/pace. Not necessarily better…just different.

I find STRYD a good source of pace and distance even when not calibrated. Others find the same. And I find the Garmin pod is alright for pace/distance when calibrated (today it was not calibrated…it didn’t auto-calibrate on for some reason…sorry!)

I expect a 935+STRYD will produce a better power figure than 935+GPS and I suspect 935+GLONASS will be worse still. I expect the 935+footpod to be closer to the power from 935+STRYD. ‘Better’ could mean that the power number is higher or lower, I suppose, I don’t know the maths. I would guess that a 935+STRYD-based pace figure would have less variability and would result in a lower power number...as I say, a guess.

Kit Combos I actually used

  1. Garmin Forerunner 235 + RunScribe
  2. Suunto Spartan Sport + STRYD (STRYD also caches the data)
  3. Garmin Forerunner 935 + Running Power App
    • with Garmin Footpod (uncalibrated)
    • with STRYD (uncalibrated)
    • with GPS only
    • with GPS+GLONASS
RunPow STRYD RunScribe Comparison Review Features
.

What I didn’t do

Science

These findings might prompt someone into a more rigorous analysis.

Just imagine that I believe in the Flat Earth, fairies and Jedi Knights and take my findings with a pinch of salt. Maybe a larger pinch than you might normally take, but a pinch nevertheless.

Heads Up

STRYD data on the Suunto SPARTAN is the control data for each of the 4 combinations of GRP

I’m NOT saying STRYD is right but in my experience it is repeatably consistent for me and if it can’t be repeatable on a track with no wind then….eeesh.

So, what we are looking out for is the variance to the STRYD power on the 4:00/km kilometre and there will be a bonus on the 5:00/km mile section before that if it shows something interesting.

RunScribe power is in beta. So I won’t discuss that much.

Also note that on the 5:00/km bit my mind may have wandered off with the aforesaid fairies somewhat. And my pace may well have varied accordingly 😉 I’ve not yet looked at the data as I write this but, from memory, I expect the pace was 3:50-4:05/km apart from the first 100m of each of the faster kms as I was a bit more excited at that point. That’s not to say how rubbish or wonderful I am just saying that it was consistent enough for the purposes of this post, in my opinion.

Result 1: GRP Pace Source: Garmin Footpod (uncalibrated)

Note the colours of the lines might refer to different devices in subsequent charts…sorry.

Garmin-Running-Power-Garmin-Footpod

 

Result 2: GRP Pace Source: STRYD Footpod

Note it seems that the 935 took 2 minutes to connect to STRYD. My bad, probably. It never normally does that. I ‘ll just use the section where STRYD pace is recorded.

Garmin-Running-Power-STRYD

Result 3: GRP Pace Source GPS

Garmin-Running-Power-GPS

Result 4: GRP Pace Source GPS+GLONASS

Garmin-Running-Power-GPS-GLONASS

Result: Summary Table

IF you assume STRYD is most accurate (my personal bias). Then the 935+STRYD combo is the closest to standalone, raw STRYD data in the Suunto SPARTAN. Next comes 935+GPS; next closest comes 935+GPS+GLONASS; next comes 935+ Garmin footpod (uncalibrated)

I think the uncalibrated nature of the footpod is a problem. My bad. (I had planned to use the MilestonePOD over BLE but couldn’t get it to pair with the 935 even with a new battery)

It seems that, as the pace inputs become more accurate, then the closer GRP power comes to STRYD power. Again, I’m not saying STRYD is right.

Distance/Guide PaceMile (@5:01/km)Km (@3:47/km)
AverageAverage
GRP+Garmin Footpod346448
STRYD238304
 GRP as % of STRYD145.38%147.37%
Distance/Guide PaceMile (short @4:42/km)Km (@3:48/km)
AverageAverage
GRP+STRYD345421
STRYD255310
135.29%135.81%
Distance/Guide PaceMile (@4:54/km)Km (@3:57/km)
AverageAverage
GRP+GPS345434
STRYD247301
139.68%144.19%
Distance/Guide PaceMile (@4:54/km)Km (@3:57/km)
AverageAverage
GRP+ GPS+GLONASS362451
STRYD250309
144.80%145.95%

 

Raw Data

Data Files are here: https://www.dropbox.com/sh/psdyxm93y2m12j3/AABNlbBRsF2E3edvzqnnMPC4a?dl=0 in 12Dec2017 folder. They shows lots of circles of St Mary’s running track. The files may contain grossly wrong oHR data.

Other Garmin Running Power ‘Tests’

Summary & Thoughts

I’m not surprised.

I wouldn’t be surprised to see different numbers again with an RD-POD. I don’t have an RD-POD.

Both the GRP and standalone STRYD seem to produce broadly similar power lines. Similar in terms of their trend and variability but not in terms of their magnitude. On that basis you might superficially call either of them believable. If someone wants to analyse the data sets that would be cool (see above).

If you are thinking of trying the freebie Garmin Running Power, then I would make sure you are happy with your source of pace/speed. Furthermore I would also make super sure that you use the same sensors forever more. If you don’t your power numbers will change, I reckon, and you REALLY don’t want that to happen.

This interestingly throws open a new market for STRYD. If they can release a cut-down, cheaper version of STRYD that just outputs pace/cadence/stride length then they ease their way into the Garmin Running Power environment. Let’s face it, you wouldn’t buy a full-priced STRYD and then not use STRYD’s power metric. Such a new STRYD just focussed on accurate pace would, for sure, also attract quite a few runners who have almost zero interest in power but about 98% interest in accurate pace.

Just a thought.

Reader-Powered Content

This content is not sponsored. It’s mostly me behind the labour of love which is this site and I appreciate everyone who follows, subscribes or Buys Me A Coffee ❤️ Alternatively please buy the reviewed product from my partners. Thank you! FTC: Affiliate Disclosure: Links pay commission. As an Amazon Associate, I earn from qualifying purchases.

16 thoughts on “Garmin Running Power :: Garbage In…Garbage Out?

  1. It would be interesting to see how you get on with the runscribe providing pace over BTLE. It’s only enabled in beta firmware at the moment, but should be live soon once Tim releases the next firmware update (BTLE and ant+).
    As for the stryd vs runscribe…. If pace from runscribe is accurate (so far its ok for me, but it’s improving further as the sensor processing is refined) and power numbers consistent, I can’t see why you would buy a stryd over runscribe considering the dual sided nature, running power and all the other metrics that come with it real time via CIQ.

    1. thanks neil, TBH time is fairly tight so I ‘m just waiting until the pace development work that RS are doing now is finished then I will devote some more time to pace.

      see: https://the5krunner.com/wp-content/uploads/2017/12/STRYD-RunScribe-RunPow-Watch-Characteristics-v1-09.png

      yep RS vs STRYD in theory is a close call as that link shows and, for the reasons you say, the dual-sided capture is just going to appeal to triathletes whereas the dual-sided gait stuff will appeal to runners.
      RS will be playing catch up to stryd with the surrounding infrastructure but (you sound like me) all some people want is a sensor that integrates into whatever they have so the stryd ecosystem might not be too attractive in that respect.

      switch: i think this market will move to the stick with what you first use. technologies wont allow switching if you want your old data (that’s obvious already).

      accuracy: i can’t see at the moment how we are going to tell this. stryd seems a good proxy to effort to me right now.

      1. “all some people want is a sensor that integrates into whatever they have”

        Exactly this… I am a sporttracks.mobi user, and really hope they will start pulling runscribe data in now that it’s in the FIT file. Analysing the data in the RS dashboard is limited as you don’t have the GPS, HR etc, and Garmin Connect is very poor. Sporttracks pull in stryd data, but it’ll need runscribe to sell a reasonable number of units to make the dev worthwhile to ST.
        If the data appeared in ST, I’d likely never visit the RS again (or GC for that matter).

  2. Such a new STRYD just focussed on accurate pace would, for sure, also attract quite a few runners who have almost zero interest in power but about 98% interest in accurate pace
    OH YES!!!!

  3. You need to find a Greyhound Track with a speed controlled mechanical rabbit to Compare the different brand power measurements.
    I have been running with 2 watches Garmin 920XT and Suunto Ambit3 using a STRYD Power foot pod. I am also wearing 2 different chest HRM’s ( Garmin and Suunto for HR ). My two watches show very similar Average Power (within 5 Watts around 2.5 percent.) and Average Heart Rates ( also within 4 bpm’s around 3 percent. Over the year, I have run about 250 times. From 2 to 10 mile runs. Using an Excel Spread Sheet, I have been entering the data into a Performance Management Chart that shows among other Metrics my Average Power, Average Heart Rate, Average Pace, to chart my Critical Training Load ( CTL ) over 42 consecutive days, Acute Training Load ( ATL ) over last 7 consecutive days to determine my Training Stress Balance ( TSB ). I then made a graph of 3 Metrics, Avg Power, Avg Pace, and Avg HR. The graph was using my Average running Pace for the 230 runs eliminating any HRM or Power Sensor failure runs due to low Battery voltage. It showed me that as my Pace went down, my Power when up as well as my HR which should be the case. But with 230 points, the resulting graph showed 3 good very presentable resulting lines. I was also, from the graph, able to determine at what pace I need to become more efficient where the Power line starts to increase non linearly. For me it is around 8:30 min per mile. I am going to have to do more speed work / intervals. This will allow me to improve and move the linear Power line more to the right as I lower my Average pace times into the 8:00 pace times.
    I have been training using Jim Vance’s great book.
    This may be TMI, but my results show that the STRYD and HR using the Garmin 920XT and Suunto Ambit3 provide very good accurate numbers for me which allows me to train better using STRYD.
    My power Metrics are very reproducible. With the Garmin Power App, this may not be the case since the Garmin does not track the other 2 methods you have been using. Nuf said.
    Can you give me your thoughts on anything I have written here?
    Thanks in advance

    1. sounds like you have a lot of data points there.
      ctl/atl/tsb – the method i use (different to yours) only starts to have the maths kick in with about 40 days of history. you should be able to use thoe to moderate your training and track that you are progressively loading your body
      not sure of your RACE goal: at your speed and with a goal of ‘increasing efficiency’ you should be loading at low intensities (aerobic stuff) and may also doing hill reps at high intensities (hills will naturally make you more efficient, probably)
      garmin vs stryd – it’s a moot point for you. you’re unlikely to change the source of power.

      1. My race goal is a sub 24 min 5K. Since I am 72.5, I will not get much faster. Where I live, there are very few hills other than the Savannah River Bridge ( 170 ft elevation at 4-5% incline ) to train on.
        But you are right. Only intervals ( 3-5 Min ) and hill repeats will help me.
        My point from my previous message is … I am now dancing with the one I brought to the dance. I have
        been using STRYD foot pod for over a year now and very happy with it. The money was well spent.
        And the STRYD team has been very helpful.
        How do you keep a constant Power pace when running up a long hill ? My graphs show high power only at the start and end of my runs. It never seems to be constant over several minutes. Do you keep looking at your watch every few seconds or what ? I have no problem keeping the Power constant on the flats.
        Thanks in advance

        Peter

      2. hi
        at your age you are about 60% age graded 5k time. that shows you are likely under-trained aerobically. so there are big gains there for you for LSD work. as you know you must maintain muscle mass as best you can, so hi protein diet is important as would be weight training. hill reps will be good. i would say all of theose woul dbe your easier wins before you start with km reps and then the km reps are better than the tempo runs and threshold runs that you might throw in from time to time.
        yes going up a hill: you can’t look at your watch all the time but if you are not sure, especialyl at the start of the hill then a few quick glances can make sure you adjust your effort to the grade sooner rather than later. IF you can keep power on the flat you can keep pace on the flat 😉
        yes i agree a divorce and subscription to tinder is not as good an option as staying with the one you have 😉

Comments are closed.

wp_footer()