Here we look at the most important aspects of the accuracy of the Polar Vantage M and the Polar Vantage V. Specifically we look at the performance of the Precision Prime optical HR and of the Sony GPS chip used by both Vantage models across different types of workouts in running, cycling and swimming; although other aspects of accuracy like steps, elevation and sleep are also at least briefly covered.
This post specifically forms part of the main and detailed review of the Polar Vantage, it’s included as a separate post to aid the reading of the LONG main review.
A couple of runs?
I’ve endeavoured to give both Vantage models extensive ‘testing’ since early September and I will probably update the charts in this post as I gather yet more data based on the live firmware. My intention is to see if I can switch to the Vantage next year so I should hopefully have more to add charts below as time passes.
Some of October’s ‘tests’ are shown in the image to the right and I’m sure there are duplicates in there from both watches and from the OH1.
Polar Vantage Accuracy
The elevation accuracy is cool. No issues here.
For once I managed to put a device to test with some relatively serious vertical meterage.
Each of these charts has one elevation plot which shows the ‘correct’ elevation based on a SRTM database. However, there could be some errors with that ‘correct’ track as mountainous regions can cause issues with SRTM-based elevation plots. Having said that, everything looked great with elevation more than a month before the launch of the Vantage.
GNSS: GPS+GLONASS Accuracy
Just like some cyclists will get very upset with a power meter that is only 98.5% accurate (rather than 99%) then some runners will get upset by inaccurate GNSS (GPS) tracks. Not because the tracks don’t look pretty after the workout but rather because the GPS signal has been used to show pace/distance continuously throughout the run. There are a LOT more runners concerned about pace and distance than there will ever be cyclists who care about power accuracy. FWIW: I care about both!
I follow a formal methodology for at least one GPS test for each device but OBVIOUSLY then combine that with a view of how the device has performed over a variety of real-world scenarios in MY own training. One interesting finding is that almost ALL GPS sports watches are within +/- 1% accuracy of the median TOTAL DISTANCE value for my formal test. Which I think is pretty good and certainly NOT what I originally expected. Methodology, results and source files – here.
So if you are running a 26.2mile course then you might expect to be up to 0.262 miles out. Let’s say 400m for the sake of argument. Which will be a couple of minutes for many people. Yet this does not translate to some of the GPS horror stories you hear. One reason could be that buildings on city-based courses will disproportionately affect the pace/distance readings. ie a fully urban marathon route would probably have a MUCH larger error than +/-1%
But also lots of +0.5% and -0.5% errors could well cancel each other out. Not a problem for those of you wanting the correct overall distance (hmmm) BUT that is a problem for those of us wanting actionable instant pace. The segments on my GPS test were specifically chosen with those repeatable, ‘real world’ conditions in mind. >>>>>>>>>>Methodology, results and source files – here<<<<<<<<<<.
If you want a pretty, post-workout plot of your workout’s route then most devices are going to be fine for that. The Vantage is perfectly fine for that now.
Generally, I’d say ‘use a footpod’ if you want accurate instant pace and distance. However, some of the better, historic Polar and Suunto devices often gave GOOD instant pace from GPS.
Polar’s Vantage is making use of a new power-friendly GPS chip from Sony. Although I’ve spent many 10s of hours with the Vantages, all of that INITIALLY was with beta hardware and /or beta firmware. During the beta stages the GPS performance continually improved – I would certainly say that accuracy can still be further improved (as of 6Nov18). Then again, I would say that Garmin and Suunto can certainly improve their GPS accuracy too.
GPS accuracy when cycling is usually relatively easy. There are no arm movements and a relatively rapid speed helps smooth out the lines and bends. Also don’t forget that roads are more likely to be further away from trees and buildings which cause runners GPS reception problems.
The Vantage is perfectly fine for how I use GPS when cycling. Others may want a tad better performance.
Here are some examples
On a sweet 90k ride the Vantage performed well, although there were exception and this is a case in point (green) where it cuts one corner along with the Hammerhead. The Garmin 935 did that section fine but in the previous section the Garmin wasn’t on the road. So all was fine enough.
But here’s a much more recent example where all was not fine. Over a 3 lap course the Vantage, blue, has 2 perfect laps and then for no apparent reason does not want to play ball on the third lap (final hardware, final software). Strange
Here we have Box Hill with the Karoo (yellow) and the 935 (green, stem mounted) and the Vantage has the best track.
Around the next bend the Vantage M is fine but the Garmin is slightly better.
Here are multiple, anticlockwise passes of a roundabout from the 2012 Olympic Road route – albeit the roundabout was removed for the race (someone still managed to crash). I’m not sure that any of these tracks are particularly wonderful but they are all ‘alright’.
I am faced with somewhat of a quandary here. In a nutshell the Vantage M is of a similar, acceptably good standard as the Garmin 935. But the Vantage V really is NOT good. The Vantage M needs to be improved a bit to get the heady heights of the V800, the Vantage V needs MORE work. And that’s strange as I assume that both have the same algorithms and same chip. Perhaps the slightly heavier Vantage V with a metal case has issues because of the metal construction or maybe I have a partly faulty watch?
So I’m just going to give a few images for the V as I hope to be able to report back in the near future as improvements are made.
This is a very tricky section that involves a 270degree loop next to a high building, under trees and then under a bridge. Both V &M generally handle this well – in fact better than most other devices. But then there is a strange tooth-like jagged deviation from the route in the centre of the image, before recovering, by the V (red). Remember, I’m showing that as being indicative of behaviour elsewhere.
Here’s a typical run where the Vantage M does a very respectable suburban job. The track is not perfect but it IS GOOD (better than most) and within 5m of where I actually was – that is accurate enough if we apply sensible limits. The Vantage V (red) is out by 10m-15m in some places, which is not good.
I had a bit of a fail on this run and only managed to get two devices to work. The M in red looks fairly accurate under trees (but I can’t prove it…grrrr) and the V (blue) is not accurate.
Here is a bit of a tree cover test in Bushy Park. The M (green) is very good and the 935 (red) is nearly as good but there is also the V (blue) running its own merry dance.
So. There is no point in talking about the Vantage V anymore. I’ll say that the images above are typical.
Here both the Vantage M and COROS Pace are each having a good day with quite a lot of tree cover along the riverbanks which often confuses a naive, young watch. Not the Vantage M, Nice Job.
Sometimes open skies can be harder than you’d think and here is a long, easy curving section where the COROS Pace and Vantage M do well. I sometimes get perfection here (eg V800 IIRC) but neither the COROS nor Vantage are perfect today. Actually these long sections are quite interesting as they often catch out devices which, for some inexplicably random one-off reason, run parallel to the track either North-South or East-West, or even a combination of both. I was once told this could be due to tiny timing errors on the watch. Maybe.
Against the Garmin 935, the Vantage M performs with equal gusto for much of a 10k run in Bushy Park. The M was marginally more correct on this section and less correct later on.
Let’s try to switch to the Garmin 935’s super new GPS+Galileo precision. Here it is in red. The Vantage M (blue) is where I actually ran (near enough)!!!
Although to be fair to the Garmin it was a bit better than the M a couple of KMs earlier
But later when both are running close to houses and shops…surprise, surprise they are both rubbish; that’s GNSSS baby….it doesn’t play nicely with shops.
I thought I’d try 935+Galileo (blue) again, but this time throw in the V800 (red). The Garmin is probably nearest to the real track on the whole on this section which might surprise some of you (and me!).
Further on, in the same run, but in much easier reception conditions we get the following where they are all good enough with the Vantage going awry right near the top of the image (maybe trees).
Again, below, Galileo on the Garmin and this time the same Sony Chip on the Suunto 9 (Best GPS). This was a 9-mile run and each device was good at different points, overall a very mixed bag. You can see the Vantage (yellow, left) just starting to go off track as I ran towards the next section where it wasn’t great. Although the Vantage M later became great for a long stretch on a tree-covered riverbank. Overall they were effectively ‘the same’ – of specific note here (based on this one run alone) the Suunto 9 after all these months with the Sony chips does not seem to have improved to the heady heights of GPS awesomeness that some of the SPARTANs achieved.
‘Hopefully’ that has created a somewhat confusing picture and given some food for thought. I say ‘hopefully’ because the true situation IS somewhat confusing. My overall judgement would be that the Vantage M is in line with the 935’s overall performance and effectively as good as the 935 in GPS-only mode – from experience, the 935 is consistently worse with Glonass or Galileo enabled.
I never compared like-with-like modes ie 935+GLONASS with the Vantage M (GLONASS always on) – If I did, I am positive that the Vantage M would be better. My time is too precious to waste.
I try to compare like-with-like in the sense of best ie ‘best 935 mode’ with ‘best Vantage mode’
Take out: Effectively the Vantage M is the same as the 935 for running GPS performance
Running Pace & Distance Accuracy
Running Pace comes from GNSS/GPS accuracy and has smoothing based on wrist movements. Running at constant speeds in good reception conditions, simultaneously with both Vantage watches, gave quite inconsistent ‘instant’ pace readings – sometimes the watches were out by 40 secs/km from each other, though more likely 30 secs/km from the actual pace by the worst offender at any one time.
Over longer periods, such as KM laps, the consistency of average KM pace for both watches naturally improved with the M being generally better.
Over more than 10 miles the V was still 99% accurate in recording the total distance. Compare this to 99.87% for the M. Such distance errors seem OK but 99% or less in my tests usually means that there are other issues with pace and distance recording on devices that score so ‘low’.
Take Out: As I said, the 99% distance accuracy hides numerous, smaller-scale, over- and under-estimations. The instant pace of the Vantages are not usable. You would need to use a footpod. This is NOT unusual and I would say a similar thing about every Garmin I’ve used. A 20-30 sec/km pace variation from the ‘correct’ pace is not unusual IMO.
Sorry it’s winter. These are tracks from other reviewers. Please come back in Spring 2019 (hmmm !!)
This track is from @trinakan
and another from @dcrainmakerblog
Both look good enough for me. For OWS but it would be nice if the tracks were a bit ‘prettier’ (smoothed).
NB: Polar state GNSS Accuracy as Distance ±2%, speed ±2 km/h
Better oHR accuracy seems to be obtained by wearing the watch tightly, notably in swimming. as the weather turned colder in the autumn/fall I found that keeping my wrists warmer improved readings too. Fiddling with the watch by moving it away from the skin during a workout seems to confuse the Vantage for several minutes afterward – don’t do that 😉
Whilst many more runners train by PACE than there are cyclists who train by power, I suspect that there are EVEN more casual runners who train by heart rate. Many of them will probably have absolutely no idea that their current wrist-based optical HR is wrong a lot of the time. Polar is aiming to produce the most accurate optical HR readings of any device. Do they?
I’ve included some of my older beta results that were good as these show what Precision PRIME is obviously capable of. I’ve only included current results that need to be brought to your attention.
Running – Steady State
Sometimes both Vantage devices were able to produce near-perfect tracks for running over extended periods.
and also on the live firmware like this
Running – Intervals
Once the watch was tightened after 15 minutes (!), a good track was delivered.
This has some harder efforts but was really a test for running power. Apart from the start both Vantage units are good (beta)
Switching over to the Vantage M, this was pretty cool. It’s the Garmin in red that is not playing ball.
Again on the M with some tempo paces – the red lines are the HRM-TRI (long story) and the Vantage M just underplays some of the efforts a little too much in places.
This hour run has some slightly harder efforts. The Vantage just over-eggs the HR a tad at the key moments towards the end.
These are longer intervals at a slightly higher intensity with a few dodgy moments for the Vantage V
I also found some less good results as the weather got colder. I think for me this reduces blood flow in my skinny wrists and hence oHR suffers. However this track was with arm warmers and is pretty cool (well…pretty warm ;-0 )
This was trying what felt like hard for 2×20 with arm warmers too. Even the Suunto 9 had arm warmers on the other wrist. Hmmm.
Here we see another generally good performance but we also see a few minutes of under-reporting of HR on the Vantage M, this happened several times.
My first ‘conclusion’ a few weeks back was that the lighter-weight Vantage M seemed slightly better for running than the Vantage V. Maybe I wanted it to be? Over all my runs, in reality, they are probably equal; even if, on one day whilst wearing a watch on either arm, I might find that either one could be better.
I would also say that the Precision Prime technology can clearly be excellent. I suspect that by getting a tight fit before setting off and keeping warm I would have even better results from both Vantage devices.
Cycling – Long Ride
On good surfaces we can get apparently great results over extended periods
yep. Same again over several hours
and again. Several more hours.
This time it was even better than a chest strap which had 2 hours of madness and then started working again.
There were a few unexpected troughs on this lapped ride at a slightly higher level of exertion and on a slightly bumpier road than those above.
This is more dubious. But ‘great’ for extended periods despite the occasional dubiousity
Not bad at all
Awesome on a turbo trainer doing intervals/ramp (as expected)
And again, although not as intense, with the Vantage M
This one on a turbo is not so awesome (beta FW) but I was fiddling around quite a bit with another sensor (Hex) and had my wrist quite bent (not my usual cycling position!). I only include it to give a ‘full picture’.
But wait a minute. Let’s look again at the Vantage M. It is a lighter weight device and this might mean the M moves about LESS on the wrist and MIGHT give a better oHR reading as a result.
This certainly looks good for a relatively easy Z2-Z3 bike effort. Even the HRavg figures are the same as with the HRM-TRI. This is the SAME BUMPY route as a few of the other Vantage V results (above) that were not quite so good. Having said that, the one below IS the M on retail firmware.
There are some differences with the V & M compared to chest straps, as shown above. I tend to get BAD oHR results from cycling on mildly bumpy tarmac – or worse. Results in better conditions like a turbo or smooth roads can improve performance notably. These (and other) results from the Vantages are probably better than any previous oHR device I have tested whilst cycling. I’m still going to use a chest strap though!
Swimming – Pool
Hey…25 minutes. Tighten up that strap will ya? I did and all was much improved.
Next time maybe START with a tight strap and look what you get. This is me trying fairly hard in the pool after 25 minutes or so of faffing around with drills. Actually I’d say this was good for a swim oHR track…certainly better than Garmin’s oHR in the water (they don’t do it 😉 )
Here I used paddles for the whole 30 minutes…don’t do that.
This one is not quite as good as the earlier ones but still alright for a swim track.
But then I got this in a steady-state 40 minutes in the pool. No paddles, exact same conditions
Let’s leave on a positive note for pool swimming. You can reasonably expect this and this is alright for swimming oHR.
With Swimming oHR accuracy, Polar’s Vantage IS better than anything else. It’s still not really accurate enough for my liking but, perversely, I might use it as I REALLY don’t like wearing a strap in group pool sessions.
I have done some low-level weights over 4 sessions with the Vantages. The results were not good. I find that the same as every single other wrist-based oHR device. I never quite see the point of measuring HR when doing weights. Mine rarely goes over 100, although I can get it up to 130 if I do a silly number of squats with weights. Maybe I need to try harder?
Take Out: oHR Accuracy
I take heart rate seriously in my training and I am going to be super-critical as a result.
These tracks are some of the best I’ve seen from a wrist-based oHR device. Precision Prime is probably on a par with Garmin’s ELEVATE but it’s still not quite good enough for me. I know this is still work-in-progress.
I’m going to keep my eye very firmly on these oHR tracks and MAYBE, just maybe, Polar will be the first company to get me to bin the chest strap in the near future?
If YOU really want a wrist-based oHR then I reckon that Polar Precision PRIME is one of the best available right now AND it should, hopefully, improve to boot. I can’t see Garmin’s current iteration of ELEVATE hardware being improved by software fixes. Garmin’s ELEVATE needs a redesign to further improve.
Caveat: optical HR varies from person-to-person. You may have better, or worse, experiences.
Running Power Accuracy
The accuracy of Polar’s running power needs to be the subject of proper 3rd party lab tests. And that statement applies to all the Running Power companies.
Polar’s Running Power *IS* perfectly usable, it’s just that all the 4 companies in this space do not agree with each other. Q: Which one is ‘correct’?…A: I’ve no idea. The only caveat I would add is that both Polar and Garmin base their running power algorithms partly on pace. When their source of pace is a GPS/GLONASS/GALILEO signal then you will get the same vagaries with power as you do with pace eg as caused by tunnels, buildings and tree cover.
If you want to look at lots of charts showing 4 companies’ running power that don’t agree with each other then click-through to the following linked post 😉
I have EVEN more running power comparison charts than shown in the previous link. They ALL say the same thing. #DISAGREEMENT
My overall impression is that Polar Vantage V’s power calculation is a reasonable proxy of my effort. It is usable. Given the choice I would use either RunScribe or STRYD but I could see how many of you would not want to make that investment. In which case, give the Vantage V a go…you WILL get some benefits that help you augment how you currently run to HR/PACE/RPE.
TAKE OUT: Really. I mean really. Choose a specific way of getting running power and don’t change it. If you choose to go with the Vantage V’s internal calculations then ALWAYS stick to either a footpod or GPS+GLONASS as its source of pace. I love running with power. I’ve always used STRYD and that isn’t going to change. If you want to go for the Vantage V’s power algorithm then you will definitely get benefits from it in training/racing but STRYD or RunScribe will be more consistent (IMO) because of the variability of pace from GPS inherent in the Polar and Garmin solutions.
General Activity/Sleep Accuracy
Almost everyone buying a Vantage will be embarking on various athletic endeavours even if they might not be calling themselves an athlete. The general activity’ stuff ….steps…is not really that important. I’ve had a cursory look at steps and the like and it seems in the right ballpark for day-to-day comparisons.
Whilst I don’t look at calories I know many of you do. I have no way to professionally validate Polar’s calorie calculation. All I can say is that they have historically had the reputation for the BEST calorie estimation. Q: “Is the Vantage the best at correct calorie calculating?”…A: “I don’t know”.
Sleep tracking is one that we need to look more closely at. It is generally thought that your body adapts to your day’s exercise whilst sleeping. Poor sleep = poor adaptation. So sleep IS important. The problem is that even the ‘gold standard’ sleep lab machines are, apparently, not fully accurate.
Over a whole week (more than shown below) Polar IS getting my total hours in bed AND ASLEEP to be correct – at least to a good level.
Looking at specific nights then Polar’s SLEEP PLUS seems to closely resemble my feeling of what my last night’s sleep was.
Look again at those charts more closely. Polar has, sensibly in my opinion, chosen to avoid guessing what the sleep stages are (N1, N2, N3, etc). I use the word ‘guess’ deliberately as I suspect that is kinda what most other brands are doing. Polar’s SLEEP PLUS looks to me to be an honest and competent reflection of how much sleep you’ve had and broadly how well you slept. Is it perfect?…no. Is it useful? … probably…a little bit.
Polar also take SLEEP one step further in Recovery Pro with the addition of mood & sleep feedback in their assessment of your readiness to train. That DEFINITELY adds a layer of proper usefulness.
Polar Sleep Plus FAQs – link to polar.com