EGNOS Accuracy Field Test

Now that I'm finally picking up sat no. 33 I decided to visit my local
"test" trig point at Blackpill, Swansea to take some readings. It's located
on the sea front with a clear view in all directions, particularly over the
sea to the south.

The official Ordnance Survey coordinates (OS datum, British National Grid)
are SS 62168 90898. However, since the conversion algorithm in the GPS is
relatively simple, it's more accurate to use the WGS84 datum with lat/long
coordinates. These are listed as N51.600017, W003.991491.

Using GPS Utility to convert the WGS84 coordinates back into OS BNG
coordinates, the result is SS 62171 90904. This conversion seems to be on a
par with the internal GPS conversion.

The elevation at the trig point is 10m, which is measured at the base of the
trig. Since I record my waypoints at the top of the trig, I've subtracted 1
metre from the results below to compensate for that.

Here's a list of waypoints recorded with my GPS12 on previous visits.
Nothing fancy, just a case of stopping at the trig and marking a waypoint.
Sometimes I only stop for a few seconds during a cycle ride, just long
enough to mark a waypoint.

I've marked the grid ref followed by the date (021228 = 28th Dec 2002), then
the Easting error and Northing error, plus any notes.

GPS12 Waypoints
SS 62171 90906 - 021228. EE +0, NE +2 (5 minute average)
SS 62172 90905 - 030330. EE +1, NE +1
SS 62172 90906 - 030330. EE +1, NE +2
SS 62172 90908 - 030331. EE +1, NE +4
SS 62172 90906 - 030331. EE +1, NE +2
SS 62170 90904 - 030403. EE -1, NE +0
SS 62171 90903 - 030403. EE +0, NE -1
SS 62168 90906 - 030704. EE -3, NE +2
SS 62171 90905 - 030707. EE +0, NE +1
SS 62169 90904 - 030714. EE -2, NE +0

The average of these 10 waypoints is as follows;
SS 62170.8 90905.3 EE -0.2, NE +1.3

The Easting is usually quite accurate, but the northing is usually a bit too
far north.

When I got my 60CS I went back to take some new readings, plus my GPS12 for
comparison.

60CS Waypoints
SS 62171 90905, --.-m - 040722. EE +0, NE +1 (GPS12 - long average)
SS 62171 90905, 09.1m - 040722. EE +0, NE +1 (barometric elevation)
SS 62170 90904, 12.1m - 040722. EE -1, NE +0 (200 samples - EPE 4.4m)
SS 62171 90904, 10.9m - 040722. EE +0, NE +0 (100 samples - EPE 5.0m)

A surprisingly accurate set of readings on this visit. far better than the
EPE suggested. I was there at roughly 8pm on a dark and clear night. Don't
know if that was a factor. The only waypoint that recorded barometric
elevation read 0.9m below the listed height.

Now today. I could hardly expect better readings than the previous visit,
but the EPE was a lot more optimistic. I was usually picking up 11-12
satellites, including no. 33, and the EPE was typically 2m, sometimes
reading 1m, even in the car on the way there!

60CS with EGNOS Waypoints
SS 62170 90905, 10.0m - 060429. EE -1, NE +1 (barometric elevation - EPE
1m)
SS 62170 90902, 13.0m - 060429. EE -1, NE -2 (1000 samples - 20 mins - EPE
1.0m)
SS 62171 90904, 12.0m - 060429. EE +0, NE +0 (barometric elevation - EPE
2m)
SS 62170 90903, --.-m - 060429. EE -1. NE -1 (photo of satellite screen -
EPE 2m)

The first waypoint was marked with an EPE of 1m, and it was exactly 1m out
in both directions, but the altitude was spot on.

The second waypoint was the most inaccurate, after being averaged for 20
minutes. Even though the EPE dropped from 2.0m to 1.0m during the
averaging, the northing ended up 2m out. The GPS altitude started at 14m
and only dropped to 13m, still 3m too high. The altimeter typically gives
more accurate readings. Doesn't seem to be much advantage in averaging, not
for only 20 minutes anyway.

The third waypoint was spot on, although the altimeter had risen by 2m,
probably due to the auto-cal and the overly high GPS altitude. I noticed
that it rose from 5.9m at the car park when I arrived to 10.0m on my return.
I probably hadn't allowed the altimeter enough time to settle down with the
auto calibration.

I then took a photo of the 60CS on top of the trig point with a full
complement of 12 satellites, which you can see here;
http://www.pvs1.f2s.com/0604290043.jpg

For those of you interested in the lat/long WGS84 coordinates, today's most
accurate waypoint (number 3) is reported by GPS Utility as;
N51.600020 W003.991490
compared to the official coordinates;
N51.600017 W003.991491

Not bad eh? The GPS seems to round the last decimal place so it couldn't be
more accurate than that.

Conclusion? Well, discounting the unusually accurate readings of the
previous visit, it does seem to have sorted out the "northing too far north"
problem, but one visit obviously isn't enough for any real conclusions. I
thought I'd share the data with you though, for what it's worth.

When the accuracy is as good as the previous visit, there's not much scope
for EGNOS to improve it much further. It seems that EGNOS is more useful
when the accuracy isn't quite so good. For example, I noticed a much better
EPE than usual whilst driving through the city centre, and even though I
lost no. 33 a few times, I got it back before losing the differential
corrections. So it seems to be an advantage in the car at least.

Paul

Note that internal GPS conversion is often not the most accurate. Also that
survey results, especially if not made by GPS,, although often the legal
reference, can contain errors and mis-measurements. In your comparison,
both conversion and the original survey as sources of error can not be
excluded. Hopefully the original survey error is small, but the conversion
one might be in meter range--as such often are.
...

Five minutes is very short relative to the error correlation. In
measurements I have made over long periods, 5 minutes when correlation of
errors is considered really amounts to just a few essentially independent
fixes.
...

The EPE is only a statistical parameter. It is used to describe a
distribution where the error will often be larger or smaller than that
parameter. Your results are actually not all that surprising. I would
compare it to asking what the temperature during a full year is like at a
given location on earth and making a measurements over short part of a
single day. You would not want to conclude too much.
...

We agree. Note the GPS spec requires 30 days and I have found atleast a
couple days of continuous measurement necessary to begin to estimate the
distribution of errors.

It is so easy to think one knows something about the accuracy from a few
measurements. The true is--despite arguments otherwise, one knows very
little from a few measurements. There are those here who recommend doing
what you did; I recomment going the next step: Sit at one point. Surveyed
is best but even unserveyed is useful. Continuously log data and analyze the
result. Yes, that can be messy. But otherwise, one does not really
understand the statistical nature of the error. It is better than trying to
study the accuracy by visiting new points but is indeed beyond the
capabilities of many in either time, hardware, or statistical knowledge.

David L. Wilson wrote:


I know, which is why I've used the WGS84 coordinates instead.


True, but trig points are the key reference points in UK mapping. I know
this isn't one of the primary trig points (which are considered error free,
since they are what the OS datum is based on) but it may be one of the
secondary or tertiary points, which are accurate to less than one metre.
The coordinates in the list have been rounded up to the nearest metre, but
the source does state that accuracy can not be guaranteed.


No, you misunderstand. I know about the conversion error, so I've bypassed
it by using the WGS84 coordinates instead of the OS coordinates (they are
supplied in both datums). There's an easting error of 3m and a northing
error of 6m between the correct OS coordinates and the internal GPS
conversion. By using the WGS84 coordinates in the GPS, the "displayed" OS
coordinates (which is what I've listed) are incorrect with respect to the
map, but are correct within the GPS itself, if you follow me.

The reason for displaying the coordinates in OS BNG format is because that's
what makes sense with UK maps, but I'm actually comparing the results with
the WGS84 coordinates, not the OS coordinates.

I did list one waypoint in WGS84 lat/long format for comparison;
N51.600020 W003.991490
compared to the official coordinates;
N51.600017 W003.991491


I believe so, but there are passive and active GPS stations not too far away
from me which have been measured to a high degree of accuracy, so I'll be
visiting them soon.


Correct, but I've bypassed the conversion error as explained above.


True, but I just did it to see what difference it would make. On most
visits I didn't bother.


I know, hence my use of the word "suggested". Many people assume that it's
the actual error.


I don't, but the readings were taken during nine separate visits and most
have been pretty accurate. The average is pretty good too.


I'm not claiming this to be a scientific test, just a practical field test.


I disagree. I know what the published coordinates are. I know what my
waypoint coordinates are, and I know what the differences between them are.
Therefore I know how accurate my GPS "appears" to be with respect to this
particular trig point, based on my experiences so far. Out of 9 visits, 8
were within 3m and 7 were within 2m of the published coordinates.

I therefore conclude that for this particular trig point, the GPS "appears"
to be accurate to within 2m, 78% of the time. Irrespective of the true
accuracy of the GPS and trig point coordinates, these are my actual
practical experiences. I'm not testing absolute accuracy here, I'm just
testing what happens when I visit this trig point and press the mark button.


That's not going to happen. I just don't have the time. Besides, there are
plenty of other people who enjoy doing that sort of thing. I respect people
who have the patience to do that, and I find the results very interesting to
read about, but what concerns me is practical accuracy in the field.

The fact of the matter is that most walkers with a GPS simply walk to a
location, press the mark button, then continue walking. They don't set up
camp and stay there for two days to do a long average and a statistical
analysis. Whilst that's theoretically very interesting, it's not remotely
practical. That's not how people use GPS in the field. I'm just testing it
the way that people use it, by walking to a location and simply pressing a
button.

Many walkers use coordinates in order to guide them to a location, often
summits, which are often the sites of trig points. When a walker inputs
waypoint coordinates, what he wants to know is how close the GPS will get
him to that point. 2 metres? 5 metres? 15 metres? Most of the time it
doesn't really matter since when you get that close you can usually see your
objective anyway, but sometimes, at night, or in bad visiblity, or if the
location you're trying to find isn't particularly obvious, the accuracy may
be critical.


Few GPS users understand it, and few ever will, but that's not going to stop
people using it. Most people simply want enough accuracy to get to where
they're going, or to mark a spot accurately enough to be able to return
there at another time. Granted you rarely need 2m accuracy to do that, but
I've noticed that many people have a tendency to walk to a waypoint then
complain "hey, this waypoint is 5m out!". It would be nice to hear people
praising the accuracy of a waypoint for a change, and that's what interests
me, practical accuracy in the field (even if it's only "apparent" accuracy).


Theoretically maybe, but people don't visit new points to study accuracy,
they do it because they like to go to different places. Testing the
accuracy of one trig point is all very well, but it's not very useful to
people who want to go to a different one.

There are 6557 trig points in the UK, with published waypoints in both OS
and WGS84 formats. I frequently encounter them in my travels, and every
time I do, I mark a waypoint. I've been doing this for nearly 7 years now,
sometimes visiting the same trigs many times. When I eventually get around
to collating all this data, the results should prove interesting. It may be
that certain trigs are consistently inaccurate, whilst others, like the
Blackpill trig, are surprisingly accurate. We shall see.

This may not be very scientific and rigorous, but it's real. It's what
actually happens in the field, and that's what interests me.

Paul

...

Ok, you are right, I did not understand that--but my point may be applicable
for others doing similar.


I would not go as far as even "suggested". I do not think there is
sufficient data to say that but my training in statistics (graduate school
and taught it for 10 years at the university level is howing).


Not enough data to say that--true about your sample but too small to say it
is generally true--may be or may not be.


I guess I am arguing there is not such thing as "practical accuracy in the
field" that is different from what you mention other people are doing (I am
one of those). I wish it were not that way.


They all are likely to have similar accuracy. It is the accuracy of the
GPS system/receiver that are the main variables. Attributing them to the
accuracy at different points is probably the result of not really getting
down to understanding the true accuracy of the GOS system/receiver.


This is a good discussion. I do not fault you for what you have done--it
does at least tell you not to expect errors in 10's of meters. I would
argue the different points are "red herring" in your look at GPS accuracy
and would be cautious about making a statement on the accuracy without
qualifying what your measured accuracy means--i.e. RMS, CEP, 95% etc.
Otherwise it is like the EPE shown on the GPS, not really much use in
statistically understanding the accuracy. It is a first step, but what you
did not do (I understand) have time for, really tells you much more. But as
I mentioned, even doing it a non-survey point logging to a PC (say over a
weekend) is a uuseful set of data to see the spread in measured position.

David L. Wilson wrote:



Indeed, I suspect few people are aware of this.



Yet that's what most people think it means. The manual doesn't help, I
quote;

"...the display at the top of the page will change to indicate position
accuracy..."

"...the map displays an Accuracy Circle surrounding the Position Icon..."
"The smaller the circle, the more accurate your location".


I accept that. What's the minimum sample size that you'd consider to be
sufficient for a reasonable assessment?


So what's the point in even having an EPE if we're not supposed to trust
what it implies? Surely you're not suggesting that we assume all readings
to be inaccurate irrespective of the EPE? It must give a relative
indication of the likelihood of good accuracy at the very least? Along with
a visual assessment of the satellite constellation.


Probably, although there's a subset of trig points with zero error, while
others will have a tiny error. Probably too small to detect with a handheld
GPS anyway.


No, I was thinking in terms of accuracy variations due to topographical or
terrain features. Most trig points are built on hill tops and therefore
have excellent all round visibility and reception, but sometimes they are
built on crests rather than summits and may have blocked visibility in one
direction. There may also be trees in the vicinity. Some trig points are
now completely surrounded by dense forestry.


That's a fair point.


More than I can glean from studying other people's graphs and plots?


Probably, but I'm in a convenient position to do that. I don't have a lead
long enough from my computer to place my GPS in a position where it could
actually get a lock.

Paul