Accuracy Study

During the fall of 2016, Bad Elf commissioned a research report to test the accuracy of a Bad Elf GNSS Surveyor. The goal of this project was to determine if Bad Elf’s Bluetooth GPS receiver can produce consistent results at a 1-meter Circular Error probability (CEP).

Elizabeth Runkle, intern and GIS student at Maricopa Community College constructed a methodology to test and verify results for the Bad Elf GNSS Surveyor using Esri tools. This research exercised a complete workflow starting with geodatabase creation, creation of a map for use in field data collection, collection of field data, and production of a final analysis of the results. The software used includes Esri ArcMap, ArcCatalog, ArcGIS Online, and Collector. Two sets of results were produced that include accuracy achieved in real-time and accuracy achieved via post-processing.

The research report "Systematic Approach To A GIS Concluding With An Accuracy Analysis" can be downloaded from this link .

Discussion

The baseline definition of accuracy that Bad Elf uses is "the degree to which the result of a measurement, calculation, or specification conforms to the correct value or a standard." Contextually, the precision we report is consistent with the achievable results for a Bad Elf receiver; a numeric representation of latitude and longitude to 8 decimal places that represents the practical limit of commercial surveying.

Common measures of two-dimensional results fall into the realm of RMS, CEP, 2DRMS, and R95 as defined below. The measure of probability represents the percentage of readings that will fall into a circle of a specific radius. For example, the Bad Elf GNSS Surveyor has a CEP of 1 meter. The 1-meter CEP specifies that reading over time will be 50% inside a circle of a 1-meter radius and 50% outside the circle in ideal circumstances.

Unless an application converts the values produced by a Bad Elf receiver, the reported accuracy is based upon CEP.

RESULTS

Overall, the Bad Elf's GNSS Surveyor produced consistent results demonstrating the 1-meter CEP claim. Data collection followed the approaches defined under Bad Elf's best practices documentation for field collection.

A few points did exceed the 1-meter CEP. These points were the result of a very challenging GPS environment and were expected.

Recorded Positional Error

From the raw recordings, Bad Elf retrieved samples to analyze further. Below is an example of a 15-minute capture over point AJ3650. This recording had 88% of the points inside the 1 meter circle which far surpasses the benchmark of 1 meter CEP.

Below is a histogram depicting the actually calculated errors from the known position of AJ3650. From the analysis below, one can make the determination for this recorded session that the Bad Elf achieved a positional accuracy of ~88% within 1 meter of the actual location. 

This graph below depicts in blue the actual error between a reference point AJ3650 and the location the Bad Elf receiver produced. The line in orange represents the statistical accuracy reported on the Bad Elf. Statistical accuracy is what is delivered on the Bad Elf receiver and in the Bad Elf app. Accuracy reported by typical GPS units is a measure based purely on statistics and therefore at times can be uncorrelated with actual accuracy as depicted below.