Lesson 9 Introduction

Welcome to Lesson 9! In this lesson, you will understand and become familiar with the main parameters that need to be considered when selecting a UAS for geospatial business activities. You will also recognize the main manufacturers of UAV, aerial acquisition sensors, and processing software. There are not many materials in the course textbooks that directly deal with these subjects, but one can indirectly derive some information from them. In addition, several research studies were conducted by private or public groups on the status of market and future prediction.

Unmanned Aerial Vehicles (UAVs) are becoming the most dynamic growth sector, and based on a research study conducted by the Teal Group Corporation, it is expected that the global UAV market will top US $54 Billion in the next decade or so.

Lesson Objectives

At the successful completion of this lesson, you should be able to:

• understand the major considerations in selecting a UAS for geospatial business;
• differentiate between the main providers of UAS;
• discriminate between the main providers of aerial sensors for UAS;
• recognize the main providers of software for UAS data processing.

Lesson Readings

• Drone Analyst's "Five Things to Consider when Adopting Drones for Your Business [1]"

Lesson Activities

• Practice with the use of Pix4D software to process the sample data, and produce ortho photo and DSM and submit.
• Prepare your presentation for the final project.
• Submit your final project report.

Basic Considerations for Selecting UAS

In this section, you will understand the requirements for selecting a UAS. Selecting a UAS depends on many factors that are closely related to the intended use of the UAS. Such use requirements will determine the size and weight of the UAS, and its endurance and range of flight, among other factors. In the following sections, we will briefly discuss each of these factors.

Size and weight

Size and weight a play great role in determining payload size and weight and in limiting its range and endurance. Large UASs have the capability of carrying a larger and heavier payload, including the power source. The larger the UAS, the more fuel or battery power it can carry on board. The more power the UAS can carry on board, the better range and endurance of the UAS.

Range and Endurance

The range of a UAS is an important performance characteristic. It is dependent on a number of basic aircraft parameters and weight of the payload. Maximum UAS range and endurance can be achieved with high propeller efficiency, low fuel consumption, and large onboard fuel (or battery power) capacity. A project that requires long hours in the air will need a larger UAS. However, most UASs that are employed for geospatial mapping purposes now days have an endurance of 90 minutes and a maximum range of around 50 miles.

Stability

In physical mechanics, stability refers to the tendency of an object to stay in its present state of rest or motion despite small disturbances. An aircraft must be stable in order to remain in flight. The forces acting on the aircraft, such as thrust, weight, and aerodynamic forces, have to be in certain directions in order to restore the aircraft to its original equilibrium position after it has been disturbed by a wind or other forces. An aircraft has angular degrees of freedom. Those are rotation around the X-axis or roll, the rotation around the Y-axis, or pitch, and the rotation around the vertical to the ground, or yaw. The aircraft has to remain stable around each of these axes. The most critical rotation is the pitch, and stability about it is called longitudinal stability. Some instability can be tolerated around the roll and the yaw.

Stability is essential for aerial data such as imagery acquisition in order to achieve gap-free imaging results. The use of a gyro-stabilized mount for the camera or the imaging sensor is preferred for mapping missions, as it results in uniform coverage free of gaps.

Cost

UAS costs play a great role in the decision for acquiring one. The price of a large UAS sometime exceeds the price of a typical manned aircraft, such as various models of Cessnas, used for aerial imaging. However, the cost of a UAS is justified by the type of jobs that are expected for the use of the UAS. Smaller UAS-based aerial imaging jobs are only justified through the use of a small UAS that costs under $100,000. It is worth mentioning here that due to strict regulations by the FAA on flying UAS, there are no large jobs for the UAS at the current time within the geospatial mapping community. No one can commercially utilize UASs for money-making projects, therefore only smaller UASs are utilized by the mapping community. Once the FAA eases the regulation, we should expect larger demand for medium or large UAS.

Payload Capacity

The maximum weight that a UAS can carry on board also plays an important role in the decision of UAS selection. Different applications require different sensors and therefore different payload capacities. Current UAS used by the mapping community can carry a payload varying in weight between a few to 100 lbs. The payload capacity directly affects the cost of the UAS, as it limits the range and endurance for the UAS. UAS with longer range and endurance cost more than those that fly a maximum distance of 35 miles and for a period of 60 minutes.

In this section, you will gain an understanding of the different brands and makers of the UAV, payload sensors, and processing software.

Market survey of the Air Vehicle (UAV)

Large UAS that are used mainly for defense purposes are around for a long time and have sophisticated technologies built into them. Examples of the manufacturers of such UAS are AAI Corporation, AeroVironment, Aurora Flight Sciences, BAE Systems, Boeing, Elbit Systems, General Atomics Aeronautical Systems, Inc., Israel Aerospace Industries, Northrop Grumman, Raytheon, Rotax, Sagem, Selex Galileo, and many others. Within the last decade, many startup companies started manufacturing low-cost UAS that are mainly used for civilian purposes. Examples of those manufacturers are Trimble, Altavian, Sensefly Ltd, American Aerospace Advisors, Prioria, Uconsystem, Idetec, and many more.

The following four resources contain good information on existing systems and manufacturers:

1. Unmanned Aerial System (UAS) Survey [2]
2. GIM International Volume 28 Spring 2014 [3]
3. UAS Suppliers [4]
4. 24^th annual edition of Shephard Media’s "Unmanned Vehicles Handbook" [5]

Market survey of Payload Sensors

The sensors required for UAS that are utilized for mapping purposes are mainly limited to cameras (Visible, near-infrared, and thermal infrared). The second resource provided in the previous section offers a list of sensors manufacturers that are used for UAS payloads. UAS payloads used for the mapping community mainly include imaging cameras. Such cameras have a variety of spectral bands such as visible (Red, green, blue), near infrared (NIR) and thermal infrared. There is only one LiDAR system developed mainly for the UAS and that is the VUX-1 manufactured by Riegl, which was described in Lesson 2. The most obvious provider of digital cameras (without endorsing any of them) that are small enough to fit within UAS payloads are the following:

1. Phase One, with their multiple models of aerial cameras [6].
2. Imprex, with their latest model of Bobcat cameras  [7]
3. Nikon, with their multiple models of cameras [8]
4. Mecasense with their multiple models of multi-spectral cameras [9] or Parrot for their Sequoia [10] camera

Market survey of Processing Software 

For image-based mapping products generation, users will need efficient photogrammetric processing software. Such software should be capable of performing the following operations, among others:

• organizing the input imagery, camera calibration reports, GPS-derived camera position, IMU-derived camera orientation angles, and ground controls data in a simple database;
• having user-friendly graphical user interface (GUI);
• having good data viewers, i.e. orthos and DSM;
• handling tens of thousands of images per project in TIFF or JPEG formats;
• image coverage verification through rapid data processing mode;
• performing automatic aerial triangulation processing using simultaneous bundle block adjustment with viewing and manual editing capability;
• accepting GPS-derived camera position and IMU-derived camera orientation;
• camera self-calibration;
• modeling GPS shift and drift;
• producing quality control reports;
• exporting exterior orientation parameters for photogrammetric work station;
• performing ortho rectification;
• automatic DSM generation through auto-correlation;
• performing image mosaic and capability to edit mosaic lines;
• exporting ortho tiles according to user defined layout in shape file format;
• performing color balancing and radiometric enhancement;
• distributed processing (parallel processing) using computing farm;
• batch processing or scripting.

Among the most obvious data processing software that are optimized for UAS data processing in the market (without endorsing any of them) are the following:

1. Agisoft Metashape [11] 
2. Pix4DMapper [12]
3. Menci APS [13]
4. CORRELATOR3D^TM [14]  by SimActive
5. Trimble Inpho UASMaster [15]

Each of these five software packages meets most of the capabilities listed above. However, some of them may be more suitable than others, depending on the situation and the nature of the project.

Geospatial Data Accuracy and Quality and Mapping Standards

Introduction:

Evaluating the quality and accuracy of geospatial data is one of the most important topics among geospatial data users. Geospatial data are used for diverse applications, including engineering and positioning applications. Knowing how accurate the measurements are that are derived from geospatial data is a matter of life or death in some applications, like when locating gas pipelines. In this section, you will be introduced to various statistical concepts that are related to determining geospatial data accuracy. You will also learn about the latest map accuracy standards designed for digital geospatial data published by the American Society of Photogrammetry and Remote Sensing (ASPRS).

Metrics in Geospatial Production Process:

For any geospatial data product, collecting metrics about a dataset revolves around the following questions:

• How well does the map fit a national or a global coordinates system and datum?
• How well does the geometric and radiometric quality meet or depart from the client’s expectations or specifications?
• How well do these metrics fit a “standard” or what is considered standard within the geospatial industry?

Why we are concerned about accuracy?

Errors exist in any product we produce, no matter how accurate the instrument or the process we utilize. This is because all measuring instruments are not perfect, including laser instruments. Figure 1 illustrates the common instruments used in surveying and mapping practices and which we may think are perfect measurement devices.

Figure 1 common instruments used in surveying and mapping practices

Enter image credit here

Errors in Measurements

There are two types of errors that concern us the most in geospatial data generation, and those are random error and systematic error. The third type, which is what we call blunders, is not considered an error, but we need to understand it and deal with it appropriately.

Random Error (or accidental error) is the type of error that randomly happens in nature due to our, or the instrument’s, incapability in realizing the true value. The true value in any measurement process is elusive to us and is beyond our metaphysical power. In a measuring process, we are only estimating the true value. Random error can be reduced by training, experience, and improved quality, but it cannot be eliminated.

Systematic Error: Is the error that has a repeated constant value and follows a mathematical logic. It can be reduced through calibration.

Blunders: A blunder is not an error; it is a mistake resulting from carelessness or negligence that resembles error. Common causes of blunders in surveying and mapping are:

• Measurement taken incorrectly
• Values misread from the measuring device (i.e. screen)
• Number transposed as they are recorded (696 vs. 969)
• Miscounting grids ticks
• Handwriting that is hard to read
• Values entered incorrectly into the computer
• Using the wrong datum and/or coordinate system
• Using the wrong units (meter versus US or international foot)
• Rounding numbers in recording the data

Facts on Error and Normal Distribution:

• Errors are unavoidable, but controllable;
• Any mapping process will have some variation of errors built in;
• No combination of machine and human can produce a product that is exactly the same each time;
• Biases should be removed prior to analysis;
• Small errors are more common than large errors;
• Errors are just as likely to be positive as negative;
• Large errors seldom occur and can only be so big. Blunders can be large.

Accuracy Defined

Accuracy: The closeness of results of observations, computations, or estimates of graphic map features to their true value or position on the ground.

Precision (Repeatability): The closeness with which measurements agree with each other.

Facts about Accuracy:

• True value is the theoretically correct or exact value of a quantity. The true value is elusive to us, and it cannot be reached considering our human limitations. True value is a matter relate to metaphysics.
• Accuracy is part of the map metrics that need to be included in the metadata of any geospatial dataset.

To illustrate the concepts of accuracy and precision in a practical fashion, let us consider evaluating the results of the four shooting sessions of Figure 2 that the sharp dart shooter completed at different times. In session A, the shooter’s shots seem to be scattered around the bullseye. He/she managed to get the shots around the targeted spot, or the bullseye, but failed to land them close to each other, i.e. they are scattered apart. To evaluate such a session, we say the shooter was accurate as he/she stayed close to the bullseye, but not precise, as the shots were not close to each other. In session B, we would say the shooter managed to cluster all shots in one spot, so he/she was precise but far away from the bullseye, so he/she was not accurate. Accordingly, in session C, he/she was accurate and precise, while in session D the shooter was neither accurate nor precise. To illustrate the concept of biases in measurements, let us analyze sessions B and C. Assuming the two sessions were shot by the same shooter, it is obvious that the shooter performed perfect shots in both sessions but that his/her shots in session B were biased due to mechanical misalignment of the bow or the gun, if a gun was used. Such misalignment of the bow, the gun barrel, or the sight scope caused the shots to be systematically directed to the wrong position instead of the bullseye, causing a bias in the shots. Once proper calibration is made to these mechanical defects, the bias is then removed and all the shots will perfectly fall around the bullseye, like in session C.

Figure 2 Accuracy and Precision illustrated

Enter image credit here

To evaluate the shooter results using probability and density distribution terms, the results of session B are equivalent to the random distribution 3 of Figure 3, precise but not accurate, assuming the most probable value of the bullseye is represented by p on the x-axis. The results of session A, however, resemble the distribution 2 of Figure 3, accurate but not precise. For more information on the subject, please watch this NGS video [16].

Figure 3 Distribution of random process

Enter image credit here

The Ever-confusing Statistical Terms

To illustrate the different statistical terms we usually run into when we discuss data accuracy, let us consider the five error values (3-in., 2-in, 1-in., 5-in., and 4-in.) that were calculated on a population of data.

1. Mean (average) $=\frac{3+2+1+5+4}{5}=3.0$
2. Range = the distance between the largest error and the smallest error, i.e. Smallest = 1, Largest = 5 3)
3. Variance = measure of spread or dispersion around the mean. It is the mean square of all the errors $=\left({\left(3-3\right)}^2+{\left(2-3\right)}^2+{\left(1-3\right)}^2+{\left(5-3\right)}^2+{\left(4-3\right)}^2\right)\frac{1}{5-1}=\frac{0+1+4+4+1}{4}=\frac{10}{4}=2.5 inc{h}^2$
   Here, inch² is a meaningless unit and a better statistical term to use is the standard deviation.
4. Standard deviation, also called one-sigma, is the square root of the variance = $= \surd 2.5=1.581 inch \left(\text{is sample mean}\right)$
5. Root Mean Square Error (RMSE)
   • RMSE is not Standard deviation or sigma; they are different.
   • Root Mean Square Error (RMSE) is computed as follows:
     $RMSE=\surd \left(\Sigma {\left(Z–Z_i\right)}^2/n\right)$
     Where,
     Z = Measured Value from the data
     Z_i = Control Value (field surveyed)
     n = number of measurements

Relationship Between Standard Deviation and Root Mean Square Error (RMSE)

Facts about RMSE:

• Includes random and systematic errors
• More useful to use as it reveals biases (systematic error)
• It tells us how accurate the data is

Facts about Standard Deviation:

• Includes only random error
• Reflects only how precise the data is
• It does not tell us how accurate the data is in the presence of biases. It only tells us how precise the data is.

Table 1 illustrates the difference between standard deviation and the RMSE in revealing the presence of biases in measurements. The table represents a vertical accuracy evaluation for points cloud derived from UAS imagery by comparing it to a higher accuracy elevation model derived from a mobile lidar mapping system. The UAS-derived elevation model needed to meet 5-cm (0.164-ft) accuracy. If we used standard deviation alone, the data would meet the specifications with a value of 0.076-ft. However, looking at the high value of 0.246-ft. (7.5-cm) of the mean, it is obvious this data set contains a bias and the only way to catch it is by either evaluating the value of the mean or using the RMSE as the accuracy measure. The high value of the RMSE = 0.257-ft. (7.83-cm) will flag the data as not meeting specifications. The far right column contains the error values after removing the bias of  0.246-ft. (7.5-cm) from the measurements. Once we remove the bias, the values for the RMSE and the standard deviation are equal and they both meet the project accuracy specifications. Removing a bias from elevation data could be as simple as shifting the entire dataset up or down by the magnitude of the bias itself, such practice is called z-pump. 

Table 1 Vertical Accuracy Tabulation of Geospatial Product

Click for a text description of Table 1.

Normal Distribution Curve

In randomly distributed repeated measurements, measurements values will vary around the mean or the average, with most values being closer to the average. Deviation from such behavior indicates the presence of bias(es) or perhaps blunders in the measurements. Figure 4 shows a true random distribution of a set of measurements that do not contain biases. For the measurement’s distribution in Figure 4, we notice that 68.2% of the measured values fall within +/- 1 RMSE or +/- 1 sigma from the mean value, that is 34.1% on both sides of the mean. We also notice that 95% of the measurements fall within +/- 2 RMSE or +/- 2 sigma from the mean. Understanding such distribution is essential to understanding the map accuracy standard we are going to discuss in the following sections.

Figure 4 Random probability distribution curve, the bell curve

Enter image credit here

Common Error Estimation Terms

Table 2 lists the most common terms used to estimate errors in surveying and mapping. Probable error is the term used to describe the probability, or the confidence level, that 50% of the errors fall within, while 95% errors represents the confidence level that 95% of the measured errors values fall under.

Positional Errors and Accuracy

According to the ASPRS Positional Accuracy Standards for Digital Geospatial Data, the terms positional error and absolute and relative accuracy are defined as follow:

• Positional error – The difference between data set coordinate values and coordinate values from an independent source of higher accuracy for identical points.
• Positional accuracy – The accuracy of the position of features, including horizontal and vertical positions, with respect to a horizontal and vertical datum.
• Relative accuracy – A measure of variation in point-to-point accuracy in a data set.
• Relative accuracy – Characterizes the internal geometric quality of an elevation data set without regard to surveyed ground control.

The New ASPRS Positional Accuracy Standards for Digital Geospatial Data

In November of 2014, the American Society of Photogrammetry and Remote Sensing (ASPRS) published Edition 1 of the first ever mapping accuracy standards that are solely designed for today's digital geospatial data. Edition 2 was published on August 23, 2023 to correct some measures to suite today's technologies and processes and add six addenda on best practices and guidelines.

Motivation Behind the New Standard is:

• Legacy map accuracy standards, such as the ASPRS 1990 standard and the National Map Accuracy Standards (NMAS) of 1947, are outdated (over 30 years since ASPRS 1990 was written).

• Many of the data acquisition and mapping technologies that these standards were based on are no longer used.

• More recent advances in mapping technologies can now produce better quality and higher accuracy geospatial products and maps.

• Legacy map accuracy standards were designed to deal with plotted or drawn maps as the only medium to represent geospatial data.

• Within the past two decades (during the transition period between the hardcopy and softcopy mapping environments), most standard measures for relating GSD and map scale to the final mapping accuracy were inherited from photogrammetric practices using scanned film.

• New mapping processes and methodologies have become much more sophisticated with advances in technology and advances in our knowledge of mapping processes and mathematical modeling.

• Mapping accuracy can no longer be associated with camera geometry and flying altitude alone (focal length, xp, yp, B/H ratio, etc.).

• New map accuracy is influenced by many factors such as:
  • the quality of camera calibration parameters;
  • quality and size of a Charged Coupled Device (CCD) used in the digital camera CCD array;
  • amount of imagery overlaps;
  • quality of parallax determination or photo measurements;
  • quality of the GPS signal;
  • quality and density of ground controls;
  • quality of the aerial triangulation solution;
  • capability of the processing software to handle GPS drift and shift;
  • capability of the processing software to handle camera self-calibration,
  • the digital terrain model used for the production of orthoimagery.

These factors can vary widely from project to project, depending on the sensor used and the specific methodology. For these reasons, existing accuracy measures based on map scale, film scale, GSD, c-factor and scanning resolution no longer apply to current geospatial mapping practices.

• Elevation products from the new technologies and active sensors such as lidar, UAS, and IFSAR are not considered by the legacy mapping standards. New accuracy standards are needed to address elevation products derived from these technologies.

The New Standard Highlights

• Sensor agnostic, data driven: Positional Accuracy Thresholds which are independent of published GSD, map scale or contour interval
• It is All Metric!
• Unlimited Horizontal Accuracy Classes:
• Added additional Accuracy Measures
  • Aerial triangulation accuracy,
  • Ground controls accuracy,
  • Orthoimagery seam lines accuracy,
  • Lidar relative swath-to-swath accuracy,
  • Recommended minimum Nominal Pulse Density (NPD)
  • Horizontal accuracy of elevation data,
  • Delineation of low confidence areas for elevation data
  • Required number and spatial distribution of QA/QC check points based on project area

• Added six addenda on best practices and guidelines for:

            1) General Best Practices and Guidelines 

             2) Field Surveying of Ground Control and Checkpoints

             3) Mapping with Photogrammetry 

             4) Mapping with Lidar 

             5) Mapping with UAS

             6) Mapping with Oblique Imagery 

Advantage of Specifying the New ASPRS Positional Accuracy Standards for Digital Geospatial Data for a Project

Users of the new standards do not have to specify accuracy details for the intermediate processes in products generation. The user needs to specify the final deliverable product accuracy and the new standards will set up all accuracy specifications for intermediate processes, such as ground survey, aerial triangulation, etc., involved in the production of the final product. Figure 5 illustrates such a concept.

Figure 5 Accuracy settings according to ASPRS standards

Enter image credit here

Some of the highlights of the new ASPRS Horizontal Accuracy Standards for Geospatial Data are the following:

1. Unlimited horizontal accuracy classes:
   The new standard was designed to fit any horizontal accuracy requirement no matter what technology, current or future, is used. Table 3 represents the new ASPRS horizontal accuracy classes.

   Table 3 The new ASPRS horizontal accuracy classes

   Horizontal Accuracy Class	Absolute Accuracy RMSEH (cm)	Orthoimagery Mosaic Seamline Mismatch (cm)
   #-cm	$\le \text{\#}$	$\le 2*\text{\#}$

    RMSE_H = RMSE_x ² +RMSE_y ²                     RMSE_H =Radial RMSE=Circular RMSE=Two−dimensional RMSE of X & Y 
2. Aerial triangulation results should be twice as accurate as the generated products:
   1. Ortho and planimetric maps ONLY: $\begin{array}{l}\hfill \end{array}$RMSE_H(AT) = ½ * RMSE_H(Map) and RMSE_V(AT) = RMSE_H(Map) 
   2. For ortho, planimetric maps, and elevation maps: R M S E _{H ( A T )} =  ½  * R M S E _{H ( M a p )} and  R M S E_{V ( A T )} = R M S E _{H (DEM )} 
3. Control points for aerial triangulation should be twice as accurate as aerial triangulation:
   1. For ortho and planimetric maps ONLY: $\begin{array}{l}\hfill \end{array}$R M S E _{H (} GCP ₎  = ½ * R M S E _{H ( M a p )} and  R M S E_{V (} GCP ₎ = R M S E _{H ( M a p )}  
   2. For ortho/planimetric maps and elevation maps: R M S E _{H (} GCP ₎ = ½ * R M S E _{H ( M a p)} and  R M S E_{V (}GCP ₎ = ½ * R M S E _{V (DEM )} 

Table 4 lists common horizontal accuracy classes for geospatial mapping products.

Some of the highlights of the new ASPRS Vertical Accuracy Standards for Geospatial Data are the following:

1. Unlimited horizontal accuracy classes: The new standard was designed to fit any vertical accuracy requirement no matter what technology, current or future, is used. Table 5 represents the new ASPRS vertical accuracy classes.
   

   Table 5 The new ASPRS vertical accuracy classes

   Click for a text description of Table 5.
2. The standards introduced two vertical accuracy types, those are:
   1. Non-vegetated Vertical Accuracy (NVA) for any part of the project that is not covered by vegetation.
   2. Vegetated Vertical Accuracy (VVA) for the part of the project that is partly or fully covered by vegetation.
3. The standards introduced relative accuracy for elevation data beside the absolute accuracy.
   Table 6 lists a new accuracy term, which is the relative accuracy. It is mainly addressing the Lidar-derived elevation data. The table also also provides vertical accuracy examples and other quality criteria for ten common vertical accuracy classes 

Table 6 Vertical Accuracy/Quality Examples for Digital Elevation Data

4. The standards introduced horizontal accuracy estimation for elevation data

1. For Photogrammetric elevation data: the horizontal accuracy equates to the horizontal accuracy class that would apply to planimetric data or digital orthoimagery produced from the same source imagery, using the same aerial triangulation/INS solution.
2. For Lidar elevation data: use the following formula:

Table 7 lists some horizontal accuracy values for lidar data based on the previous formula (the GNSS horizontal accuracy is assumed to be equal to 0.10 m, the IMU error is assumed to be 10.0 arc-seconds for the roll and pitch and 15.0 arc-seconds for the heading)

5. The Standards Introduced a Formal Accuracy Testing Statement:
For the first time, the new standards provide users with formal data evaluation statements to be used by the data users and data producers. The following statement are examples of the accuracy statement of an elevation dataset:

5.1 Accuracy Reporting by Data User or Consultant

This type of reporting should only be based on a set of independent checkpoints. The positional accuracy of digital orthoimagery, planimetric data, and elevation data products shall be reported in the metadata in one of the manners listed below. For projects with NVA and VVA requirements, two three-dimensional positional accuracy values should be reported based on the use of NVA and VVA, respectively.

5.1.1 Accuracy Testing Meets ASPRS Standard Requirements

If testing is performed using a minimum of thirty (30) checkpoints, accuracy assessment results should be reported in the form of the following statements:

• • Reporting Horizontal Positional Accuracy

“This data set was tested to meet ASPRS Positional Accuracy Standards for Digital Geospatial Data, Edition 2 (2023) for a __(cm) RMSE_H horizontal positional accuracy class. The tested horizontal positional accuracy was found to be RMSE_H = __(cm)”.

• • Reporting Vertical Positional Accuracy

“This data set was tested to meet ASPRS Positional Accuracy Standards for Digital Geospatial Data, Edition 2 (2023) for a __(cm) RMSE_V Vertical Accuracy Class. NVA accuracy was found to be RMSE_V = __(cm).” VVA accuracy was found to be RMSE_V = __(cm).”

• • Reporting Three-Dimensional Positional Accuracy

“This data set was tested to meet ASPRS Positional Accuracy Standards for Digital Geospatial Data, Edition 2 (2023) for a ___ (cm) RMSE_3D three-dimensional positional accuracy class. The tested three-dimensional accuracy was found to be RMSE_3D = ___(cm).”

5.1.2 Accuracy Testing Does Not Meet ASPRS Standard Requirements

If testing is performed using fewer than thirty (30) checkpoints, accuracy assessment results should be reported in the form of the following statements:

• • Reporting Horizontal Positional Accuracy

“This data set was tested as required by ASPRS Positional Accuracy Standards for Digital Geospatial Data, Edition 2 (2023). Although the Standards call for a minimum of thirty (30) checkpoints, this test was performed using ONLY __ checkpoints. This data set was produced to meet a ___(cm) RMSE_H horizontal positional accuracy class. The tested horizontal positional accuracy was found to be RMSE_H = ___(cm) using the reduced number of checkpoints.”

• • Reporting Vertical Positional Accuracy

“This data set was tested as required by ASPRS Positional Accuracy Standards for Digital Geospatial Data, Edition 2 (2023). Although the Standards call for a minimum of thirty (30) checkpoints, this test was performed using ONLY __ checkpoints. This data set was produced to meet a ___(cm) RMSE_V vertical positional accuracy class. The tested vertical positional accuracy was found to be RMSE_V = ___(cm) using the reduced number of checkpoints.”

• • Reporting Three-Dimensional Positional Accuracy

“This data set was tested as required by ASPRS Positional Accuracy Standards for Digital Geospatial Data, Edition 2 (2023). Although the Standards call for a minimum of thirty (30) checkpoints, this test was performed using ONLY __ checkpoints. This data set was produced to meet a ___(cm) RMSE_3D three-dimensional positional accuracy class. The tested three-dimensional positional accuracy was found to be RMSE_3D = ___(cm) using the reduced number of checkpoints.”

5.2 Accuracy Reporting by Data Producer

In most cases, data producers do not have access to independent checkpoints to assess product accuracy. If rigorous testing is not performed by the data producer due to the absence of independent checkpoints, accuracy statements should specify that the data was “produced to meet” a stated accuracy. This “produced to meet’’ statement is equivalent to the “compiled to meet” statement used by prior Standards when referring to cartographic maps. The “produced to meet’’ statement is appropriate for data producers who employ mature technologies, and who follow best practices and guidelines through established and documented procedures during project design, data processing and quality control. However, if enough independent checkpoints are available to the data producer to assess product accuracy, it will do no harm to report the accuracy using the statement provided in section 4.1 above.

If not enough checkpoints are available, but the data producer has demonstrated that they are able to produce repeatable, reliable results and thus able to guarantee the produced-to-meet accuracy, they may report product accuracy in the form of the following statements:

• • Reporting Horizontal Positional Accuracy

“This data set was produced to meet ASPRS Positional Accuracy Standards for Digital Geospatial Data, Edition 2 (2023) for a __(cm) RMSE_H horizontal positional accuracy class.

• • Reporting Vertical Positional Accuracy

“This data set was produced to meet ASPRS Positional Accuracy Standards for Digital Geospatial Data, Edition 2 (2023) for a __(cm) RMSE_V vertical accuracy class.

• • Reporting Three-Dimensional Positional Accuracy

“This data set was produced to meet ASPRS Positional Accuracy Standards for Digital Geospatial Data, Edition 2 (2023) for a ___ (cm) RMSE_3D three-dimensional positional accuracy class

6. The Standards Introduced a new accuracy term, the Three-Dimensional Positional Accuracy:

The following formula defines the three-dimensional accuracy standard for any three-dimensional digital data as a combination of horizontal and vertical radial error. RMSE_3D is derived from the horizontal and vertical components of error according to the following formula:

7. The Standards Introduced a new approach for assessing product accuracy by factoring in the accuracy of the surveyed check points when computing product accuracy:

As we are producing more accurate products, errors in surveying techniques of the checkpoints used to assess product accuracy, although it is small, can no longer be neglected and it should be represented in computing the product accuracy. Currently, we quantify products accuracy ignoring the errors in the surveyed check points. In such practice, our surveying techniques approximates the datum, i.e., producing pseudo datum and therefore, we are evaluating the closeness of data to the pseudo datum and not the true datum. The following figure illustrates the current practices and the new one proposed in Edition 2 of the ASPRS standards.

Figure 6 Factoring in the accuracy of the surveyed check points when computing product accuracy

Best Practices in Determining Product Accuracy*

1. Check data should not be used in calibrating the tested products:
   • Totally independent check points
2. Check data must be more accurate than tested data:
   • Two times more accurate
3. Check data must be well distributed around the project area:
4. Check data must be a valid statistical sample:
   • Minimum of 30 check points for orthos
   • Minimum of 30 check points for elevation data

* according to the ASPRS Positional Accuracy Standards for Digital Geospatial Data, Edition 2 of 2023 (https://publicdocuments.asprs.org/PositionalAccuracyStd-Ed2-V1 [17])

The new ASPRs Standards and number of check points

The new standards provide Table 8 for the recommended number of check points required for validating product accuracies. For project areas that are larger than 10,000 square kilometers, use only 120 checkpoints.

Find more information about the new ASPRS standards here [18].

Download a copy of the standards here [19].

Summary

Congratulations! You have just completed Lesson 9. You may have noticed from the different sections of the lessons that the UAS market is growing rapidly. There are quite a few manufacturers for the civilian UAS, as well as software and sensor producers. User requirements will drive the selection process for the UAS and the processing software that is right for the job. Required UAS endurance, range and payload capacity will be different from one application to another. However, most applications will prefer more endurance, longer range, and heavier payload if the price is right.

In this lesson, you also learned about the value of evaluating data quality and accuracy and how to use the new ASPRS standards to report such quality and accuracy factors. 

By now, you must be finishing the products generation of ortho photo and digital elevation model using Pix4D and the sample imagery. Samples of the products need to be submitted with your project report and presented next week during your presentation.

Final Tasks