Raster Management

Esri offers a number of different options for managing raster data in an enterprise geodatabase. One area that receives a good deal of attention is managing collections of adjacent raster data sets (e.g., aerial photos). The options for dealing with such collections range from working with the raster data sets individually to merging them together into one large data set. In between is something Esri calls a mosaic dataset which attempts to provide the best of both worlds, the ability to work with multiple raster data sets as one combined layer or to break them up into their individual components. We'll talk about a couple of these approaches in this section of the lesson.

A. Raster datasets

Let's see how to bring a raster data set into a geodatabase.

Download this zipped raster image of the earth at night.
Open a Windows Remote Desktop Connection to your enterprise geodatabase instance if you haven't already.
Copy the earthatnight.zip file to the C:\data folder on your remote instance and unzip it.
Open ArcGIS Pro and open the Geoprocessing pane.
Open the Raster to Geodatabase tool (Conversion Tools > To Geodatabase).
For the top Input Rasters parameter, browse to your C:\data folder and select your earthatnight.bil raster.
For the Output Geodatabase parameter, browse to one of your data loading users (e.g., dbo_egdb.sde).
Click Run to begin the import.
After the import has completed, refresh the connection's listing if necessary, and add the raster to a map to confirm that it imported properly.

Now, let's take a moment to look at how the imported earthatnight raster looks from within SQL Server.
Close ArcMap and open SQL Server Management Studio (SSMS).
In SSMS, browse to Databases > egdb > Tables.
Open the EARTHATNIGHT table, and note that it contains just one row. This table doesn't store the raster data itself, but rather the footprint of the raster (a polygon representing the area that it covers). Other information associated with the raster can be found in the following supporting tables:
- SDE_blk_1: pixel data
- SDE_bnd_1: metadata on the raster's bands
- SDE_aux_1: statistics (similar to those used to improve vector dataset performance) and color maps
The "_1" part of these table names comes from the rastercolumn_id value assigned to the raster, found in the SDE_raster_columns repository table. If the earthatnight raster instead had a rastercolumn_id of 2, its pixel data would be stored in SDE_blk_2, for example.

Raster datasets can hold either integer or floating-point data. They can also be comprised of multiple bands. If you have a single-band integer raster dataset, a value attribute table (VAT) can be built that stores the number of cells associated with each integer value. The earthatnight raster holds integer values, but it is comprised of three bands. If it were a single-band integer raster dataset, we would see an SDE_vat_1 table in addition to the other tables.

B. Mosaic datasets

As mentioned at the beginning of this section, it is common in GIS to deal with collections of adjacent raster datasets. Esri's mosaic dataset can be used to treat such collections as a single unit while still having the ability to work with the individual files as needed. Let's create a mosaic dataset to manage some elevation raster data sets for the State College, PA, area (found in the doqs folder from the Lesson7_data download).

First, you need to upload the DOQ data mentioned above from your machine to the C:/data folder on your instance.

On your local machine, unzip the Lesson7_data.zip archive that you downloaded at the beginning of the lesson.

Open a Remote Desktop Connection to your instance if you need to. Then, as you did for the earthatnight zip file above, Copy and Paste the Lesson7_data\doqs folder from your computer to the C:\data folder of your instance. This will take a few minutes.
Next, in your instance, start a new ArcGIS Pro project. For simplicity's sake, we'll just continue working as the dbo user.
In the Catalog pane, activate the dbo_egdb connection, right-click on it, and select New > Mosaic Dataset.

Confirm that the Output Location is set to your dbo_egdb database connection.

Set the Mosaic Dataset Name to state_college_mosaic.

Open the Coordinate System dialog, and Import the coordinate system definition from one of the three images in the doqs folder.

Leave the Product Definition parameter set to None since we're dealing with generic black-and-white orthophotos rather than imagery coming from a particular product. But take a moment to look over the options for this parameter, especially if you commonly work with imagery.

Click Run to create the mosaic dataset. Once it has been created, ArcMap will add it to the data frame as a group layer. We haven't added images to the mosaic dataset yet, so the group layer has nothing to show at this point.

Before adding images to the mosaic dataset, you should first build pyramids and calculate statistics for the images. Pyramids are simply lower-resolution versions of raster datasets that improve drawing performance at smaller scales. Without pyramids, a raster will be drawn at its highest resolution at all scales; users will wait longer than necessary for the drawing of image detail that can't even be appreciated. An important point to remember about pyramids is that they're used purely for display purposes. When analysis is conducted using the raster as an input, its normal resolution will be used.
In the Geoprocessing pane, open the Build Pyramids And Statistics tool (found under Data Management Tools > Raster > Raster Properties).

For the Input Data or Workspace parameter, browse to the doqs folder, select it, and click OK.

Before building pyramids for a raster, you should give some thought to two components of the process – the resampling and compression methods. ArcGIS provides three resampling methods:
- nearest neighbor
- bilinear interpolation
- cubic convolution
The nearest neighbor method is best for discrete raster datasets (like a land use or soils grid) and for scanned maps. Bilinear interpolation and cubic convolution are better suited for continuous raster datasets (like an elevation grid) and for satellite imagery and aerial photography. Bilinear interpolation is faster at pyramid creation time than cubic convolution, but on the flip side, cubic convolution typically produces the most visually pleasing output.

The pixel data created by the pyramid building process can be compressed to reduce storage requirements and improve performance. Higher levels of compression can be achieved with different methods, though care should be taken to match the data's use to an appropriate compression method.

The LZ77 method is referred to as a loss-less compression method because it results in no degradation of the input data. It should be used when the highest accuracy possible is required. The other method, JPEG, can produce a significantly higher level of compression, though at the expense of some degradation of the input data. Thus, it is referred to as a lossy compression method. The JPEG method can be used in situations when the highest level of spatial accuracy is not really necessary.

Click the Environments button, then click on the Raster Storage heading to access the resampling and compression options. Since we're dealing with aerial photography, let's use the Bilinear Interpolation resampling option; select Bilinear from the Resampling technique list.
A bit of degradation in the raster quality in the pyramid data is acceptable, so let's go with the JPEG compression option.

Choose JPEG from the Compression type dropdown list, and accept the default Quality value of 75.

In the Raster Statistics part of the dialog, you'll see a couple of "skip factor" options, one for the x dimension and one for the y. These values specify how many rows/columns to skip when computing statistics on the raster. The default skip factor value is 1 for each dimension, which means that cell values are retrieved for every other row and every other column. This decreases the time required to calculate the statistics, though it also decreases the accuracy of the statistics. In most cases, a skip value of 1 should produce statistics that are "good enough".

The Statistics ignore value is a value or list of values that should not be included in the statistics calculations. For example, if you used a value like -9999 for missing data, you would want to specify that as the ignore value to avoid generating distorted statistics.

Accept all of the defaults in the Raster Statistics part of the dialog. Click OK to close the Environment Settings window.
Click Run to build pyramids and statistics for each of the orthophoto rasters. This will take a few minutes - note the ticker-tape at the bottom of the window. Once the pyramid and statistics building process is complete, you're ready to add the raster data sets to the mosaic dataset.
In the Geoprocessing pane, open the Add Rasters to Mosaic Dataset tool, found under Data Management Tools > Raster > Mosaic Dataset.

For the Mosaic Dataset parameter, select egdb.DBO.state_college_mosaic from the dropdown list.

Leave the Raster Type and Processing Templates parameters set to their default values of Raster Dataset and Default, respectively.

Set the Input Data parameter to Folder because you're going to add all images from a folder.

Click the browse icon button just beneath the Input Data dropdown menu, navigate to your doqs folder, select it and click OK.

Click Run to proceed with adding the rasters to the mosaic dataset (state_college_mosaic).
When Pro has finished adding the images, you should see them displayed as a single unit (a group layer) which can be turned on/off by checking its box in the Display pane. You may need to refresh the view by using the Full Extent tool or by right-clicking on the group layer and choosing Zoom to Layer. Note that the Boundary layer shows the outline of the mosaicked data, while the Footprint layer shows the area covered by the individual input rasters.

After creating a mosaic dataset, it is good practice to run an Analyze on it as you would do with a new vector dataset.
In the Geoprocessing pane, open the Analyze Mosaic Dataset tool (found at the same Data Management Tools > Raster > Mosaic Dataset path), select the state_college_mosaic dataset, and click Run.

Now, let's have a look at the dataset's attribute table.
Right-click on the state_college_mosaic group layer in the Display pane and select Open Table > Attribute Table. This opens the Footprint layer attribute table.
Note that the table's Name field includes references to the original raster datasets. An important point to remember about mosaic datasets is that the input rasters remain in their original location. They are not transferred or stored in the database unless they had started out that way (e.g., like the earthatnight raster).
In the Display pane, confirm that the Boundary layer is toggled off and the Footprint and Image layers are toggled on.

As mentioned above, a mosaic dataset combines raster datasets together but still allows for working with the original individual raster datasets. We'll wrap up this section by displaying just two of the original images.
Activate the Select tool, and click-drag a rectangle that touches on two of the DOQ images.

Note what is selected. The actual image pixels are not selectable. Go ahead and turn off the visibility of the Image layer if you want.

Look in the open Footprint table at what records have been selected. It should be two of the named DOQs.
Now, right-click the state_college_mosaic layer in the Display pane, and click Selection > Add to Map.

In the resulting dialog, enter My Rasters for the Group layer name and confirm that Name is selected from the Layer name based on dropdown list. This simply specifies that your raster(s) will be labeled based on the value in the Name field in the mosaic dataset's attribute table.

Click OK to add your raster(s) to the map.
Turn off the state_college_mosaic group layer visibility, and turn on the new My Rasters group layer (the boxes for each raster need to be checked).

Always remember to Stop your Instance when you finish or when you take a long break.

Raster Management