PrintAt some point during this course, you've hopefully felt "the lightbulb go on" regarding how you might apply the lesson material to your own tasks in the GIS workplace. To conclude this course, just as we did in Geog 485, you will be expected to complete an individual project that uses Python automation to make some GIS task easier, faster, or more accurate.
The project goal is up to you, but it is preferably one that relates to your current field of work or a field in which you have a personal interest. We don't mind what packages or libraries you use – it can be an arcpy project, a Jupyter notebook, a QGIS program or plugin (to be discussed in Lesson 4), a standalone program, or some combination of the above.
What we are expecting is that you will use the advanced concepts that you have learned in the class where appropriate, so perhaps that means using list comprehension or multiprocessing or regular expressions (or all of them) depending on the problem that you're trying to solve. Feel free to use any of the packages that we've used already in the class (PyQt5, pandas, GDAL, etc.) or any others that interest you.
Since you're defining the requirements from the beginning, there is no "over and above" credit factored into this project grade. The number of lines of code you write is not as important as the problem you solve. However, we ask you to propose a project that meets or even slightly exceeds your relative level of experience with programming and that is significantly more complex than what you did as a term project for Geog 485.
You will have two weeks at the end of the term to dedicate completely toward the project. This is your chance to apply what you've learned about Advanced Python to a problem that really interests you. As with Geog 485, if you're implementing a solution for a work problem – that's fine – feel free to use synthetic (made up) data if that's necessary to ensure confidentiality.
One week into Lesson 4 you are required to submit a project proposal to the Final Project Proposal Drop Box in Canvas. This proposal must clearly explain:
- The task you intend to accomplish using Python
- How your proposed solution will make the task easier, faster, and/or more accurate. Also explain why your task could not simply be accomplished using the "out-of-the-box" tools from Esri, QGIS etc., or why your script gives a particular advantage over those tools
- What kind of tool this will be (arcpy script, Jupyter notebook, QGIS plugin, standalone Python program, …) and why you have decided to take this approach, as well as what other packages/libraries you expect to use for the project
- The deliverables you will submit for the project. Well-documented code is expected. If the script/tool/application requires data, describe how the instructors will be able to evaluate your script. Possible solutions are to zip a sample dataset for the instructors, demonstrate your script during a recorded video, or make the script flexible enough that it could be used with any dataset
The proposal will contribute toward 10% of your Final Project grade so please put some effort in describing your idea as clearly and in as much detail as possible. The proposal will also be used to help grade the rest of your project. Your proposal must be approved by the instructors before you move forward with coding the project. We may also offer some guidance on how to approach your particular task, and we'll provide thoughts on whether you are taking on too much or too little work to be successful.
As you work on your project, you're encouraged to seek help from all resources discussed in this class, including existing code samples and scripts on the Internet. If you re-use any sections of code that you found on the Internet, please thoroughly explain in your project writeup how you found it, tested it, and extracted only the parts you needed.
You will be graded based on the level of difficulty of your project, relative to your skills, the quality of the code, documentation and writeup, whether or not the code solves the problem you defined, and the elegance of your solution (is it efficient and well crafted; there are some tips on how to write elegant code here - Writing Elegant and Readable Code).
Project Ideas
If you're having trouble thinking up a project, you can derive a proposal from one of the suggestions here which are broadly the same as in Geog 485. But now we expect that you'll extend and expand on them significantly using what you know about various packages, libraries and delivery methods (Jupyter Notebooks, QGIS plugins and building your own GUI tools). So please see these only as core ideas to develop a project around, not as complete task specifications for a project.
You may have to spend a little bit of time acquiring or making up some test datasets to fit these project ideas. I also suggest that you read through the Lesson 4 material before selecting a project, just so you have a better idea of what types of things are possible with Python.
- Compare dataset statistics: Make a tool or script that takes two feature classes or other datasets as input, along with field names. The tool should check whether or not the field data is the same in both data sources and exists in both feature classes. If both these conditions are met, the tool should calculate statistics for that field for both feature classes and report the difference. Statistics could be sum, average, standard deviation, etc. and produce an output file.
- Compare existence of features in two datasets: Make a tool or script that reads two feature classes based on a key field (such as OBJECTID). The tool should figure out which features only appear in one of the feature classes and write them to a third feature class. As a variation on this, the tool could figure out which features appear in both feature classes and write them to a third feature class. You should allow the tool user to set a parameter to determine this – perhaps even with a custom interface.
- Calculate and compare areas: Make a tool or script that tallies the areas of all geometries in a feature class, or subsets of geometries based on a query and reports the difference. For example, this tool might compare "Acres of privately owned wetlands in 2008" and "Acres of privately owned wetlands in 2009." Make your tool available in a number of locations or accessible in a number of ways (e.g. as an arcpy / ArcGIS script, QGIS plugin and on github as well).
- Find and replace: Make a tool flexible enough to search for any term in any field in a feature class and replace it with another user-provided term. Ensure in your code that users cannot modify the critical fields' OBJECTIDs or SHAPEs. Also ensure that partial strings are supported, such that if the search term is found anywhere within a string, it will be replaced while leaving the rest of the string intact.
- Process rasters to meet an organizational need: Write a tool or script that takes a raw elevation dataset (such as a DEM), clips it to your study area, creates both hillshade and slope rasters, and projects them into your organization's most commonly used projection. Expose the study area feature class to the end user as a parameter and perhaps perform all of this analysis in a Jupyter notebook and display the results.
- Parse raw textual data and write to feature class: Find some data available on the Internet that has lat/lon locations but is in text-based format with no Esri feature class available (for example, weather station readings or GPS tracks). If you need to, you can copy the HTML out of the Web page and paste it in a .txt file to help you get started. Read the .txt file and write the data to a new feature class. We've covered this in the lesson so you'll really need to extend this idea to make it interesting.
- Do a performance comparison between arcpy and gdal tools (for example) or numpy arrays and R code to perform comparable tasks (e.g. calculating slope / aspect for a raster) or updating values in a shapefile / table.
- Map data using arcpy mapping and incorporate plotly graphs of statistics into the map book with custom layouts, color schema or graphs depending on the type of analysis a user specifies (either from an Arc toolbox or via a pyqt5 interface).