Primer on Threat Scores

As it turns out, the Hydrometeorological Prediction Center keep score with regard to their predictions for heavy precipitation. For a given event, forecasters compute threat scores by comparing the area where they predicted heavy precipitation with the area that ultimately received heavy rain or heavy snow.

You can think of a threat score as the ratio of the area where the forecast was accurate to the area where the forecast didn't verify. In the figure below, the hatched area, C, represents the region where the forecast for heavy precipitation was correct. F, the red shaded area, is the region for which HPC forecasted heavy precipitation. OB, the green shaded area, indicates the region where heavy precipitation fell (observed). By definition, the threat score (T) is calculated using the equation: T = C/(F + OB - C)

A schematic defining the areas used to compute a threat score (T). The hatched area, C, represents the region where the forecast for heavy precipitation verified. F, the red shaded area, is the region for which HPC forecasters predicted heavy precipitation. OB, the green shaded area, indicates the region where heavy precipitation fell.

Thus, threat score for a perfect forecast is 1, while a completely busted forecast gets a big fat "0". Read more.

The figure below shows the trend in annual threat scores based on predictions for one inch of rain (or a liquid-equivalent of one inch) during each year. To put this graph in proper context, I point out that HPC bases all of QPF verifications on the 12Z-to-12Z period. As for the graph itself, please note that Day 1 represents the forecast period from 12 to 36 hours (HPC calculates forecast hours based on the 00Z model runs). Day 2 forecasts cover 36 to 60 hours, while Day 3 spans from 60 to 84 hours.

The annual threat scores for predicting one inch of rain or liquid equivalent from 1960 to 2005.

These threat scores indicate that HPC typically scores less than 0.24 for Day-2 forecasts of one inch of rain (or liquid equivalent). Such a score translates to HPC getting less than 40% of the predicted area of heavy rain or snow (one inch of rain or liquid equivalent) correct 36 to 60 hours in advance. And the forecasters at HPC are very good. I can't imagine what Hale Stone's threat scores would be.

Lesson learned: The track record for pinpointing regions of heavy snow more than 48 hours in advance of the event is not very good. We're getting better (as the upward trend on the graph indicates), but forecasting 6 to 10 inches of rain from Hurricane Charley more than 48 hours in advance was simply "grandstanding" and totally irresponsible.