Ozone Forecast Statistics Calculator

Introduction : This calculator calculates verification statistics for a two-category forecast. In this example, we are looking at whether or not we have exceeded an ozone concentration level of 85 ppb (an "event") or if we have not exceeded 85 ppb (a "non-event"). This data shows the results of observing for an event or non-event for 180 days, and forecasting for the event and non-event for 180 days. The statistics this calculator calculates are:

  1. Accuracy: percentage of forecasts that correctly predicted the event or non-event
  2. Bias: indicates if the forecasts are underpredicted (false negatives) or overpredicted (false positives)
  3. False alarm rate (FAR): percentage of times a forecast of high ozone did not actually occur
  4. Critical success index (CSI): describes how well the high ozone events were predicted. This value is unaffected by a large number of correctly forecasted, low-ozone events
  5. Probability of detection (POD): ability to predict high ozone events
  6. Skill score: percentage improvement of a forecast with respect to a reference forecast, typically a climatology or persistence forecast. In this calculator, we have set the reference accuracy at 50% (basically chance, we have a 50-50 chance of getting it right!)
The data in the default calculator shows these values:

Observed: 138 (130+8) days were "non-events", meaning that the ozone levels were below 85 ppb. 42 (20 +22) days were observed as being at or above 85 ppb.

Forecasted: we predicted that 150 (130+20) days would be non-events, while 30 (8 +22) days would have concentrations at or greater than 85 ppb.

How did we do?

For this dataset, our accuracy is 84%. With a bias of 0.71, we have under-forecasted a high-ozone event. We did pretty well with the false alarm rate, we only predicted a bad ozone day 27% of the time, and it didn't happen. With a critical success index of 44%, slightly less than half of our forecasts for high ozone were correct. With a score of 52% for the probability of detection, roughly five (5) out of every 10 forecasted high ozone events actually occurred. Our skill score of 68% indicates our forecast was 68% better than our forecast using 50-50 chance.

Ozone Forecast Statistics Calculator

Input Values:

  Forecast Exceedences
Observed Exceedences
  Forecasted low Forecasted high
Observed low
Observed high
Reference Accuracy (Aref)

Results:

Accuracy
Bias
False Alarm Rate (FAR)
Critical Success Index (CSI)
Probability of Detection (POD)
Skill Score (SS)