Skip to contents

Calculate Forecast Skill Scores

skill_scores.Rd

This function calculates various skill scores and statistics for evaluating the performance of binary forecasts based on a 2x2 contingency table.

Usage

skill_scores(hit, miss, false_alarm, quiescent)

Arguments

hit

Number of hits (correct positive forecasts)

miss

Number of misses (false negative forecasts)

false_alarm

Number of false alarms (false positive forecasts)

quiescent

Number of correct rejections (correct negative forecasts)

Value

A named list containing the following components:

  • hit: Number of hits (input)

  • miss: Number of misses (input)

  • false_alarm: Number of false alarms (input)

  • quiescent: Number of correct rejections (input)

  • EF: Event forecasts (hit + false_alarm)

  • NEF: Non-event forecasts (miss + quiescent)

  • E: Total events (hit + miss)

  • NE: Total non-events (false_alarm + quiescent)

  • SS: Sample size (total observations)

  • POD: Probability of Detection (hit rate)

  • FOM: Frequency of Misses (miss rate)

  • FAR: False Alarm Ratio

  • FOH: Frequency of Hits (hit rate among forecasts)

  • PON: Probability of Null (correct rejection rate)

  • POFD: Probability of False Detection (false alarm rate)

  • DFR: Detection Failure Ratio

  • FOCN: Frequency of Correct Null forecasts

  • CSI: Critical Success Index

  • TSS: True Skill Statistic

  • HSS: Heidke Skill Score

Details

The function calculates skill scores commonly used in meteorology and other forecasting domains. The contingency table structure is:


                   Observed
                Yes       No
Forecast  Yes   hit   false_alarm
          No   miss   quiescent

Correspondence to ML terminology:

Meteorology/ForecastingML/Statistics
POD (Probability of Detection)Recall/Sensitivity/TPR
POFD (Prob. of False Detection)False Positive Rate
FAR (False Alarm Ratio)1 - Precision
FOH (Frequency of Hits)Precision/PPV
PON (Probability of Null)Specificity/TNR
TSS (True Skill Statistic)Sensitivity + Specificity - 1
CSI (Critical Success Index)Jaccard Index
HSS (Heidke Skill Score)Cohen's Kappa (similar)