Metrics selection

September 13, 2017

Definitions

• A type I error occurs when the null hypothesis (H0) is true, but is rejected. Also known as false positive, where positive means rejection of a null hypothesis.
• A type II error occurs when the null hypothesis is false, but erroneously fails to be rejected. Also called as false negative, where negative refers to retaining a null hypothesis.
• TPR (True Positive Rate) = ( # True positives \over # positives ) = Recall = ( TP \over (TP+FN) )
• FPR (False Positive Rate) = ( # False Positives \over # negatives ) = ( FP \over (FP+TN) )
• Precision = (# True positives \over # predicted positive ) = ( TP \over (TP+FP) )

reference

1. What metrics should be used for evaluating a model on an imbalanced data set? (precision + recall or ROC=TPR+FPR)