Classifier. In a machine learning classification procedure the items are classif
ID: 3306462 • Letter: C
Question
Classifier. In a machine learning classification procedure the items are classified as 1 or 0. Based on a training sample of size 120 in which there are 65 1's and 55 0's, the classifier predicts 70 1's and 50 0's. Out of 70 items predicted by the classifier as 1, 52 are correctly classified. (a) What are the sensitivity and specificity of the classifier? (b) From the population of items where the proportion of 0-labels is 99% (and 1-labels 1%), an item is selected at random. What is the probability that the item is of label 1, if the classifier says it is.Explanation / Answer
a) Here we are given that out of 70 items predicted as 1, 52 are correctly predicted. Therefore the number of 1's which are correctly predicted as 1 are 52 and there were a total of 65 1's here. Therefore the sensitivity of the test is computed as:
= P( tested 1 | 1 ) = 52 / 65 = 0.8
Therefore the sensitivity of the test is 80%
Now the specificity of the test is computed as:
= Number of 0's that are predicted as 0's / Total number of 0's
= ( 55 - 18 ) / 55
Note that 18 0's are predicted to be 1. Therefore the specificity is computed as:
= 37 / 55
= 0.6727
Therefore the specoficity of the test is 67.27%
b) Here we are given that P(1) = 0.01 and P(0) = 0.99
Probability of having a test as 1 is computed using the law of total probability as:
P( tested 1 )= P( tested 1 | 1) P(1) + P( tested 1 | 0) P(0)
P( tested 1 )= 0.8*0.01 + (1 - 0.6727)*0.99 = 0.008 + 0.324 = 0.332
Now given that the classifier says it is 1, probability that the item is a label 1 is given as:
= P( tested 1 | 1) P(1) / P( tested 1 ) = 0.008 / 0.332 = 0.0241
Therefore 0.0241 is the required probability here.