Next: Leave-one-out Error Estimation [4 Up: MLA_Exercises_160106 Previous: Binary Decision Trees [2

# Confidence Intervals [3 P]

a)
Hypothesis makes mistakes on a sample of randomly drawn test instances. Calculate the two-sided 95% confidence interval for the true error. Also calculate the 95% one-sided confidence interval (i.e. an upper bound such that the true error is smaller than with 95% confidence).
b)
You know that the true error of hypothesis lies between 0.1 and 0.4. How many test examples would you need at least to make the width of the two-sided 95% confidence interval smaller than 0.05?
c)
A nearest-neighbour and a SVM-classifier were trained for a classification problem on the same training set. The nearest-neighbour classifier makes 9 mistakes on test set , which consists of 85 instances. The SVM-classifier makes 5 mistakes on an independent test set of 65 examples. With what probability is the SVM hypothesis better than the nearest-neighbour hypothesis? Can you decide from the given information whether SVMs are more suitable for this problem than nearest-neighbour algorithms? Explain your answer.

Next: Leave-one-out Error Estimation [4 Up: MLA_Exercises_160106 Previous: Binary Decision Trees [2
Pfeiffer Michael 2006-01-18