IN MATLAB, first correct answer will recieve thumbs up 2. Bias vs Variance in Es
ID: 3887120 • Letter: I
Question
IN MATLAB, first correct answer will recieve thumbs up
2. Bias vs Variance in Estimation: Consider the problem of estimating the real value parameter w from the dataset Di with N points Here ni - 1.2, .....N are identical independently distributed zero-mean Gaussian noise samples of variance 2. Note that .. 2 and hence we can solve (2) with ri = I. It can also be shown that this corresponds to the maximum likelihood estimate, and hence we will denote the solution as We will analyze the performance of biased and unbiased estimators. Assume that w 2-0.1 for your MATLAB simulations. 0.04 andExplanation / Answer
Understanding how different sources of error lead to bias and variance helps us improve the data fitting process resulting in more accurate models. We define bias and variance in three ways: conceptually, graphically and mathematically.
Conceptual Definition
Graphical Definition
We can create a graphical visualization of bias and variance using a bulls-eye diagram. Imagine that the center of the target is a model that perfectly predicts the correct values. As we move away from the bulls-eye, our predictions get worse and worse. Imagine we can repeat our entire model building process to get a number of separate hits on the target. Each hit represents an individual realization of our model, given the chance variability in the training data we gather. Sometimes we will get a good distribution of training data so we predict very well and we are close to the bulls-eye, while sometimes our training data might be full of outliers or non-standard values resulting in poorer predictions. These different realizations result in a scatter of hits on the target.
We can plot four different cases representing combinations of both high and low bias and variance.
Graphical illustration of bias and variance.
Mathematical Definition
after Hastie, et al. 2009 1
If we denote the variable we are trying to predict as YY and our covariates as XX, we may assume that there is a relationship relating one to the other such as Y=f(X)+Y=f(X)+ where the error term is normally distributed with a mean of zero like so N(0,)N(0,).
We may estimate a model f^(X)f^(X) of f(X)f(X) using linear regressions or another modeling technique. In this case, the expected squared prediction error at a point xx is:
Err(x)=E[(Yf^(x))2]Err(x)=E[(Yf^(x))2]
This error may then be decomposed into bias and variance components:
Err(x)=(E[f^(x)]f(x))2+E[(f^(x)E[f^(x)])2]+2eErr(x)=(E[f^(x)]f(x))2+E[(f^(x)E[f^(x)])2]+e2
Err(x)=Bias2+Variance+Irreducible Error
Low Variance High Variance Low Bias High Bias