Suppose we fit a Lasso regression to a data set which has 100 features (x1; ...

ID: 3327707 • Letter: S

Question

Suppose we fit a Lasso regression to a data set which has 100 features (x1; ... ; x100 ).

We now rescale the feature x1 by multiplying it by 10, and then refit Lasso regression with the same regularization parameter.

Multiple Choice: Which of the following option will be correct?

(a) It is more likely for x1 to be excluded from the model

(b) It is more likely for x1 to be included in the model

(d) None of these

Please explain your reasoning. Does scaling feature x1 by multiplying it by 10 make the coefficients larger or smaller?

Solution: B

It is more likely for x1 to be included in the model

Big feature values = smaller coefficients = less lasso penalty = more likely to have be kept