机器学习7-SVM

http://pan.baidu.com/s/1bnyCFIB

Question 1

Suppose you have trained an SVM classifier with a Gaussian kernel, and it learned the following decision boundary on the training set:

You suspect that the SVM is underfitting (==high bais)your dataset. Should you try increasing or decreasingC? Increasing or decreasing σ2?

Your Answer ScoreExplanationIt would be reasonable to try decreasing C. It would also be reasonable to try decreasing σ2.   It would be reasonable to try increasing C. It would also be reasonable to try decreasing σ2.Correct1.00The figure shows a decision boundary that is underfit to the training set, so we'd like to lower the bias / increase the variance of the SVM. We can do so by either increasing the parameterC or decreasing σ2.It would be reasonable to try increasing C. It would also be reasonable to try increasing σ2.   It would be reasonable to try decreasing C. It would also be reasonable to try increasing σ2.   Total 1.00 / 1.00 

Question 2

The formula for the Gaussian kernel is given bysimilarity(x,l(1))=exp(−||x−l(1)||22σ2) . The figure below shows a plot of f1=similarity(x,l(1)) when σ2=1.

Which of the following is a plot of f1 when σ2=0.25?

Your Answer ScoreExplanation         Correct1.00This figure shows a "narrower" Gaussian kernel centered at the same location which is the effect of decreasingσ2.Total 1.00 / 1.00 

Question 3

The SVM solves minθ C∑mi=1y(i)cost1(θTx(i))+(1−y(i))cost0(θTx(i))+∑nj=1θ2j where the functions cost0(z) and cost1(z) look like this:

The first term in the objective is: C∑mi=1y(i)cost1(θTx(i))+(1−y(i))cost0(θTx(i)). This first term will be zero if two of the following four conditions hold true. Which are the two conditions that would guarantee that this term equals zero?

Your Answer ScoreExplanationFor every example with y(i)=0, we have that θTx(i)≤0.Correct0.25cost0(θTx(i)) is still non-zero for inputs between -1 and 0, so being less than or equal to 0 is insufficient.For every example with y(i)=0, we have that θTx(i)≤−1.Correct0.25For examples with y(i)=0, only the cost0(θTx(i)) term is present. As you can see in the graph, this will be zero for all inputs less than or equal to -1.For every example with y(i)=1, we have that θTx(i)≥0.Correct0.25cost1(θTx(i)) is still non-zero for inputs between 0 and 1, so being greater than or equal to 0 is insufficient.For every example with y(i)=1, we have that θTx(i)≥1.Correct0.25For examples with y(i)=1, only the cost1(θTx(i)) term is present. As you can see in the graph, this will be zero for all inputs greater than or equal to 1.Total 1.00 / 1.00 

Question 4 

Suppose you have a dataset with n = 10 features and m = 5000 examples. After training your logistic regression classifier with gradient descent, you find that it hasunderfit the training set and does not achieve the desired performance on the training or cross validation sets. Which of the following might be promising steps to take? Check all that apply.

Your Answer ScoreExplanationUse a different optimization method since using gradient descent to train logistic regression might result in a local minimum.Correct0.25The logistic regression cost function is convex, so gradient descent will always find the global minimum.Try using a neural network with a large number of hidden units.Correct0.25A neural network with many hidden units is a more complex (higher variance) model than logistic regression, so it is less likely to underfit the data.Reduce the number of examples in the training set.Correct0.25While you can improve accuracy on the training set by removing examples, doing so results in a worse model that will not generalize as well.Use an SVM with a Gaussian Kernel.Correct0.25By using a Gaussian kernel, your model will have greater complexity and can avoid underfitting the data.Total 1.00 / 1.00 

总结: 当feature较多，训练集合较大，出现欠拟合，使用神经网络和带有高斯核的支持向量机

Question 5

Which of the following statements are true? Check all that apply.

Your Answer ScoreExplanationIf the data are linearly separable, an SVM using a linear kernel willreturn the same parametersθ regardless of the chosen value ofC (i.e., the resulting value of θ does not depend on C).Correct0.25A linearly separable dataset can usually be separated by many different lines. Varying the parameterC will cause the SVM's decision boundary to vary among these possibilities. For example, for a very large value ofC, it might learn larger values of θ in order to increase the margin on certain examples.Suppose you are using SVMs to do multi-class classification andwould like to use the one-vs-all approach. If you haveK differentclasses, you will train K - 1 different SVMs.Correct0.25The one-vs-all method requires that we have a separate classifier for every class, so you will trainK different SVMs.The maximum value of the Gaussian kernel (i.e., sim(x,l(1))) is 1.Correct0.25When x=l(1), the Gaussian kernel has value exp(0)=1, and it is less than 1 otherwise.It is important to perform feature normalization before using the Gaussian kernel.Correct0.25The similarity measure used by the Gaussian kernel expects that the data lie in approximately the same range.