Machine Learning Demo

1. Spam Email Detection using SVM

用的是LibSVM库，最开始write in Matlab，后把其用在之前做的Web Demo里面

基本过程：

（1）打乱样本，分成train，validation，test三部分

（2）确认kernel type，如果用高斯核，就用validation set确认最佳的C，Gamma

（3）用最佳的参数进行训练

clear allclcload data.mat[N,M] = size(data);p = randperm(N); %直接打乱了训练样本numberOfSamplesForTraining = N * 0.1;Training = data(p(1:numberOfSamplesForTraining), :);Testing  = data(p(numberOfSamplesForTraining+1:end), :);xTraining = Training(:, 1:6);yTraining = Training(:, 7);xTesting = Testing(:, 1:6);yTesting = Testing(:, 7);%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%SVM Gaussian kernel %Search for the optimal C and gamma, K(x1,x2) = exp{-||x1-x2||^2/gamma} to%make the recognition rate maximum. %Firstly, search C and gamma in a crude scale (as recommended in 'A practical Guide to Support Vector Classification'))CScale = [-5, -3, -1, 1, 3, 5,7,9,11,13,15];gammaScale = [-15,-13,-11,-9,-7,-5,-3,-1,1,3];C = 2.^CScale;gamma = 2.^gammaScale;maxRecognitionRate = 0;for i = 1:length(C)    for j = 1:length(gamma)        cmd=['-t 2 -c ',num2str(C(i)),' -g ',num2str(gamma(j)),' -v 5'];        recognitionRate = svmtrain(yTraining,xTraining,cmd);        if recognitionRate>maxRecognitionRate            maxRecognitionRate = recognitionRate            maxCIndex = i;            maxGammaIndex = j;        end;    end;end;%Then search for optimal C and gamma in a refined scale. n = 10;minCScale = 0.5*(CScale(max(1,maxCIndex-1))+CScale(maxCIndex));maxCScale = 0.5*(CScale(min(length(CScale),maxCIndex+1))+CScale(maxCIndex));newCScale = [minCScale:(maxCScale-minCScale)/n:maxCScale];minGammaScale = 0.5*(gammaScale(max(1,maxGammaIndex-1))+gammaScale(maxGammaIndex));maxGammaScale = 0.5*(gammaScale(min(length(gammaScale),maxGammaIndex+1))+gammaScale(maxGammaIndex));newGammaScale = [minGammaScale:(maxGammaScale-minGammaScale)/n:maxGammaScale];newC = 2.^newCScale;newGamma = 2.^newGammaScale;maxRecognitionRate = 0;for i = 1:length(newC)    for j = 1:length(newGamma)        cmd=['-t 2 -c ',num2str(newC(i)),' -g ',num2str(newGamma(j)),' -v 5'];        recognitionRate = svmtrain(yTraining,xTraining,cmd);        if recognitionRate>maxRecognitionRate            maxRecognitionRate = recognitionRate            maxC = newC(i);            maxGamma = newGamma(j);        end;    end;end;%Train the SVM model by the optimal C and gamma.cmd=['-t 2 -c ',num2str(maxC),' -g ',num2str(maxGamma)];model = svmtrain(yTraining,xTraining,cmd);save model.mat model;%Test the model on the remaining testing data and obtain the recognition rate.load model.mat;[yPred,accuracy,decisionValues] = svmpredict(yTesting,xTesting,model);  save yPred.mat yPred;save decisionValues.mat decisionValues;save xTesting.mat xTesting;save yTesting.mat yTesting;

在Java里面写的时候，先把模型训练好，保存起来，收到邮件就先load model，再对其predict，把结果一并储存在DataBase中

2. 兵王问题 using SVM && NN

SVM就不说了，类似之前的，用NN的话也是用Matlab里面的神经网络toolbox中的newpr，用newff效果不好

% using newpr: it is doing a lot better[N,M] = size(xapp);X = xapp(:, 1 : floor(M*0.5));y = yapp(:, 1 : floor(M*0.5));net = newpr(X, y, 20 * ones(1, 10));net.trainParam.epochs=1000;net.trainParam.lr=0.1;net.trainParam.goal=0.00004;net=train(net,X,y);predict = sim(net, xapp);[t, index] = max(predict);[t yy] = max(yapp);ratio = sum(yy==index) / length(yy)

而且从结果来看：SVM只需要少量的数据样本就可以达到比较高的accuracy

3. Recommend System using collaborative filtering

直接用的之前Coursera上ML的代码，把电影feature，用户喜欢什么类型电影feature放到一个Object function里面进行求极值

因为最后rating要求是1-5的整数，所以算出double矩阵后四舍五入，这时发现train accuracy竟然只有50%，后来允许有1的误差就达到了95%，感觉准确率还行

4. 图片压缩 using K-Means

implement by myself in Java & 集成到之前的Web Demo

遇到的一个坑爹问题就是数组的应用，直接把center引用指向原始二维数组的话，当累加二维数组时，center也相当于累加了，所以最后就出现了灰度值大于255的情况

import java.awt.image.BufferedImage;import java.io.File;import java.util.Arrays;import java.util.Random;import javax.imageio.ImageIO;/* * k cluster  * convert to double  */public class Cluster {public static double[][] image;public static double[][] center;public static int[] assign;public static double[][] distance;static int h, w;static int K = 16;public static void main(String[] args) throws Exception {run("bird_small.png");for(double[] c : center)System.out.println(Arrays.toString(c));}public  static void run(String file) throws Exception {readImg(file);initial();runKmeans();}private static void readImg(String file) throws Exception {File f = new File(file);BufferedImage img = ImageIO.read(f);w = img.getWidth();h = img.getHeight();//System.out.println(img.getType());image =  new double[w*h][3];for(int i=0; i<w; i++)for(int j=0; j<h; j++) {int rgb = img.getRGB(i, j);String s = Integer.toBinaryString(rgb);int id = i*h+j;image[id][0] = Integer.valueOf(s.substring(24, 32), 2)/255.0;image[id][1] = Integer.valueOf(s.substring(16, 24), 2)/255.0;image[id][2] = Integer.valueOf(s.substring(8, 16), 2)/255.0;}}private static void initial() {center = new double[K][3];assign = new int[image.length];distance = new double[image.length][K];Random random = new Random();for(int i=0; i<K; i++)center[i] = image[random.nextInt(image.length)];}private static void runKmeans() {// based on max iterationsfor(int i=0; i<10; i++) {System.out.println(i+1 + " iteration ... ");findColsest();reCalCenter();}}private static void findColsest() {// compute distance to centerfor(int i=0; i<distance.length; i++) {for(int j=0; j<distance[0].length; j++) {double d = (image[i][0]-center[j][0])*(image[i][0]-center[j][0]) + (image[i][1]-center[j][1])*(image[i][1]-center[j][1])+ (image[i][2]-center[j][2])*(image[i][2]-center[j][2]);distance[i][j] = d;}}for(int i=0; i<assign.length; i++) {double min = distance[i][0];int minIdx = 0;for(int j=1; j<center.length; j++) {if(distance[i][j] < min) {min = distance[i][j];minIdx = j;}}assign[i] = minIdx;}}private static void reCalCenter() {for(int i=0; i<K; i++) {center[i][0]=0;center[i][1]=0;center[i][2]=0;int cnt = 0;for(int j=0; j<assign.length; j++) {if(assign[j] == i) {center[i][0]+=image[j][0];center[i][1]+=image[j][1];center[i][2]+=image[j][2];cnt++;}}if(cnt == 0) {Random random = new Random();center[i] = image[random.nextInt(image.length)];} else {center[i][0] /= cnt;center[i][1] /= cnt;center[i][2] /= cnt;}}}}

5. MNIST using NN && CNN

用的是TensorFlow，就是官网的例程，Deep Learning已经碾压了其他的方法

COursera上也有用NN实现的版本