3.1、随机森林之随机森林实例
来源：互联网 发布：厦门二手房成交数据 编辑：程序博客网 时间：2024/04/29 12:44
随机森林
junjun
2016年2月8日
随机森林实例
Markdown脚本及数据集：http://pan.baidu.com/s/1bnY6ar9
实例一、用随机森林对鸢尾花数据进行分类
#1、加载数据并查看data("iris")summary(iris)## Sepal.Length Sepal.Width Petal.Length Petal.Width ## Min. :4.300 Min. :2.000 Min. :1.000 Min. :0.100 ## 1st Qu.:5.100 1st Qu.:2.800 1st Qu.:1.600 1st Qu.:0.300 ## Median :5.800 Median :3.000 Median :4.350 Median :1.300 ## Mean :5.843 Mean :3.057 Mean :3.758 Mean :1.199 ## 3rd Qu.:6.400 3rd Qu.:3.300 3rd Qu.:5.100 3rd Qu.:1.800 ## Max. :7.900 Max. :4.400 Max. :6.900 Max. :2.500 ## Species ## setosa :50 ## versicolor:50 ## virginica :50 ## ## ## str(iris)## 'data.frame': 150 obs. of 5 variables:## $ Sepal.Length: num 5.1 4.9 4.7 4.6 5 5.4 4.6 5 4.4 4.9 ...## $ Sepal.Width : num 3.5 3 3.2 3.1 3.6 3.9 3.4 3.4 2.9 3.1 ...## $ Petal.Length: num 1.4 1.4 1.3 1.5 1.4 1.7 1.4 1.5 1.4 1.5 ...## $ Petal.Width : num 0.2 0.2 0.2 0.2 0.2 0.4 0.3 0.2 0.2 0.1 ...## $ Species : Factor w/ 3 levels "setosa","versicolor",..: 1 1 1 1 1 1 1 1 1 1 ...#2、创建训练集和测试集数据set.seed(2001)library(caret)## Loading required package: lattice## Loading required package: ggplot2## Warning: package 'ggplot2' was built under R version 3.2.3index <- createDataPartition(iris$Species, p=0.7, list=F)train_iris <- iris[index, ]test_iris <- iris[-index, ]#3、建模 library(randomForest)## randomForest 4.6-12## Type rfNews() to see new features/changes/bug fixes.## ## Attaching package: 'randomForest'## The following object is masked from 'package:ggplot2':## ## marginmodel_iris <- randomForest(Species~., data=train_iris, ntree=50, nPerm=10, mtry=3, proximity=T, importance=T)#4、模型评估model_iris## ## Call:## randomForest(formula = Species ~ ., data = train_iris, ntree = 50, nPerm = 10, mtry = 3, proximity = T, importance = T) ## Type of random forest: classification## Number of trees: 50## No. of variables tried at each split: 3## ## OOB estimate of error rate: 4.76%## Confusion matrix:## setosa versicolor virginica class.error## setosa 35 0 0 0.00000000## versicolor 0 32 3 0.08571429## virginica 0 2 33 0.05714286str(model_iris)## List of 19## $ call : language randomForest(formula = Species ~ ., data = train_iris, ntree = 50, nPerm = 10, mtry = 3, proximity = T, importance = T)## $ type : chr "classification"## $ predicted : Factor w/ 3 levels "setosa","versicolor",..: 1 1 1 1 1 1 1 1 1 1 ...## ..- attr(*, "names")= chr [1:105] "5" "7" "8" "11" ...## $ err.rate : num [1:50, 1:4] 0.0513 0.0758 0.0741 0.0435 0.0505 ...## ..- attr(*, "dimnames")=List of 2## .. ..$ : NULL## .. ..$ : chr [1:4] "OOB" "setosa" "versicolor" "virginica"## $ confusion : num [1:3, 1:4] 35 0 0 0 32 2 0 3 33 0 ...## ..- attr(*, "dimnames")=List of 2## .. ..$ : chr [1:3] "setosa" "versicolor" "virginica"## .. ..$ : chr [1:4] "setosa" "versicolor" "virginica" "class.error"## $ votes : matrix [1:105, 1:3] 1 1 1 1 1 1 1 1 1 1 ...## ..- attr(*, "dimnames")=List of 2## .. ..$ : chr [1:105] "5" "7" "8" "11" ...## .. ..$ : chr [1:3] "setosa" "versicolor" "virginica"## ..- attr(*, "class")= chr [1:2] "matrix" "votes"## $ oob.times : num [1:105] 15 23 22 16 17 11 20 20 17 19 ...## $ classes : chr [1:3] "setosa" "versicolor" "virginica"## $ importance : num [1:4, 1:5] 0 0 0.3417 0.34918 -0.00518 ...## ..- attr(*, "dimnames")=List of 2## .. ..$ : chr [1:4] "Sepal.Length" "Sepal.Width" "Petal.Length" "Petal.Width"## .. ..$ : chr [1:5] "setosa" "versicolor" "virginica" "MeanDecreaseAccuracy" ...## $ importanceSD : num [1:4, 1:4] 0 0 0.04564 0.04711 0.00395 ...## ..- attr(*, "dimnames")=List of 2## .. ..$ : chr [1:4] "Sepal.Length" "Sepal.Width" "Petal.Length" "Petal.Width"## .. ..$ : chr [1:4] "setosa" "versicolor" "virginica" "MeanDecreaseAccuracy"## $ localImportance: NULL## $ proximity : num [1:105, 1:105] 1 1 1 1 1 1 1 1 1 1 ...## ..- attr(*, "dimnames")=List of 2## .. ..$ : chr [1:105] "5" "7" "8" "11" ...## .. ..$ : chr [1:105] "5" "7" "8" "11" ...## $ ntree : num 50## $ mtry : num 3## $ forest :List of 14## ..$ ndbigtree : int [1:50] 11 5 9 9 9 9 9 11 11 9 ...## ..$ nodestatus: int [1:17, 1:50] 1 -1 1 1 1 -1 -1 1 -1 -1 ...## ..$ bestvar : int [1:17, 1:50] 4 0 4 3 3 0 0 1 0 0 ...## ..$ treemap : int [1:17, 1:2, 1:50] 2 0 4 6 8 0 0 10 0 0 ...## ..$ nodepred : int [1:17, 1:50] 0 1 0 0 0 2 3 0 3 2 ...## ..$ xbestsplit: num [1:17, 1:50] 0.8 0 1.65 5.25 4.85 0 0 6.05 0 0 ...## ..$ pid : num [1:3] 1 1 1## ..$ cutoff : num [1:3] 0.333 0.333 0.333## ..$ ncat : Named int [1:4] 1 1 1 1## .. ..- attr(*, "names")= chr [1:4] "Sepal.Length" "Sepal.Width" "Petal.Length" "Petal.Width"## ..$ maxcat : int 1## ..$ nrnodes : int 17## ..$ ntree : num 50## ..$ nclass : int 3## ..$ xlevels :List of 4## .. ..$ Sepal.Length: num 0## .. ..$ Sepal.Width : num 0## .. ..$ Petal.Length: num 0## .. ..$ Petal.Width : num 0## $ y : Factor w/ 3 levels "setosa","versicolor",..: 1 1 1 1 1 1 1 1 1 1 ...## ..- attr(*, "names")= chr [1:105] "5" "7" "8" "11" ...## $ test : NULL## $ inbag : NULL## $ terms :Classes 'terms', 'formula' length 3 Species ~ Sepal.Length + Sepal.Width + Petal.Length + Petal.Width## .. ..- attr(*, "variables")= language list(Species, Sepal.Length, Sepal.Width, Petal.Length, Petal.Width)## .. ..- attr(*, "factors")= int [1:5, 1:4] 0 1 0 0 0 0 0 1 0 0 ...## .. .. ..- attr(*, "dimnames")=List of 2## .. .. .. ..$ : chr [1:5] "Species" "Sepal.Length" "Sepal.Width" "Petal.Length" ...## .. .. .. ..$ : chr [1:4] "Sepal.Length" "Sepal.Width" "Petal.Length" "Petal.Width"## .. ..- attr(*, "term.labels")= chr [1:4] "Sepal.Length" "Sepal.Width" "Petal.Length" "Petal.Width"## .. ..- attr(*, "order")= int [1:4] 1 1 1 1## .. ..- attr(*, "intercept")= num 0## .. ..- attr(*, "response")= int 1## .. ..- attr(*, ".Environment")=<environment: R_GlobalEnv> ## .. ..- attr(*, "predvars")= language list(Species, Sepal.Length, Sepal.Width, Petal.Length, Petal.Width)## .. ..- attr(*, "dataClasses")= Named chr [1:5] "factor" "numeric" "numeric" "numeric" ...## .. .. ..- attr(*, "names")= chr [1:5] "Species" "Sepal.Length" "Sepal.Width" "Petal.Length" ...## - attr(*, "class")= chr [1:2] "randomForest.formula" "randomForest"pred <- predict(model_iris, train_iris)mean(pred==train_iris[, 5])## [1] 1#5、预测pred_iris <- predict(model_iris, test_iris)table(pred_iris, test_iris[, 5])## ## pred_iris setosa versicolor virginica## setosa 15 0 0## versicolor 0 13 2## virginica 0 2 13mean(pred_iris==test_iris[, 5])## [1] 0.9111111library(gmodels)CrossTable(pred_iris, test_iris[, 5])## ## ## Cell Contents## |-------------------------|## | N |## | Chi-square contribution |## | N / Row Total |## | N / Col Total |## | N / Table Total |## |-------------------------|## ## ## Total Observations in Table: 45 ## ## ## | test_iris[, 5] ## pred_iris | setosa | versicolor | virginica | Row Total | ## -------------|------------|------------|------------|------------|## setosa | 15 | 0 | 0 | 15 | ## | 20.000 | 5.000 | 5.000 | | ## | 1.000 | 0.000 | 0.000 | 0.333 | ## | 1.000 | 0.000 | 0.000 | | ## | 0.333 | 0.000 | 0.000 | | ## -------------|------------|------------|------------|------------|## versicolor | 0 | 13 | 2 | 15 | ## | 5.000 | 12.800 | 1.800 | | ## | 0.000 | 0.867 | 0.133 | 0.333 | ## | 0.000 | 0.867 | 0.133 | | ## | 0.000 | 0.289 | 0.044 | | ## -------------|------------|------------|------------|------------|## virginica | 0 | 2 | 13 | 15 | ## | 5.000 | 1.800 | 12.800 | | ## | 0.000 | 0.133 | 0.867 | 0.333 | ## | 0.000 | 0.133 | 0.867 | | ## | 0.000 | 0.044 | 0.289 | | ## -------------|------------|------------|------------|------------|## Column Total | 15 | 15 | 15 | 45 | ## | 0.333 | 0.333 | 0.333 | | ## -------------|------------|------------|------------|------------|## ## 实例二、用坦泰尼克号乘客是否存活数据应用到随机森林算法中
在随机森林算法的函数randomForest()中有两个非常重要的参数，而这两个参数又将影响模型的准确性，它们分别是mtry和ntree。一般对mtry的选择是逐一尝试，直到找到比较理想的值，ntree的选择可通过图形大致判断模型内误差稳定时的值。 randomForest包中的randomForest(formula, data, ntree, nPerm, mtry, proximity, importace)函数：随机森林分类与回归。ntree表示生成决策树的数目（不应设置太小，默认为 500）；nPerm表示计算importance时的重复次数，数量大于1给出了比较稳定的估计，但不是很有效（目前只实现了回归）；mtry表示选择的分裂属性的个数；proximity表示是否生成邻近矩阵，为T表示生成邻近矩阵；importance表示输出分裂属性的重要性。
下面使用坦泰尼克号乘客是否存活数据应用到随机森林算法中，看看模型的准确性如何。
#1、加载数据并查看：同时读取训练样本和测试样本集train <- read.table("F:\\R\\Rworkspace\\RandomForest/train.csv", header=T, sep=",")test <- read.table("F:\\R\\Rworkspace\\RandomForest/test.csv", header=T, sep=",")#注意：训练集和测试集数据来自不同的数据集，一定要注意测试集和训练集的factor的levels相同，否则，在利用训练集训练的模型对测试集进行预测时，会报错！！！str(train)## 'data.frame': 891 obs. of 8 variables:## $ Survived: int 0 1 1 1 0 0 0 0 1 1 ...## $ Pclass : int 3 1 3 1 3 3 1 3 3 2 ...## $ Sex : Factor w/ 2 levels "female","male": 2 1 1 1 2 2 2 2 1 1 ...## $ Age : num 22 38 26 35 35 NA 54 2 27 14 ...## $ SibSp : int 1 1 0 1 0 0 0 3 0 1 ...## $ Parch : int 0 0 0 0 0 0 0 1 2 0 ...## $ Fare : num 7.25 71.28 7.92 53.1 8.05 ...## $ Embarked: Factor w/ 4 levels "","C","Q","S": 4 2 4 4 4 3 4 4 4 2 ...str(test)## 'data.frame': 418 obs. of 7 variables:## $ Pclass : int 3 3 2 3 3 3 3 2 3 3 ...## $ Sex : Factor w/ 2 levels "female","male": 2 1 2 2 1 2 1 2 1 2 ...## $ Age : num 34.5 47 62 27 22 14 30 26 18 21 ...## $ SibSp : int 0 1 0 0 1 0 0 1 0 2 ...## $ Parch : int 0 0 0 0 1 0 0 1 0 0 ...## $ Fare : num 7.83 7 9.69 8.66 12.29 ...## $ Embarked: Factor w/ 3 levels "C","Q","S": 2 3 2 3 3 3 2 3 1 3 ...#从上可知：训练集数据共891条记录，8个变量，Embarked因子水平为4；测试集数据共418条记录，7个变量，Embarked因子水平为3；训练集中存在缺失数据；Survived因变量为数字类型，测试集数据无因变量#2、数据清洗#1）调整测试集与训练基地因子水平levels(train$Embarked)## [1] "" "C" "Q" "S"levels(test$Embarked)## [1] "C" "Q" "S"levels(test$Embarked) <- levels(train$Embarked)#2)把因变量转化为因子类型train$Survived <- as.factor(train$Survived)#3)使用rfImpute()函数补齐训练集的缺失值NAlibrary(randomForest)train_impute <- rfImpute(Survived~., data=train)## ntree OOB 1 2## 300: 16.39% 7.83% 30.12%## ntree OOB 1 2## 300: 16.50% 8.93% 28.65%## ntree OOB 1 2## 300: 16.72% 8.74% 29.53%## ntree OOB 1 2## 300: 16.50% 8.56% 29.24%## ntree OOB 1 2## 300: 17.28% 9.47% 29.82%#4)补齐测试集的缺失值：对待测样本进行预测,发现待测样本中存在缺失值，这里使用多重插补法将缺失值补齐summary(test)## Pclass Sex Age SibSp ## Min. :1.000 female:152 Min. : 0.17 Min. :0.0000 ## 1st Qu.:1.000 male :266 1st Qu.:21.00 1st Qu.:0.0000 ## Median :3.000 Median :27.00 Median :0.0000 ## Mean :2.266 Mean :30.27 Mean :0.4474 ## 3rd Qu.:3.000 3rd Qu.:39.00 3rd Qu.:1.0000 ## Max. :3.000 Max. :76.00 Max. :8.0000 ## NA's :86 ## Parch Fare Embarked## Min. :0.0000 Min. : 0.000 :102 ## 1st Qu.:0.0000 1st Qu.: 7.896 C: 46 ## Median :0.0000 Median : 14.454 Q:270 ## Mean :0.3923 Mean : 35.627 S: 0 ## 3rd Qu.:0.0000 3rd Qu.: 31.500 ## Max. :9.0000 Max. :512.329 ## NA's :1#可是看出测试集数据存在缺失值NA,Age和Fare的数据有NA#多重插补法填充缺失值：library(mice)## Loading required package: Rcpp## mice 2.25 2015-11-09imput <- mice(data=test, m=10)## ## iter imp variable## 1 1 Age Fare## 1 2 Age Fare## 1 3 Age Fare## 1 4 Age Fare## 1 5 Age Fare## 1 6 Age Fare## 1 7 Age Fare## 1 8 Age Fare## 1 9 Age Fare## 1 10 Age Fare## 2 1 Age Fare## 2 2 Age Fare## 2 3 Age Fare## 2 4 Age Fare## 2 5 Age Fare## 2 6 Age Fare## 2 7 Age Fare## 2 8 Age Fare## 2 9 Age Fare## 2 10 Age Fare## 3 1 Age Fare## 3 2 Age Fare## 3 3 Age Fare## 3 4 Age Fare## 3 5 Age Fare## 3 6 Age Fare## 3 7 Age Fare## 3 8 Age Fare## 3 9 Age Fare## 3 10 Age Fare## 4 1 Age Fare## 4 2 Age Fare## 4 3 Age Fare## 4 4 Age Fare## 4 5 Age Fare## 4 6 Age Fare## 4 7 Age Fare## 4 8 Age Fare## 4 9 Age Fare## 4 10 Age Fare## 5 1 Age Fare## 5 2 Age Fare## 5 3 Age Fare## 5 4 Age Fare## 5 5 Age Fare## 5 6 Age Fare## 5 7 Age Fare## 5 8 Age Fare## 5 9 Age Fare## 5 10 Age FareAge <- data.frame(Age=apply(imput$imp$Age, 1, mean))Fare <- data.frame(Fare=apply(imput$imp$Fare, 1, mean))#添加行标号：test$Id <- row.names(test)Age$Id <- row.names(Age)Fare$Id <- row.names(Fare)#替换缺失值：test[test$Id %in% Age$Id, 'Age'] <- Age$Agetest[test$Id %in% Fare$Id, 'Fare'] <- Fare$Faresummary(test)## Pclass Sex Age SibSp ## Min. :1.000 female:152 Min. : 0.17 Min. :0.0000 ## 1st Qu.:1.000 male :266 1st Qu.:22.00 1st Qu.:0.0000 ## Median :3.000 Median :26.19 Median :0.0000 ## Mean :2.266 Mean :29.41 Mean :0.4474 ## 3rd Qu.:3.000 3rd Qu.:36.65 3rd Qu.:1.0000 ## Max. :3.000 Max. :76.00 Max. :8.0000 ## Parch Fare Embarked Id ## Min. :0.0000 Min. : 0.000 :102 Length:418 ## 1st Qu.:0.0000 1st Qu.: 7.896 C: 46 Class :character ## Median :0.0000 Median : 14.454 Q:270 Mode :character ## Mean :0.3923 Mean : 35.583 S: 0 ## 3rd Qu.:0.0000 3rd Qu.: 31.472 ## Max. :9.0000 Max. :512.329#从上可知：测试数据集中已经没有了NA值。#3、选着随机森林的mtry和ntree值#1）选着mtry(n <- length(names(train)))## [1] 8library(randomForest)for(i in 1:n) { model <- randomForest(Survived~., data=train_impute, mtry=i) err <- mean(model$err.rate) print(err)}## [1] 0.2100028## [1] 0.1889116## [1] 0.1776607## [1] 0.1902606## [1] 0.1960938## [1] 0.1953451## [1] 0.1951303## [1] 0.2018745#从上可知：mtry=2或者mtry=3时，模型内评价误差最小，故确定参数mtry=2或者mtry=3#2)选着ntreeset.seed(2002)model <- randomForest(Survived~., data=train_impute, mtry=2, ntree=1000)plot(model)#从上图可知：ntree在400左右时，模型内误差基本稳定，故取ntree=400#4、建模model_fit <- randomForest(Survived~., data=train_impute, mtry=2, ntree=400, importance=T)#5、模型评估model_fit## ## Call:## randomForest(formula = Survived ~ ., data = train_impute, mtry = 2, ntree = 400, importance = T) ## Type of random forest: classification## Number of trees: 400## No. of variables tried at each split: 2## ## OOB estimate of error rate: 16.61%## Confusion matrix:## 0 1 class.error## 0 500 49 0.08925319## 1 99 243 0.28947368#查看变量的重要性(importance <- importance(x=model_fit))## 0 1 MeanDecreaseAccuracy MeanDecreaseGini## Pclass 16.766454 28.241508 32.16125 33.15984## Sex 46.578191 76.145306 72.42624 100.74843## Age 19.882605 24.586274 30.52032 60.85186## SibSp 19.070707 2.834303 18.95690 16.11720## Parch 10.366140 8.380559 13.18282 12.28725## Fare 18.649672 20.967558 29.43262 66.31489## Embarked 7.904436 11.479919 14.18780 12.68924#绘制变量的重要性图varImpPlot(model_fit)#从上图可知：模型中乘客的性别最为重要，接下来的是Pclass,age,Fare和Fare,age,Pclass。#6、预测#1）对训练集数据预测：train_pred <- predict(model_fit, train_impute)mean(train_pred==train_impute$Survived)## [1] 0.9135802table(train_pred, train_impute$Survived)## ## train_pred 0 1## 0 535 63## 1 14 279#模型的预测精度在90%以上#2)对测试集数据预测：test_pred <- predict(model_fit, test[, 1:7])head(test_pred)## 1 2 3 4 5 6 ## 0 0 0 0 1 0 ## Levels: 0 1 0 0
- 3.1、随机森林之随机森林实例
- 决策树之随机森林
- 随机森林
- 随机森林
- 随机森林
- 随机森林
- 随机森林
- 随机森林
- 随机森林
- 随机森林
- 随机森林
- 随机森林
- 随机森林
- 随机森林
- 随机森林
- 随机森林
- 随机森林
- 随机森林
- ubuntu tftp服务器安装
- 关于线程并发的问题
- Java Web 错误/异常处理页面（更新）
- hust 1626 Cutting rope 概率论
- hdu1789 Doing Homework again（贪心）
- 3.1、随机森林之随机森林实例
- 1012. 数字分类
- 安装交叉工具链
- 利用线程本地存储降低服务器处理延时
- hdu 1075 What Are You Talking About【字典树】
- hust 1628 LowerBound 莫队算法
- Android报错：Your content must have a ListView whose id attribute is 'android.R.id.list'
- Linux netstat命令详解
- CCM原理
