RHadoop 例子WordCount测试
来源:互联网 发布:淘宝改差评技巧 编辑:程序博客网 时间:2024/05/20 19:46
安装完RHadoop,当然要进行一下例子测试,看了网上相关的关于wordcount的例子,还是有不少,有些还比较模糊,于是就把自己下载的代码与编译结果记录一下:
欢迎交流学习:sparkexpert@sina.com
library(rmr2)
library(rhdfs)
hdfs.init()
rmr.options(backend = "local")
# Word count --------------------------------------------------------------
ebookLocation <- "/home/ndscbigdata/wofile.txt"
m <- mapreduce(input = ebookLocation,
input.format = "text",
map = function(k, v){
words <- unlist(strsplit(v, split = "[[:space:][:punct:]]"))
words <- tolower(words)
words <- gsub("[0-9]", "", words)
words <- words[words != ""]
wordcount <- table(words)
keyval(
key = names(wordcount),
val = as.numeric(wordcount)
)
},
reduce = function(k, counts){
keyval(key = k,
val = sum(counts))
}
)
# Retrieve results and prepare to plot ------------------------------------
x <- from.dfs(m)
dat <- data.frame(
word = keys(x),
count = values(x)
)
dat <- dat[order(dat$count, decreasing=TRUE), ]
head(dat, 50)
with(head(dat, 25), plot(count, names = word))
其结果呈现在RStudio示例如下:
- RHadoop 例子WordCount测试
- 配置RHadoop与运行WordCount例子
- RHadoop实现wordcount
- Rhadoop的第一个实例wordcount
- rhadoop
- RHADOOP
- rhadoop
- RHadoop
- Hadoop的测试例子WordCount(含效果图)
- Hadoop的测试例子WordCount(含效果图)
- wordcount例子程序
- mapreduce-wordcount例子
- MapReduce例子1--wordcount
- hadoop执行wordcount例子
- hadoop的WordCount例子
- storm运行wordcount例子
- Hadoop 例子之 WordCount
- Trident wordCount例子解读
- 三、RemoteViews
- 剑指offer第二十一题【栈的压入、弹出序列】c++实现
- BZOJ1037 生日聚会
- Jquery iframe获取父窗口中的元素
- 1769493476
- RHadoop 例子WordCount测试
- cadence设置丝印字体大小
- NYOJ——64 鸡兔同笼
- HTML学习笔记之表格的使用
- HDU 5504 GT and sequence(给一堆数,求最大值,坑比较多)
- ACM算法学习
- 寄存器与缓存
- git install 报错处理
- BestCoder Round #60 5505GT and numbers