spark1.6从hdfs上读取文件运行wordcount

从hdfs上读取文件并运行wordcount

[root@hadoop14 app]# hadoop fs -put word.txt /yuhui

scala> val file=sc.textFile("hdfs://hadoop14:9000/yuhui/word.txt")

scala> val count=file.flatMap(line => line.split(" ")).map(word => (word,1)).reduceByKey(_+_)

scala> count.collect()

结果：

res1: Array[(String, Int)] = Array((cccc,2), (aaaa,2), (bbb,2), (ddd,1))