WordCount的实例

来源：互联网发布：网络游戏破解软件大全编辑：程序博客网时间：2024/06/02 00:53

scala> sc.textFile("hdfs://hadoop1:9000/sparktest/hello.txt").flatMap( line => line.split("\t")).collectres8: Array[String] = Array(you, jump, i, jump)scala> sc.textFile("hdfs://hadoop1:9000/sparktest/hello.txt").flatMap( line => line.split("\t")).map( word => (word,1)).collectres9: Array[(String, Int)] = Array((you,1), (jump,1), (i,1), (jump,1))scala> sc.textFile("hdfs://hadoop1:9000/sparktest/hello.txt").flatMap( line => line.split("\t")).map( word => (word,1)).reduceByKey( (x:Int,y:Int) => x + y).collectres10: Array[(String, Int)] = Array((jump,2), (you,1), (i,1))scala> sc.textFile("hdfs://hadoop1:9000/sparktest/hello.txt").flatMap( line => line.split("\t")).map( word => (word,1)).reduceByKey( (_+_)).collectres11: Array[(String, Int)] = Array((jump,2), (you,1), (i,1))scala> sc.textFile("hdfs://hadoop1:9000/sparktest/hello.txt").flatMap( _.split("\t")).map( (_,1)).reduceByKey( (_+_)).collectres12: Array[(String, Int)] = Array((jump,2), (you,1), (i,1))

阅读全文

0 0