Scala-Actor并行wordcount

来源:互联网 发布:淘宝什么是转化率 编辑:程序博客网 时间:2024/05/16 01:55

scala-2.10.6


文件:

"d://word.txt", "d://word.log"

文件内容:



代码:

package cn.zengmg.day26.actorimport scala.actors.{Actor, Future}import scala.collection.mutable.ListBufferimport scala.io.Sourcecase class SubmitFile(filePath: String)class ActorWordCount extends Actor {  override def act(): Unit = {    loop {      react {        case SubmitFile(filePath) => {          val words = Source.fromFile(filePath).getLines()          val result = words.toList.flatMap(_.split(" ")).map((_, 1)).groupBy(_._1).mapValues(_.foldLeft(0)(_ + _._2))          sender ! result        }      }    }  }}object ActorWordCount {  def main(args: Array[String]): Unit = {    //发送异步消息,返回值是 Future[Any]。    val futures = new ListBuffer[Future[Any]]    //存放每个文件统计得结果    val results = new ListBuffer[Map[String, Int]]    val files = Array("d://word.txt", "d://word.log")    for (f <- files) {      val t = new ActorWordCount      t.start()      //调用就会返回值,但该值不是真正的计算结果,可以理解为放结果的一个容器,isSet的值为false。      val future=t !! SubmitFile(f)      futures+=future    }    while(futures.size>0){      val validResult=futures.filter(_.isSet)      if(validResult.size>0){        //result 类型 Future[Any]。        for(result<-validResult){          results+=result.apply().asInstanceOf[Map[String, Int]]          futures-=result        }      }    }    val finalRsult = results.flatten.groupBy(_._1).mapValues(_.foldLeft(0)(_+_._2))    println("未排序:"+finalRsult)    val sortFinalRsult=finalRsult.toList.sortBy(_._2).reverse    println("排序:"+sortFinalRsult)  }}


运行结果:


未排序:Map(wecki -> 7,  -> 1, rerex -> 7, kljk -> 7, oppyed -> 7, weopc -> 7, tom -> 8, aadkj; -> 7, sdlkjfsfsad -> 7, xe;q -> 7, kl;xjljks -> 7, djkdsfsadf -> 7, hello -> 16, jerry -> 8, weweioe -> 7, wiot -> 7)


排序:List((hello,16), (jerry,8), (tom,8), (wiot,7), (weweioe,7), (djkdsfsadf,7), (kl;xjljks,7), (xe;q,7), (sdlkjfsfsad,7), (aadkj;,7), (weopc,7), (oppyed,7), (kljk,7), (rerex,7), (wecki,7), (,1))



原创粉丝点击