Spark之Scala学习之路（三）

来源：互联网发布：淘宝助理怎么导出图片编辑：程序博客网时间：2024/05/19 23:56

函数式编程:
函数定义：

def fun1(name:String){println(name)}val fun1_v=fun1 _    //名称 空格 下划线//fun1_v: String => Unit = <function1> 左边变量进行右边的加工fun1(1)fun1_v    //调用方法

匿名函数使用（无名称）

val fun2=(content:String) =>println(content)val hiScala=(content:String) =>println(content)def bigData(func:(String) =>Unit,content:String){func(content)}bigData(hiScala,"spark")  //spark//传入一个函数进来，要求输入是字符串，返回类型是Unitval array=Array(1,2,3,4,5,6,7,8,9)array.map(item => 2*item )

函数的返回值是函数

def func_Returned(content:String)=(message:String) => println(message)func_Returned("spark")def func_Returned(content:String)=(message:String) => println(content+"   "+message)val returned=func_Returned("Spark")returned("Scala")  //Spark  Scala

简化：如果在函数的函数体内只使用一次函数的输入的参数值参数的名称省略掉用下划线代替

def spark(func:(content:String)=>Unit,name:String){func(name)}def spark(func:(String)=>Unit,name:String){func(name)}spark((name:String) => println(name),"Scala")  //以下为简化方法spark((name) => println(name),"Scala")spark(name => println(name),"Scala")spark(println(_),"Scala")spark(println,"Scala")array.map(2*_)array.map(2*_).foreach(println)array.map(2*_).filter(_>10).foreach(println)

闭包：函数变量超出有效作用域，还能对函数内部进行访问

def scala(content:String) =(message:String)=>println(content+":"+message)val funcResult=scala("Spark")funcResult("Flink")

柯里化

def sum(x:Int,y:Int) =x+ysum(1,2)def sum_Curring(x:Int) = (y:Int) =>x+ysum_Curring(1)(2)def sum_Curring_Butter(x:Int)(y:Int) =x+ysum_Curring_Butter(1)(2)(1 to 100).reduceLeft(_+_)val list=List("scala","spark","hadoop")val cal=list.map("the content is" +_)cal.flatMap(_.split(" "))list.zip(List(10,6,5)) //List[(String, Int)] = List((scala,10), (spark,6), (hadoop,5))

统计对文件夹中所有文本的单词个数

import scala.io.Source._import scala.io.Sourceimport java.io._object WordCount {  var map = Map.empty[String, Int]  def main(args: Array[String]): Unit = {    try{      Make(new File("E:/test"))    }catch {       case _:FileNotFoundException=>println("error")    }    map.foreach(println)  }  def Make(file: File) = {    file.listFiles.foreach { file =>      if (file.isFile()) {        txtFile(file)        println(file)      }    }  }  def txtFile(file: File) = {    lazy val content = Source.fromFile(file)    for (line <- content.getLines()) {      count(line)    }  }  def count(line: String) = {    for (word <- line.split(" ")) {      if (map.contains(word))        map += (word -> (map(word) + 1))      else        map += (word -> 1)    }  }}

0 0