Spark Scala 二次排序

1.输入

1 52 43 51 82 13 25 16 2

2. 代码实现

2.1 自定义比较器

/** * User:leen * Date:2017/10/18 0018 * Time:14:16 * * 比较器： * 1. 继承 Ordered[T] 和Serializeable 方法 * 2. 重写 compare 方法 * 3. 需要传入参数 */class SecondarySortKey1(val first: Int, val second: Int) extends Ordered[SecondarySortKey1] with Serializable {  override def compare(that: SecondarySortKey1): Int = {    if (this.first - that.first != 0) {      this.first - that.first    //第一列 排序：升序    } else {      that.second - this.second   //第二列 排序：降序    }  }}

2.2调用比较器实现二次排序

import org.apache.spark.{SparkContext, SparkConf}/** * User:leen * Date:2017/10/18 0018 * Time:14:23 */object SecondarySort1 {  def main(args: Array[String]) {    val conf = new SparkConf().setAppName("SecondarySort1").setMaster("local");    val sc = new SparkContext(conf)    val lines = sc.textFile("C:\\Users\\leen\\Desktop\\sort.txt")    //与自定义Key组合返回Tuple2    val words = lines.map(line => {      val ws = line.split(" ")      (new SecondarySortKey1(ws(0).toInt, ws(1).toInt), line)    })    val sortRes = words.sortByKey()    sortRes.foreach(pair => println(pair._2))    sc.stop()  }}

3.结果输出

1 81 52 42 13 53 25 16 2