Spark组件之GraphX学习16--最短路径ShortestPaths

更多代码请见：https://github.com/xubo245/SparkLearning

1解释

求图中的最短路径，更多的请见参考【3】，这篇写的很详细

2.代码：

/** * @author xubo * ref http://spark.apache.org/docs/1.5.2/graphx-programming-guide.html * time 20160503 */package org.apache.spark.graphx.learningimport org.apache.spark.SparkConfimport org.apache.spark.SparkContextimport org.apache.spark.graphx.Graphimport org.apache.spark.graphx.Graph.graphToGraphOpsimport org.apache.spark.graphx.lib.ShortestPathsobject ShortPaths {  def main(args: Array[String]): Unit = {    val conf = new SparkConf().setAppName("ShortPaths").setMaster("local[4]")    val sc = new SparkContext(conf)    // 测试的真实结果，后面用于对比    val shortestPaths = Set(      (1, Map(1 -> 0, 4 -> 2)), (2, Map(1 -> 1, 4 -> 2)), (3, Map(1 -> 2, 4 -> 1)),      (4, Map(1 -> 2, 4 -> 0)), (5, Map(1 -> 1, 4 -> 1)), (6, Map(1 -> 3, 4 -> 1)))    // 构造无向图的边序列    val edgeSeq = Seq((1, 2), (1, 5), (2, 3), (2, 5), (3, 4), (4, 5), (4, 6)).flatMap {      case e => Seq(e, e.swap)    }    // 构造无向图    val edges = sc.parallelize(edgeSeq).map { case (v1, v2) => (v1.toLong, v2.toLong) }    val graph = Graph.fromEdgeTuples(edges, 1)    // 要求最短路径的点集合    val landmarks = Seq(1, 4).map(_.toLong)    // 计算最短路径    val results = ShortestPaths.run(graph, landmarks).vertices.collect.map {      case (v, spMap) => (v, spMap.mapValues(i => i))    }    val shortestPath1 = ShortestPaths.run(graph, landmarks)    // 与真实结果对比    println("\ngraph edges");    println("edges:");    graph.edges.collect.foreach(println)    //    graph.edges.collect.foreach(println)    println("vertices:");    graph.vertices.collect.foreach(println)    //    println("triplets:");    //    graph.triplets.collect.foreach(println)    println();    println("\n shortestPath1");    println("edges:");    shortestPath1.edges.collect.foreach(println)    println("vertices:");    shortestPath1.vertices.collect.foreach(println)    //    println("vertices:")    assert(results.toSet == shortestPaths)    println("results.toSet:" + results.toSet);    println("end");    sc.stop()  }}

图分析：其实是无向图，但是存储的时候GraphX存的是有向图

3.结果：

分析：返回的是

(1,Map(1 -> 0, 4 -> 2))(5,Map(1 -> 1, 4 -> 1))(6,Map(4 -> 1, 1 -> 3))

节点的属性存的是到某几点的最短路径，比如

(1,Map(1 -> 0, 4 -> 2))

表明的是1节点到1节点路径为0，到4节点为2

同理

(6,Map(4 -> 1, 1 -> 3))

6号节点到4为1，到1为3，途中可以看得出来

全部结果：

graph edgesedges:Edge(1,2,1)Edge(1,5,1)Edge(2,1,1)Edge(2,3,1)Edge(2,5,1)Edge(3,2,1)Edge(5,1,1)Edge(3,4,1)Edge(4,3,1)Edge(5,2,1)Edge(4,5,1)Edge(4,6,1)Edge(5,4,1)Edge(6,4,1)vertices:(4,1)(1,1)(5,1)(6,1)(2,1)(3,1) shortestPath1edges:Edge(1,2,1)Edge(1,5,1)Edge(2,1,1)Edge(2,3,1)Edge(2,5,1)Edge(3,2,1)Edge(5,1,1)Edge(3,4,1)Edge(4,3,1)Edge(5,2,1)Edge(4,5,1)Edge(4,6,1)Edge(5,4,1)Edge(6,4,1)vertices:(4,Map(4 -> 0, 1 -> 2))(1,Map(1 -> 0, 4 -> 2))(5,Map(1 -> 1, 4 -> 1))(6,Map(4 -> 1, 1 -> 3))(2,Map(1 -> 1, 4 -> 2))(3,Map(4 -> 1, 1 -> 2))results.toSet:Set((1,Map(1 -> 0, 4 -> 2)), (5,Map(1 -> 1, 4 -> 1)), (2,Map(1 -> 1, 4 -> 2)), (6,Map(4 -> 1, 1 -> 3)), (4,Map(4 -> 0, 1 -> 2)), (3,Map(4 -> 1, 1 -> 2)))end

如果改为全部节点，则为：

vertices:(4,Map(5 -> 1, 1 -> 2, 6 -> 1, 2 -> 2, 3 -> 1, 4 -> 0))(1,Map(5 -> 1, 1 -> 0, 6 -> 3, 2 -> 1, 3 -> 2, 4 -> 2))(5,Map(5 -> 0, 1 -> 1, 6 -> 2, 2 -> 1, 3 -> 2, 4 -> 1))(6,Map(5 -> 2, 1 -> 3, 6 -> 0, 2 -> 3, 3 -> 2, 4 -> 1))(2,Map(5 -> 1, 1 -> 1, 6 -> 3, 2 -> 0, 3 -> 1, 4 -> 2))(3,Map(5 -> 2, 1 -> 2, 6 -> 2, 2 -> 1, 3 -> 0, 4 -> 1))

results.toSet:Set((6,Map(5 -> 2, 1 -> 3, 6 -> 0, 2 -> 3, 3 -> 2, 4 -> 1)), (4,Map(5 -> 1, 1 -> 2, 6 -> 1, 2 -> 2, 3 -> 1, 4 -> 0)), (3,Map(5 -> 2, 1 -> 2, 6 -> 2, 2 -> 1, 3 -> 0, 4 -> 1)), (2,Map(5 -> 1, 1 -> 1, 6 -> 3, 2 -> 0, 3 -> 1, 4 -> 2)), (1,Map(5 -> 1, 1 -> 0, 6 -> 3, 2 -> 1, 3 -> 2, 4 -> 2)), (5,Map(5 -> 0, 1 -> 1, 6 -> 2, 2 -> 1, 3 -> 2, 4 -> 1)))

参考

【1】 http://spark.apache.org/docs/1.5.2/graphx-programming-guide.html

【2】https://github.com/xubo245/SparkLearning

【3】http://blog.csdn.net/zcf1002797280/article/details/50007913