GraphX增加BFS测试用例
来源:互联网 发布:电脑怎么修改淘宝评价 编辑:程序博客网 时间:2024/06/05 07:01
基于Analytics.scala,在其中增加:
case "bfs" => val outFname = options.remove("output").getOrElse("") options.foreach { case (opt, _) => throw new IllegalArgumentException("Invalid option: " + opt) } println("======================================") println("| BFS |") println("======================================") val sc = new SparkContext(conf.setAppName("ConnectedComponents(" + fname + ")")) val unpartitionedGraph = GraphLoader.edgeListFile(sc, fname, numEdgePartitions = numEPart, edgeStorageLevel = edgeStorageLevel, vertexStorageLevel = vertexStorageLevel).cache() val graph = partitionStrategy.foldLeft(unpartitionedGraph)(_.partitionBy(_)) //unpartitionedGraph是初始值 val bfs = BFS.run(graph)// println("BFS: " + cc.vertices.map { case (vid, data) => data }.distinct()) if (!outFname.isEmpty) { //修改使输出结果 logWarning("Saving pageranks of pages to " + outFname) bfs.vertices.map { case (vid, data) => vid + "\t" + data }.saveAsTextFile(outFname) } sc.stop()
新建bfs.scala
/* * Licensed to the Apache Software Foundation (ASF) under one or more * contributor license agreements. See the NOTICE file distributed with * this work for additional information regarding copyright ownership. * The ASF licenses this file to You under the Apache License, Version 2.0 * (the "License"); you may not use this file except in compliance with * the License. You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. */package org.apache.spark.graphx.libimport scala.reflect.ClassTagimport org.apache.spark.graphx.PartitionStrategy._import org.apache.spark.graphx._import org.apache.spark._object BFS { def run[VD: ClassTag, ED: ClassTag](graph: Graph[VD, ED]): Graph[Double, ED] = { val srcVertex = 1 val bfsGraph = graph.mapVertices((id, _) => if (id == srcVertex) 0.0 else Double.PositiveInfinity) Pregel(bfsGraph, Double.PositiveInfinity, activeDirection = EdgeDirection.Either)( vprog = (id, attr, msg) => math.min(attr, msg), sendMsg = edge => { if (edge.srcAttr != Double.PositiveInfinity) { Iterator((edge.dstId, edge.srcAttr+1)) } else { Iterator.empty } }, mergeMsg = (a,b) => math.min(a,b)) } // end of bfs}
阅读全文
0 0
- GraphX增加BFS测试用例
- GraphX测试要点
- 增加操作测试用例
- App测试和传统软件测试有哪些区别?应该增加哪些方面的测试用例?
- junit测试用例增加@Transaction注解的作用
- 增加测试用例状态的精确度(软件测试过程管理-读书笔记)
- graphx操作实例05-VertexRDD和EdgeRDD属性测试
- 用graphx实现Bronkerbosch计算极大团
- 增加测试手机链接
- 【玩一玩】用java程序测试一下增加blog的访问量!!
- 用 LDA 做主题模型:当 MLlib 邂逅 GraphX
- 用 LDA 做主题模型:当 MLlib 邂逅 GraphX
- 用 LDA 做主题模型:当 MLlib 邂逅 GraphX
- GraphX笔记
- 浅谈GraphX
- GraphX PageRank
- 学习GraphX
- Spark GraphX
- docker安装和使用过程中遇到的问题
- oss 罗列Bucket所有Object
- Hadoop-使用MRUnit来写单元测试
- list的简单剖析
- [免下载]Sublime Text 3 (Build 3143) 最新注册码,亲测可用!
- GraphX增加BFS测试用例
- Fiddler实现手机抓包
- 传输层-3、流水机制和滑动窗口协议
- js 适配器模式
- mybatis在xml文件中处理大于号小于号的方法
- jenkins-git下载代码超时
- Git命令大全思维导图
- Matlab 折线图转化成柱状图
- 并发编程复习(二):volatile关键字使用