transformation action
来源:互联网 发布:网络与手机失泄密 编辑:程序博客网 时间:2024/03/29 02:42
val num = sc.parallelize(1 to 10,3)
指定3个分片(cpu)
//transformation
val doublenum = num.map(_*2)
val threenum = doublenum.filter(_ % 3 ==0)
//action
doublenum.collect()
threenum.collect()
action
reduce collect count first take(前几行) takeSample saveAsTextFile saveAsSequenceFile
countByKey foreach cache
transformation
map filter flatMap mapPartitions mapPartitionsWithSplit sample union distinct
groupByKey reduceByKey sortByKey join union cogroup((k,v1)(k,v2)->(k,v1,v2))
cartesian
join和 groupByKey 略有不同 组内合并
flatMap 组间合并
cache
threenum.cache() // 延后执行 action
指定3个分片(cpu)
//transformation
val doublenum = num.map(_*2)
val threenum = doublenum.filter(_ % 3 ==0)
//action
doublenum.collect()
threenum.collect()
action
reduce collect count first take(前几行) takeSample saveAsTextFile saveAsSequenceFile
countByKey foreach cache
transformation
map filter flatMap mapPartitions mapPartitionsWithSplit sample union distinct
groupByKey reduceByKey sortByKey join union cogroup((k,v1)(k,v2)->(k,v1,v2))
cartesian
join和 groupByKey 略有不同 组内合并
flatMap 组间合并
cache
threenum.cache() // 延后执行 action
threenum.unpersist() // 立即执行
process_local
node_local
0 0
- transformation action
- Spark:Transformation和Action
- Spark: Transformation和Action
- Spark: Transformation和Action
- Spark Transformation和Action
- spark中的action和transformation
- spark中的action和transformation
- spark中的action和transformation
- spark中的action和transformation
- spark transformation和action算子
- transformation and action of spark
- Spark的action和transformation
- spark的transformation与action
- spark中RDD的transformation&action
- spark RDD transformation和action操作
- spark常用函数:transformation和action
- spark中RDD的transformation&action
- spark常用函数:transformation和action
- mysql
- Zenefits 第一轮编程题
- [leetcode] 253. Meeting Rooms II 解题报告
- 周赛题解报告-2013 亚洲南京区域赛
- 8VC Venture Cup 2016 - Final Round (Div. 2 Edition) D635 - Factory Repairs 线段树
- transformation action
- hive 结构
- 学习Javascript闭包(Closure)讲的很深入透彻
- DE 27 Sketching Solutions of 2x2 Homogeneous Linear System with Constant Coefficients
- DE28 Matrix Methods for Inhomogeneous Systems
- [leetcode] 49. Group Anagrams 解题报告
- Hive QL
- vector的详细用法---线程安全
- 带侧边栏字母索引的列表--重写Button类