Spark实战-Spark SQL（一）
来源：互联网 发布：淘宝详情模板下载 编辑：程序博客网 时间：2024/05/21 17:47
本文是按照周志胡老师的博客实践的，再次特别感谢周老师！
Spark修炼之道（进阶篇）——Spark入门到精通：第十节 Spark SQL案例实战（一）
http://blog.csdn.net/lovehuangjiaju/article/details/50639287
分为以下主要的几步：
获取数据 创建DataFrame DataFrame方法实战 DataFrame注册成临时表使用实战
首先，启动hadoop集群，咱们简单点，就直接 ./start-all.sh启动hdsf和yarn
获取数据
文章通过将最近github的Python练习项目的git日志作为数据，对SparkSQL的内容进行详细介绍
数据获取命令如下：
进入相应本地版本仓库 （.git）目录
root@hjr:/usr/local/developTools/pycharmProjects/python3Learning# git log --pretty=format:'{"commit":"%H","author":"%an","author_email":"%ae","date":"%ad","meaasge":"%f"}' > sparktest.json格式化日志内容，输出如下：
root@hjr:/usr/local/developTools/pycharmProjects/python3Learning# head -1 sparktest.json{"commit":"b40dc4452931232f17fdd749eb43e5e66c99f240","author":"yiyou","author_email":"1627383642@qq.com","date":"Wed Feb 3 19:05:04 2016 +0800","meaasge":"commit-NO3"}
使用put命令将sparktest.json文件上传到HDFS上的/hjr/目录下面：
root@hjr:/usr/local/developTools/pycharmProjects/python3Learning# hadoop dfs -put sparktest.json /hjr/
创建DataFrame
启动spark-shell：
使用我们上传到hdfs上的/hjr/sparktest.json的数据创建DataFrame:
scala> val df = sqlContext.read.json("/hjr/sparktest.json")16/02/06 15:42:40 INFO json.JSONRelation: Listing hdfs://localhost:9000/hjr/sparktest.json on driver16/02/06 15:42:43 INFO storage.MemoryStore: Block broadcast_0 stored as values in memory (estimated size 212.5 KB, free 212.5 KB)16/02/06 15:42:44 INFO storage.MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 19.5 KB, free 232.1 KB)16/02/06 15:42:44 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on localhost:45391 (size: 19.5 KB, free: 511.1 MB)16/02/06 15:42:48 INFO spark.SparkContext: Created broadcast 0 from json at <console>:2516/02/06 15:42:49 INFO mapred.FileInputFormat: Total input paths to process : 116/02/06 15:42:49 INFO spark.SparkContext: Starting job: json at <console>:2516/02/06 15:42:50 INFO scheduler.DAGScheduler: Got job 0 (json at <console>:25) with 2 output partitions16/02/06 15:42:50 INFO scheduler.DAGScheduler: Final stage: ResultStage 0 (json at <console>:25)16/02/06 15:42:50 INFO scheduler.DAGScheduler: Parents of final stage: List()16/02/06 15:42:50 INFO scheduler.DAGScheduler: Missing parents: List()16/02/06 15:42:50 INFO scheduler.DAGScheduler: Submitting ResultStage 0 (MapPartitionsRDD[3] at json at <console>:25), which has no missing parents16/02/06 15:42:50 INFO storage.MemoryStore: Block broadcast_1 stored as values in memory (estimated size 4.3 KB, free 236.4 KB)16/02/06 15:42:50 INFO storage.MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 2.4 KB, free 238.8 KB)16/02/06 15:42:50 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on localhost:45391 (size: 2.4 KB, free: 511.1 MB)16/02/06 15:42:50 INFO spark.SparkContext: Created broadcast 1 from broadcast at DAGScheduler.scala:100616/02/06 15:42:50 INFO scheduler.DAGScheduler: Submitting 2 missing tasks from ResultStage 0 (MapPartitionsRDD[3] at json at <console>:25)16/02/06 15:42:50 INFO scheduler.TaskSchedulerImpl: Adding task set 0.0 with 2 tasks16/02/06 15:42:50 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0, localhost, partition 0,ANY, 2142 bytes)16/02/06 15:42:50 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 0.0 (TID 1, localhost, partition 1,ANY, 2142 bytes)16/02/06 15:42:51 INFO executor.Executor: Running task 1.0 in stage 0.0 (TID 1)16/02/06 15:42:51 INFO executor.Executor: Running task 0.0 in stage 0.0 (TID 0)16/02/06 15:42:51 INFO rdd.HadoopRDD: Input split: hdfs://localhost:9000/hjr/sparktest.json:339+34016/02/06 15:42:51 INFO rdd.HadoopRDD: Input split: hdfs://localhost:9000/hjr/sparktest.json:0+33916/02/06 15:42:51 INFO Configuration.deprecation: mapred.task.id is deprecated. Instead, use mapreduce.task.attempt.id16/02/06 15:42:51 INFO Configuration.deprecation: mapred.task.is.map is deprecated. Instead, use mapreduce.task.ismap16/02/06 15:42:51 INFO Configuration.deprecation: mapred.task.partition is deprecated. Instead, use mapreduce.task.partition16/02/06 15:42:51 INFO Configuration.deprecation: mapred.job.id is deprecated. Instead, use mapreduce.job.id16/02/06 15:42:51 INFO Configuration.deprecation: mapred.tip.id is deprecated. Instead, use mapreduce.task.id16/02/06 15:42:53 INFO executor.Executor: Finished task 0.0 in stage 0.0 (TID 0). 2903 bytes result sent to driver16/02/06 15:42:53 INFO executor.Executor: Finished task 1.0 in stage 0.0 (TID 1). 2903 bytes result sent to driver16/02/06 15:42:53 INFO scheduler.TaskSetManager: Finished task 1.0 in stage 0.0 (TID 1) in 2326 ms on localhost (1/2)16/02/06 15:42:53 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in 2467 ms on localhost (2/2)16/02/06 15:42:53 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks have all completed, from pool 16/02/06 15:42:53 INFO scheduler.DAGScheduler: ResultStage 0 (json at <console>:25) finished in 2.547 s16/02/06 15:42:53 INFO scheduler.DAGScheduler: Job 0 finished: json at <console>:25, took 3.589878 sdf: org.apache.spark.sql.DataFrame = [author: string, author_email: string, commit: string, date: string, meaasge: string]
查看其模式：
scala> df.printSchemaroot |-- author: string (nullable = true) |-- author_email: string (nullable = true) |-- commit: string (nullable = true) |-- date: string (nullable = true) |-- meaasge: string (nullable = true)
DataFrame方法实战
显示数据：
df.show或者显示前两条数据：df.show(2)scala> df.show16/02/06 15:43:34 INFO storage.MemoryStore: Block broadcast_2 stored as values in memory (estimated size 61.8 KB, free 300.6 KB)16/02/06 15:43:34 INFO storage.MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 19.5 KB, free 320.1 KB)16/02/06 15:43:34 INFO storage.BlockManagerInfo: Added broadcast_2_piece0 in memory on localhost:45391 (size: 19.5 KB, free: 511.1 MB)16/02/06 15:43:34 INFO spark.SparkContext: Created broadcast 2 from show at <console>:2816/02/06 15:43:34 INFO storage.MemoryStore: Block broadcast_3 stored as values in memory (estimated size 212.5 KB, free 532.6 KB)16/02/06 15:43:35 INFO storage.MemoryStore: Block broadcast_3_piece0 stored as bytes in memory (estimated size 19.5 KB, free 552.2 KB)16/02/06 15:43:35 INFO storage.BlockManagerInfo: Added broadcast_3_piece0 in memory on localhost:45391 (size: 19.5 KB, free: 511.1 MB)16/02/06 15:43:35 INFO spark.SparkContext: Created broadcast 3 from show at <console>:2816/02/06 15:43:36 INFO mapred.FileInputFormat: Total input paths to process : 116/02/06 15:43:36 INFO spark.SparkContext: Starting job: show at <console>:2816/02/06 15:43:36 INFO scheduler.DAGScheduler: Got job 1 (show at <console>:28) with 1 output partitions16/02/06 15:43:36 INFO scheduler.DAGScheduler: Final stage: ResultStage 1 (show at <console>:28)16/02/06 15:43:36 INFO scheduler.DAGScheduler: Parents of final stage: List()16/02/06 15:43:36 INFO scheduler.DAGScheduler: Missing parents: List()16/02/06 15:43:36 INFO scheduler.DAGScheduler: Submitting ResultStage 1 (MapPartitionsRDD[9] at show at <console>:28), which has no missing parents16/02/06 15:43:36 INFO storage.MemoryStore: Block broadcast_4 stored as values in memory (estimated size 5.9 KB, free 558.0 KB)16/02/06 15:43:36 INFO storage.MemoryStore: Block broadcast_4_piece0 stored as bytes in memory (estimated size 3.3 KB, free 561.3 KB)16/02/06 15:43:36 INFO storage.BlockManagerInfo: Added broadcast_4_piece0 in memory on localhost:45391 (size: 3.3 KB, free: 511.1 MB)16/02/06 15:43:36 INFO spark.SparkContext: Created broadcast 4 from broadcast at DAGScheduler.scala:100616/02/06 15:43:36 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 1 (MapPartitionsRDD[9] at show at <console>:28)16/02/06 15:43:36 INFO scheduler.TaskSchedulerImpl: Adding task set 1.0 with 1 tasks16/02/06 15:43:36 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 1.0 (TID 2, localhost, partition 0,ANY, 2142 bytes)16/02/06 15:43:36 INFO executor.Executor: Running task 0.0 in stage 1.0 (TID 2)16/02/06 15:43:36 INFO rdd.HadoopRDD: Input split: hdfs://localhost:9000/hjr/sparktest.json:0+33916/02/06 15:43:38 INFO codegen.GenerateUnsafeProjection: Code generated in 1530.569751 ms16/02/06 15:43:38 INFO codegen.GenerateSafeProjection: Code generated in 136.022342 ms16/02/06 15:43:38 INFO executor.Executor: Finished task 0.0 in stage 1.0 (TID 2). 2857 bytes result sent to driver16/02/06 15:43:38 INFO scheduler.DAGScheduler: ResultStage 1 (show at <console>:28) finished in 2.126 s16/02/06 15:43:38 INFO scheduler.DAGScheduler: Job 1 finished: show at <console>:28, took 2.448737 s16/02/06 15:43:38 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 1.0 (TID 2) in 2127 ms on localhost (1/1)16/02/06 15:43:38 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 1.0, whose tasks have all completed, from pool 16/02/06 15:43:38 INFO spark.SparkContext: Starting job: show at <console>:2816/02/06 15:43:38 INFO scheduler.DAGScheduler: Got job 2 (show at <console>:28) with 1 output partitions16/02/06 15:43:38 INFO scheduler.DAGScheduler: Final stage: ResultStage 2 (show at <console>:28)16/02/06 15:43:38 INFO scheduler.DAGScheduler: Parents of final stage: List()16/02/06 15:43:38 INFO scheduler.DAGScheduler: Missing parents: List()16/02/06 15:43:38 INFO scheduler.DAGScheduler: Submitting ResultStage 2 (MapPartitionsRDD[9] at show at <console>:28), which has no missing parents16/02/06 15:43:38 INFO storage.MemoryStore: Block broadcast_5 stored as values in memory (estimated size 5.9 KB, free 567.2 KB)16/02/06 15:43:39 INFO storage.MemoryStore: Block broadcast_5_piece0 stored as bytes in memory (estimated size 3.3 KB, free 570.5 KB)16/02/06 15:43:39 INFO storage.BlockManagerInfo: Added broadcast_5_piece0 in memory on localhost:45391 (size: 3.3 KB, free: 511.1 MB)16/02/06 15:43:39 INFO spark.SparkContext: Created broadcast 5 from broadcast at DAGScheduler.scala:100616/02/06 15:43:39 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 2 (MapPartitionsRDD[9] at show at <console>:28)16/02/06 15:43:39 INFO scheduler.TaskSchedulerImpl: Adding task set 2.0 with 1 tasks16/02/06 15:43:39 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 2.0 (TID 3, localhost, partition 1,ANY, 2142 bytes)16/02/06 15:43:39 INFO executor.Executor: Running task 0.0 in stage 2.0 (TID 3)16/02/06 15:43:39 INFO rdd.HadoopRDD: Input split: hdfs://localhost:9000/hjr/sparktest.json:339+34016/02/06 15:43:39 INFO executor.Executor: Finished task 0.0 in stage 2.0 (TID 3). 2501 bytes result sent to driver16/02/06 15:43:39 INFO scheduler.DAGScheduler: ResultStage 2 (show at <console>:28) finished in 0.180 s16/02/06 15:43:39 INFO scheduler.DAGScheduler: Job 2 finished: show at <console>:28, took 0.618656 s16/02/06 15:43:39 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 2.0 (TID 3) in 177 ms on localhost (1/1)16/02/06 15:43:39 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 2.0, whose tasks have all completed, from pool +---------+-----------------+--------------------+--------------------+--------------+| author| author_email| commit| date| meaasge|+---------+-----------------+--------------------+--------------------+--------------+| yiyou|1627383642@qq.com|b40dc4452931232f1...|Wed Feb 3 19:05:0...| commit-NO3|| yiyou|1627383642@qq.com|d9fa805c7d1c00f5c...|Mon Feb 1 23:56:1...| commit-NO2|| yiyou|1627383642@qq.com|66123d4f98e0e5a0f...|Mon Feb 1 15:04:5...| commit-NO1||jingruhou|1627383642@qq.com|56a1fbd42a6122aa2...|Mon Feb 1 14:34:5...|Initial-commit|+---------+-----------------+--------------------+--------------------+--------------+
计算总提交数：
df.countscala> df.count16/02/06 15:43:52 INFO storage.MemoryStore: Block broadcast_6 stored as values in memory (estimated size 212.1 KB, free 782.6 KB)16/02/06 15:43:52 INFO storage.MemoryStore: Block broadcast_6_piece0 stored as bytes in memory (estimated size 19.5 KB, free 802.1 KB)16/02/06 15:43:52 INFO storage.BlockManagerInfo: Added broadcast_6_piece0 in memory on localhost:45391 (size: 19.5 KB, free: 511.0 MB)16/02/06 15:43:52 INFO spark.SparkContext: Created broadcast 6 from count at <console>:2816/02/06 15:43:53 INFO storage.MemoryStore: Block broadcast_7 stored as values in memory (estimated size 212.5 KB, free 1014.6 KB)16/02/06 15:43:53 INFO storage.MemoryStore: Block broadcast_7_piece0 stored as bytes in memory (estimated size 19.5 KB, free 1034.1 KB)16/02/06 15:43:53 INFO storage.BlockManagerInfo: Added broadcast_7_piece0 in memory on localhost:45391 (size: 19.5 KB, free: 511.0 MB)16/02/06 15:43:53 INFO spark.SparkContext: Created broadcast 7 from count at <console>:2816/02/06 15:43:54 INFO mapred.FileInputFormat: Total input paths to process : 116/02/06 15:43:54 INFO spark.SparkContext: Starting job: count at <console>:2816/02/06 15:43:54 INFO scheduler.DAGScheduler: Registering RDD 15 (count at <console>:28)16/02/06 15:43:54 INFO scheduler.DAGScheduler: Got job 3 (count at <console>:28) with 1 output partitions16/02/06 15:43:54 INFO scheduler.DAGScheduler: Final stage: ResultStage 4 (count at <console>:28)16/02/06 15:43:54 INFO scheduler.DAGScheduler: Parents of final stage: List(ShuffleMapStage 3)16/02/06 15:43:54 INFO scheduler.DAGScheduler: Missing parents: List(ShuffleMapStage 3)16/02/06 15:43:54 INFO scheduler.DAGScheduler: Submitting ShuffleMapStage 3 (MapPartitionsRDD[15] at count at <console>:28), which has no missing parents16/02/06 15:43:54 INFO storage.MemoryStore: Block broadcast_8 stored as values in memory (estimated size 10.4 KB, free 1044.5 KB)16/02/06 15:43:54 INFO storage.MemoryStore: Block broadcast_8_piece0 stored as bytes in memory (estimated size 5.3 KB, free 1049.8 KB)16/02/06 15:43:54 INFO storage.BlockManagerInfo: Added broadcast_8_piece0 in memory on localhost:45391 (size: 5.3 KB, free: 511.0 MB)16/02/06 15:43:54 INFO spark.SparkContext: Created broadcast 8 from broadcast at DAGScheduler.scala:100616/02/06 15:43:54 INFO scheduler.DAGScheduler: Submitting 2 missing tasks from ShuffleMapStage 3 (MapPartitionsRDD[15] at count at <console>:28)16/02/06 15:43:54 INFO scheduler.TaskSchedulerImpl: Adding task set 3.0 with 2 tasks16/02/06 15:43:54 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 3.0 (TID 4, localhost, partition 0,ANY, 2131 bytes)16/02/06 15:43:54 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 3.0 (TID 5, localhost, partition 1,ANY, 2131 bytes)16/02/06 15:43:54 INFO executor.Executor: Running task 0.0 in stage 3.0 (TID 4)16/02/06 15:43:54 INFO executor.Executor: Running task 1.0 in stage 3.0 (TID 5)16/02/06 15:43:54 INFO rdd.HadoopRDD: Input split: hdfs://localhost:9000/hjr/sparktest.json:339+34016/02/06 15:43:54 INFO rdd.HadoopRDD: Input split: hdfs://localhost:9000/hjr/sparktest.json:0+33916/02/06 15:43:55 INFO codegen.GenerateUnsafeProjection: Code generated in 144.98594 ms16/02/06 15:43:55 INFO codegen.GenerateMutableProjection: Code generated in 123.515589 ms16/02/06 15:43:55 INFO codegen.GenerateMutableProjection: Code generated in 134.29397 ms16/02/06 15:43:56 INFO codegen.GenerateUnsafeRowJoiner: Code generated in 91.839487 ms16/02/06 15:43:56 INFO codegen.GenerateUnsafeProjection: Code generated in 292.836142 ms16/02/06 15:43:57 INFO executor.Executor: Finished task 0.0 in stage 3.0 (TID 4). 2500 bytes result sent to driver16/02/06 15:43:57 INFO executor.Executor: Finished task 1.0 in stage 3.0 (TID 5). 2500 bytes result sent to driver16/02/06 15:43:57 INFO scheduler.TaskSetManager: Finished task 1.0 in stage 3.0 (TID 5) in 2913 ms on localhost (1/2)16/02/06 15:43:57 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 3.0 (TID 4) in 2928 ms on localhost (2/2)16/02/06 15:43:57 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 3.0, whose tasks have all completed, from pool 16/02/06 15:43:57 INFO scheduler.DAGScheduler: ShuffleMapStage 3 (count at <console>:28) finished in 2.928 s16/02/06 15:43:57 INFO scheduler.DAGScheduler: looking for newly runnable stages16/02/06 15:43:57 INFO scheduler.DAGScheduler: running: Set()16/02/06 15:43:57 INFO scheduler.DAGScheduler: waiting: Set(ResultStage 4)16/02/06 15:43:57 INFO scheduler.DAGScheduler: failed: Set()16/02/06 15:43:57 INFO scheduler.DAGScheduler: Submitting ResultStage 4 (MapPartitionsRDD[18] at count at <console>:28), which has no missing parents16/02/06 15:43:57 INFO storage.MemoryStore: Block broadcast_9 stored as values in memory (estimated size 11.5 KB, free 1061.3 KB)16/02/06 15:43:57 INFO storage.MemoryStore: Block broadcast_9_piece0 stored as bytes in memory (estimated size 5.8 KB, free 1067.1 KB)16/02/06 15:43:57 INFO storage.BlockManagerInfo: Added broadcast_9_piece0 in memory on localhost:45391 (size: 5.8 KB, free: 511.0 MB)16/02/06 15:43:57 INFO spark.SparkContext: Created broadcast 9 from broadcast at DAGScheduler.scala:100616/02/06 15:43:57 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 4 (MapPartitionsRDD[18] at count at <console>:28)16/02/06 15:43:57 INFO scheduler.TaskSchedulerImpl: Adding task set 4.0 with 1 tasks16/02/06 15:43:57 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 4.0 (TID 6, localhost, partition 0,NODE_LOCAL, 1999 bytes)16/02/06 15:43:57 INFO executor.Executor: Running task 0.0 in stage 4.0 (TID 6)16/02/06 15:43:58 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:43:58 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 29 ms16/02/06 15:43:58 INFO codegen.GenerateMutableProjection: Code generated in 66.448896 ms16/02/06 15:43:58 INFO codegen.GenerateMutableProjection: Code generated in 39.551418 ms16/02/06 15:44:02 INFO executor.Executor: Finished task 0.0 in stage 4.0 (TID 6). 1830 bytes result sent to driver16/02/06 15:44:02 INFO scheduler.DAGScheduler: ResultStage 4 (count at <console>:28) finished in 4.435 s16/02/06 15:44:02 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 4.0 (TID 6) in 4430 ms on localhost (1/1)16/02/06 15:44:02 INFO scheduler.DAGScheduler: Job 3 finished: count at <console>:28, took 3.981684 s16/02/06 15:44:02 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 4.0, whose tasks have all completed, from pool res2: Long = 4
按提交次数进行降序排序：
scala> df.groupBy("author").count.sort($"count".desc).show16/02/06 15:45:16 INFO spark.ContextCleaner: Cleaned accumulator 1016/02/06 15:45:16 INFO spark.ContextCleaner: Cleaned accumulator 916/02/06 15:45:16 INFO spark.ContextCleaner: Cleaned accumulator 816/02/06 15:45:16 INFO spark.ContextCleaner: Cleaned accumulator 716/02/06 15:45:16 INFO spark.ContextCleaner: Cleaned accumulator 616/02/06 15:45:16 INFO spark.ContextCleaner: Cleaned accumulator 516/02/06 15:45:17 INFO storage.BlockManagerInfo: Removed broadcast_7_piece0 on localhost:45391 in memory (size: 19.5 KB, free: 511.0 MB)16/02/06 15:45:17 INFO storage.BlockManagerInfo: Removed broadcast_6_piece0 on localhost:45391 in memory (size: 19.5 KB, free: 511.0 MB)16/02/06 15:45:17 INFO storage.BlockManagerInfo: Removed broadcast_5_piece0 on localhost:45391 in memory (size: 3.3 KB, free: 511.1 MB)16/02/06 15:45:17 INFO spark.ContextCleaner: Cleaned accumulator 316/02/06 15:45:17 INFO storage.BlockManagerInfo: Removed broadcast_4_piece0 on localhost:45391 in memory (size: 3.3 KB, free: 511.1 MB)16/02/06 15:45:17 INFO spark.ContextCleaner: Cleaned accumulator 216/02/06 15:45:17 INFO storage.BlockManagerInfo: Removed broadcast_3_piece0 on localhost:45391 in memory (size: 19.5 KB, free: 511.1 MB)16/02/06 15:45:17 INFO storage.BlockManagerInfo: Removed broadcast_2_piece0 on localhost:45391 in memory (size: 19.5 KB, free: 511.1 MB)16/02/06 15:45:17 INFO storage.BlockManagerInfo: Removed broadcast_1_piece0 on localhost:45391 in memory (size: 2.4 KB, free: 511.1 MB)16/02/06 15:45:17 INFO spark.ContextCleaner: Cleaned accumulator 116/02/06 15:45:17 INFO storage.BlockManagerInfo: Removed broadcast_0_piece0 on localhost:45391 in memory (size: 19.5 KB, free: 511.1 MB)16/02/06 15:45:17 INFO storage.BlockManagerInfo: Removed broadcast_9_piece0 on localhost:45391 in memory (size: 5.8 KB, free: 511.1 MB)16/02/06 15:45:17 INFO spark.ContextCleaner: Cleaned accumulator 1416/02/06 15:45:17 INFO storage.BlockManagerInfo: Removed broadcast_8_piece0 on localhost:45391 in memory (size: 5.3 KB, free: 511.1 MB)16/02/06 15:45:17 INFO spark.ContextCleaner: Cleaned accumulator 1316/02/06 15:45:17 INFO spark.ContextCleaner: Cleaned shuffle 016/02/06 15:45:17 INFO spark.ContextCleaner: Cleaned accumulator 1216/02/06 15:45:17 INFO spark.ContextCleaner: Cleaned accumulator 1116/02/06 15:45:18 INFO storage.MemoryStore: Block broadcast_10 stored as values in memory (estimated size 212.1 KB, free 212.1 KB)16/02/06 15:45:18 INFO storage.MemoryStore: Block broadcast_10_piece0 stored as bytes in memory (estimated size 19.5 KB, free 231.5 KB)16/02/06 15:45:18 INFO storage.BlockManagerInfo: Added broadcast_10_piece0 in memory on localhost:45391 (size: 19.5 KB, free: 511.1 MB)16/02/06 15:45:18 INFO spark.SparkContext: Created broadcast 10 from show at <console>:2816/02/06 15:45:18 INFO storage.MemoryStore: Block broadcast_11 stored as values in memory (estimated size 212.5 KB, free 444.1 KB)16/02/06 15:45:19 INFO storage.MemoryStore: Block broadcast_11_piece0 stored as bytes in memory (estimated size 19.5 KB, free 463.6 KB)16/02/06 15:45:19 INFO storage.BlockManagerInfo: Added broadcast_11_piece0 in memory on localhost:45391 (size: 19.5 KB, free: 511.1 MB)16/02/06 15:45:19 INFO spark.SparkContext: Created broadcast 11 from show at <console>:2816/02/06 15:45:19 INFO mapred.FileInputFormat: Total input paths to process : 116/02/06 15:45:19 INFO spark.SparkContext: Starting job: show at <console>:2816/02/06 15:45:19 INFO scheduler.DAGScheduler: Registering RDD 24 (show at <console>:28)16/02/06 15:45:19 INFO scheduler.DAGScheduler: Got job 4 (show at <console>:28) with 200 output partitions16/02/06 15:45:19 INFO scheduler.DAGScheduler: Final stage: ResultStage 6 (show at <console>:28)16/02/06 15:45:19 INFO scheduler.DAGScheduler: Parents of final stage: List(ShuffleMapStage 5)16/02/06 15:45:19 INFO scheduler.DAGScheduler: Missing parents: List(ShuffleMapStage 5)16/02/06 15:45:19 INFO scheduler.DAGScheduler: Submitting ShuffleMapStage 5 (MapPartitionsRDD[24] at show at <console>:28), which has no missing parents16/02/06 15:45:19 INFO storage.MemoryStore: Block broadcast_12 stored as values in memory (estimated size 10.8 KB, free 474.4 KB)16/02/06 15:45:19 INFO storage.MemoryStore: Block broadcast_12_piece0 stored as bytes in memory (estimated size 5.5 KB, free 479.9 KB)16/02/06 15:45:19 INFO storage.BlockManagerInfo: Added broadcast_12_piece0 in memory on localhost:45391 (size: 5.5 KB, free: 511.1 MB)16/02/06 15:45:19 INFO spark.SparkContext: Created broadcast 12 from broadcast at DAGScheduler.scala:100616/02/06 15:45:19 INFO scheduler.DAGScheduler: Submitting 2 missing tasks from ShuffleMapStage 5 (MapPartitionsRDD[24] at show at <console>:28)16/02/06 15:45:19 INFO scheduler.TaskSchedulerImpl: Adding task set 5.0 with 2 tasks16/02/06 15:45:19 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 5.0 (TID 7, localhost, partition 0,ANY, 2131 bytes)16/02/06 15:45:19 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 5.0 (TID 8, localhost, partition 1,ANY, 2131 bytes)16/02/06 15:45:19 INFO executor.Executor: Running task 0.0 in stage 5.0 (TID 7)16/02/06 15:45:19 INFO executor.Executor: Running task 1.0 in stage 5.0 (TID 8)16/02/06 15:45:19 INFO rdd.HadoopRDD: Input split: hdfs://localhost:9000/hjr/sparktest.json:0+33916/02/06 15:45:19 INFO rdd.HadoopRDD: Input split: hdfs://localhost:9000/hjr/sparktest.json:339+34016/02/06 15:45:19 INFO codegen.GenerateUnsafeProjection: Code generated in 168.897247 ms16/02/06 15:45:19 INFO codegen.GenerateUnsafeRowJoiner: Code generated in 90.377235 ms16/02/06 15:45:20 INFO codegen.GenerateMutableProjection: Code generated in 50.775065 ms16/02/06 15:45:20 INFO executor.Executor: Finished task 0.0 in stage 5.0 (TID 7). 2702 bytes result sent to driver16/02/06 15:45:20 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 5.0 (TID 7) in 1222 ms on localhost (1/2)16/02/06 15:45:20 INFO executor.Executor: Finished task 1.0 in stage 5.0 (TID 8). 2702 bytes result sent to driver16/02/06 15:45:20 INFO scheduler.DAGScheduler: ShuffleMapStage 5 (show at <console>:28) finished in 1.265 s16/02/06 15:45:20 INFO scheduler.DAGScheduler: looking for newly runnable stages16/02/06 15:45:20 INFO scheduler.DAGScheduler: running: Set()16/02/06 15:45:20 INFO scheduler.DAGScheduler: waiting: Set(ResultStage 6)16/02/06 15:45:20 INFO scheduler.DAGScheduler: failed: Set()16/02/06 15:45:20 INFO scheduler.TaskSetManager: Finished task 1.0 in stage 5.0 (TID 8) in 1263 ms on localhost (2/2)16/02/06 15:45:20 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 5.0, whose tasks have all completed, from pool 16/02/06 15:45:20 INFO scheduler.DAGScheduler: Submitting ResultStage 6 (MapPartitionsRDD[29] at show at <console>:28), which has no missing parents16/02/06 15:45:20 INFO storage.MemoryStore: Block broadcast_13 stored as values in memory (estimated size 13.9 KB, free 493.8 KB)16/02/06 15:45:21 INFO storage.MemoryStore: Block broadcast_13_piece0 stored as bytes in memory (estimated size 7.2 KB, free 501.0 KB)16/02/06 15:45:21 INFO storage.BlockManagerInfo: Added broadcast_13_piece0 in memory on localhost:45391 (size: 7.2 KB, free: 511.1 MB)16/02/06 15:45:21 INFO spark.SparkContext: Created broadcast 13 from broadcast at DAGScheduler.scala:100616/02/06 15:45:21 INFO scheduler.DAGScheduler: Submitting 200 missing tasks from ResultStage 6 (MapPartitionsRDD[29] at show at <console>:28)16/02/06 15:45:21 INFO scheduler.TaskSchedulerImpl: Adding task set 6.0 with 200 tasks16/02/06 15:45:21 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 6.0 (TID 9, localhost, partition 0,NODE_LOCAL, 1999 bytes)16/02/06 15:45:21 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 6.0 (TID 10, localhost, partition 1,NODE_LOCAL, 1999 bytes)16/02/06 15:45:21 INFO executor.Executor: Running task 0.0 in stage 6.0 (TID 9)16/02/06 15:45:21 INFO executor.Executor: Running task 1.0 in stage 6.0 (TID 10)16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:21 INFO codegen.GenerateSafeProjection: Code generated in 71.975262 ms16/02/06 15:45:21 INFO executor.Executor: Finished task 1.0 in stage 6.0 (TID 10). 2676 bytes result sent to driver16/02/06 15:45:21 INFO scheduler.TaskSetManager: Starting task 2.0 in stage 6.0 (TID 11, localhost, partition 2,NODE_LOCAL, 1999 bytes)16/02/06 15:45:21 INFO executor.Executor: Finished task 0.0 in stage 6.0 (TID 9). 2676 bytes result sent to driver16/02/06 15:45:21 INFO scheduler.TaskSetManager: Finished task 1.0 in stage 6.0 (TID 10) in 266 ms on localhost (1/200)16/02/06 15:45:21 INFO scheduler.TaskSetManager: Starting task 3.0 in stage 6.0 (TID 12, localhost, partition 3,NODE_LOCAL, 1999 bytes)16/02/06 15:45:21 INFO executor.Executor: Running task 2.0 in stage 6.0 (TID 11)16/02/06 15:45:21 INFO executor.Executor: Running task 3.0 in stage 6.0 (TID 12)16/02/06 15:45:21 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 6.0 (TID 9) in 276 ms on localhost (2/200)16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 4 ms16/02/06 15:45:21 INFO executor.Executor: Finished task 2.0 in stage 6.0 (TID 11). 2676 bytes result sent to driver16/02/06 15:45:21 INFO scheduler.TaskSetManager: Starting task 4.0 in stage 6.0 (TID 13, localhost, partition 4,NODE_LOCAL, 1999 bytes)16/02/06 15:45:21 INFO executor.Executor: Running task 4.0 in stage 6.0 (TID 13)16/02/06 15:45:21 INFO scheduler.TaskSetManager: Finished task 2.0 in stage 6.0 (TID 11) in 55 ms on localhost (3/200)16/02/06 15:45:21 INFO executor.Executor: Finished task 3.0 in stage 6.0 (TID 12). 2676 bytes result sent to driver16/02/06 15:45:21 INFO scheduler.TaskSetManager: Starting task 5.0 in stage 6.0 (TID 14, localhost, partition 5,NODE_LOCAL, 1999 bytes)16/02/06 15:45:21 INFO executor.Executor: Running task 5.0 in stage 6.0 (TID 14)16/02/06 15:45:21 INFO scheduler.TaskSetManager: Finished task 3.0 in stage 6.0 (TID 12) in 65 ms on localhost (4/200)16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 6 ms16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:21 INFO executor.Executor: Finished task 4.0 in stage 6.0 (TID 13). 2676 bytes result sent to driver16/02/06 15:45:21 INFO scheduler.TaskSetManager: Starting task 6.0 in stage 6.0 (TID 15, localhost, partition 6,NODE_LOCAL, 1999 bytes)16/02/06 15:45:21 INFO executor.Executor: Running task 6.0 in stage 6.0 (TID 15)16/02/06 15:45:21 INFO scheduler.TaskSetManager: Finished task 4.0 in stage 6.0 (TID 13) in 58 ms on localhost (5/200)16/02/06 15:45:21 INFO executor.Executor: Finished task 5.0 in stage 6.0 (TID 14). 2676 bytes result sent to driver16/02/06 15:45:21 INFO scheduler.TaskSetManager: Starting task 7.0 in stage 6.0 (TID 16, localhost, partition 7,NODE_LOCAL, 1999 bytes)16/02/06 15:45:21 INFO executor.Executor: Running task 7.0 in stage 6.0 (TID 16)16/02/06 15:45:21 INFO scheduler.TaskSetManager: Finished task 5.0 in stage 6.0 (TID 14) in 53 ms on localhost (6/200)16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 4 ms16/02/06 15:45:21 INFO executor.Executor: Finished task 7.0 in stage 6.0 (TID 16). 2676 bytes result sent to driver16/02/06 15:45:21 INFO scheduler.TaskSetManager: Starting task 8.0 in stage 6.0 (TID 17, localhost, partition 8,NODE_LOCAL, 1999 bytes)16/02/06 15:45:21 INFO scheduler.TaskSetManager: Finished task 7.0 in stage 6.0 (TID 16) in 91 ms on localhost (7/200)16/02/06 15:45:21 INFO executor.Executor: Running task 8.0 in stage 6.0 (TID 17)16/02/06 15:45:21 INFO executor.Executor: Finished task 6.0 in stage 6.0 (TID 15). 2676 bytes result sent to driver16/02/06 15:45:21 INFO scheduler.TaskSetManager: Starting task 9.0 in stage 6.0 (TID 18, localhost, partition 9,NODE_LOCAL, 1999 bytes)16/02/06 15:45:21 INFO executor.Executor: Running task 9.0 in stage 6.0 (TID 18)16/02/06 15:45:21 INFO scheduler.TaskSetManager: Finished task 6.0 in stage 6.0 (TID 15) in 125 ms on localhost (8/200)16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 8 ms16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 8 ms16/02/06 15:45:21 INFO executor.Executor: Finished task 8.0 in stage 6.0 (TID 17). 2676 bytes result sent to driver16/02/06 15:45:21 INFO scheduler.TaskSetManager: Starting task 10.0 in stage 6.0 (TID 19, localhost, partition 10,NODE_LOCAL, 1999 bytes)16/02/06 15:45:21 INFO executor.Executor: Running task 10.0 in stage 6.0 (TID 19)16/02/06 15:45:21 INFO scheduler.TaskSetManager: Finished task 8.0 in stage 6.0 (TID 17) in 121 ms on localhost (9/200)16/02/06 15:45:21 INFO executor.Executor: Finished task 9.0 in stage 6.0 (TID 18). 2676 bytes result sent to driver16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 17 ms16/02/06 15:45:21 INFO scheduler.TaskSetManager: Starting task 11.0 in stage 6.0 (TID 20, localhost, partition 11,NODE_LOCAL, 1999 bytes)16/02/06 15:45:21 INFO scheduler.TaskSetManager: Finished task 9.0 in stage 6.0 (TID 18) in 146 ms on localhost (10/200)16/02/06 15:45:21 INFO executor.Executor: Running task 11.0 in stage 6.0 (TID 20)16/02/06 15:45:21 INFO executor.Executor: Finished task 10.0 in stage 6.0 (TID 19). 2676 bytes result sent to driver16/02/06 15:45:21 INFO scheduler.TaskSetManager: Starting task 12.0 in stage 6.0 (TID 21, localhost, partition 12,NODE_LOCAL, 1999 bytes)16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms16/02/06 15:45:21 INFO scheduler.TaskSetManager: Finished task 10.0 in stage 6.0 (TID 19) in 82 ms on localhost (11/200)16/02/06 15:45:21 INFO executor.Executor: Running task 12.0 in stage 6.0 (TID 21)16/02/06 15:45:21 INFO executor.Executor: Finished task 11.0 in stage 6.0 (TID 20). 2676 bytes result sent to driver16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 6 ms16/02/06 15:45:21 INFO scheduler.TaskSetManager: Starting task 13.0 in stage 6.0 (TID 22, localhost, partition 13,NODE_LOCAL, 1999 bytes)16/02/06 15:45:21 INFO executor.Executor: Running task 13.0 in stage 6.0 (TID 22)16/02/06 15:45:21 INFO scheduler.TaskSetManager: Finished task 11.0 in stage 6.0 (TID 20) in 100 ms on localhost (12/200)16/02/06 15:45:21 INFO executor.Executor: Finished task 12.0 in stage 6.0 (TID 21). 2676 bytes result sent to driver16/02/06 15:45:21 INFO scheduler.TaskSetManager: Starting task 14.0 in stage 6.0 (TID 23, localhost, partition 14,NODE_LOCAL, 1999 bytes)16/02/06 15:45:21 INFO scheduler.TaskSetManager: Finished task 12.0 in stage 6.0 (TID 21) in 85 ms on localhost (13/200)16/02/06 15:45:21 INFO executor.Executor: Running task 14.0 in stage 6.0 (TID 23)16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:21 INFO executor.Executor: Finished task 13.0 in stage 6.0 (TID 22). 2676 bytes result sent to driver16/02/06 15:45:21 INFO scheduler.TaskSetManager: Starting task 15.0 in stage 6.0 (TID 24, localhost, partition 15,NODE_LOCAL, 1999 bytes)16/02/06 15:45:21 INFO executor.Executor: Running task 15.0 in stage 6.0 (TID 24)16/02/06 15:45:21 INFO scheduler.TaskSetManager: Finished task 13.0 in stage 6.0 (TID 22) in 72 ms on localhost (14/200)16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 7 ms16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 7 ms16/02/06 15:45:21 INFO executor.Executor: Finished task 15.0 in stage 6.0 (TID 24). 2676 bytes result sent to driver16/02/06 15:45:21 INFO executor.Executor: Finished task 14.0 in stage 6.0 (TID 23). 2676 bytes result sent to driver16/02/06 15:45:21 INFO scheduler.TaskSetManager: Starting task 16.0 in stage 6.0 (TID 25, localhost, partition 16,NODE_LOCAL, 1999 bytes)16/02/06 15:45:21 INFO executor.Executor: Running task 16.0 in stage 6.0 (TID 25)16/02/06 15:45:21 INFO scheduler.TaskSetManager: Finished task 15.0 in stage 6.0 (TID 24) in 63 ms on localhost (15/200)16/02/06 15:45:21 INFO scheduler.TaskSetManager: Starting task 17.0 in stage 6.0 (TID 26, localhost, partition 17,NODE_LOCAL, 1999 bytes)16/02/06 15:45:21 INFO executor.Executor: Running task 17.0 in stage 6.0 (TID 26)16/02/06 15:45:21 INFO scheduler.TaskSetManager: Finished task 14.0 in stage 6.0 (TID 23) in 125 ms on localhost (16/200)16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 4 ms16/02/06 15:45:21 INFO executor.Executor: Finished task 16.0 in stage 6.0 (TID 25). 2676 bytes result sent to driver16/02/06 15:45:21 INFO scheduler.TaskSetManager: Starting task 18.0 in stage 6.0 (TID 27, localhost, partition 18,NODE_LOCAL, 1999 bytes)16/02/06 15:45:21 INFO executor.Executor: Running task 18.0 in stage 6.0 (TID 27)16/02/06 15:45:21 INFO scheduler.TaskSetManager: Finished task 16.0 in stage 6.0 (TID 25) in 65 ms on localhost (17/200)16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:21 INFO executor.Executor: Finished task 18.0 in stage 6.0 (TID 27). 2676 bytes result sent to driver16/02/06 15:45:21 INFO scheduler.TaskSetManager: Starting task 19.0 in stage 6.0 (TID 28, localhost, partition 19,NODE_LOCAL, 1999 bytes)16/02/06 15:45:21 INFO executor.Executor: Running task 19.0 in stage 6.0 (TID 28)16/02/06 15:45:21 INFO scheduler.TaskSetManager: Finished task 18.0 in stage 6.0 (TID 27) in 48 ms on localhost (18/200)16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 19 ms16/02/06 15:45:21 INFO executor.Executor: Finished task 17.0 in stage 6.0 (TID 26). 2676 bytes result sent to driver16/02/06 15:45:21 INFO scheduler.TaskSetManager: Starting task 20.0 in stage 6.0 (TID 29, localhost, partition 20,NODE_LOCAL, 1999 bytes)16/02/06 15:45:21 INFO executor.Executor: Running task 20.0 in stage 6.0 (TID 29)16/02/06 15:45:21 INFO scheduler.TaskSetManager: Finished task 17.0 in stage 6.0 (TID 26) in 99 ms on localhost (19/200)16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:21 INFO executor.Executor: Finished task 19.0 in stage 6.0 (TID 28). 2676 bytes result sent to driver16/02/06 15:45:21 INFO scheduler.TaskSetManager: Starting task 21.0 in stage 6.0 (TID 30, localhost, partition 21,NODE_LOCAL, 1999 bytes)16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:21 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 8 ms16/02/06 15:45:22 INFO scheduler.TaskSetManager: Finished task 19.0 in stage 6.0 (TID 28) in 47 ms on localhost (20/200)16/02/06 15:45:22 INFO executor.Executor: Finished task 20.0 in stage 6.0 (TID 29). 2676 bytes result sent to driver16/02/06 15:45:22 INFO executor.Executor: Running task 21.0 in stage 6.0 (TID 30)16/02/06 15:45:22 INFO scheduler.TaskSetManager: Starting task 22.0 in stage 6.0 (TID 31, localhost, partition 22,NODE_LOCAL, 1999 bytes)16/02/06 15:45:22 INFO executor.Executor: Running task 22.0 in stage 6.0 (TID 31)16/02/06 15:45:22 INFO scheduler.TaskSetManager: Finished task 20.0 in stage 6.0 (TID 29) in 34 ms on localhost (21/200)16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 4 ms16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 10 ms16/02/06 15:45:22 INFO executor.Executor: Finished task 22.0 in stage 6.0 (TID 31). 2676 bytes result sent to driver16/02/06 15:45:22 INFO executor.Executor: Finished task 21.0 in stage 6.0 (TID 30). 2676 bytes result sent to driver16/02/06 15:45:22 INFO scheduler.TaskSetManager: Starting task 23.0 in stage 6.0 (TID 32, localhost, partition 23,NODE_LOCAL, 1999 bytes)16/02/06 15:45:22 INFO scheduler.TaskSetManager: Finished task 22.0 in stage 6.0 (TID 31) in 71 ms on localhost (22/200)16/02/06 15:45:22 INFO executor.Executor: Running task 23.0 in stage 6.0 (TID 32)16/02/06 15:45:22 INFO scheduler.TaskSetManager: Finished task 21.0 in stage 6.0 (TID 30) in 85 ms on localhost (23/200)16/02/06 15:45:22 INFO scheduler.TaskSetManager: Starting task 24.0 in stage 6.0 (TID 33, localhost, partition 24,NODE_LOCAL, 1999 bytes)16/02/06 15:45:22 INFO executor.Executor: Running task 24.0 in stage 6.0 (TID 33)16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:22 INFO executor.Executor: Finished task 24.0 in stage 6.0 (TID 33). 2676 bytes result sent to driver16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 12 ms16/02/06 15:45:22 INFO executor.Executor: Finished task 23.0 in stage 6.0 (TID 32). 2676 bytes result sent to driver16/02/06 15:45:22 INFO scheduler.TaskSetManager: Starting task 25.0 in stage 6.0 (TID 34, localhost, partition 25,NODE_LOCAL, 1999 bytes)16/02/06 15:45:22 INFO scheduler.TaskSetManager: Finished task 24.0 in stage 6.0 (TID 33) in 52 ms on localhost (24/200)16/02/06 15:45:22 INFO executor.Executor: Running task 25.0 in stage 6.0 (TID 34)16/02/06 15:45:22 INFO scheduler.TaskSetManager: Starting task 26.0 in stage 6.0 (TID 35, localhost, partition 26,NODE_LOCAL, 1999 bytes)16/02/06 15:45:22 INFO scheduler.TaskSetManager: Finished task 23.0 in stage 6.0 (TID 32) in 80 ms on localhost (25/200)16/02/06 15:45:22 INFO executor.Executor: Running task 26.0 in stage 6.0 (TID 35)16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 4 ms16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms16/02/06 15:45:22 INFO executor.Executor: Finished task 25.0 in stage 6.0 (TID 34). 2676 bytes result sent to driver16/02/06 15:45:22 INFO executor.Executor: Finished task 26.0 in stage 6.0 (TID 35). 2676 bytes result sent to driver16/02/06 15:45:22 INFO scheduler.TaskSetManager: Starting task 27.0 in stage 6.0 (TID 36, localhost, partition 27,NODE_LOCAL, 1999 bytes)16/02/06 15:45:22 INFO executor.Executor: Running task 27.0 in stage 6.0 (TID 36)16/02/06 15:45:22 INFO scheduler.TaskSetManager: Finished task 25.0 in stage 6.0 (TID 34) in 56 ms on localhost (26/200)16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:22 INFO scheduler.TaskSetManager: Starting task 28.0 in stage 6.0 (TID 37, localhost, partition 28,NODE_LOCAL, 1999 bytes)16/02/06 15:45:22 INFO scheduler.TaskSetManager: Finished task 26.0 in stage 6.0 (TID 35) in 55 ms on localhost (27/200)16/02/06 15:45:22 INFO executor.Executor: Running task 28.0 in stage 6.0 (TID 37)16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 13 ms16/02/06 15:45:22 INFO executor.Executor: Finished task 27.0 in stage 6.0 (TID 36). 2676 bytes result sent to driver16/02/06 15:45:22 INFO scheduler.TaskSetManager: Starting task 29.0 in stage 6.0 (TID 38, localhost, partition 29,NODE_LOCAL, 1999 bytes)16/02/06 15:45:22 INFO executor.Executor: Running task 29.0 in stage 6.0 (TID 38)16/02/06 15:45:22 INFO scheduler.TaskSetManager: Finished task 27.0 in stage 6.0 (TID 36) in 39 ms on localhost (28/200)16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 16 ms16/02/06 15:45:22 INFO executor.Executor: Finished task 28.0 in stage 6.0 (TID 37). 2676 bytes result sent to driver16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:22 INFO executor.Executor: Finished task 29.0 in stage 6.0 (TID 38). 2676 bytes result sent to driver16/02/06 15:45:22 INFO scheduler.TaskSetManager: Starting task 30.0 in stage 6.0 (TID 39, localhost, partition 30,NODE_LOCAL, 1999 bytes)16/02/06 15:45:22 INFO scheduler.TaskSetManager: Finished task 28.0 in stage 6.0 (TID 37) in 50 ms on localhost (29/200)16/02/06 15:45:22 INFO executor.Executor: Running task 30.0 in stage 6.0 (TID 39)16/02/06 15:45:22 INFO scheduler.TaskSetManager: Starting task 31.0 in stage 6.0 (TID 40, localhost, partition 31,NODE_LOCAL, 1999 bytes)16/02/06 15:45:22 INFO executor.Executor: Running task 31.0 in stage 6.0 (TID 40)16/02/06 15:45:22 INFO scheduler.TaskSetManager: Finished task 29.0 in stage 6.0 (TID 38) in 46 ms on localhost (30/200)16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 7 ms16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 15 ms16/02/06 15:45:22 INFO executor.Executor: Finished task 30.0 in stage 6.0 (TID 39). 2676 bytes result sent to driver16/02/06 15:45:22 INFO executor.Executor: Finished task 31.0 in stage 6.0 (TID 40). 2676 bytes result sent to driver16/02/06 15:45:22 INFO scheduler.TaskSetManager: Starting task 32.0 in stage 6.0 (TID 41, localhost, partition 32,NODE_LOCAL, 1999 bytes)16/02/06 15:45:22 INFO executor.Executor: Running task 32.0 in stage 6.0 (TID 41)16/02/06 15:45:22 INFO scheduler.TaskSetManager: Finished task 30.0 in stage 6.0 (TID 39) in 71 ms on localhost (31/200)16/02/06 15:45:22 INFO scheduler.TaskSetManager: Starting task 33.0 in stage 6.0 (TID 42, localhost, partition 33,NODE_LOCAL, 1999 bytes)16/02/06 15:45:22 INFO executor.Executor: Running task 33.0 in stage 6.0 (TID 42)16/02/06 15:45:22 INFO scheduler.TaskSetManager: Finished task 31.0 in stage 6.0 (TID 40) in 127 ms on localhost (32/200)16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 17 ms16/02/06 15:45:22 INFO executor.Executor: Finished task 32.0 in stage 6.0 (TID 41). 2676 bytes result sent to driver16/02/06 15:45:22 INFO scheduler.TaskSetManager: Starting task 34.0 in stage 6.0 (TID 43, localhost, partition 34,NODE_LOCAL, 1999 bytes)16/02/06 15:45:22 INFO executor.Executor: Running task 34.0 in stage 6.0 (TID 43)16/02/06 15:45:22 INFO scheduler.TaskSetManager: Finished task 32.0 in stage 6.0 (TID 41) in 191 ms on localhost (33/200)16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:22 INFO executor.Executor: Finished task 33.0 in stage 6.0 (TID 42). 2676 bytes result sent to driver16/02/06 15:45:22 INFO scheduler.TaskSetManager: Starting task 35.0 in stage 6.0 (TID 44, localhost, partition 35,NODE_LOCAL, 1999 bytes)16/02/06 15:45:22 INFO executor.Executor: Running task 35.0 in stage 6.0 (TID 44)16/02/06 15:45:22 INFO scheduler.TaskSetManager: Finished task 33.0 in stage 6.0 (TID 42) in 200 ms on localhost (34/200)16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms16/02/06 15:45:22 INFO executor.Executor: Finished task 34.0 in stage 6.0 (TID 43). 2676 bytes result sent to driver16/02/06 15:45:22 INFO scheduler.TaskSetManager: Starting task 36.0 in stage 6.0 (TID 45, localhost, partition 36,NODE_LOCAL, 1999 bytes)16/02/06 15:45:22 INFO executor.Executor: Running task 36.0 in stage 6.0 (TID 45)16/02/06 15:45:22 INFO scheduler.TaskSetManager: Finished task 34.0 in stage 6.0 (TID 43) in 67 ms on localhost (35/200)16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 19 ms16/02/06 15:45:22 INFO executor.Executor: Finished task 35.0 in stage 6.0 (TID 44). 2676 bytes result sent to driver16/02/06 15:45:22 INFO scheduler.TaskSetManager: Starting task 37.0 in stage 6.0 (TID 46, localhost, partition 37,NODE_LOCAL, 1999 bytes)16/02/06 15:45:22 INFO executor.Executor: Running task 37.0 in stage 6.0 (TID 46)16/02/06 15:45:22 INFO scheduler.TaskSetManager: Finished task 35.0 in stage 6.0 (TID 44) in 46 ms on localhost (36/200)16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 14 ms16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms16/02/06 15:45:22 INFO executor.Executor: Finished task 37.0 in stage 6.0 (TID 46). 2676 bytes result sent to driver16/02/06 15:45:22 INFO scheduler.TaskSetManager: Starting task 38.0 in stage 6.0 (TID 47, localhost, partition 38,NODE_LOCAL, 1999 bytes)16/02/06 15:45:22 INFO executor.Executor: Running task 38.0 in stage 6.0 (TID 47)16/02/06 15:45:22 INFO scheduler.TaskSetManager: Finished task 37.0 in stage 6.0 (TID 46) in 25 ms on localhost (37/200)16/02/06 15:45:22 INFO executor.Executor: Finished task 36.0 in stage 6.0 (TID 45). 2676 bytes result sent to driver16/02/06 15:45:22 INFO scheduler.TaskSetManager: Starting task 39.0 in stage 6.0 (TID 48, localhost, partition 39,NODE_LOCAL, 1999 bytes)16/02/06 15:45:22 INFO executor.Executor: Running task 39.0 in stage 6.0 (TID 48)16/02/06 15:45:22 INFO scheduler.TaskSetManager: Finished task 36.0 in stage 6.0 (TID 45) in 61 ms on localhost (38/200)16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 16 ms16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 19 ms16/02/06 15:45:22 INFO executor.Executor: Finished task 38.0 in stage 6.0 (TID 47). 2676 bytes result sent to driver16/02/06 15:45:22 INFO scheduler.TaskSetManager: Starting task 40.0 in stage 6.0 (TID 49, localhost, partition 40,NODE_LOCAL, 1999 bytes)16/02/06 15:45:22 INFO executor.Executor: Running task 40.0 in stage 6.0 (TID 49)16/02/06 15:45:22 INFO scheduler.TaskSetManager: Finished task 38.0 in stage 6.0 (TID 47) in 57 ms on localhost (39/200)16/02/06 15:45:22 INFO executor.Executor: Finished task 39.0 in stage 6.0 (TID 48). 2676 bytes result sent to driver16/02/06 15:45:22 INFO scheduler.TaskSetManager: Starting task 41.0 in stage 6.0 (TID 50, localhost, partition 41,NODE_LOCAL, 1999 bytes)16/02/06 15:45:22 INFO executor.Executor: Running task 41.0 in stage 6.0 (TID 50)16/02/06 15:45:22 INFO scheduler.TaskSetManager: Finished task 39.0 in stage 6.0 (TID 48) in 75 ms on localhost (40/200)16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 4 ms16/02/06 15:45:22 INFO executor.Executor: Finished task 40.0 in stage 6.0 (TID 49). 2676 bytes result sent to driver16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:22 INFO scheduler.TaskSetManager: Starting task 42.0 in stage 6.0 (TID 51, localhost, partition 42,NODE_LOCAL, 1999 bytes)16/02/06 15:45:22 INFO scheduler.TaskSetManager: Finished task 40.0 in stage 6.0 (TID 49) in 67 ms on localhost (41/200)16/02/06 15:45:22 INFO executor.Executor: Running task 42.0 in stage 6.0 (TID 51)16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 15 ms16/02/06 15:45:22 INFO executor.Executor: Finished task 41.0 in stage 6.0 (TID 50). 2676 bytes result sent to driver16/02/06 15:45:22 INFO scheduler.TaskSetManager: Starting task 43.0 in stage 6.0 (TID 52, localhost, partition 43,NODE_LOCAL, 1999 bytes)16/02/06 15:45:22 INFO executor.Executor: Running task 43.0 in stage 6.0 (TID 52)16/02/06 15:45:22 INFO scheduler.TaskSetManager: Finished task 41.0 in stage 6.0 (TID 50) in 77 ms on localhost (42/200)16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 7 ms16/02/06 15:45:22 INFO executor.Executor: Finished task 42.0 in stage 6.0 (TID 51). 2676 bytes result sent to driver16/02/06 15:45:22 INFO executor.Executor: Finished task 43.0 in stage 6.0 (TID 52). 2676 bytes result sent to driver16/02/06 15:45:22 INFO scheduler.TaskSetManager: Starting task 44.0 in stage 6.0 (TID 53, localhost, partition 44,NODE_LOCAL, 1999 bytes)16/02/06 15:45:22 INFO executor.Executor: Running task 44.0 in stage 6.0 (TID 53)16/02/06 15:45:22 INFO scheduler.TaskSetManager: Finished task 42.0 in stage 6.0 (TID 51) in 108 ms on localhost (43/200)16/02/06 15:45:22 INFO scheduler.TaskSetManager: Starting task 45.0 in stage 6.0 (TID 54, localhost, partition 45,NODE_LOCAL, 1999 bytes)16/02/06 15:45:22 INFO executor.Executor: Running task 45.0 in stage 6.0 (TID 54)16/02/06 15:45:22 INFO scheduler.TaskSetManager: Finished task 43.0 in stage 6.0 (TID 52) in 114 ms on localhost (44/200)16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms16/02/06 15:45:22 INFO executor.Executor: Finished task 44.0 in stage 6.0 (TID 53). 2676 bytes result sent to driver16/02/06 15:45:22 INFO scheduler.TaskSetManager: Starting task 46.0 in stage 6.0 (TID 55, localhost, partition 46,NODE_LOCAL, 1999 bytes)16/02/06 15:45:22 INFO executor.Executor: Running task 46.0 in stage 6.0 (TID 55)16/02/06 15:45:22 INFO scheduler.TaskSetManager: Finished task 44.0 in stage 6.0 (TID 53) in 90 ms on localhost (45/200)16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:22 INFO executor.Executor: Finished task 45.0 in stage 6.0 (TID 54). 2676 bytes result sent to driver16/02/06 15:45:22 INFO scheduler.TaskSetManager: Starting task 47.0 in stage 6.0 (TID 56, localhost, partition 47,NODE_LOCAL, 1999 bytes)16/02/06 15:45:22 INFO executor.Executor: Running task 47.0 in stage 6.0 (TID 56)16/02/06 15:45:22 INFO scheduler.TaskSetManager: Finished task 45.0 in stage 6.0 (TID 54) in 80 ms on localhost (46/200)16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:22 INFO executor.Executor: Finished task 46.0 in stage 6.0 (TID 55). 2676 bytes result sent to driver16/02/06 15:45:22 INFO scheduler.TaskSetManager: Starting task 48.0 in stage 6.0 (TID 57, localhost, partition 48,NODE_LOCAL, 1999 bytes)16/02/06 15:45:22 INFO executor.Executor: Running task 48.0 in stage 6.0 (TID 57)16/02/06 15:45:22 INFO scheduler.TaskSetManager: Finished task 46.0 in stage 6.0 (TID 55) in 67 ms on localhost (47/200)16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 7 ms16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:22 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms16/02/06 15:45:22 INFO executor.Executor: Finished task 47.0 in stage 6.0 (TID 56). 2676 bytes result sent to driver16/02/06 15:45:22 INFO scheduler.TaskSetManager: Starting task 49.0 in stage 6.0 (TID 58, localhost, partition 49,NODE_LOCAL, 1999 bytes)16/02/06 15:45:22 INFO scheduler.TaskSetManager: Finished task 47.0 in stage 6.0 (TID 56) in 63 ms on localhost (48/200)16/02/06 15:45:22 INFO executor.Executor: Running task 49.0 in stage 6.0 (TID 58)16/02/06 15:45:23 INFO executor.Executor: Finished task 48.0 in stage 6.0 (TID 57). 2676 bytes result sent to driver16/02/06 15:45:23 INFO scheduler.TaskSetManager: Starting task 50.0 in stage 6.0 (TID 59, localhost, partition 50,NODE_LOCAL, 1999 bytes)16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:23 INFO executor.Executor: Running task 50.0 in stage 6.0 (TID 59)16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:23 INFO executor.Executor: Finished task 49.0 in stage 6.0 (TID 58). 2676 bytes result sent to driver16/02/06 15:45:23 INFO scheduler.TaskSetManager: Finished task 48.0 in stage 6.0 (TID 57) in 61 ms on localhost (49/200)16/02/06 15:45:23 INFO scheduler.TaskSetManager: Starting task 51.0 in stage 6.0 (TID 60, localhost, partition 51,NODE_LOCAL, 1999 bytes)16/02/06 15:45:23 INFO executor.Executor: Running task 51.0 in stage 6.0 (TID 60)16/02/06 15:45:23 INFO scheduler.TaskSetManager: Finished task 49.0 in stage 6.0 (TID 58) in 55 ms on localhost (50/200)16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:23 INFO executor.Executor: Finished task 50.0 in stage 6.0 (TID 59). 2676 bytes result sent to driver16/02/06 15:45:23 INFO scheduler.TaskSetManager: Starting task 52.0 in stage 6.0 (TID 61, localhost, partition 52,NODE_LOCAL, 1999 bytes)16/02/06 15:45:23 INFO scheduler.TaskSetManager: Finished task 50.0 in stage 6.0 (TID 59) in 78 ms on localhost (51/200)16/02/06 15:45:23 INFO executor.Executor: Running task 52.0 in stage 6.0 (TID 61)16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms16/02/06 15:45:23 INFO executor.Executor: Finished task 52.0 in stage 6.0 (TID 61). 2676 bytes result sent to driver16/02/06 15:45:23 INFO scheduler.TaskSetManager: Starting task 53.0 in stage 6.0 (TID 62, localhost, partition 53,NODE_LOCAL, 1999 bytes)16/02/06 15:45:23 INFO executor.Executor: Running task 53.0 in stage 6.0 (TID 62)16/02/06 15:45:23 INFO scheduler.TaskSetManager: Finished task 52.0 in stage 6.0 (TID 61) in 63 ms on localhost (52/200)16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms16/02/06 15:45:23 INFO executor.Executor: Finished task 53.0 in stage 6.0 (TID 62). 2676 bytes result sent to driver16/02/06 15:45:23 INFO codegen.GenerateMutableProjection: Code generated in 90.906767 ms16/02/06 15:45:23 INFO scheduler.TaskSetManager: Starting task 54.0 in stage 6.0 (TID 63, localhost, partition 54,NODE_LOCAL, 1999 bytes)16/02/06 15:45:23 INFO scheduler.TaskSetManager: Finished task 53.0 in stage 6.0 (TID 62) in 70 ms on localhost (53/200)16/02/06 15:45:23 INFO executor.Executor: Running task 54.0 in stage 6.0 (TID 63)16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 10 ms16/02/06 15:45:23 INFO executor.Executor: Finished task 54.0 in stage 6.0 (TID 63). 2676 bytes result sent to driver16/02/06 15:45:23 INFO scheduler.TaskSetManager: Starting task 55.0 in stage 6.0 (TID 64, localhost, partition 55,NODE_LOCAL, 1999 bytes)16/02/06 15:45:23 INFO executor.Executor: Running task 55.0 in stage 6.0 (TID 64)16/02/06 15:45:23 INFO scheduler.TaskSetManager: Finished task 54.0 in stage 6.0 (TID 63) in 87 ms on localhost (54/200)16/02/06 15:45:23 INFO codegen.GenerateUnsafeProjection: Code generated in 67.988504 ms16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms16/02/06 15:45:23 INFO executor.Executor: Finished task 51.0 in stage 6.0 (TID 60). 2987 bytes result sent to driver16/02/06 15:45:23 INFO scheduler.TaskSetManager: Starting task 56.0 in stage 6.0 (TID 65, localhost, partition 56,NODE_LOCAL, 1999 bytes)16/02/06 15:45:23 INFO executor.Executor: Running task 56.0 in stage 6.0 (TID 65)16/02/06 15:45:23 INFO scheduler.TaskSetManager: Finished task 51.0 in stage 6.0 (TID 60) in 345 ms on localhost (55/200)16/02/06 15:45:23 INFO executor.Executor: Finished task 55.0 in stage 6.0 (TID 64). 2983 bytes result sent to driver16/02/06 15:45:23 INFO scheduler.TaskSetManager: Starting task 57.0 in stage 6.0 (TID 66, localhost, partition 57,NODE_LOCAL, 1999 bytes)16/02/06 15:45:23 INFO scheduler.TaskSetManager: Finished task 55.0 in stage 6.0 (TID 64) in 120 ms on localhost (56/200)16/02/06 15:45:23 INFO executor.Executor: Running task 57.0 in stage 6.0 (TID 66)16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 8 ms16/02/06 15:45:23 INFO executor.Executor: Finished task 56.0 in stage 6.0 (TID 65). 2676 bytes result sent to driver16/02/06 15:45:23 INFO scheduler.TaskSetManager: Starting task 58.0 in stage 6.0 (TID 67, localhost, partition 58,NODE_LOCAL, 1999 bytes)16/02/06 15:45:23 INFO executor.Executor: Running task 58.0 in stage 6.0 (TID 67)16/02/06 15:45:23 INFO scheduler.TaskSetManager: Finished task 56.0 in stage 6.0 (TID 65) in 71 ms on localhost (57/200)16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 4 ms16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms16/02/06 15:45:23 INFO executor.Executor: Finished task 57.0 in stage 6.0 (TID 66). 2676 bytes result sent to driver16/02/06 15:45:23 INFO scheduler.TaskSetManager: Starting task 59.0 in stage 6.0 (TID 68, localhost, partition 59,NODE_LOCAL, 1999 bytes)16/02/06 15:45:23 INFO executor.Executor: Running task 59.0 in stage 6.0 (TID 68)16/02/06 15:45:23 INFO scheduler.TaskSetManager: Finished task 57.0 in stage 6.0 (TID 66) in 93 ms on localhost (58/200)16/02/06 15:45:23 INFO executor.Executor: Finished task 58.0 in stage 6.0 (TID 67). 2676 bytes result sent to driver16/02/06 15:45:23 INFO scheduler.TaskSetManager: Starting task 60.0 in stage 6.0 (TID 69, localhost, partition 60,NODE_LOCAL, 1999 bytes)16/02/06 15:45:23 INFO executor.Executor: Running task 60.0 in stage 6.0 (TID 69)16/02/06 15:45:23 INFO scheduler.TaskSetManager: Finished task 58.0 in stage 6.0 (TID 67) in 89 ms on localhost (59/200)16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:23 INFO executor.Executor: Finished task 59.0 in stage 6.0 (TID 68). 2676 bytes result sent to driver16/02/06 15:45:23 INFO scheduler.TaskSetManager: Starting task 61.0 in stage 6.0 (TID 70, localhost, partition 61,NODE_LOCAL, 1999 bytes)16/02/06 15:45:23 INFO executor.Executor: Running task 61.0 in stage 6.0 (TID 70)16/02/06 15:45:23 INFO scheduler.TaskSetManager: Finished task 59.0 in stage 6.0 (TID 68) in 73 ms on localhost (60/200)16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 5 ms16/02/06 15:45:23 INFO executor.Executor: Finished task 61.0 in stage 6.0 (TID 70). 2676 bytes result sent to driver16/02/06 15:45:23 INFO scheduler.TaskSetManager: Starting task 62.0 in stage 6.0 (TID 71, localhost, partition 62,NODE_LOCAL, 1999 bytes)16/02/06 15:45:23 INFO executor.Executor: Running task 62.0 in stage 6.0 (TID 71)16/02/06 15:45:23 INFO scheduler.TaskSetManager: Finished task 61.0 in stage 6.0 (TID 70) in 48 ms on localhost (61/200)16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:23 INFO executor.Executor: Finished task 60.0 in stage 6.0 (TID 69). 2676 bytes result sent to driver16/02/06 15:45:23 INFO scheduler.TaskSetManager: Starting task 63.0 in stage 6.0 (TID 72, localhost, partition 63,NODE_LOCAL, 1999 bytes)16/02/06 15:45:23 INFO scheduler.TaskSetManager: Finished task 60.0 in stage 6.0 (TID 69) in 109 ms on localhost (62/200)16/02/06 15:45:23 INFO executor.Executor: Running task 63.0 in stage 6.0 (TID 72)16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 8 ms16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 5 ms16/02/06 15:45:23 INFO executor.Executor: Finished task 62.0 in stage 6.0 (TID 71). 2676 bytes result sent to driver16/02/06 15:45:23 INFO scheduler.TaskSetManager: Starting task 64.0 in stage 6.0 (TID 73, localhost, partition 64,NODE_LOCAL, 1999 bytes)16/02/06 15:45:23 INFO executor.Executor: Running task 64.0 in stage 6.0 (TID 73)16/02/06 15:45:23 INFO scheduler.TaskSetManager: Finished task 62.0 in stage 6.0 (TID 71) in 68 ms on localhost (63/200)16/02/06 15:45:23 INFO executor.Executor: Finished task 63.0 in stage 6.0 (TID 72). 2676 bytes result sent to driver16/02/06 15:45:23 INFO scheduler.TaskSetManager: Starting task 65.0 in stage 6.0 (TID 74, localhost, partition 65,NODE_LOCAL, 1999 bytes)16/02/06 15:45:23 INFO executor.Executor: Running task 65.0 in stage 6.0 (TID 74)16/02/06 15:45:23 INFO scheduler.TaskSetManager: Finished task 63.0 in stage 6.0 (TID 72) in 79 ms on localhost (64/200)16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 4 ms16/02/06 15:45:23 INFO executor.Executor: Finished task 64.0 in stage 6.0 (TID 73). 2676 bytes result sent to driver16/02/06 15:45:23 INFO scheduler.TaskSetManager: Starting task 66.0 in stage 6.0 (TID 75, localhost, partition 66,NODE_LOCAL, 1999 bytes)16/02/06 15:45:23 INFO executor.Executor: Running task 66.0 in stage 6.0 (TID 75)16/02/06 15:45:23 INFO scheduler.TaskSetManager: Finished task 64.0 in stage 6.0 (TID 73) in 61 ms on localhost (65/200)16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms16/02/06 15:45:23 INFO executor.Executor: Finished task 65.0 in stage 6.0 (TID 74). 2676 bytes result sent to driver16/02/06 15:45:23 INFO scheduler.TaskSetManager: Starting task 67.0 in stage 6.0 (TID 76, localhost, partition 67,NODE_LOCAL, 1999 bytes)16/02/06 15:45:23 INFO executor.Executor: Running task 67.0 in stage 6.0 (TID 76)16/02/06 15:45:23 INFO executor.Executor: Finished task 66.0 in stage 6.0 (TID 75). 2676 bytes result sent to driver16/02/06 15:45:23 INFO scheduler.TaskSetManager: Finished task 65.0 in stage 6.0 (TID 74) in 79 ms on localhost (66/200)16/02/06 15:45:23 INFO scheduler.TaskSetManager: Starting task 68.0 in stage 6.0 (TID 77, localhost, partition 68,NODE_LOCAL, 1999 bytes)16/02/06 15:45:23 INFO scheduler.TaskSetManager: Finished task 66.0 in stage 6.0 (TID 75) in 69 ms on localhost (67/200)16/02/06 15:45:23 INFO executor.Executor: Running task 68.0 in stage 6.0 (TID 77)16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms16/02/06 15:45:23 INFO executor.Executor: Finished task 67.0 in stage 6.0 (TID 76). 2676 bytes result sent to driver16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 4 ms16/02/06 15:45:23 INFO executor.Executor: Finished task 68.0 in stage 6.0 (TID 77). 2676 bytes result sent to driver16/02/06 15:45:23 INFO scheduler.TaskSetManager: Starting task 69.0 in stage 6.0 (TID 78, localhost, partition 69,NODE_LOCAL, 1999 bytes)16/02/06 15:45:23 INFO scheduler.TaskSetManager: Starting task 70.0 in stage 6.0 (TID 79, localhost, partition 70,NODE_LOCAL, 1999 bytes)16/02/06 15:45:23 INFO executor.Executor: Running task 70.0 in stage 6.0 (TID 79)16/02/06 15:45:23 INFO scheduler.TaskSetManager: Finished task 68.0 in stage 6.0 (TID 77) in 44 ms on localhost (68/200)16/02/06 15:45:23 INFO scheduler.TaskSetManager: Finished task 67.0 in stage 6.0 (TID 76) in 73 ms on localhost (69/200)16/02/06 15:45:23 INFO executor.Executor: Running task 69.0 in stage 6.0 (TID 78)16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 4 ms16/02/06 15:45:23 INFO executor.Executor: Finished task 70.0 in stage 6.0 (TID 79). 2676 bytes result sent to driver16/02/06 15:45:23 INFO scheduler.TaskSetManager: Starting task 71.0 in stage 6.0 (TID 80, localhost, partition 71,NODE_LOCAL, 1999 bytes)16/02/06 15:45:23 INFO executor.Executor: Running task 71.0 in stage 6.0 (TID 80)16/02/06 15:45:23 INFO scheduler.TaskSetManager: Finished task 70.0 in stage 6.0 (TID 79) in 45 ms on localhost (70/200)16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms16/02/06 15:45:23 INFO executor.Executor: Finished task 71.0 in stage 6.0 (TID 80). 2676 bytes result sent to driver16/02/06 15:45:23 INFO scheduler.TaskSetManager: Starting task 72.0 in stage 6.0 (TID 81, localhost, partition 72,NODE_LOCAL, 1999 bytes)16/02/06 15:45:23 INFO executor.Executor: Running task 72.0 in stage 6.0 (TID 81)16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 9 ms16/02/06 15:45:23 INFO executor.Executor: Finished task 69.0 in stage 6.0 (TID 78). 2676 bytes result sent to driver16/02/06 15:45:23 INFO scheduler.TaskSetManager: Finished task 71.0 in stage 6.0 (TID 80) in 52 ms on localhost (71/200)16/02/06 15:45:23 INFO scheduler.TaskSetManager: Starting task 73.0 in stage 6.0 (TID 82, localhost, partition 73,NODE_LOCAL, 1999 bytes)16/02/06 15:45:23 INFO executor.Executor: Running task 73.0 in stage 6.0 (TID 82)16/02/06 15:45:23 INFO scheduler.TaskSetManager: Finished task 69.0 in stage 6.0 (TID 78) in 110 ms on localhost (72/200)16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 5 ms16/02/06 15:45:23 INFO executor.Executor: Finished task 72.0 in stage 6.0 (TID 81). 2676 bytes result sent to driver16/02/06 15:45:23 INFO scheduler.TaskSetManager: Starting task 74.0 in stage 6.0 (TID 83, localhost, partition 74,NODE_LOCAL, 1999 bytes)16/02/06 15:45:23 INFO executor.Executor: Running task 74.0 in stage 6.0 (TID 83)16/02/06 15:45:23 INFO scheduler.TaskSetManager: Finished task 72.0 in stage 6.0 (TID 81) in 57 ms on localhost (73/200)16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 6 ms16/02/06 15:45:23 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 11 ms16/02/06 15:45:23 INFO executor.Executor: Finished task 74.0 in stage 6.0 (TID 83). 2676 bytes result sent to driver16/02/06 15:45:23 INFO executor.Executor: Finished task 73.0 in stage 6.0 (TID 82). 2676 bytes result sent to driver16/02/06 15:45:23 INFO scheduler.TaskSetManager: Starting task 75.0 in stage 6.0 (TID 84, localhost, partition 75,NODE_LOCAL, 1999 bytes)16/02/06 15:45:23 INFO executor.Executor: Running task 75.0 in stage 6.0 (TID 84)16/02/06 15:45:24 INFO scheduler.TaskSetManager: Finished task 74.0 in stage 6.0 (TID 83) in 51 ms on localhost (74/200)16/02/06 15:45:24 INFO scheduler.TaskSetManager: Starting task 76.0 in stage 6.0 (TID 85, localhost, partition 76,NODE_LOCAL, 1999 bytes)16/02/06 15:45:24 INFO scheduler.TaskSetManager: Finished task 73.0 in stage 6.0 (TID 82) in 99 ms on localhost (75/200)16/02/06 15:45:24 INFO executor.Executor: Running task 76.0 in stage 6.0 (TID 85)16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:24 INFO executor.Executor: Finished task 75.0 in stage 6.0 (TID 84). 2676 bytes result sent to driver16/02/06 15:45:24 INFO scheduler.TaskSetManager: Starting task 77.0 in stage 6.0 (TID 86, localhost, partition 77,NODE_LOCAL, 1999 bytes)16/02/06 15:45:24 INFO executor.Executor: Running task 77.0 in stage 6.0 (TID 86)16/02/06 15:45:24 INFO scheduler.TaskSetManager: Finished task 75.0 in stage 6.0 (TID 84) in 75 ms on localhost (76/200)16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:24 INFO executor.Executor: Finished task 77.0 in stage 6.0 (TID 86). 2676 bytes result sent to driver16/02/06 15:45:24 INFO scheduler.TaskSetManager: Starting task 78.0 in stage 6.0 (TID 87, localhost, partition 78,NODE_LOCAL, 1999 bytes)16/02/06 15:45:24 INFO executor.Executor: Running task 78.0 in stage 6.0 (TID 87)16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:24 INFO scheduler.TaskSetManager: Finished task 77.0 in stage 6.0 (TID 86) in 42 ms on localhost (77/200)16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 14 ms16/02/06 15:45:24 INFO executor.Executor: Finished task 76.0 in stage 6.0 (TID 85). 2676 bytes result sent to driver16/02/06 15:45:24 INFO scheduler.TaskSetManager: Starting task 79.0 in stage 6.0 (TID 88, localhost, partition 79,NODE_LOCAL, 1999 bytes)16/02/06 15:45:24 INFO executor.Executor: Running task 79.0 in stage 6.0 (TID 88)16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:24 INFO scheduler.TaskSetManager: Finished task 76.0 in stage 6.0 (TID 85) in 130 ms on localhost (78/200)16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:24 INFO executor.Executor: Finished task 78.0 in stage 6.0 (TID 87). 2676 bytes result sent to driver16/02/06 15:45:24 INFO executor.Executor: Finished task 79.0 in stage 6.0 (TID 88). 2676 bytes result sent to driver16/02/06 15:45:24 INFO scheduler.TaskSetManager: Starting task 80.0 in stage 6.0 (TID 89, localhost, partition 80,NODE_LOCAL, 1999 bytes)16/02/06 15:45:24 INFO executor.Executor: Running task 80.0 in stage 6.0 (TID 89)16/02/06 15:45:24 INFO scheduler.TaskSetManager: Starting task 81.0 in stage 6.0 (TID 90, localhost, partition 81,NODE_LOCAL, 1999 bytes)16/02/06 15:45:24 INFO executor.Executor: Running task 81.0 in stage 6.0 (TID 90)16/02/06 15:45:24 INFO scheduler.TaskSetManager: Finished task 79.0 in stage 6.0 (TID 88) in 50 ms on localhost (79/200)16/02/06 15:45:24 INFO scheduler.TaskSetManager: Finished task 78.0 in stage 6.0 (TID 87) in 86 ms on localhost (80/200)16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 13 ms16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 7 ms16/02/06 15:45:24 INFO executor.Executor: Finished task 81.0 in stage 6.0 (TID 90). 2676 bytes result sent to driver16/02/06 15:45:24 INFO executor.Executor: Finished task 80.0 in stage 6.0 (TID 89). 2676 bytes result sent to driver16/02/06 15:45:24 INFO scheduler.TaskSetManager: Starting task 82.0 in stage 6.0 (TID 91, localhost, partition 82,NODE_LOCAL, 1999 bytes)16/02/06 15:45:24 INFO executor.Executor: Running task 82.0 in stage 6.0 (TID 91)16/02/06 15:45:24 INFO scheduler.TaskSetManager: Starting task 83.0 in stage 6.0 (TID 92, localhost, partition 83,NODE_LOCAL, 1999 bytes)16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 19 ms16/02/06 15:45:24 INFO executor.Executor: Running task 83.0 in stage 6.0 (TID 92)16/02/06 15:45:24 INFO scheduler.TaskSetManager: Finished task 81.0 in stage 6.0 (TID 90) in 78 ms on localhost (81/200)16/02/06 15:45:24 INFO scheduler.TaskSetManager: Finished task 80.0 in stage 6.0 (TID 89) in 99 ms on localhost (82/200)16/02/06 15:45:24 INFO executor.Executor: Finished task 82.0 in stage 6.0 (TID 91). 2676 bytes result sent to driver16/02/06 15:45:24 INFO scheduler.TaskSetManager: Starting task 84.0 in stage 6.0 (TID 93, localhost, partition 84,NODE_LOCAL, 1999 bytes)16/02/06 15:45:24 INFO executor.Executor: Running task 84.0 in stage 6.0 (TID 93)16/02/06 15:45:24 INFO scheduler.TaskSetManager: Finished task 82.0 in stage 6.0 (TID 91) in 65 ms on localhost (83/200)16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:24 INFO executor.Executor: Finished task 84.0 in stage 6.0 (TID 93). 2676 bytes result sent to driver16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:24 INFO scheduler.TaskSetManager: Starting task 85.0 in stage 6.0 (TID 94, localhost, partition 85,NODE_LOCAL, 1999 bytes)16/02/06 15:45:24 INFO scheduler.TaskSetManager: Finished task 84.0 in stage 6.0 (TID 93) in 61 ms on localhost (84/200)16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 25 ms16/02/06 15:45:24 INFO executor.Executor: Running task 85.0 in stage 6.0 (TID 94)16/02/06 15:45:24 INFO executor.Executor: Finished task 83.0 in stage 6.0 (TID 92). 2676 bytes result sent to driver16/02/06 15:45:24 INFO scheduler.TaskSetManager: Starting task 86.0 in stage 6.0 (TID 95, localhost, partition 86,NODE_LOCAL, 1999 bytes)16/02/06 15:45:24 INFO executor.Executor: Running task 86.0 in stage 6.0 (TID 95)16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:24 INFO scheduler.TaskSetManager: Finished task 83.0 in stage 6.0 (TID 92) in 136 ms on localhost (85/200)16/02/06 15:45:24 INFO executor.Executor: Finished task 85.0 in stage 6.0 (TID 94). 2676 bytes result sent to driver16/02/06 15:45:24 INFO scheduler.TaskSetManager: Starting task 87.0 in stage 6.0 (TID 96, localhost, partition 87,NODE_LOCAL, 1999 bytes)16/02/06 15:45:24 INFO executor.Executor: Running task 87.0 in stage 6.0 (TID 96)16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:24 INFO scheduler.TaskSetManager: Finished task 85.0 in stage 6.0 (TID 94) in 62 ms on localhost (86/200)16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 12 ms16/02/06 15:45:24 INFO executor.Executor: Finished task 86.0 in stage 6.0 (TID 95). 2676 bytes result sent to driver16/02/06 15:45:24 INFO scheduler.TaskSetManager: Starting task 88.0 in stage 6.0 (TID 97, localhost, partition 88,NODE_LOCAL, 1999 bytes)16/02/06 15:45:24 INFO executor.Executor: Running task 88.0 in stage 6.0 (TID 97)16/02/06 15:45:24 INFO scheduler.TaskSetManager: Finished task 86.0 in stage 6.0 (TID 95) in 65 ms on localhost (87/200)16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms16/02/06 15:45:24 INFO executor.Executor: Finished task 88.0 in stage 6.0 (TID 97). 2676 bytes result sent to driver16/02/06 15:45:24 INFO scheduler.TaskSetManager: Starting task 89.0 in stage 6.0 (TID 98, localhost, partition 89,NODE_LOCAL, 1999 bytes)16/02/06 15:45:24 INFO executor.Executor: Running task 89.0 in stage 6.0 (TID 98)16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:24 INFO scheduler.TaskSetManager: Finished task 88.0 in stage 6.0 (TID 97) in 46 ms on localhost (88/200)16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 6 ms16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms16/02/06 15:45:24 INFO executor.Executor: Finished task 87.0 in stage 6.0 (TID 96). 2676 bytes result sent to driver16/02/06 15:45:24 INFO scheduler.TaskSetManager: Starting task 90.0 in stage 6.0 (TID 99, localhost, partition 90,NODE_LOCAL, 1999 bytes)16/02/06 15:45:24 INFO executor.Executor: Running task 90.0 in stage 6.0 (TID 99)16/02/06 15:45:24 INFO scheduler.TaskSetManager: Finished task 87.0 in stage 6.0 (TID 96) in 118 ms on localhost (89/200)16/02/06 15:45:24 INFO executor.Executor: Finished task 89.0 in stage 6.0 (TID 98). 2676 bytes result sent to driver16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 8 ms16/02/06 15:45:24 INFO scheduler.TaskSetManager: Starting task 91.0 in stage 6.0 (TID 100, localhost, partition 91,NODE_LOCAL, 1999 bytes)16/02/06 15:45:24 INFO executor.Executor: Running task 91.0 in stage 6.0 (TID 100)16/02/06 15:45:24 INFO scheduler.TaskSetManager: Finished task 89.0 in stage 6.0 (TID 98) in 123 ms on localhost (90/200)16/02/06 15:45:24 INFO executor.Executor: Finished task 90.0 in stage 6.0 (TID 99). 2676 bytes result sent to driver16/02/06 15:45:24 INFO scheduler.TaskSetManager: Starting task 92.0 in stage 6.0 (TID 101, localhost, partition 92,NODE_LOCAL, 1999 bytes)16/02/06 15:45:24 INFO executor.Executor: Running task 92.0 in stage 6.0 (TID 101)16/02/06 15:45:24 INFO scheduler.TaskSetManager: Finished task 90.0 in stage 6.0 (TID 99) in 112 ms on localhost (91/200)16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 6 ms16/02/06 15:45:24 INFO executor.Executor: Finished task 92.0 in stage 6.0 (TID 101). 2676 bytes result sent to driver16/02/06 15:45:24 INFO executor.Executor: Finished task 91.0 in stage 6.0 (TID 100). 2676 bytes result sent to driver16/02/06 15:45:24 INFO scheduler.TaskSetManager: Starting task 93.0 in stage 6.0 (TID 102, localhost, partition 93,NODE_LOCAL, 1999 bytes)16/02/06 15:45:24 INFO scheduler.TaskSetManager: Finished task 92.0 in stage 6.0 (TID 101) in 38 ms on localhost (92/200)16/02/06 15:45:24 INFO executor.Executor: Running task 93.0 in stage 6.0 (TID 102)16/02/06 15:45:24 INFO scheduler.TaskSetManager: Starting task 94.0 in stage 6.0 (TID 103, localhost, partition 94,NODE_LOCAL, 1999 bytes)16/02/06 15:45:24 INFO executor.Executor: Running task 94.0 in stage 6.0 (TID 103)16/02/06 15:45:24 INFO scheduler.TaskSetManager: Finished task 91.0 in stage 6.0 (TID 100) in 89 ms on localhost (93/200)16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 8 ms16/02/06 15:45:24 INFO executor.Executor: Finished task 94.0 in stage 6.0 (TID 103). 2676 bytes result sent to driver16/02/06 15:45:24 INFO scheduler.TaskSetManager: Starting task 95.0 in stage 6.0 (TID 104, localhost, partition 95,NODE_LOCAL, 1999 bytes)16/02/06 15:45:24 INFO executor.Executor: Running task 95.0 in stage 6.0 (TID 104)16/02/06 15:45:24 INFO scheduler.TaskSetManager: Finished task 94.0 in stage 6.0 (TID 103) in 37 ms on localhost (94/200)16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 11 ms16/02/06 15:45:24 INFO executor.Executor: Finished task 93.0 in stage 6.0 (TID 102). 2676 bytes result sent to driver16/02/06 15:45:24 INFO executor.Executor: Finished task 95.0 in stage 6.0 (TID 104). 2676 bytes result sent to driver16/02/06 15:45:24 INFO scheduler.TaskSetManager: Starting task 96.0 in stage 6.0 (TID 105, localhost, partition 96,NODE_LOCAL, 1999 bytes)16/02/06 15:45:24 INFO executor.Executor: Running task 96.0 in stage 6.0 (TID 105)16/02/06 15:45:24 INFO scheduler.TaskSetManager: Starting task 97.0 in stage 6.0 (TID 106, localhost, partition 97,NODE_LOCAL, 1999 bytes)16/02/06 15:45:24 INFO scheduler.TaskSetManager: Finished task 93.0 in stage 6.0 (TID 102) in 94 ms on localhost (95/200)16/02/06 15:45:24 INFO executor.Executor: Running task 97.0 in stage 6.0 (TID 106)16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms16/02/06 15:45:24 INFO scheduler.TaskSetManager: Finished task 95.0 in stage 6.0 (TID 104) in 67 ms on localhost (96/200)16/02/06 15:45:24 INFO executor.Executor: Finished task 96.0 in stage 6.0 (TID 105). 2676 bytes result sent to driver16/02/06 15:45:24 INFO scheduler.TaskSetManager: Starting task 98.0 in stage 6.0 (TID 107, localhost, partition 98,NODE_LOCAL, 1999 bytes)16/02/06 15:45:24 INFO executor.Executor: Running task 98.0 in stage 6.0 (TID 107)16/02/06 15:45:24 INFO scheduler.TaskSetManager: Finished task 96.0 in stage 6.0 (TID 105) in 43 ms on localhost (97/200)16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 4 ms16/02/06 15:45:24 INFO executor.Executor: Finished task 97.0 in stage 6.0 (TID 106). 2676 bytes result sent to driver16/02/06 15:45:24 INFO scheduler.TaskSetManager: Starting task 99.0 in stage 6.0 (TID 108, localhost, partition 99,NODE_LOCAL, 1999 bytes)16/02/06 15:45:24 INFO executor.Executor: Running task 99.0 in stage 6.0 (TID 108)16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 10 ms16/02/06 15:45:24 INFO scheduler.TaskSetManager: Finished task 97.0 in stage 6.0 (TID 106) in 76 ms on localhost (98/200)16/02/06 15:45:24 INFO executor.Executor: Finished task 98.0 in stage 6.0 (TID 107). 2676 bytes result sent to driver16/02/06 15:45:24 INFO scheduler.TaskSetManager: Starting task 100.0 in stage 6.0 (TID 109, localhost, partition 100,NODE_LOCAL, 1999 bytes)16/02/06 15:45:24 INFO scheduler.TaskSetManager: Finished task 98.0 in stage 6.0 (TID 107) in 77 ms on localhost (99/200)16/02/06 15:45:24 INFO executor.Executor: Running task 100.0 in stage 6.0 (TID 109)16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:24 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 11 ms16/02/06 15:45:24 INFO executor.Executor: Finished task 99.0 in stage 6.0 (TID 108). 2676 bytes result sent to driver16/02/06 15:45:24 INFO scheduler.TaskSetManager: Starting task 101.0 in stage 6.0 (TID 110, localhost, partition 101,NODE_LOCAL, 1999 bytes)16/02/06 15:45:24 INFO executor.Executor: Running task 101.0 in stage 6.0 (TID 110)16/02/06 15:45:25 INFO scheduler.TaskSetManager: Finished task 99.0 in stage 6.0 (TID 108) in 227 ms on localhost (100/200)16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 5 ms16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 4 ms16/02/06 15:45:25 INFO executor.Executor: Finished task 100.0 in stage 6.0 (TID 109). 2676 bytes result sent to driver16/02/06 15:45:25 INFO executor.Executor: Finished task 101.0 in stage 6.0 (TID 110). 2676 bytes result sent to driver16/02/06 15:45:25 INFO scheduler.TaskSetManager: Starting task 102.0 in stage 6.0 (TID 111, localhost, partition 102,NODE_LOCAL, 1999 bytes)16/02/06 15:45:25 INFO executor.Executor: Running task 102.0 in stage 6.0 (TID 111)16/02/06 15:45:25 INFO scheduler.TaskSetManager: Finished task 100.0 in stage 6.0 (TID 109) in 214 ms on localhost (101/200)16/02/06 15:45:25 INFO scheduler.TaskSetManager: Starting task 103.0 in stage 6.0 (TID 112, localhost, partition 103,NODE_LOCAL, 1999 bytes)16/02/06 15:45:25 INFO executor.Executor: Running task 103.0 in stage 6.0 (TID 112)16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 10 ms16/02/06 15:45:25 INFO executor.Executor: Finished task 102.0 in stage 6.0 (TID 111). 2676 bytes result sent to driver16/02/06 15:45:25 INFO scheduler.TaskSetManager: Starting task 104.0 in stage 6.0 (TID 113, localhost, partition 104,NODE_LOCAL, 1999 bytes)16/02/06 15:45:25 INFO executor.Executor: Running task 104.0 in stage 6.0 (TID 113)16/02/06 15:45:25 INFO scheduler.TaskSetManager: Finished task 102.0 in stage 6.0 (TID 111) in 51 ms on localhost (102/200)16/02/06 15:45:25 INFO scheduler.TaskSetManager: Finished task 101.0 in stage 6.0 (TID 110) in 120 ms on localhost (103/200)16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 5 ms16/02/06 15:45:25 INFO executor.Executor: Finished task 103.0 in stage 6.0 (TID 112). 2676 bytes result sent to driver16/02/06 15:45:25 INFO executor.Executor: Finished task 104.0 in stage 6.0 (TID 113). 2676 bytes result sent to driver16/02/06 15:45:25 INFO scheduler.TaskSetManager: Starting task 105.0 in stage 6.0 (TID 114, localhost, partition 105,NODE_LOCAL, 1999 bytes)16/02/06 15:45:25 INFO scheduler.TaskSetManager: Starting task 106.0 in stage 6.0 (TID 115, localhost, partition 106,NODE_LOCAL, 1999 bytes)16/02/06 15:45:25 INFO executor.Executor: Running task 105.0 in stage 6.0 (TID 114)16/02/06 15:45:25 INFO executor.Executor: Running task 106.0 in stage 6.0 (TID 115)16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms16/02/06 15:45:25 INFO scheduler.TaskSetManager: Finished task 103.0 in stage 6.0 (TID 112) in 97 ms on localhost (104/200)16/02/06 15:45:25 INFO scheduler.TaskSetManager: Finished task 104.0 in stage 6.0 (TID 113) in 76 ms on localhost (105/200)16/02/06 15:45:25 INFO executor.Executor: Finished task 105.0 in stage 6.0 (TID 114). 2676 bytes result sent to driver16/02/06 15:45:25 INFO scheduler.TaskSetManager: Starting task 107.0 in stage 6.0 (TID 116, localhost, partition 107,NODE_LOCAL, 1999 bytes)16/02/06 15:45:25 INFO executor.Executor: Running task 107.0 in stage 6.0 (TID 116)16/02/06 15:45:25 INFO scheduler.TaskSetManager: Finished task 105.0 in stage 6.0 (TID 114) in 66 ms on localhost (106/200)16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 12 ms16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 4 ms16/02/06 15:45:25 INFO executor.Executor: Finished task 106.0 in stage 6.0 (TID 115). 2676 bytes result sent to driver16/02/06 15:45:25 INFO scheduler.TaskSetManager: Starting task 108.0 in stage 6.0 (TID 117, localhost, partition 108,NODE_LOCAL, 1999 bytes)16/02/06 15:45:25 INFO executor.Executor: Finished task 107.0 in stage 6.0 (TID 116). 2676 bytes result sent to driver16/02/06 15:45:25 INFO scheduler.TaskSetManager: Finished task 106.0 in stage 6.0 (TID 115) in 102 ms on localhost (107/200)16/02/06 15:45:25 INFO executor.Executor: Running task 108.0 in stage 6.0 (TID 117)16/02/06 15:45:25 INFO scheduler.TaskSetManager: Starting task 109.0 in stage 6.0 (TID 118, localhost, partition 109,NODE_LOCAL, 1999 bytes)16/02/06 15:45:25 INFO executor.Executor: Running task 109.0 in stage 6.0 (TID 118)16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:25 INFO scheduler.TaskSetManager: Finished task 107.0 in stage 6.0 (TID 116) in 77 ms on localhost (108/200)16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 10 ms16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms16/02/06 15:45:25 INFO executor.Executor: Finished task 109.0 in stage 6.0 (TID 118). 2676 bytes result sent to driver16/02/06 15:45:25 INFO scheduler.TaskSetManager: Starting task 110.0 in stage 6.0 (TID 119, localhost, partition 110,NODE_LOCAL, 1999 bytes)16/02/06 15:45:25 INFO executor.Executor: Running task 110.0 in stage 6.0 (TID 119)16/02/06 15:45:25 INFO executor.Executor: Finished task 108.0 in stage 6.0 (TID 117). 2676 bytes result sent to driver16/02/06 15:45:25 INFO scheduler.TaskSetManager: Finished task 109.0 in stage 6.0 (TID 118) in 48 ms on localhost (109/200)16/02/06 15:45:25 INFO scheduler.TaskSetManager: Starting task 111.0 in stage 6.0 (TID 120, localhost, partition 111,NODE_LOCAL, 1999 bytes)16/02/06 15:45:25 INFO executor.Executor: Running task 111.0 in stage 6.0 (TID 120)16/02/06 15:45:25 INFO scheduler.TaskSetManager: Finished task 108.0 in stage 6.0 (TID 117) in 77 ms on localhost (110/200)16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:25 INFO executor.Executor: Finished task 110.0 in stage 6.0 (TID 119). 2676 bytes result sent to driver16/02/06 15:45:25 INFO scheduler.TaskSetManager: Starting task 112.0 in stage 6.0 (TID 121, localhost, partition 112,NODE_LOCAL, 1999 bytes)16/02/06 15:45:25 INFO executor.Executor: Running task 112.0 in stage 6.0 (TID 121)16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 22 ms16/02/06 15:45:25 INFO scheduler.TaskSetManager: Finished task 110.0 in stage 6.0 (TID 119) in 68 ms on localhost (111/200)16/02/06 15:45:25 INFO executor.Executor: Finished task 111.0 in stage 6.0 (TID 120). 2676 bytes result sent to driver16/02/06 15:45:25 INFO scheduler.TaskSetManager: Starting task 113.0 in stage 6.0 (TID 122, localhost, partition 113,NODE_LOCAL, 1999 bytes)16/02/06 15:45:25 INFO executor.Executor: Running task 113.0 in stage 6.0 (TID 122)16/02/06 15:45:25 INFO scheduler.TaskSetManager: Finished task 111.0 in stage 6.0 (TID 120) in 94 ms on localhost (112/200)16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:25 INFO executor.Executor: Finished task 112.0 in stage 6.0 (TID 121). 2676 bytes result sent to driver16/02/06 15:45:25 INFO scheduler.TaskSetManager: Starting task 114.0 in stage 6.0 (TID 123, localhost, partition 114,NODE_LOCAL, 1999 bytes)16/02/06 15:45:25 INFO executor.Executor: Running task 114.0 in stage 6.0 (TID 123)16/02/06 15:45:25 INFO scheduler.TaskSetManager: Finished task 112.0 in stage 6.0 (TID 121) in 65 ms on localhost (113/200)16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms16/02/06 15:45:25 INFO executor.Executor: Finished task 114.0 in stage 6.0 (TID 123). 2676 bytes result sent to driver16/02/06 15:45:25 INFO scheduler.TaskSetManager: Starting task 115.0 in stage 6.0 (TID 124, localhost, partition 115,NODE_LOCAL, 1999 bytes)16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 25 ms16/02/06 15:45:25 INFO executor.Executor: Running task 115.0 in stage 6.0 (TID 124)16/02/06 15:45:25 INFO scheduler.TaskSetManager: Finished task 114.0 in stage 6.0 (TID 123) in 21 ms on localhost (114/200)16/02/06 15:45:25 INFO executor.Executor: Finished task 113.0 in stage 6.0 (TID 122). 2676 bytes result sent to driver16/02/06 15:45:25 INFO scheduler.TaskSetManager: Starting task 116.0 in stage 6.0 (TID 125, localhost, partition 116,NODE_LOCAL, 1999 bytes)16/02/06 15:45:25 INFO scheduler.TaskSetManager: Finished task 113.0 in stage 6.0 (TID 122) in 94 ms on localhost (115/200)16/02/06 15:45:25 INFO executor.Executor: Running task 116.0 in stage 6.0 (TID 125)16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 7 ms16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:25 INFO executor.Executor: Finished task 115.0 in stage 6.0 (TID 124). 2676 bytes result sent to driver16/02/06 15:45:25 INFO scheduler.TaskSetManager: Starting task 117.0 in stage 6.0 (TID 126, localhost, partition 117,NODE_LOCAL, 1999 bytes)16/02/06 15:45:25 INFO executor.Executor: Running task 117.0 in stage 6.0 (TID 126)16/02/06 15:45:25 INFO scheduler.TaskSetManager: Finished task 115.0 in stage 6.0 (TID 124) in 70 ms on localhost (116/200)16/02/06 15:45:25 INFO executor.Executor: Finished task 116.0 in stage 6.0 (TID 125). 2676 bytes result sent to driver16/02/06 15:45:25 INFO scheduler.TaskSetManager: Starting task 118.0 in stage 6.0 (TID 127, localhost, partition 118,NODE_LOCAL, 1999 bytes)16/02/06 15:45:25 INFO scheduler.TaskSetManager: Finished task 116.0 in stage 6.0 (TID 125) in 52 ms on localhost (117/200)16/02/06 15:45:25 INFO executor.Executor: Running task 118.0 in stage 6.0 (TID 127)16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms16/02/06 15:45:25 INFO executor.Executor: Finished task 117.0 in stage 6.0 (TID 126). 2676 bytes result sent to driver16/02/06 15:45:25 INFO scheduler.TaskSetManager: Starting task 119.0 in stage 6.0 (TID 128, localhost, partition 119,NODE_LOCAL, 1999 bytes)16/02/06 15:45:25 INFO executor.Executor: Running task 119.0 in stage 6.0 (TID 128)16/02/06 15:45:25 INFO scheduler.TaskSetManager: Finished task 117.0 in stage 6.0 (TID 126) in 40 ms on localhost (118/200)16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 10 ms16/02/06 15:45:25 INFO executor.Executor: Finished task 119.0 in stage 6.0 (TID 128). 2676 bytes result sent to driver16/02/06 15:45:25 INFO scheduler.TaskSetManager: Starting task 120.0 in stage 6.0 (TID 129, localhost, partition 120,NODE_LOCAL, 1999 bytes)16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 31 ms16/02/06 15:45:25 INFO executor.Executor: Running task 120.0 in stage 6.0 (TID 129)16/02/06 15:45:25 INFO scheduler.TaskSetManager: Finished task 119.0 in stage 6.0 (TID 128) in 66 ms on localhost (119/200)16/02/06 15:45:25 INFO executor.Executor: Finished task 118.0 in stage 6.0 (TID 127). 2676 bytes result sent to driver16/02/06 15:45:25 INFO scheduler.TaskSetManager: Starting task 121.0 in stage 6.0 (TID 130, localhost, partition 121,NODE_LOCAL, 1999 bytes)16/02/06 15:45:25 INFO executor.Executor: Running task 121.0 in stage 6.0 (TID 130)16/02/06 15:45:25 INFO scheduler.TaskSetManager: Finished task 118.0 in stage 6.0 (TID 127) in 99 ms on localhost (120/200)16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:25 INFO executor.Executor: Finished task 121.0 in stage 6.0 (TID 130). 2676 bytes result sent to driver16/02/06 15:45:25 INFO scheduler.TaskSetManager: Starting task 122.0 in stage 6.0 (TID 131, localhost, partition 122,NODE_LOCAL, 1999 bytes)16/02/06 15:45:25 INFO executor.Executor: Running task 122.0 in stage 6.0 (TID 131)16/02/06 15:45:25 INFO scheduler.TaskSetManager: Finished task 121.0 in stage 6.0 (TID 130) in 51 ms on localhost (121/200)16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:25 INFO executor.Executor: Finished task 120.0 in stage 6.0 (TID 129). 2676 bytes result sent to driver16/02/06 15:45:25 INFO scheduler.TaskSetManager: Starting task 123.0 in stage 6.0 (TID 132, localhost, partition 123,NODE_LOCAL, 1999 bytes)16/02/06 15:45:25 INFO executor.Executor: Running task 123.0 in stage 6.0 (TID 132)16/02/06 15:45:25 INFO scheduler.TaskSetManager: Finished task 120.0 in stage 6.0 (TID 129) in 108 ms on localhost (122/200)16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 20 ms16/02/06 15:45:25 INFO executor.Executor: Finished task 122.0 in stage 6.0 (TID 131). 2676 bytes result sent to driver16/02/06 15:45:25 INFO scheduler.TaskSetManager: Starting task 124.0 in stage 6.0 (TID 133, localhost, partition 124,NODE_LOCAL, 1999 bytes)16/02/06 15:45:25 INFO scheduler.TaskSetManager: Finished task 122.0 in stage 6.0 (TID 131) in 59 ms on localhost (123/200)16/02/06 15:45:25 INFO executor.Executor: Running task 124.0 in stage 6.0 (TID 133)16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 18 ms16/02/06 15:45:25 INFO executor.Executor: Finished task 123.0 in stage 6.0 (TID 132). 2676 bytes result sent to driver16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:25 INFO scheduler.TaskSetManager: Starting task 125.0 in stage 6.0 (TID 134, localhost, partition 125,NODE_LOCAL, 1999 bytes)16/02/06 15:45:25 INFO executor.Executor: Running task 125.0 in stage 6.0 (TID 134)16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 13 ms16/02/06 15:45:25 INFO scheduler.TaskSetManager: Finished task 123.0 in stage 6.0 (TID 132) in 99 ms on localhost (124/200)16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:25 INFO executor.Executor: Finished task 124.0 in stage 6.0 (TID 133). 2676 bytes result sent to driver16/02/06 15:45:25 INFO scheduler.TaskSetManager: Starting task 126.0 in stage 6.0 (TID 135, localhost, partition 126,NODE_LOCAL, 1999 bytes)16/02/06 15:45:25 INFO executor.Executor: Running task 126.0 in stage 6.0 (TID 135)16/02/06 15:45:25 INFO scheduler.TaskSetManager: Finished task 124.0 in stage 6.0 (TID 133) in 84 ms on localhost (125/200)16/02/06 15:45:25 INFO executor.Executor: Finished task 125.0 in stage 6.0 (TID 134). 2676 bytes result sent to driver16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:25 INFO scheduler.TaskSetManager: Starting task 127.0 in stage 6.0 (TID 136, localhost, partition 127,NODE_LOCAL, 1999 bytes)16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 6 ms16/02/06 15:45:25 INFO executor.Executor: Running task 127.0 in stage 6.0 (TID 136)16/02/06 15:45:25 INFO scheduler.TaskSetManager: Finished task 125.0 in stage 6.0 (TID 134) in 78 ms on localhost (126/200)16/02/06 15:45:25 INFO executor.Executor: Finished task 126.0 in stage 6.0 (TID 135). 2676 bytes result sent to driver16/02/06 15:45:25 INFO scheduler.TaskSetManager: Starting task 128.0 in stage 6.0 (TID 137, localhost, partition 128,NODE_LOCAL, 1999 bytes)16/02/06 15:45:25 INFO executor.Executor: Running task 128.0 in stage 6.0 (TID 137)16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:25 INFO scheduler.TaskSetManager: Finished task 126.0 in stage 6.0 (TID 135) in 97 ms on localhost (127/200)16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 5 ms16/02/06 15:45:25 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 270 ms16/02/06 15:45:26 INFO executor.Executor: Finished task 127.0 in stage 6.0 (TID 136). 2676 bytes result sent to driver16/02/06 15:45:26 INFO scheduler.TaskSetManager: Starting task 129.0 in stage 6.0 (TID 138, localhost, partition 129,NODE_LOCAL, 1999 bytes)16/02/06 15:45:26 INFO executor.Executor: Finished task 128.0 in stage 6.0 (TID 137). 2676 bytes result sent to driver16/02/06 15:45:26 INFO scheduler.TaskSetManager: Finished task 127.0 in stage 6.0 (TID 136) in 367 ms on localhost (128/200)16/02/06 15:45:26 INFO executor.Executor: Running task 129.0 in stage 6.0 (TID 138)16/02/06 15:45:26 INFO scheduler.TaskSetManager: Starting task 130.0 in stage 6.0 (TID 139, localhost, partition 130,NODE_LOCAL, 1999 bytes)16/02/06 15:45:26 INFO storage.BlockManagerInfo: Removed broadcast_10_piece0 on localhost:45391 in memory (size: 19.5 KB, free: 511.1 MB)16/02/06 15:45:26 INFO scheduler.TaskSetManager: Finished task 128.0 in stage 6.0 (TID 137) in 333 ms on localhost (129/200)16/02/06 15:45:26 INFO executor.Executor: Running task 130.0 in stage 6.0 (TID 139)16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 39 ms16/02/06 15:45:26 INFO storage.BlockManagerInfo: Removed broadcast_12_piece0 on localhost:45391 in memory (size: 5.5 KB, free: 511.1 MB)16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 4 ms16/02/06 15:45:26 INFO executor.Executor: Finished task 129.0 in stage 6.0 (TID 138). 2676 bytes result sent to driver16/02/06 15:45:26 INFO scheduler.TaskSetManager: Starting task 131.0 in stage 6.0 (TID 140, localhost, partition 131,NODE_LOCAL, 1999 bytes)16/02/06 15:45:26 INFO executor.Executor: Running task 131.0 in stage 6.0 (TID 140)16/02/06 15:45:26 INFO scheduler.TaskSetManager: Finished task 129.0 in stage 6.0 (TID 138) in 89 ms on localhost (130/200)16/02/06 15:45:26 INFO executor.Executor: Finished task 130.0 in stage 6.0 (TID 139). 2676 bytes result sent to driver16/02/06 15:45:26 INFO scheduler.TaskSetManager: Starting task 132.0 in stage 6.0 (TID 141, localhost, partition 132,NODE_LOCAL, 1999 bytes)16/02/06 15:45:26 INFO scheduler.TaskSetManager: Finished task 130.0 in stage 6.0 (TID 139) in 98 ms on localhost (131/200)16/02/06 15:45:26 INFO executor.Executor: Running task 132.0 in stage 6.0 (TID 141)16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 9 ms16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 11 ms16/02/06 15:45:26 INFO executor.Executor: Finished task 132.0 in stage 6.0 (TID 141). 2676 bytes result sent to driver16/02/06 15:45:26 INFO scheduler.TaskSetManager: Starting task 133.0 in stage 6.0 (TID 142, localhost, partition 133,NODE_LOCAL, 1999 bytes)16/02/06 15:45:26 INFO executor.Executor: Running task 133.0 in stage 6.0 (TID 142)16/02/06 15:45:26 INFO scheduler.TaskSetManager: Finished task 132.0 in stage 6.0 (TID 141) in 52 ms on localhost (132/200)16/02/06 15:45:26 INFO executor.Executor: Finished task 131.0 in stage 6.0 (TID 140). 2676 bytes result sent to driver16/02/06 15:45:26 INFO scheduler.TaskSetManager: Starting task 134.0 in stage 6.0 (TID 143, localhost, partition 134,NODE_LOCAL, 1999 bytes)16/02/06 15:45:26 INFO scheduler.TaskSetManager: Finished task 131.0 in stage 6.0 (TID 140) in 92 ms on localhost (133/200)16/02/06 15:45:26 INFO executor.Executor: Running task 134.0 in stage 6.0 (TID 143)16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 9 ms16/02/06 15:45:26 INFO executor.Executor: Finished task 133.0 in stage 6.0 (TID 142). 2676 bytes result sent to driver16/02/06 15:45:26 INFO scheduler.TaskSetManager: Starting task 135.0 in stage 6.0 (TID 144, localhost, partition 135,NODE_LOCAL, 1999 bytes)16/02/06 15:45:26 INFO executor.Executor: Running task 135.0 in stage 6.0 (TID 144)16/02/06 15:45:26 INFO scheduler.TaskSetManager: Finished task 133.0 in stage 6.0 (TID 142) in 53 ms on localhost (134/200)16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 31 ms16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms16/02/06 15:45:26 INFO executor.Executor: Finished task 135.0 in stage 6.0 (TID 144). 2676 bytes result sent to driver16/02/06 15:45:26 INFO scheduler.TaskSetManager: Starting task 136.0 in stage 6.0 (TID 145, localhost, partition 136,NODE_LOCAL, 1999 bytes)16/02/06 15:45:26 INFO executor.Executor: Running task 136.0 in stage 6.0 (TID 145)16/02/06 15:45:26 INFO scheduler.TaskSetManager: Finished task 135.0 in stage 6.0 (TID 144) in 37 ms on localhost (135/200)16/02/06 15:45:26 INFO executor.Executor: Finished task 134.0 in stage 6.0 (TID 143). 2676 bytes result sent to driver16/02/06 15:45:26 INFO scheduler.TaskSetManager: Starting task 137.0 in stage 6.0 (TID 146, localhost, partition 137,NODE_LOCAL, 1999 bytes)16/02/06 15:45:26 INFO scheduler.TaskSetManager: Finished task 134.0 in stage 6.0 (TID 143) in 75 ms on localhost (136/200)16/02/06 15:45:26 INFO executor.Executor: Running task 137.0 in stage 6.0 (TID 146)16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 8 ms16/02/06 15:45:26 INFO executor.Executor: Finished task 136.0 in stage 6.0 (TID 145). 2676 bytes result sent to driver16/02/06 15:45:26 INFO scheduler.TaskSetManager: Starting task 138.0 in stage 6.0 (TID 147, localhost, partition 138,NODE_LOCAL, 1999 bytes)16/02/06 15:45:26 INFO scheduler.TaskSetManager: Finished task 136.0 in stage 6.0 (TID 145) in 51 ms on localhost (137/200)16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:26 INFO executor.Executor: Running task 138.0 in stage 6.0 (TID 147)16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 28 ms16/02/06 15:45:26 INFO executor.Executor: Finished task 137.0 in stage 6.0 (TID 146). 2676 bytes result sent to driver16/02/06 15:45:26 INFO scheduler.TaskSetManager: Starting task 139.0 in stage 6.0 (TID 148, localhost, partition 139,NODE_LOCAL, 1999 bytes)16/02/06 15:45:26 INFO executor.Executor: Running task 139.0 in stage 6.0 (TID 148)16/02/06 15:45:26 INFO scheduler.TaskSetManager: Finished task 137.0 in stage 6.0 (TID 146) in 82 ms on localhost (138/200)16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 20 ms16/02/06 15:45:26 INFO executor.Executor: Finished task 138.0 in stage 6.0 (TID 147). 2676 bytes result sent to driver16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:26 INFO executor.Executor: Finished task 139.0 in stage 6.0 (TID 148). 2676 bytes result sent to driver16/02/06 15:45:26 INFO scheduler.TaskSetManager: Starting task 140.0 in stage 6.0 (TID 149, localhost, partition 140,NODE_LOCAL, 1999 bytes)16/02/06 15:45:26 INFO scheduler.TaskSetManager: Finished task 138.0 in stage 6.0 (TID 147) in 65 ms on localhost (139/200)16/02/06 15:45:26 INFO executor.Executor: Running task 140.0 in stage 6.0 (TID 149)16/02/06 15:45:26 INFO scheduler.TaskSetManager: Starting task 141.0 in stage 6.0 (TID 150, localhost, partition 141,NODE_LOCAL, 1999 bytes)16/02/06 15:45:26 INFO executor.Executor: Running task 141.0 in stage 6.0 (TID 150)16/02/06 15:45:26 INFO scheduler.TaskSetManager: Finished task 139.0 in stage 6.0 (TID 148) in 40 ms on localhost (140/200)16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 22 ms16/02/06 15:45:26 INFO executor.Executor: Finished task 141.0 in stage 6.0 (TID 150). 2676 bytes result sent to driver16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 6 ms16/02/06 15:45:26 INFO executor.Executor: Finished task 140.0 in stage 6.0 (TID 149). 2676 bytes result sent to driver16/02/06 15:45:26 INFO scheduler.TaskSetManager: Starting task 142.0 in stage 6.0 (TID 151, localhost, partition 142,NODE_LOCAL, 1999 bytes)16/02/06 15:45:26 INFO scheduler.TaskSetManager: Finished task 141.0 in stage 6.0 (TID 150) in 66 ms on localhost (141/200)16/02/06 15:45:26 INFO executor.Executor: Running task 142.0 in stage 6.0 (TID 151)16/02/06 15:45:26 INFO scheduler.TaskSetManager: Starting task 143.0 in stage 6.0 (TID 152, localhost, partition 143,NODE_LOCAL, 1999 bytes)16/02/06 15:45:26 INFO scheduler.TaskSetManager: Finished task 140.0 in stage 6.0 (TID 149) in 88 ms on localhost (142/200)16/02/06 15:45:26 INFO executor.Executor: Running task 143.0 in stage 6.0 (TID 152)16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 6 ms16/02/06 15:45:26 INFO executor.Executor: Finished task 143.0 in stage 6.0 (TID 152). 2676 bytes result sent to driver16/02/06 15:45:26 INFO scheduler.TaskSetManager: Starting task 144.0 in stage 6.0 (TID 153, localhost, partition 144,NODE_LOCAL, 1999 bytes)16/02/06 15:45:26 INFO scheduler.TaskSetManager: Finished task 143.0 in stage 6.0 (TID 152) in 37 ms on localhost (143/200)16/02/06 15:45:26 INFO executor.Executor: Running task 144.0 in stage 6.0 (TID 153)16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 18 ms16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 37 ms16/02/06 15:45:26 INFO executor.Executor: Finished task 142.0 in stage 6.0 (TID 151). 2676 bytes result sent to driver16/02/06 15:45:26 INFO scheduler.TaskSetManager: Starting task 145.0 in stage 6.0 (TID 154, localhost, partition 145,NODE_LOCAL, 1999 bytes)16/02/06 15:45:26 INFO executor.Executor: Running task 145.0 in stage 6.0 (TID 154)16/02/06 15:45:26 INFO scheduler.TaskSetManager: Finished task 142.0 in stage 6.0 (TID 151) in 93 ms on localhost (144/200)16/02/06 15:45:26 INFO executor.Executor: Finished task 144.0 in stage 6.0 (TID 153). 2676 bytes result sent to driver16/02/06 15:45:26 INFO scheduler.TaskSetManager: Starting task 146.0 in stage 6.0 (TID 155, localhost, partition 146,NODE_LOCAL, 1999 bytes)16/02/06 15:45:26 INFO executor.Executor: Running task 146.0 in stage 6.0 (TID 155)16/02/06 15:45:26 INFO scheduler.TaskSetManager: Finished task 144.0 in stage 6.0 (TID 153) in 79 ms on localhost (145/200)16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms16/02/06 15:45:26 INFO executor.Executor: Finished task 145.0 in stage 6.0 (TID 154). 2676 bytes result sent to driver16/02/06 15:45:26 INFO scheduler.TaskSetManager: Starting task 147.0 in stage 6.0 (TID 156, localhost, partition 147,NODE_LOCAL, 1999 bytes)16/02/06 15:45:26 INFO executor.Executor: Running task 147.0 in stage 6.0 (TID 156)16/02/06 15:45:26 INFO executor.Executor: Finished task 146.0 in stage 6.0 (TID 155). 2676 bytes result sent to driver16/02/06 15:45:26 INFO scheduler.TaskSetManager: Finished task 145.0 in stage 6.0 (TID 154) in 58 ms on localhost (146/200)16/02/06 15:45:26 INFO scheduler.TaskSetManager: Starting task 148.0 in stage 6.0 (TID 157, localhost, partition 148,NODE_LOCAL, 1999 bytes)16/02/06 15:45:26 INFO scheduler.TaskSetManager: Finished task 146.0 in stage 6.0 (TID 155) in 55 ms on localhost (147/200)16/02/06 15:45:26 INFO executor.Executor: Running task 148.0 in stage 6.0 (TID 157)16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms16/02/06 15:45:26 INFO executor.Executor: Finished task 147.0 in stage 6.0 (TID 156). 2676 bytes result sent to driver16/02/06 15:45:26 INFO scheduler.TaskSetManager: Starting task 149.0 in stage 6.0 (TID 158, localhost, partition 149,NODE_LOCAL, 1999 bytes)16/02/06 15:45:26 INFO executor.Executor: Running task 149.0 in stage 6.0 (TID 158)16/02/06 15:45:26 INFO scheduler.TaskSetManager: Finished task 147.0 in stage 6.0 (TID 156) in 76 ms on localhost (148/200)16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 7 ms16/02/06 15:45:26 INFO executor.Executor: Finished task 148.0 in stage 6.0 (TID 157). 2676 bytes result sent to driver16/02/06 15:45:26 INFO executor.Executor: Finished task 149.0 in stage 6.0 (TID 158). 2676 bytes result sent to driver16/02/06 15:45:26 INFO scheduler.TaskSetManager: Starting task 150.0 in stage 6.0 (TID 159, localhost, partition 150,NODE_LOCAL, 1999 bytes)16/02/06 15:45:26 INFO executor.Executor: Running task 150.0 in stage 6.0 (TID 159)16/02/06 15:45:26 INFO scheduler.TaskSetManager: Finished task 148.0 in stage 6.0 (TID 157) in 103 ms on localhost (149/200)16/02/06 15:45:26 INFO scheduler.TaskSetManager: Starting task 151.0 in stage 6.0 (TID 160, localhost, partition 151,NODE_LOCAL, 1999 bytes)16/02/06 15:45:26 INFO scheduler.TaskSetManager: Finished task 149.0 in stage 6.0 (TID 158) in 50 ms on localhost (150/200)16/02/06 15:45:26 INFO executor.Executor: Running task 151.0 in stage 6.0 (TID 160)16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 11 ms16/02/06 15:45:26 INFO executor.Executor: Finished task 150.0 in stage 6.0 (TID 159). 2676 bytes result sent to driver16/02/06 15:45:26 INFO scheduler.TaskSetManager: Starting task 152.0 in stage 6.0 (TID 161, localhost, partition 152,NODE_LOCAL, 1999 bytes)16/02/06 15:45:26 INFO executor.Executor: Running task 152.0 in stage 6.0 (TID 161)16/02/06 15:45:26 INFO scheduler.TaskSetManager: Finished task 150.0 in stage 6.0 (TID 159) in 41 ms on localhost (151/200)16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 8 ms16/02/06 15:45:26 INFO executor.Executor: Finished task 152.0 in stage 6.0 (TID 161). 2676 bytes result sent to driver16/02/06 15:45:26 INFO scheduler.TaskSetManager: Starting task 153.0 in stage 6.0 (TID 162, localhost, partition 153,NODE_LOCAL, 1999 bytes)16/02/06 15:45:26 INFO executor.Executor: Running task 153.0 in stage 6.0 (TID 162)16/02/06 15:45:26 INFO scheduler.TaskSetManager: Finished task 152.0 in stage 6.0 (TID 161) in 37 ms on localhost (152/200)16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 8 ms16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 5 ms16/02/06 15:45:26 INFO executor.Executor: Finished task 151.0 in stage 6.0 (TID 160). 2676 bytes result sent to driver16/02/06 15:45:26 INFO scheduler.TaskSetManager: Starting task 154.0 in stage 6.0 (TID 163, localhost, partition 154,NODE_LOCAL, 1999 bytes)16/02/06 15:45:26 INFO executor.Executor: Running task 154.0 in stage 6.0 (TID 163)16/02/06 15:45:26 INFO scheduler.TaskSetManager: Finished task 151.0 in stage 6.0 (TID 160) in 87 ms on localhost (153/200)16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:26 INFO executor.Executor: Finished task 153.0 in stage 6.0 (TID 162). 2676 bytes result sent to driver16/02/06 15:45:26 INFO scheduler.TaskSetManager: Starting task 155.0 in stage 6.0 (TID 164, localhost, partition 155,NODE_LOCAL, 1999 bytes)16/02/06 15:45:26 INFO executor.Executor: Running task 155.0 in stage 6.0 (TID 164)16/02/06 15:45:26 INFO scheduler.TaskSetManager: Finished task 153.0 in stage 6.0 (TID 162) in 54 ms on localhost (154/200)16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:26 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:26 INFO executor.Executor: Finished task 154.0 in stage 6.0 (TID 163). 2676 bytes result sent to driver16/02/06 15:45:26 INFO scheduler.TaskSetManager: Starting task 156.0 in stage 6.0 (TID 165, localhost, partition 156,NODE_LOCAL, 1999 bytes)16/02/06 15:45:26 INFO executor.Executor: Running task 156.0 in stage 6.0 (TID 165)16/02/06 15:45:26 INFO scheduler.TaskSetManager: Finished task 154.0 in stage 6.0 (TID 163) in 53 ms on localhost (155/200)16/02/06 15:45:26 INFO executor.Executor: Finished task 155.0 in stage 6.0 (TID 164). 2676 bytes result sent to driver16/02/06 15:45:26 INFO scheduler.TaskSetManager: Starting task 157.0 in stage 6.0 (TID 166, localhost, partition 157,NODE_LOCAL, 1999 bytes)16/02/06 15:45:26 INFO executor.Executor: Running task 157.0 in stage 6.0 (TID 166)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Finished task 155.0 in stage 6.0 (TID 164) in 64 ms on localhost (156/200)16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 6 ms16/02/06 15:45:27 INFO executor.Executor: Finished task 156.0 in stage 6.0 (TID 165). 2676 bytes result sent to driver16/02/06 15:45:27 INFO scheduler.TaskSetManager: Starting task 158.0 in stage 6.0 (TID 167, localhost, partition 158,NODE_LOCAL, 1999 bytes)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Finished task 156.0 in stage 6.0 (TID 165) in 72 ms on localhost (157/200)16/02/06 15:45:27 INFO executor.Executor: Running task 158.0 in stage 6.0 (TID 167)16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:27 INFO executor.Executor: Finished task 157.0 in stage 6.0 (TID 166). 2676 bytes result sent to driver16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms16/02/06 15:45:27 INFO scheduler.TaskSetManager: Starting task 159.0 in stage 6.0 (TID 168, localhost, partition 159,NODE_LOCAL, 1999 bytes)16/02/06 15:45:27 INFO executor.Executor: Running task 159.0 in stage 6.0 (TID 168)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Finished task 157.0 in stage 6.0 (TID 166) in 73 ms on localhost (158/200)16/02/06 15:45:27 INFO executor.Executor: Finished task 158.0 in stage 6.0 (TID 167). 2676 bytes result sent to driver16/02/06 15:45:27 INFO scheduler.TaskSetManager: Starting task 160.0 in stage 6.0 (TID 169, localhost, partition 160,NODE_LOCAL, 1999 bytes)16/02/06 15:45:27 INFO executor.Executor: Running task 160.0 in stage 6.0 (TID 169)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Finished task 158.0 in stage 6.0 (TID 167) in 61 ms on localhost (159/200)16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms16/02/06 15:45:27 INFO executor.Executor: Finished task 159.0 in stage 6.0 (TID 168). 2676 bytes result sent to driver16/02/06 15:45:27 INFO scheduler.TaskSetManager: Starting task 161.0 in stage 6.0 (TID 170, localhost, partition 161,NODE_LOCAL, 1999 bytes)16/02/06 15:45:27 INFO executor.Executor: Running task 161.0 in stage 6.0 (TID 170)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Finished task 159.0 in stage 6.0 (TID 168) in 92 ms on localhost (160/200)16/02/06 15:45:27 INFO executor.Executor: Finished task 160.0 in stage 6.0 (TID 169). 2676 bytes result sent to driver16/02/06 15:45:27 INFO scheduler.TaskSetManager: Starting task 162.0 in stage 6.0 (TID 171, localhost, partition 162,NODE_LOCAL, 1999 bytes)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Finished task 160.0 in stage 6.0 (TID 169) in 73 ms on localhost (161/200)16/02/06 15:45:27 INFO executor.Executor: Running task 162.0 in stage 6.0 (TID 171)16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms16/02/06 15:45:27 INFO executor.Executor: Finished task 162.0 in stage 6.0 (TID 171). 2676 bytes result sent to driver16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms16/02/06 15:45:27 INFO executor.Executor: Finished task 161.0 in stage 6.0 (TID 170). 2676 bytes result sent to driver16/02/06 15:45:27 INFO scheduler.TaskSetManager: Starting task 163.0 in stage 6.0 (TID 172, localhost, partition 163,NODE_LOCAL, 1999 bytes)16/02/06 15:45:27 INFO executor.Executor: Running task 163.0 in stage 6.0 (TID 172)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Finished task 162.0 in stage 6.0 (TID 171) in 59 ms on localhost (162/200)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Starting task 164.0 in stage 6.0 (TID 173, localhost, partition 164,NODE_LOCAL, 1999 bytes)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Finished task 161.0 in stage 6.0 (TID 170) in 87 ms on localhost (163/200)16/02/06 15:45:27 INFO executor.Executor: Running task 164.0 in stage 6.0 (TID 173)16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 4 ms16/02/06 15:45:27 INFO executor.Executor: Finished task 164.0 in stage 6.0 (TID 173). 2676 bytes result sent to driver16/02/06 15:45:27 INFO scheduler.TaskSetManager: Starting task 165.0 in stage 6.0 (TID 174, localhost, partition 165,NODE_LOCAL, 1999 bytes)16/02/06 15:45:27 INFO executor.Executor: Running task 165.0 in stage 6.0 (TID 174)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Finished task 164.0 in stage 6.0 (TID 173) in 38 ms on localhost (164/200)16/02/06 15:45:27 INFO executor.Executor: Finished task 163.0 in stage 6.0 (TID 172). 2676 bytes result sent to driver16/02/06 15:45:27 INFO scheduler.TaskSetManager: Starting task 166.0 in stage 6.0 (TID 175, localhost, partition 166,NODE_LOCAL, 1999 bytes)16/02/06 15:45:27 INFO executor.Executor: Running task 166.0 in stage 6.0 (TID 175)16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 5 ms16/02/06 15:45:27 INFO executor.Executor: Finished task 165.0 in stage 6.0 (TID 174). 2676 bytes result sent to driver16/02/06 15:45:27 INFO scheduler.TaskSetManager: Starting task 167.0 in stage 6.0 (TID 176, localhost, partition 167,NODE_LOCAL, 1999 bytes)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Finished task 163.0 in stage 6.0 (TID 172) in 88 ms on localhost (165/200)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Finished task 165.0 in stage 6.0 (TID 174) in 40 ms on localhost (166/200)16/02/06 15:45:27 INFO executor.Executor: Running task 167.0 in stage 6.0 (TID 176)16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms16/02/06 15:45:27 INFO executor.Executor: Finished task 166.0 in stage 6.0 (TID 175). 2676 bytes result sent to driver16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms16/02/06 15:45:27 INFO scheduler.TaskSetManager: Starting task 168.0 in stage 6.0 (TID 177, localhost, partition 168,NODE_LOCAL, 1999 bytes)16/02/06 15:45:27 INFO executor.Executor: Running task 168.0 in stage 6.0 (TID 177)16/02/06 15:45:27 INFO executor.Executor: Finished task 167.0 in stage 6.0 (TID 176). 2676 bytes result sent to driver16/02/06 15:45:27 INFO scheduler.TaskSetManager: Finished task 166.0 in stage 6.0 (TID 175) in 67 ms on localhost (167/200)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Starting task 169.0 in stage 6.0 (TID 178, localhost, partition 169,NODE_LOCAL, 1999 bytes)16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:27 INFO scheduler.TaskSetManager: Finished task 167.0 in stage 6.0 (TID 176) in 55 ms on localhost (168/200)16/02/06 15:45:27 INFO executor.Executor: Running task 169.0 in stage 6.0 (TID 178)16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 5 ms16/02/06 15:45:27 INFO executor.Executor: Finished task 168.0 in stage 6.0 (TID 177). 2676 bytes result sent to driver16/02/06 15:45:27 INFO scheduler.TaskSetManager: Starting task 170.0 in stage 6.0 (TID 179, localhost, partition 170,NODE_LOCAL, 1999 bytes)16/02/06 15:45:27 INFO executor.Executor: Running task 170.0 in stage 6.0 (TID 179)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Finished task 168.0 in stage 6.0 (TID 177) in 32 ms on localhost (169/200)16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:27 INFO executor.Executor: Finished task 170.0 in stage 6.0 (TID 179). 2676 bytes result sent to driver16/02/06 15:45:27 INFO scheduler.TaskSetManager: Starting task 171.0 in stage 6.0 (TID 180, localhost, partition 171,NODE_LOCAL, 1999 bytes)16/02/06 15:45:27 INFO executor.Executor: Running task 171.0 in stage 6.0 (TID 180)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Finished task 170.0 in stage 6.0 (TID 179) in 41 ms on localhost (170/200)16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 10 ms16/02/06 15:45:27 INFO executor.Executor: Finished task 169.0 in stage 6.0 (TID 178). 2676 bytes result sent to driver16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:27 INFO scheduler.TaskSetManager: Starting task 172.0 in stage 6.0 (TID 181, localhost, partition 172,NODE_LOCAL, 1999 bytes)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Finished task 169.0 in stage 6.0 (TID 178) in 81 ms on localhost (171/200)16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 8 ms16/02/06 15:45:27 INFO executor.Executor: Running task 172.0 in stage 6.0 (TID 181)16/02/06 15:45:27 INFO executor.Executor: Finished task 171.0 in stage 6.0 (TID 180). 2676 bytes result sent to driver16/02/06 15:45:27 INFO scheduler.TaskSetManager: Starting task 173.0 in stage 6.0 (TID 182, localhost, partition 173,NODE_LOCAL, 1999 bytes)16/02/06 15:45:27 INFO executor.Executor: Running task 173.0 in stage 6.0 (TID 182)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Finished task 171.0 in stage 6.0 (TID 180) in 59 ms on localhost (172/200)16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:27 INFO executor.Executor: Finished task 172.0 in stage 6.0 (TID 181). 2676 bytes result sent to driver16/02/06 15:45:27 INFO scheduler.TaskSetManager: Starting task 174.0 in stage 6.0 (TID 183, localhost, partition 174,NODE_LOCAL, 1999 bytes)16/02/06 15:45:27 INFO executor.Executor: Running task 174.0 in stage 6.0 (TID 183)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Finished task 172.0 in stage 6.0 (TID 181) in 55 ms on localhost (173/200)16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 11 ms16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 4 ms16/02/06 15:45:27 INFO executor.Executor: Finished task 174.0 in stage 6.0 (TID 183). 2676 bytes result sent to driver16/02/06 15:45:27 INFO executor.Executor: Finished task 173.0 in stage 6.0 (TID 182). 2676 bytes result sent to driver16/02/06 15:45:27 INFO scheduler.TaskSetManager: Starting task 175.0 in stage 6.0 (TID 184, localhost, partition 175,NODE_LOCAL, 1999 bytes)16/02/06 15:45:27 INFO executor.Executor: Running task 175.0 in stage 6.0 (TID 184)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Finished task 174.0 in stage 6.0 (TID 183) in 47 ms on localhost (174/200)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Starting task 176.0 in stage 6.0 (TID 185, localhost, partition 176,NODE_LOCAL, 1999 bytes)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Finished task 173.0 in stage 6.0 (TID 182) in 92 ms on localhost (175/200)16/02/06 15:45:27 INFO executor.Executor: Running task 176.0 in stage 6.0 (TID 185)16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:27 INFO executor.Executor: Finished task 176.0 in stage 6.0 (TID 185). 2676 bytes result sent to driver16/02/06 15:45:27 INFO scheduler.TaskSetManager: Starting task 177.0 in stage 6.0 (TID 186, localhost, partition 177,NODE_LOCAL, 1999 bytes)16/02/06 15:45:27 INFO executor.Executor: Running task 177.0 in stage 6.0 (TID 186)16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 5 ms16/02/06 15:45:27 INFO executor.Executor: Finished task 175.0 in stage 6.0 (TID 184). 2676 bytes result sent to driver16/02/06 15:45:27 INFO scheduler.TaskSetManager: Finished task 176.0 in stage 6.0 (TID 185) in 51 ms on localhost (176/200)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Starting task 178.0 in stage 6.0 (TID 187, localhost, partition 178,NODE_LOCAL, 1999 bytes)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Finished task 175.0 in stage 6.0 (TID 184) in 88 ms on localhost (177/200)16/02/06 15:45:27 INFO executor.Executor: Running task 178.0 in stage 6.0 (TID 187)16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 10 ms16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 12 ms16/02/06 15:45:27 INFO executor.Executor: Finished task 177.0 in stage 6.0 (TID 186). 2676 bytes result sent to driver16/02/06 15:45:27 INFO executor.Executor: Finished task 178.0 in stage 6.0 (TID 187). 2676 bytes result sent to driver16/02/06 15:45:27 INFO scheduler.TaskSetManager: Starting task 179.0 in stage 6.0 (TID 188, localhost, partition 179,NODE_LOCAL, 1999 bytes)16/02/06 15:45:27 INFO executor.Executor: Running task 179.0 in stage 6.0 (TID 188)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Starting task 180.0 in stage 6.0 (TID 189, localhost, partition 180,NODE_LOCAL, 1999 bytes)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Finished task 177.0 in stage 6.0 (TID 186) in 92 ms on localhost (178/200)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Finished task 178.0 in stage 6.0 (TID 187) in 66 ms on localhost (179/200)16/02/06 15:45:27 INFO executor.Executor: Running task 180.0 in stage 6.0 (TID 189)16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 4 ms16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:27 INFO executor.Executor: Finished task 180.0 in stage 6.0 (TID 189). 2676 bytes result sent to driver16/02/06 15:45:27 INFO scheduler.TaskSetManager: Starting task 181.0 in stage 6.0 (TID 190, localhost, partition 181,NODE_LOCAL, 1999 bytes)16/02/06 15:45:27 INFO executor.Executor: Running task 181.0 in stage 6.0 (TID 190)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Finished task 180.0 in stage 6.0 (TID 189) in 63 ms on localhost (180/200)16/02/06 15:45:27 INFO executor.Executor: Finished task 179.0 in stage 6.0 (TID 188). 2676 bytes result sent to driver16/02/06 15:45:27 INFO scheduler.TaskSetManager: Starting task 182.0 in stage 6.0 (TID 191, localhost, partition 182,NODE_LOCAL, 1999 bytes)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Finished task 179.0 in stage 6.0 (TID 188) in 91 ms on localhost (181/200)16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:27 INFO executor.Executor: Running task 182.0 in stage 6.0 (TID 191)16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 16 ms16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 16 ms16/02/06 15:45:27 INFO executor.Executor: Finished task 181.0 in stage 6.0 (TID 190). 2676 bytes result sent to driver16/02/06 15:45:27 INFO scheduler.TaskSetManager: Starting task 183.0 in stage 6.0 (TID 192, localhost, partition 183,NODE_LOCAL, 1999 bytes)16/02/06 15:45:27 INFO executor.Executor: Running task 183.0 in stage 6.0 (TID 192)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Finished task 181.0 in stage 6.0 (TID 190) in 64 ms on localhost (182/200)16/02/06 15:45:27 INFO executor.Executor: Finished task 182.0 in stage 6.0 (TID 191). 2676 bytes result sent to driver16/02/06 15:45:27 INFO scheduler.TaskSetManager: Starting task 184.0 in stage 6.0 (TID 193, localhost, partition 184,NODE_LOCAL, 1999 bytes)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Finished task 182.0 in stage 6.0 (TID 191) in 66 ms on localhost (183/200)16/02/06 15:45:27 INFO executor.Executor: Running task 184.0 in stage 6.0 (TID 193)16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 4 ms16/02/06 15:45:27 INFO executor.Executor: Finished task 183.0 in stage 6.0 (TID 192). 2676 bytes result sent to driver16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:27 INFO scheduler.TaskSetManager: Starting task 185.0 in stage 6.0 (TID 194, localhost, partition 185,NODE_LOCAL, 1999 bytes)16/02/06 15:45:27 INFO executor.Executor: Running task 185.0 in stage 6.0 (TID 194)16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 11 ms16/02/06 15:45:27 INFO executor.Executor: Finished task 184.0 in stage 6.0 (TID 193). 2676 bytes result sent to driver16/02/06 15:45:27 INFO scheduler.TaskSetManager: Starting task 186.0 in stage 6.0 (TID 195, localhost, partition 186,NODE_LOCAL, 1999 bytes)16/02/06 15:45:27 INFO executor.Executor: Running task 186.0 in stage 6.0 (TID 195)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Finished task 183.0 in stage 6.0 (TID 192) in 90 ms on localhost (184/200)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Finished task 184.0 in stage 6.0 (TID 193) in 68 ms on localhost (185/200)16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 4 ms16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms16/02/06 15:45:27 INFO executor.Executor: Finished task 186.0 in stage 6.0 (TID 195). 2676 bytes result sent to driver16/02/06 15:45:27 INFO executor.Executor: Finished task 185.0 in stage 6.0 (TID 194). 2676 bytes result sent to driver16/02/06 15:45:27 INFO scheduler.TaskSetManager: Starting task 187.0 in stage 6.0 (TID 196, localhost, partition 187,NODE_LOCAL, 1999 bytes)16/02/06 15:45:27 INFO executor.Executor: Running task 187.0 in stage 6.0 (TID 196)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Finished task 185.0 in stage 6.0 (TID 194) in 72 ms on localhost (186/200)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Starting task 188.0 in stage 6.0 (TID 197, localhost, partition 188,NODE_LOCAL, 1999 bytes)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Finished task 186.0 in stage 6.0 (TID 195) in 69 ms on localhost (187/200)16/02/06 15:45:27 INFO executor.Executor: Running task 188.0 in stage 6.0 (TID 197)16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 4 ms16/02/06 15:45:27 INFO executor.Executor: Finished task 187.0 in stage 6.0 (TID 196). 2676 bytes result sent to driver16/02/06 15:45:27 INFO scheduler.TaskSetManager: Starting task 189.0 in stage 6.0 (TID 198, localhost, partition 189,NODE_LOCAL, 1999 bytes)16/02/06 15:45:27 INFO executor.Executor: Running task 189.0 in stage 6.0 (TID 198)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Finished task 187.0 in stage 6.0 (TID 196) in 58 ms on localhost (188/200)16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 6 ms16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms16/02/06 15:45:27 INFO executor.Executor: Finished task 188.0 in stage 6.0 (TID 197). 2676 bytes result sent to driver16/02/06 15:45:27 INFO executor.Executor: Finished task 189.0 in stage 6.0 (TID 198). 2676 bytes result sent to driver16/02/06 15:45:27 INFO scheduler.TaskSetManager: Starting task 190.0 in stage 6.0 (TID 199, localhost, partition 190,NODE_LOCAL, 1999 bytes)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Starting task 191.0 in stage 6.0 (TID 200, localhost, partition 191,NODE_LOCAL, 1999 bytes)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Finished task 188.0 in stage 6.0 (TID 197) in 74 ms on localhost (189/200)16/02/06 15:45:27 INFO scheduler.TaskSetManager: Finished task 189.0 in stage 6.0 (TID 198) in 41 ms on localhost (190/200)16/02/06 15:45:27 INFO executor.Executor: Running task 191.0 in stage 6.0 (TID 200)16/02/06 15:45:27 INFO executor.Executor: Running task 190.0 in stage 6.0 (TID 199)16/02/06 15:45:28 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:28 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:28 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:28 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms16/02/06 15:45:28 INFO executor.Executor: Finished task 190.0 in stage 6.0 (TID 199). 2676 bytes result sent to driver16/02/06 15:45:28 INFO executor.Executor: Finished task 191.0 in stage 6.0 (TID 200). 2676 bytes result sent to driver16/02/06 15:45:28 INFO scheduler.TaskSetManager: Starting task 192.0 in stage 6.0 (TID 201, localhost, partition 192,NODE_LOCAL, 1999 bytes)16/02/06 15:45:28 INFO scheduler.TaskSetManager: Finished task 190.0 in stage 6.0 (TID 199) in 67 ms on localhost (191/200)16/02/06 15:45:28 INFO executor.Executor: Running task 192.0 in stage 6.0 (TID 201)16/02/06 15:45:28 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:28 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:28 INFO scheduler.TaskSetManager: Starting task 193.0 in stage 6.0 (TID 202, localhost, partition 193,NODE_LOCAL, 1999 bytes)16/02/06 15:45:28 INFO scheduler.TaskSetManager: Finished task 191.0 in stage 6.0 (TID 200) in 85 ms on localhost (192/200)16/02/06 15:45:28 INFO executor.Executor: Running task 193.0 in stage 6.0 (TID 202)16/02/06 15:45:28 INFO executor.Executor: Finished task 192.0 in stage 6.0 (TID 201). 2676 bytes result sent to driver16/02/06 15:45:28 INFO scheduler.TaskSetManager: Starting task 194.0 in stage 6.0 (TID 203, localhost, partition 194,NODE_LOCAL, 1999 bytes)16/02/06 15:45:28 INFO scheduler.TaskSetManager: Finished task 192.0 in stage 6.0 (TID 201) in 43 ms on localhost (193/200)16/02/06 15:45:28 INFO executor.Executor: Running task 194.0 in stage 6.0 (TID 203)16/02/06 15:45:28 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:28 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 5 ms16/02/06 15:45:28 INFO executor.Executor: Finished task 193.0 in stage 6.0 (TID 202). 2676 bytes result sent to driver16/02/06 15:45:28 INFO scheduler.TaskSetManager: Starting task 195.0 in stage 6.0 (TID 204, localhost, partition 195,NODE_LOCAL, 1999 bytes)16/02/06 15:45:28 INFO scheduler.TaskSetManager: Finished task 193.0 in stage 6.0 (TID 202) in 64 ms on localhost (194/200)16/02/06 15:45:28 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:28 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms16/02/06 15:45:28 INFO executor.Executor: Running task 195.0 in stage 6.0 (TID 204)16/02/06 15:45:28 INFO executor.Executor: Finished task 194.0 in stage 6.0 (TID 203). 2676 bytes result sent to driver16/02/06 15:45:28 INFO scheduler.TaskSetManager: Starting task 196.0 in stage 6.0 (TID 205, localhost, partition 196,NODE_LOCAL, 1999 bytes)16/02/06 15:45:28 INFO scheduler.TaskSetManager: Finished task 194.0 in stage 6.0 (TID 203) in 70 ms on localhost (195/200)16/02/06 15:45:28 INFO executor.Executor: Running task 196.0 in stage 6.0 (TID 205)16/02/06 15:45:28 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:28 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 13 ms16/02/06 15:45:28 INFO executor.Executor: Finished task 195.0 in stage 6.0 (TID 204). 2676 bytes result sent to driver16/02/06 15:45:28 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:28 INFO scheduler.TaskSetManager: Starting task 197.0 in stage 6.0 (TID 206, localhost, partition 197,NODE_LOCAL, 1999 bytes)16/02/06 15:45:28 INFO scheduler.TaskSetManager: Finished task 195.0 in stage 6.0 (TID 204) in 59 ms on localhost (196/200)16/02/06 15:45:28 INFO executor.Executor: Running task 197.0 in stage 6.0 (TID 206)16/02/06 15:45:28 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 22 ms16/02/06 15:45:28 INFO executor.Executor: Finished task 196.0 in stage 6.0 (TID 205). 2676 bytes result sent to driver16/02/06 15:45:28 INFO scheduler.TaskSetManager: Starting task 198.0 in stage 6.0 (TID 207, localhost, partition 198,NODE_LOCAL, 1999 bytes)16/02/06 15:45:28 INFO executor.Executor: Running task 198.0 in stage 6.0 (TID 207)16/02/06 15:45:28 INFO scheduler.TaskSetManager: Finished task 196.0 in stage 6.0 (TID 205) in 70 ms on localhost (197/200)16/02/06 15:45:28 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:28 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 5 ms16/02/06 15:45:28 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:28 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:45:28 INFO executor.Executor: Finished task 197.0 in stage 6.0 (TID 206). 2676 bytes result sent to driver16/02/06 15:45:28 INFO executor.Executor: Finished task 198.0 in stage 6.0 (TID 207). 2676 bytes result sent to driver16/02/06 15:45:28 INFO scheduler.TaskSetManager: Starting task 199.0 in stage 6.0 (TID 208, localhost, partition 199,NODE_LOCAL, 1999 bytes)16/02/06 15:45:28 INFO executor.Executor: Running task 199.0 in stage 6.0 (TID 208)16/02/06 15:45:28 INFO scheduler.TaskSetManager: Finished task 197.0 in stage 6.0 (TID 206) in 66 ms on localhost (198/200)16/02/06 15:45:28 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:45:28 INFO scheduler.TaskSetManager: Finished task 198.0 in stage 6.0 (TID 207) in 42 ms on localhost (199/200)16/02/06 15:45:28 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 8 ms16/02/06 15:45:28 INFO executor.Executor: Finished task 199.0 in stage 6.0 (TID 208). 2676 bytes result sent to driver16/02/06 15:45:28 INFO scheduler.DAGScheduler: ResultStage 6 (show at <console>:28) finished in 7.223 s16/02/06 15:45:28 INFO scheduler.DAGScheduler: Job 4 finished: show at <console>:28, took 8.829114 s16/02/06 15:45:28 INFO scheduler.TaskSetManager: Finished task 199.0 in stage 6.0 (TID 208) in 33 ms on localhost (200/200)16/02/06 15:45:28 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 6.0, whose tasks have all completed, from pool +---------+-----+| author|count|+---------+-----+| yiyou| 3||jingruhou| 1|+---------+-----+
DataFrame注册成临时表使用
使用下列语句将DataFrame注册成表：
scala> val commitlog=df.registerTempTable("commitlog")commitlog: Unit = ()显示数据：
scala> sqlContext.sql("SELECT * FROM commitlog").show16/02/06 15:47:53 INFO parse.ParseDriver: Parsing command: SELECT * FROM commitlog16/02/06 15:47:57 INFO parse.ParseDriver: Parse Completed16/02/06 15:47:57 INFO storage.MemoryStore: Block broadcast_14 stored as values in memory (estimated size 212.1 KB, free 465.3 KB)16/02/06 15:47:57 INFO storage.MemoryStore: Block broadcast_14_piece0 stored as bytes in memory (estimated size 19.5 KB, free 484.7 KB)16/02/06 15:47:57 INFO storage.BlockManagerInfo: Added broadcast_14_piece0 in memory on localhost:45391 (size: 19.5 KB, free: 511.1 MB)16/02/06 15:47:57 INFO spark.SparkContext: Created broadcast 14 from show at <console>:2616/02/06 15:47:57 INFO storage.MemoryStore: Block broadcast_15 stored as values in memory (estimated size 212.5 KB, free 697.3 KB)16/02/06 15:47:57 INFO storage.MemoryStore: Block broadcast_15_piece0 stored as bytes in memory (estimated size 19.5 KB, free 716.8 KB)16/02/06 15:47:57 INFO storage.BlockManagerInfo: Added broadcast_15_piece0 in memory on localhost:45391 (size: 19.5 KB, free: 511.1 MB)16/02/06 15:47:57 INFO spark.SparkContext: Created broadcast 15 from show at <console>:2616/02/06 15:47:58 INFO mapred.FileInputFormat: Total input paths to process : 116/02/06 15:47:58 INFO spark.SparkContext: Starting job: show at <console>:2616/02/06 15:47:58 INFO scheduler.DAGScheduler: Got job 5 (show at <console>:26) with 1 output partitions16/02/06 15:47:58 INFO scheduler.DAGScheduler: Final stage: ResultStage 7 (show at <console>:26)16/02/06 15:47:58 INFO scheduler.DAGScheduler: Parents of final stage: List()16/02/06 15:47:58 INFO scheduler.DAGScheduler: Missing parents: List()16/02/06 15:47:58 INFO scheduler.DAGScheduler: Submitting ResultStage 7 (MapPartitionsRDD[35] at show at <console>:26), which has no missing parents16/02/06 15:47:58 INFO storage.MemoryStore: Block broadcast_16 stored as values in memory (estimated size 6.1 KB, free 722.9 KB)16/02/06 15:47:58 INFO storage.MemoryStore: Block broadcast_16_piece0 stored as bytes in memory (estimated size 3.4 KB, free 726.3 KB)16/02/06 15:47:58 INFO storage.BlockManagerInfo: Added broadcast_16_piece0 in memory on localhost:45391 (size: 3.4 KB, free: 511.1 MB)16/02/06 15:47:58 INFO spark.SparkContext: Created broadcast 16 from broadcast at DAGScheduler.scala:100616/02/06 15:47:58 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 7 (MapPartitionsRDD[35] at show at <console>:26)16/02/06 15:47:58 INFO scheduler.TaskSchedulerImpl: Adding task set 7.0 with 1 tasks16/02/06 15:47:58 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 7.0 (TID 209, localhost, partition 0,ANY, 2142 bytes)16/02/06 15:47:58 INFO executor.Executor: Running task 0.0 in stage 7.0 (TID 209)16/02/06 15:47:58 INFO rdd.HadoopRDD: Input split: hdfs://localhost:9000/hjr/sparktest.json:0+33916/02/06 15:47:58 INFO executor.Executor: Finished task 0.0 in stage 7.0 (TID 209). 2857 bytes result sent to driver16/02/06 15:47:58 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 7.0 (TID 209) in 87 ms on localhost (1/1)16/02/06 15:47:58 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 7.0, whose tasks have all completed, from pool 16/02/06 15:47:58 INFO scheduler.DAGScheduler: ResultStage 7 (show at <console>:26) finished in 0.087 s16/02/06 15:47:58 INFO scheduler.DAGScheduler: Job 5 finished: show at <console>:26, took 0.131679 s16/02/06 15:47:58 INFO spark.SparkContext: Starting job: show at <console>:2616/02/06 15:47:58 INFO scheduler.DAGScheduler: Got job 6 (show at <console>:26) with 1 output partitions16/02/06 15:47:58 INFO scheduler.DAGScheduler: Final stage: ResultStage 8 (show at <console>:26)16/02/06 15:47:58 INFO scheduler.DAGScheduler: Parents of final stage: List()16/02/06 15:47:58 INFO scheduler.DAGScheduler: Missing parents: List()16/02/06 15:47:58 INFO scheduler.DAGScheduler: Submitting ResultStage 8 (MapPartitionsRDD[35] at show at <console>:26), which has no missing parents16/02/06 15:47:58 INFO storage.MemoryStore: Block broadcast_17 stored as values in memory (estimated size 6.1 KB, free 732.4 KB)16/02/06 15:47:58 INFO storage.MemoryStore: Block broadcast_17_piece0 stored as bytes in memory (estimated size 3.4 KB, free 735.8 KB)16/02/06 15:47:58 INFO storage.BlockManagerInfo: Added broadcast_17_piece0 in memory on localhost:45391 (size: 3.4 KB, free: 511.1 MB)16/02/06 15:47:58 INFO spark.SparkContext: Created broadcast 17 from broadcast at DAGScheduler.scala:100616/02/06 15:47:58 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 8 (MapPartitionsRDD[35] at show at <console>:26)16/02/06 15:47:58 INFO scheduler.TaskSchedulerImpl: Adding task set 8.0 with 1 tasks16/02/06 15:47:58 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 8.0 (TID 210, localhost, partition 1,ANY, 2142 bytes)16/02/06 15:47:58 INFO executor.Executor: Running task 0.0 in stage 8.0 (TID 210)16/02/06 15:47:58 INFO rdd.HadoopRDD: Input split: hdfs://localhost:9000/hjr/sparktest.json:339+34016/02/06 15:47:58 INFO executor.Executor: Finished task 0.0 in stage 8.0 (TID 210). 2501 bytes result sent to driver16/02/06 15:47:58 INFO scheduler.DAGScheduler: ResultStage 8 (show at <console>:26) finished in 0.106 s16/02/06 15:47:58 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 8.0 (TID 210) in 102 ms on localhost (1/1)16/02/06 15:47:58 INFO scheduler.DAGScheduler: Job 6 finished: show at <console>:26, took 0.147341 s16/02/06 15:47:58 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 8.0, whose tasks have all completed, from pool +---------+-----------------+--------------------+--------------------+--------------+| author| author_email| commit| date| meaasge|+---------+-----------------+--------------------+--------------------+--------------+| yiyou|1627383642@qq.com|b40dc4452931232f1...|Wed Feb 3 19:05:0...| commit-NO3|| yiyou|1627383642@qq.com|d9fa805c7d1c00f5c...|Mon Feb 1 23:56:1...| commit-NO2|| yiyou|1627383642@qq.com|66123d4f98e0e5a0f...|Mon Feb 1 15:04:5...| commit-NO1||jingruhou|1627383642@qq.com|56a1fbd42a6122aa2...|Mon Feb 1 14:34:5...|Initial-commit|+---------+-----------------+--------------------+--------------------+--------------+
计算总提交次数
scala> sqlContext.sql("SELECT count(*) as TotalCommitNumber FROM commitlog").show16/02/06 15:49:02 INFO parse.ParseDriver: Parsing command: SELECT count(*) as TotalCommitNumber FROM commitlog16/02/06 15:49:02 INFO parse.ParseDriver: Parse Completed16/02/06 15:49:02 INFO storage.MemoryStore: Block broadcast_18 stored as values in memory (estimated size 212.1 KB, free 947.9 KB)16/02/06 15:49:02 INFO storage.MemoryStore: Block broadcast_18_piece0 stored as bytes in memory (estimated size 19.5 KB, free 967.3 KB)16/02/06 15:49:02 INFO storage.BlockManagerInfo: Added broadcast_18_piece0 in memory on localhost:45391 (size: 19.5 KB, free: 511.0 MB)16/02/06 15:49:02 INFO spark.SparkContext: Created broadcast 18 from show at <console>:2616/02/06 15:49:02 INFO storage.MemoryStore: Block broadcast_19 stored as values in memory (estimated size 212.5 KB, free 1179.9 KB)16/02/06 15:49:02 INFO storage.MemoryStore: Block broadcast_19_piece0 stored as bytes in memory (estimated size 19.5 KB, free 1199.4 KB)16/02/06 15:49:02 INFO storage.BlockManagerInfo: Added broadcast_19_piece0 in memory on localhost:45391 (size: 19.5 KB, free: 511.0 MB)16/02/06 15:49:02 INFO spark.SparkContext: Created broadcast 19 from show at <console>:2616/02/06 15:49:02 INFO mapred.FileInputFormat: Total input paths to process : 116/02/06 15:49:03 INFO spark.SparkContext: Starting job: show at <console>:2616/02/06 15:49:03 INFO scheduler.DAGScheduler: Registering RDD 41 (show at <console>:26)16/02/06 15:49:03 INFO scheduler.DAGScheduler: Got job 7 (show at <console>:26) with 1 output partitions16/02/06 15:49:03 INFO scheduler.DAGScheduler: Final stage: ResultStage 10 (show at <console>:26)16/02/06 15:49:03 INFO scheduler.DAGScheduler: Parents of final stage: List(ShuffleMapStage 9)16/02/06 15:49:03 INFO scheduler.DAGScheduler: Missing parents: List(ShuffleMapStage 9)16/02/06 15:49:03 INFO scheduler.DAGScheduler: Submitting ShuffleMapStage 9 (MapPartitionsRDD[41] at show at <console>:26), which has no missing parents16/02/06 15:49:03 INFO storage.MemoryStore: Block broadcast_20 stored as values in memory (estimated size 10.4 KB, free 1209.8 KB)16/02/06 15:49:03 INFO storage.MemoryStore: Block broadcast_20_piece0 stored as bytes in memory (estimated size 5.3 KB, free 1215.2 KB)16/02/06 15:49:03 INFO storage.BlockManagerInfo: Added broadcast_20_piece0 in memory on localhost:45391 (size: 5.3 KB, free: 511.0 MB)16/02/06 15:49:03 INFO spark.SparkContext: Created broadcast 20 from broadcast at DAGScheduler.scala:100616/02/06 15:49:03 INFO scheduler.DAGScheduler: Submitting 2 missing tasks from ShuffleMapStage 9 (MapPartitionsRDD[41] at show at <console>:26)16/02/06 15:49:03 INFO scheduler.TaskSchedulerImpl: Adding task set 9.0 with 2 tasks16/02/06 15:49:03 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 9.0 (TID 211, localhost, partition 0,ANY, 2131 bytes)16/02/06 15:49:03 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 9.0 (TID 212, localhost, partition 1,ANY, 2131 bytes)16/02/06 15:49:03 INFO executor.Executor: Running task 0.0 in stage 9.0 (TID 211)16/02/06 15:49:03 INFO rdd.HadoopRDD: Input split: hdfs://localhost:9000/hjr/sparktest.json:0+33916/02/06 15:49:03 INFO executor.Executor: Running task 1.0 in stage 9.0 (TID 212)16/02/06 15:49:03 INFO rdd.HadoopRDD: Input split: hdfs://localhost:9000/hjr/sparktest.json:339+34016/02/06 15:49:03 INFO executor.Executor: Finished task 0.0 in stage 9.0 (TID 211). 2500 bytes result sent to driver16/02/06 15:49:03 INFO executor.Executor: Finished task 1.0 in stage 9.0 (TID 212). 2500 bytes result sent to driver16/02/06 15:49:03 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 9.0 (TID 211) in 234 ms on localhost (1/2)16/02/06 15:49:03 INFO scheduler.TaskSetManager: Finished task 1.0 in stage 9.0 (TID 212) in 236 ms on localhost (2/2)16/02/06 15:49:03 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 9.0, whose tasks have all completed, from pool 16/02/06 15:49:03 INFO scheduler.DAGScheduler: ShuffleMapStage 9 (show at <console>:26) finished in 0.240 s16/02/06 15:49:03 INFO scheduler.DAGScheduler: looking for newly runnable stages16/02/06 15:49:03 INFO scheduler.DAGScheduler: running: Set()16/02/06 15:49:03 INFO scheduler.DAGScheduler: waiting: Set(ResultStage 10)16/02/06 15:49:03 INFO scheduler.DAGScheduler: failed: Set()16/02/06 15:49:03 INFO scheduler.DAGScheduler: Submitting ResultStage 10 (MapPartitionsRDD[45] at show at <console>:26), which has no missing parents16/02/06 15:49:03 INFO storage.MemoryStore: Block broadcast_21 stored as values in memory (estimated size 11.8 KB, free 1226.9 KB)16/02/06 15:49:03 INFO storage.MemoryStore: Block broadcast_21_piece0 stored as bytes in memory (estimated size 5.9 KB, free 1232.8 KB)16/02/06 15:49:03 INFO storage.BlockManagerInfo: Added broadcast_21_piece0 in memory on localhost:45391 (size: 5.9 KB, free: 511.0 MB)16/02/06 15:49:03 INFO spark.SparkContext: Created broadcast 21 from broadcast at DAGScheduler.scala:100616/02/06 15:49:03 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 10 (MapPartitionsRDD[45] at show at <console>:26)16/02/06 15:49:03 INFO scheduler.TaskSchedulerImpl: Adding task set 10.0 with 1 tasks16/02/06 15:49:03 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 10.0 (TID 213, localhost, partition 0,NODE_LOCAL, 1999 bytes)16/02/06 15:49:03 INFO executor.Executor: Running task 0.0 in stage 10.0 (TID 213)16/02/06 15:49:03 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:49:03 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms16/02/06 15:49:03 INFO codegen.GenerateSafeProjection: Code generated in 90.242927 ms16/02/06 15:49:03 INFO executor.Executor: Finished task 0.0 in stage 10.0 (TID 213). 1894 bytes result sent to driver16/02/06 15:49:03 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 10.0 (TID 213) in 146 ms on localhost (1/1)16/02/06 15:49:03 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 10.0, whose tasks have all completed, from pool 16/02/06 15:49:03 INFO scheduler.DAGScheduler: ResultStage 10 (show at <console>:26) finished in 0.148 s16/02/06 15:49:03 INFO scheduler.DAGScheduler: Job 7 finished: show at <console>:26, took 0.534126 s+-----------------+|TotalCommitNumber|+-----------------+| 4|+-----------------+
按提交次数进行降序排序：
scala> sqlContext.sql("SELECT author,count(*) as countNumber FROM commitlog GROUP BY author ORDER BY CountNumber DESC").show16/02/06 15:51:41 INFO parse.ParseDriver: Parsing command: SELECT author,count(*) as countNumber FROM commitlog GROUP BY author ORDER BY CountNumber DESC16/02/06 15:51:41 INFO parse.ParseDriver: Parse Completed16/02/06 15:51:41 INFO storage.MemoryStore: Block broadcast_27 stored as values in memory (estimated size 212.1 KB, free 499.9 KB)16/02/06 15:51:41 INFO storage.MemoryStore: Block broadcast_27_piece0 stored as bytes in memory (estimated size 19.5 KB, free 519.3 KB)16/02/06 15:51:41 INFO storage.BlockManagerInfo: Added broadcast_27_piece0 in memory on localhost:45391 (size: 19.5 KB, free: 511.1 MB)16/02/06 15:51:41 INFO spark.SparkContext: Created broadcast 27 from show at <console>:2616/02/06 15:51:41 INFO storage.MemoryStore: Block broadcast_28 stored as values in memory (estimated size 212.5 KB, free 731.9 KB)16/02/06 15:51:41 INFO storage.MemoryStore: Block broadcast_28_piece0 stored as bytes in memory (estimated size 19.5 KB, free 751.4 KB)16/02/06 15:51:41 INFO storage.BlockManagerInfo: Added broadcast_28_piece0 in memory on localhost:45391 (size: 19.5 KB, free: 511.0 MB)16/02/06 15:51:41 INFO spark.SparkContext: Created broadcast 28 from show at <console>:2616/02/06 15:51:41 INFO mapred.FileInputFormat: Total input paths to process : 116/02/06 15:51:41 INFO spark.SparkContext: Starting job: show at <console>:2616/02/06 15:51:41 INFO scheduler.DAGScheduler: Registering RDD 61 (show at <console>:26)16/02/06 15:51:41 INFO scheduler.DAGScheduler: Got job 10 (show at <console>:26) with 200 output partitions16/02/06 15:51:41 INFO scheduler.DAGScheduler: Final stage: ResultStage 16 (show at <console>:26)16/02/06 15:51:41 INFO scheduler.DAGScheduler: Parents of final stage: List(ShuffleMapStage 15)16/02/06 15:51:41 INFO scheduler.DAGScheduler: Missing parents: List(ShuffleMapStage 15)16/02/06 15:51:41 INFO scheduler.DAGScheduler: Submitting ShuffleMapStage 15 (MapPartitionsRDD[61] at show at <console>:26), which has no missing parents16/02/06 15:51:41 INFO storage.MemoryStore: Block broadcast_29 stored as values in memory (estimated size 10.7 KB, free 762.2 KB)16/02/06 15:51:41 INFO storage.MemoryStore: Block broadcast_29_piece0 stored as bytes in memory (estimated size 5.5 KB, free 767.7 KB)16/02/06 15:51:41 INFO storage.BlockManagerInfo: Added broadcast_29_piece0 in memory on localhost:45391 (size: 5.5 KB, free: 511.0 MB)16/02/06 15:51:41 INFO spark.SparkContext: Created broadcast 29 from broadcast at DAGScheduler.scala:100616/02/06 15:51:41 INFO scheduler.DAGScheduler: Submitting 2 missing tasks from ShuffleMapStage 15 (MapPartitionsRDD[61] at show at <console>:26)16/02/06 15:51:41 INFO scheduler.TaskSchedulerImpl: Adding task set 15.0 with 2 tasks16/02/06 15:51:41 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 15.0 (TID 416, localhost, partition 0,ANY, 2131 bytes)16/02/06 15:51:42 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 15.0 (TID 417, localhost, partition 1,ANY, 2131 bytes)16/02/06 15:51:42 INFO executor.Executor: Running task 0.0 in stage 15.0 (TID 416)16/02/06 15:51:42 INFO executor.Executor: Running task 1.0 in stage 15.0 (TID 417)16/02/06 15:51:42 INFO rdd.HadoopRDD: Input split: hdfs://localhost:9000/hjr/sparktest.json:339+34016/02/06 15:51:42 INFO rdd.HadoopRDD: Input split: hdfs://localhost:9000/hjr/sparktest.json:0+33916/02/06 15:51:42 INFO storage.BlockManagerInfo: Removed broadcast_23_piece0 on localhost:45391 in memory (size: 19.5 KB, free: 511.1 MB)16/02/06 15:51:42 INFO storage.BlockManagerInfo: Removed broadcast_27_piece0 on localhost:45391 in memory (size: 19.5 KB, free: 511.1 MB)16/02/06 15:51:42 INFO storage.BlockManagerInfo: Removed broadcast_26_piece0 on localhost:45391 in memory (size: 6.8 KB, free: 511.1 MB)16/02/06 15:51:42 INFO spark.ContextCleaner: Cleaned accumulator 4716/02/06 15:51:42 INFO storage.BlockManagerInfo: Removed broadcast_25_piece0 on localhost:45391 in memory (size: 6.8 KB, free: 511.1 MB)16/02/06 15:51:42 INFO spark.ContextCleaner: Cleaned accumulator 4616/02/06 15:51:42 INFO storage.BlockManagerInfo: Removed broadcast_24_piece0 on localhost:45391 in memory (size: 5.5 KB, free: 511.1 MB)16/02/06 15:51:42 INFO spark.ContextCleaner: Cleaned accumulator 4516/02/06 15:51:42 INFO spark.ContextCleaner: Cleaned shuffle 316/02/06 15:51:42 INFO spark.ContextCleaner: Cleaned accumulator 4416/02/06 15:51:42 INFO spark.ContextCleaner: Cleaned accumulator 4316/02/06 15:51:42 INFO spark.ContextCleaner: Cleaned accumulator 4216/02/06 15:51:42 INFO spark.ContextCleaner: Cleaned accumulator 4116/02/06 15:51:42 INFO spark.ContextCleaner: Cleaned accumulator 4016/02/06 15:51:42 INFO spark.ContextCleaner: Cleaned accumulator 3916/02/06 15:51:42 INFO spark.ContextCleaner: Cleaned accumulator 3816/02/06 15:51:42 INFO spark.ContextCleaner: Cleaned accumulator 3716/02/06 15:51:43 INFO executor.Executor: Finished task 1.0 in stage 15.0 (TID 417). 2702 bytes result sent to driver16/02/06 15:51:43 INFO scheduler.TaskSetManager: Finished task 1.0 in stage 15.0 (TID 417) in 1504 ms on localhost (1/2)16/02/06 15:51:43 INFO executor.Executor: Finished task 0.0 in stage 15.0 (TID 416). 2702 bytes result sent to driver16/02/06 15:51:43 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 15.0 (TID 416) in 1523 ms on localhost (2/2)16/02/06 15:51:43 INFO scheduler.DAGScheduler: ShuffleMapStage 15 (show at <console>:26) finished in 1.524 s16/02/06 15:51:43 INFO scheduler.DAGScheduler: looking for newly runnable stages16/02/06 15:51:43 INFO scheduler.DAGScheduler: running: Set()16/02/06 15:51:43 INFO scheduler.DAGScheduler: waiting: Set(ResultStage 16)16/02/06 15:51:43 INFO scheduler.DAGScheduler: failed: Set()16/02/06 15:51:43 INFO scheduler.DAGScheduler: Submitting ResultStage 16 (MapPartitionsRDD[66] at show at <console>:26), which has no missing parents16/02/06 15:51:43 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 15.0, whose tasks have all completed, from pool 16/02/06 15:51:43 INFO storage.MemoryStore: Block broadcast_30 stored as values in memory (estimated size 13.9 KB, free 262.2 KB)16/02/06 15:51:43 INFO storage.MemoryStore: Block broadcast_30_piece0 stored as bytes in memory (estimated size 7.3 KB, free 269.5 KB)16/02/06 15:51:43 INFO storage.BlockManagerInfo: Added broadcast_30_piece0 in memory on localhost:45391 (size: 7.3 KB, free: 511.1 MB)16/02/06 15:51:43 INFO spark.SparkContext: Created broadcast 30 from broadcast at DAGScheduler.scala:100616/02/06 15:51:43 INFO scheduler.DAGScheduler: Submitting 200 missing tasks from ResultStage 16 (MapPartitionsRDD[66] at show at <console>:26)16/02/06 15:51:43 INFO scheduler.TaskSchedulerImpl: Adding task set 16.0 with 200 tasks16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 16.0 (TID 418, localhost, partition 0,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 16.0 (TID 419, localhost, partition 1,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO executor.Executor: Running task 0.0 in stage 16.0 (TID 418)16/02/06 15:51:47 INFO executor.Executor: Running task 1.0 in stage 16.0 (TID 419)16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms16/02/06 15:51:47 INFO executor.Executor: Finished task 0.0 in stage 16.0 (TID 418). 2676 bytes result sent to driver16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 2.0 in stage 16.0 (TID 420, localhost, partition 2,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO executor.Executor: Running task 2.0 in stage 16.0 (TID 420)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 16.0 (TID 418) in 46 ms on localhost (1/200)16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms16/02/06 15:51:47 INFO executor.Executor: Finished task 2.0 in stage 16.0 (TID 420). 2676 bytes result sent to driver16/02/06 15:51:47 INFO executor.Executor: Finished task 1.0 in stage 16.0 (TID 419). 2676 bytes result sent to driver16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 3.0 in stage 16.0 (TID 421, localhost, partition 3,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 4.0 in stage 16.0 (TID 422, localhost, partition 4,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 2.0 in stage 16.0 (TID 420) in 36 ms on localhost (2/200)16/02/06 15:51:47 INFO executor.Executor: Running task 4.0 in stage 16.0 (TID 422)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 1.0 in stage 16.0 (TID 419) in 78 ms on localhost (3/200)16/02/06 15:51:47 INFO executor.Executor: Running task 3.0 in stage 16.0 (TID 421)16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:47 INFO executor.Executor: Finished task 4.0 in stage 16.0 (TID 422). 2676 bytes result sent to driver16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 5.0 in stage 16.0 (TID 423, localhost, partition 5,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO executor.Executor: Running task 5.0 in stage 16.0 (TID 423)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 4.0 in stage 16.0 (TID 422) in 35 ms on localhost (4/200)16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:47 INFO executor.Executor: Finished task 3.0 in stage 16.0 (TID 421). 2676 bytes result sent to driver16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 6.0 in stage 16.0 (TID 424, localhost, partition 6,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO executor.Executor: Running task 6.0 in stage 16.0 (TID 424)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 3.0 in stage 16.0 (TID 421) in 62 ms on localhost (5/200)16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 9 ms16/02/06 15:51:47 INFO executor.Executor: Finished task 5.0 in stage 16.0 (TID 423). 2676 bytes result sent to driver16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 7.0 in stage 16.0 (TID 425, localhost, partition 7,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO executor.Executor: Running task 7.0 in stage 16.0 (TID 425)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 5.0 in stage 16.0 (TID 423) in 44 ms on localhost (6/200)16/02/06 15:51:47 INFO executor.Executor: Finished task 6.0 in stage 16.0 (TID 424). 2676 bytes result sent to driver16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 8.0 in stage 16.0 (TID 426, localhost, partition 8,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO executor.Executor: Running task 8.0 in stage 16.0 (TID 426)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 6.0 in stage 16.0 (TID 424) in 34 ms on localhost (7/200)16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 8 ms16/02/06 15:51:47 INFO executor.Executor: Finished task 7.0 in stage 16.0 (TID 425). 2676 bytes result sent to driver16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 9.0 in stage 16.0 (TID 427, localhost, partition 9,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO executor.Executor: Finished task 8.0 in stage 16.0 (TID 426). 2676 bytes result sent to driver16/02/06 15:51:47 INFO executor.Executor: Running task 9.0 in stage 16.0 (TID 427)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 7.0 in stage 16.0 (TID 425) in 53 ms on localhost (8/200)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 10.0 in stage 16.0 (TID 428, localhost, partition 10,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 8.0 in stage 16.0 (TID 426) in 59 ms on localhost (9/200)16/02/06 15:51:47 INFO executor.Executor: Running task 10.0 in stage 16.0 (TID 428)16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 11 ms16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:47 INFO executor.Executor: Finished task 9.0 in stage 16.0 (TID 427). 2676 bytes result sent to driver16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 11.0 in stage 16.0 (TID 429, localhost, partition 11,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO executor.Executor: Running task 11.0 in stage 16.0 (TID 429)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 9.0 in stage 16.0 (TID 427) in 54 ms on localhost (10/200)16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:47 INFO executor.Executor: Finished task 10.0 in stage 16.0 (TID 428). 2676 bytes result sent to driver16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 12.0 in stage 16.0 (TID 430, localhost, partition 12,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 10.0 in stage 16.0 (TID 428) in 56 ms on localhost (11/200)16/02/06 15:51:47 INFO executor.Executor: Running task 12.0 in stage 16.0 (TID 430)16/02/06 15:51:47 INFO executor.Executor: Finished task 11.0 in stage 16.0 (TID 429). 2676 bytes result sent to driver16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 13.0 in stage 16.0 (TID 431, localhost, partition 13,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO executor.Executor: Running task 13.0 in stage 16.0 (TID 431)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 11.0 in stage 16.0 (TID 429) in 51 ms on localhost (12/200)16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms16/02/06 15:51:47 INFO executor.Executor: Finished task 13.0 in stage 16.0 (TID 431). 2676 bytes result sent to driver16/02/06 15:51:47 INFO executor.Executor: Finished task 12.0 in stage 16.0 (TID 430). 2676 bytes result sent to driver16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 14.0 in stage 16.0 (TID 432, localhost, partition 14,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO executor.Executor: Running task 14.0 in stage 16.0 (TID 432)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 13.0 in stage 16.0 (TID 431) in 23 ms on localhost (13/200)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 15.0 in stage 16.0 (TID 433, localhost, partition 15,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 12.0 in stage 16.0 (TID 430) in 60 ms on localhost (14/200)16/02/06 15:51:47 INFO executor.Executor: Running task 15.0 in stage 16.0 (TID 433)16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms16/02/06 15:51:47 INFO executor.Executor: Finished task 14.0 in stage 16.0 (TID 432). 2676 bytes result sent to driver16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 16.0 in stage 16.0 (TID 434, localhost, partition 16,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO executor.Executor: Running task 16.0 in stage 16.0 (TID 434)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 14.0 in stage 16.0 (TID 432) in 26 ms on localhost (15/200)16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 16 ms16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 12 ms16/02/06 15:51:47 INFO executor.Executor: Finished task 15.0 in stage 16.0 (TID 433). 2676 bytes result sent to driver16/02/06 15:51:47 INFO executor.Executor: Finished task 16.0 in stage 16.0 (TID 434). 2676 bytes result sent to driver16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 17.0 in stage 16.0 (TID 435, localhost, partition 17,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO executor.Executor: Running task 17.0 in stage 16.0 (TID 435)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 15.0 in stage 16.0 (TID 433) in 42 ms on localhost (16/200)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 18.0 in stage 16.0 (TID 436, localhost, partition 18,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 16.0 in stage 16.0 (TID 434) in 42 ms on localhost (17/200)16/02/06 15:51:47 INFO executor.Executor: Running task 18.0 in stage 16.0 (TID 436)16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms16/02/06 15:51:47 INFO executor.Executor: Finished task 17.0 in stage 16.0 (TID 435). 2676 bytes result sent to driver16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 4 ms16/02/06 15:51:47 INFO executor.Executor: Finished task 18.0 in stage 16.0 (TID 436). 2676 bytes result sent to driver16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 19.0 in stage 16.0 (TID 437, localhost, partition 19,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO executor.Executor: Running task 19.0 in stage 16.0 (TID 437)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 20.0 in stage 16.0 (TID 438, localhost, partition 20,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO executor.Executor: Running task 20.0 in stage 16.0 (TID 438)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 18.0 in stage 16.0 (TID 436) in 68 ms on localhost (18/200)16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 5 ms16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 17.0 in stage 16.0 (TID 435) in 88 ms on localhost (19/200)16/02/06 15:51:47 INFO executor.Executor: Finished task 20.0 in stage 16.0 (TID 438). 2676 bytes result sent to driver16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms16/02/06 15:51:47 INFO executor.Executor: Finished task 19.0 in stage 16.0 (TID 437). 2676 bytes result sent to driver16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 21.0 in stage 16.0 (TID 439, localhost, partition 21,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 22.0 in stage 16.0 (TID 440, localhost, partition 22,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 20.0 in stage 16.0 (TID 438) in 52 ms on localhost (20/200)16/02/06 15:51:47 INFO executor.Executor: Running task 22.0 in stage 16.0 (TID 440)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 19.0 in stage 16.0 (TID 437) in 57 ms on localhost (21/200)16/02/06 15:51:47 INFO executor.Executor: Running task 21.0 in stage 16.0 (TID 439)16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 5 ms16/02/06 15:51:47 INFO executor.Executor: Finished task 22.0 in stage 16.0 (TID 440). 2676 bytes result sent to driver16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 23.0 in stage 16.0 (TID 441, localhost, partition 23,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO executor.Executor: Running task 23.0 in stage 16.0 (TID 441)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 22.0 in stage 16.0 (TID 440) in 26 ms on localhost (22/200)16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 7 ms16/02/06 15:51:47 INFO executor.Executor: Finished task 21.0 in stage 16.0 (TID 439). 2676 bytes result sent to driver16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 24.0 in stage 16.0 (TID 442, localhost, partition 24,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 21.0 in stage 16.0 (TID 439) in 39 ms on localhost (23/200)16/02/06 15:51:47 INFO executor.Executor: Running task 24.0 in stage 16.0 (TID 442)16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 17 ms16/02/06 15:51:47 INFO executor.Executor: Finished task 23.0 in stage 16.0 (TID 441). 2676 bytes result sent to driver16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 25.0 in stage 16.0 (TID 443, localhost, partition 25,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 23.0 in stage 16.0 (TID 441) in 35 ms on localhost (24/200)16/02/06 15:51:47 INFO executor.Executor: Running task 25.0 in stage 16.0 (TID 443)16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 21 ms16/02/06 15:51:47 INFO executor.Executor: Finished task 24.0 in stage 16.0 (TID 442). 2676 bytes result sent to driver16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 26.0 in stage 16.0 (TID 444, localhost, partition 26,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 24.0 in stage 16.0 (TID 442) in 47 ms on localhost (25/200)16/02/06 15:51:47 INFO executor.Executor: Running task 26.0 in stage 16.0 (TID 444)16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 14 ms16/02/06 15:51:47 INFO executor.Executor: Finished task 25.0 in stage 16.0 (TID 443). 2676 bytes result sent to driver16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 27.0 in stage 16.0 (TID 445, localhost, partition 27,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO executor.Executor: Running task 27.0 in stage 16.0 (TID 445)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 25.0 in stage 16.0 (TID 443) in 49 ms on localhost (26/200)16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 12 ms16/02/06 15:51:47 INFO executor.Executor: Finished task 26.0 in stage 16.0 (TID 444). 2676 bytes result sent to driver16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 28.0 in stage 16.0 (TID 446, localhost, partition 28,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 26.0 in stage 16.0 (TID 444) in 38 ms on localhost (27/200)16/02/06 15:51:47 INFO executor.Executor: Running task 28.0 in stage 16.0 (TID 446)16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 10 ms16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms16/02/06 15:51:47 INFO executor.Executor: Finished task 27.0 in stage 16.0 (TID 445). 2676 bytes result sent to driver16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 29.0 in stage 16.0 (TID 447, localhost, partition 29,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 27.0 in stage 16.0 (TID 445) in 40 ms on localhost (28/200)16/02/06 15:51:47 INFO executor.Executor: Running task 29.0 in stage 16.0 (TID 447)16/02/06 15:51:47 INFO executor.Executor: Finished task 28.0 in stage 16.0 (TID 446). 2676 bytes result sent to driver16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 30.0 in stage 16.0 (TID 448, localhost, partition 30,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 28.0 in stage 16.0 (TID 446) in 42 ms on localhost (29/200)16/02/06 15:51:47 INFO executor.Executor: Running task 30.0 in stage 16.0 (TID 448)16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 9 ms16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 11 ms16/02/06 15:51:47 INFO executor.Executor: Finished task 29.0 in stage 16.0 (TID 447). 2676 bytes result sent to driver16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 31.0 in stage 16.0 (TID 449, localhost, partition 31,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO executor.Executor: Running task 31.0 in stage 16.0 (TID 449)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 29.0 in stage 16.0 (TID 447) in 51 ms on localhost (30/200)16/02/06 15:51:47 INFO executor.Executor: Finished task 30.0 in stage 16.0 (TID 448). 2676 bytes result sent to driver16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 32.0 in stage 16.0 (TID 450, localhost, partition 32,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO executor.Executor: Running task 32.0 in stage 16.0 (TID 450)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 30.0 in stage 16.0 (TID 448) in 42 ms on localhost (31/200)16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms16/02/06 15:51:47 INFO executor.Executor: Finished task 31.0 in stage 16.0 (TID 449). 2676 bytes result sent to driver16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 33.0 in stage 16.0 (TID 451, localhost, partition 33,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO executor.Executor: Running task 33.0 in stage 16.0 (TID 451)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 31.0 in stage 16.0 (TID 449) in 33 ms on localhost (32/200)16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 5 ms16/02/06 15:51:47 INFO executor.Executor: Finished task 33.0 in stage 16.0 (TID 451). 2676 bytes result sent to driver16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 34.0 in stage 16.0 (TID 452, localhost, partition 34,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 33.0 in stage 16.0 (TID 451) in 25 ms on localhost (33/200)16/02/06 15:51:47 INFO executor.Executor: Running task 34.0 in stage 16.0 (TID 452)16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 9 ms16/02/06 15:51:47 INFO executor.Executor: Finished task 32.0 in stage 16.0 (TID 450). 2676 bytes result sent to driver16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 35.0 in stage 16.0 (TID 453, localhost, partition 35,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO executor.Executor: Running task 35.0 in stage 16.0 (TID 453)16/02/06 15:51:47 INFO executor.Executor: Finished task 34.0 in stage 16.0 (TID 452). 2676 bytes result sent to driver16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 32.0 in stage 16.0 (TID 450) in 72 ms on localhost (34/200)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 36.0 in stage 16.0 (TID 454, localhost, partition 36,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 34.0 in stage 16.0 (TID 452) in 36 ms on localhost (35/200)16/02/06 15:51:47 INFO executor.Executor: Running task 36.0 in stage 16.0 (TID 454)16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms16/02/06 15:51:47 INFO executor.Executor: Finished task 35.0 in stage 16.0 (TID 453). 2676 bytes result sent to driver16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 37.0 in stage 16.0 (TID 455, localhost, partition 37,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO executor.Executor: Running task 37.0 in stage 16.0 (TID 455)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 35.0 in stage 16.0 (TID 453) in 43 ms on localhost (36/200)16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:47 INFO executor.Executor: Finished task 37.0 in stage 16.0 (TID 455). 2676 bytes result sent to driver16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 38.0 in stage 16.0 (TID 456, localhost, partition 38,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 37.0 in stage 16.0 (TID 455) in 17 ms on localhost (37/200)16/02/06 15:51:47 INFO executor.Executor: Running task 38.0 in stage 16.0 (TID 456)16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 8 ms16/02/06 15:51:47 INFO executor.Executor: Finished task 38.0 in stage 16.0 (TID 456). 2676 bytes result sent to driver16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 39.0 in stage 16.0 (TID 457, localhost, partition 39,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO executor.Executor: Running task 39.0 in stage 16.0 (TID 457)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 38.0 in stage 16.0 (TID 456) in 25 ms on localhost (38/200)16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 10 ms16/02/06 15:51:47 INFO executor.Executor: Finished task 36.0 in stage 16.0 (TID 454). 2676 bytes result sent to driver16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 40.0 in stage 16.0 (TID 458, localhost, partition 40,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO executor.Executor: Running task 40.0 in stage 16.0 (TID 458)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 36.0 in stage 16.0 (TID 454) in 81 ms on localhost (39/200)16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 8 ms16/02/06 15:51:47 INFO executor.Executor: Finished task 39.0 in stage 16.0 (TID 457). 2676 bytes result sent to driver16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 41.0 in stage 16.0 (TID 459, localhost, partition 41,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO executor.Executor: Running task 41.0 in stage 16.0 (TID 459)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 39.0 in stage 16.0 (TID 457) in 34 ms on localhost (40/200)16/02/06 15:51:47 INFO executor.Executor: Finished task 40.0 in stage 16.0 (TID 458). 2676 bytes result sent to driver16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 42.0 in stage 16.0 (TID 460, localhost, partition 42,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO executor.Executor: Running task 42.0 in stage 16.0 (TID 460)16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 40.0 in stage 16.0 (TID 458) in 44 ms on localhost (41/200)16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 10 ms16/02/06 15:51:47 INFO executor.Executor: Finished task 42.0 in stage 16.0 (TID 460). 2676 bytes result sent to driver16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 10 ms16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 43.0 in stage 16.0 (TID 461, localhost, partition 43,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 42.0 in stage 16.0 (TID 460) in 37 ms on localhost (42/200)16/02/06 15:51:47 INFO executor.Executor: Running task 43.0 in stage 16.0 (TID 461)16/02/06 15:51:47 INFO executor.Executor: Finished task 41.0 in stage 16.0 (TID 459). 2676 bytes result sent to driver16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 44.0 in stage 16.0 (TID 462, localhost, partition 44,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO executor.Executor: Running task 44.0 in stage 16.0 (TID 462)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 41.0 in stage 16.0 (TID 459) in 73 ms on localhost (43/200)16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:47 INFO executor.Executor: Finished task 44.0 in stage 16.0 (TID 462). 2676 bytes result sent to driver16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 45.0 in stage 16.0 (TID 463, localhost, partition 45,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO executor.Executor: Running task 45.0 in stage 16.0 (TID 463)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 44.0 in stage 16.0 (TID 462) in 13 ms on localhost (44/200)16/02/06 15:51:47 INFO executor.Executor: Finished task 43.0 in stage 16.0 (TID 461). 2676 bytes result sent to driver16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 46.0 in stage 16.0 (TID 464, localhost, partition 46,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO executor.Executor: Running task 46.0 in stage 16.0 (TID 464)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 43.0 in stage 16.0 (TID 461) in 56 ms on localhost (45/200)16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 23 ms16/02/06 15:51:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms16/02/06 15:51:47 INFO executor.Executor: Finished task 45.0 in stage 16.0 (TID 463). 2676 bytes result sent to driver16/02/06 15:51:47 INFO executor.Executor: Finished task 46.0 in stage 16.0 (TID 464). 2676 bytes result sent to driver16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 47.0 in stage 16.0 (TID 465, localhost, partition 47,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Starting task 48.0 in stage 16.0 (TID 466, localhost, partition 48,NODE_LOCAL, 1999 bytes)16/02/06 15:51:47 INFO executor.Executor: Running task 48.0 in stage 16.0 (TID 466)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 45.0 in stage 16.0 (TID 463) in 52 ms on localhost (46/200)16/02/06 15:51:47 INFO executor.Executor: Running task 47.0 in stage 16.0 (TID 465)16/02/06 15:51:47 INFO scheduler.TaskSetManager: Finished task 46.0 in stage 16.0 (TID 464) in 42 ms on localhost (47/200)16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms16/02/06 15:51:48 INFO executor.Executor: Finished task 48.0 in stage 16.0 (TID 466). 2676 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 49.0 in stage 16.0 (TID 467, localhost, partition 49,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO executor.Executor: Running task 49.0 in stage 16.0 (TID 467)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 48.0 in stage 16.0 (TID 466) in 35 ms on localhost (48/200)16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms16/02/06 15:51:48 INFO executor.Executor: Finished task 47.0 in stage 16.0 (TID 465). 2676 bytes result sent to driver16/02/06 15:51:48 INFO executor.Executor: Finished task 49.0 in stage 16.0 (TID 467). 2676 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 50.0 in stage 16.0 (TID 468, localhost, partition 50,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO executor.Executor: Running task 50.0 in stage 16.0 (TID 468)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 47.0 in stage 16.0 (TID 465) in 48 ms on localhost (49/200)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 51.0 in stage 16.0 (TID 469, localhost, partition 51,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO executor.Executor: Running task 51.0 in stage 16.0 (TID 469)16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 11 ms16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 49.0 in stage 16.0 (TID 467) in 31 ms on localhost (50/200)16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms16/02/06 15:51:48 INFO executor.Executor: Finished task 50.0 in stage 16.0 (TID 468). 2676 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 52.0 in stage 16.0 (TID 470, localhost, partition 52,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO executor.Executor: Running task 52.0 in stage 16.0 (TID 470)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 50.0 in stage 16.0 (TID 468) in 43 ms on localhost (51/200)16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 9 ms16/02/06 15:51:48 INFO executor.Executor: Finished task 52.0 in stage 16.0 (TID 470). 2676 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 53.0 in stage 16.0 (TID 471, localhost, partition 53,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 52.0 in stage 16.0 (TID 470) in 31 ms on localhost (52/200)16/02/06 15:51:48 INFO executor.Executor: Running task 53.0 in stage 16.0 (TID 471)16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms16/02/06 15:51:48 INFO executor.Executor: Finished task 51.0 in stage 16.0 (TID 469). 2987 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 54.0 in stage 16.0 (TID 472, localhost, partition 54,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO executor.Executor: Running task 54.0 in stage 16.0 (TID 472)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 51.0 in stage 16.0 (TID 469) in 73 ms on localhost (53/200)16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:48 INFO executor.Executor: Finished task 54.0 in stage 16.0 (TID 472). 2676 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 55.0 in stage 16.0 (TID 473, localhost, partition 55,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO executor.Executor: Finished task 53.0 in stage 16.0 (TID 471). 2676 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 56.0 in stage 16.0 (TID 474, localhost, partition 56,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 54.0 in stage 16.0 (TID 472) in 48 ms on localhost (54/200)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 53.0 in stage 16.0 (TID 471) in 71 ms on localhost (55/200)16/02/06 15:51:48 INFO executor.Executor: Running task 55.0 in stage 16.0 (TID 473)16/02/06 15:51:48 INFO executor.Executor: Running task 56.0 in stage 16.0 (TID 474)16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms16/02/06 15:51:48 INFO executor.Executor: Finished task 56.0 in stage 16.0 (TID 474). 2676 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 57.0 in stage 16.0 (TID 475, localhost, partition 57,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 56.0 in stage 16.0 (TID 474) in 44 ms on localhost (56/200)16/02/06 15:51:48 INFO executor.Executor: Running task 57.0 in stage 16.0 (TID 475)16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:48 INFO executor.Executor: Finished task 57.0 in stage 16.0 (TID 475). 2676 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 58.0 in stage 16.0 (TID 476, localhost, partition 58,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO executor.Executor: Running task 58.0 in stage 16.0 (TID 476)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 57.0 in stage 16.0 (TID 475) in 42 ms on localhost (57/200)16/02/06 15:51:48 INFO executor.Executor: Finished task 55.0 in stage 16.0 (TID 473). 2983 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 59.0 in stage 16.0 (TID 477, localhost, partition 59,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 55.0 in stage 16.0 (TID 473) in 101 ms on localhost (58/200)16/02/06 15:51:48 INFO executor.Executor: Running task 59.0 in stage 16.0 (TID 477)16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms16/02/06 15:51:48 INFO executor.Executor: Finished task 58.0 in stage 16.0 (TID 476). 2676 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 60.0 in stage 16.0 (TID 478, localhost, partition 60,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO executor.Executor: Running task 60.0 in stage 16.0 (TID 478)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 58.0 in stage 16.0 (TID 476) in 56 ms on localhost (59/200)16/02/06 15:51:48 INFO executor.Executor: Finished task 59.0 in stage 16.0 (TID 477). 2676 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 61.0 in stage 16.0 (TID 479, localhost, partition 61,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:48 INFO executor.Executor: Running task 61.0 in stage 16.0 (TID 479)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 59.0 in stage 16.0 (TID 477) in 39 ms on localhost (60/200)16/02/06 15:51:48 INFO executor.Executor: Finished task 60.0 in stage 16.0 (TID 478). 2676 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 62.0 in stage 16.0 (TID 480, localhost, partition 62,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO executor.Executor: Running task 62.0 in stage 16.0 (TID 480)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 60.0 in stage 16.0 (TID 478) in 15 ms on localhost (61/200)16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 20 ms16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms16/02/06 15:51:48 INFO executor.Executor: Finished task 61.0 in stage 16.0 (TID 479). 2676 bytes result sent to driver16/02/06 15:51:48 INFO executor.Executor: Finished task 62.0 in stage 16.0 (TID 480). 2676 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 63.0 in stage 16.0 (TID 481, localhost, partition 63,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO executor.Executor: Running task 63.0 in stage 16.0 (TID 481)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 64.0 in stage 16.0 (TID 482, localhost, partition 64,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO executor.Executor: Running task 64.0 in stage 16.0 (TID 482)16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 5 ms16/02/06 15:51:48 INFO executor.Executor: Finished task 63.0 in stage 16.0 (TID 481). 2676 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 65.0 in stage 16.0 (TID 483, localhost, partition 65,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO executor.Executor: Running task 65.0 in stage 16.0 (TID 483)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 63.0 in stage 16.0 (TID 481) in 24 ms on localhost (62/200)16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 5 ms16/02/06 15:51:48 INFO executor.Executor: Finished task 64.0 in stage 16.0 (TID 482). 2676 bytes result sent to driver16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 62.0 in stage 16.0 (TID 480) in 72 ms on localhost (63/200)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 66.0 in stage 16.0 (TID 484, localhost, partition 66,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO executor.Executor: Running task 66.0 in stage 16.0 (TID 484)16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 5 ms16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 61.0 in stage 16.0 (TID 479) in 88 ms on localhost (64/200)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 64.0 in stage 16.0 (TID 482) in 41 ms on localhost (65/200)16/02/06 15:51:48 INFO executor.Executor: Finished task 65.0 in stage 16.0 (TID 483). 2676 bytes result sent to driver16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:48 INFO executor.Executor: Finished task 66.0 in stage 16.0 (TID 484). 2676 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 67.0 in stage 16.0 (TID 485, localhost, partition 67,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO executor.Executor: Running task 67.0 in stage 16.0 (TID 485)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 65.0 in stage 16.0 (TID 483) in 40 ms on localhost (66/200)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 68.0 in stage 16.0 (TID 486, localhost, partition 68,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO executor.Executor: Running task 68.0 in stage 16.0 (TID 486)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 66.0 in stage 16.0 (TID 484) in 27 ms on localhost (67/200)16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms16/02/06 15:51:48 INFO executor.Executor: Finished task 68.0 in stage 16.0 (TID 486). 2676 bytes result sent to driver16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms16/02/06 15:51:48 INFO executor.Executor: Finished task 67.0 in stage 16.0 (TID 485). 2676 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 69.0 in stage 16.0 (TID 487, localhost, partition 69,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO executor.Executor: Running task 69.0 in stage 16.0 (TID 487)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 70.0 in stage 16.0 (TID 488, localhost, partition 70,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 68.0 in stage 16.0 (TID 486) in 22 ms on localhost (68/200)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 67.0 in stage 16.0 (TID 485) in 31 ms on localhost (69/200)16/02/06 15:51:48 INFO executor.Executor: Running task 70.0 in stage 16.0 (TID 488)16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms16/02/06 15:51:48 INFO executor.Executor: Finished task 70.0 in stage 16.0 (TID 488). 2676 bytes result sent to driver16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms16/02/06 15:51:48 INFO executor.Executor: Finished task 69.0 in stage 16.0 (TID 487). 2676 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 71.0 in stage 16.0 (TID 489, localhost, partition 71,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO executor.Executor: Running task 71.0 in stage 16.0 (TID 489)16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:48 INFO executor.Executor: Finished task 71.0 in stage 16.0 (TID 489). 2676 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 69.0 in stage 16.0 (TID 487) in 35 ms on localhost (70/200)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 70.0 in stage 16.0 (TID 488) in 48 ms on localhost (71/200)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 72.0 in stage 16.0 (TID 490, localhost, partition 72,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 73.0 in stage 16.0 (TID 491, localhost, partition 73,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO executor.Executor: Running task 73.0 in stage 16.0 (TID 491)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 71.0 in stage 16.0 (TID 489) in 27 ms on localhost (72/200)16/02/06 15:51:48 INFO executor.Executor: Running task 72.0 in stage 16.0 (TID 490)16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:48 INFO executor.Executor: Finished task 72.0 in stage 16.0 (TID 490). 2676 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 74.0 in stage 16.0 (TID 492, localhost, partition 74,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO executor.Executor: Running task 74.0 in stage 16.0 (TID 492)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 72.0 in stage 16.0 (TID 490) in 28 ms on localhost (73/200)16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:48 INFO executor.Executor: Finished task 73.0 in stage 16.0 (TID 491). 2676 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 75.0 in stage 16.0 (TID 493, localhost, partition 75,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO executor.Executor: Running task 75.0 in stage 16.0 (TID 493)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 73.0 in stage 16.0 (TID 491) in 37 ms on localhost (74/200)16/02/06 15:51:48 INFO executor.Executor: Finished task 74.0 in stage 16.0 (TID 492). 2676 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 76.0 in stage 16.0 (TID 494, localhost, partition 76,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO executor.Executor: Running task 76.0 in stage 16.0 (TID 494)16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 7 ms16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 74.0 in stage 16.0 (TID 492) in 59 ms on localhost (75/200)16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 36 ms16/02/06 15:51:48 INFO executor.Executor: Finished task 75.0 in stage 16.0 (TID 493). 2676 bytes result sent to driver16/02/06 15:51:48 INFO executor.Executor: Finished task 76.0 in stage 16.0 (TID 494). 2676 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 77.0 in stage 16.0 (TID 495, localhost, partition 77,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 78.0 in stage 16.0 (TID 496, localhost, partition 78,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 76.0 in stage 16.0 (TID 494) in 47 ms on localhost (76/200)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 75.0 in stage 16.0 (TID 493) in 95 ms on localhost (77/200)16/02/06 15:51:48 INFO executor.Executor: Running task 78.0 in stage 16.0 (TID 496)16/02/06 15:51:48 INFO executor.Executor: Running task 77.0 in stage 16.0 (TID 495)16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 4 ms16/02/06 15:51:48 INFO executor.Executor: Finished task 77.0 in stage 16.0 (TID 495). 2676 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 79.0 in stage 16.0 (TID 497, localhost, partition 79,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO executor.Executor: Running task 79.0 in stage 16.0 (TID 497)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 77.0 in stage 16.0 (TID 495) in 31 ms on localhost (78/200)16/02/06 15:51:48 INFO executor.Executor: Finished task 78.0 in stage 16.0 (TID 496). 2676 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 80.0 in stage 16.0 (TID 498, localhost, partition 80,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 78.0 in stage 16.0 (TID 496) in 42 ms on localhost (79/200)16/02/06 15:51:48 INFO executor.Executor: Running task 80.0 in stage 16.0 (TID 498)16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 4 ms16/02/06 15:51:48 INFO executor.Executor: Finished task 79.0 in stage 16.0 (TID 497). 2676 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 81.0 in stage 16.0 (TID 499, localhost, partition 81,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO executor.Executor: Running task 81.0 in stage 16.0 (TID 499)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 79.0 in stage 16.0 (TID 497) in 44 ms on localhost (80/200)16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:48 INFO executor.Executor: Finished task 80.0 in stage 16.0 (TID 498). 2676 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 82.0 in stage 16.0 (TID 500, localhost, partition 82,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 80.0 in stage 16.0 (TID 498) in 47 ms on localhost (81/200)16/02/06 15:51:48 INFO executor.Executor: Running task 82.0 in stage 16.0 (TID 500)16/02/06 15:51:48 INFO executor.Executor: Finished task 81.0 in stage 16.0 (TID 499). 2676 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 83.0 in stage 16.0 (TID 501, localhost, partition 83,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO executor.Executor: Running task 83.0 in stage 16.0 (TID 501)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 81.0 in stage 16.0 (TID 499) in 36 ms on localhost (82/200)16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 31 ms16/02/06 15:51:48 INFO executor.Executor: Finished task 82.0 in stage 16.0 (TID 500). 2676 bytes result sent to driver16/02/06 15:51:48 INFO executor.Executor: Finished task 83.0 in stage 16.0 (TID 501). 2676 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 84.0 in stage 16.0 (TID 502, localhost, partition 84,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO executor.Executor: Running task 84.0 in stage 16.0 (TID 502)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 85.0 in stage 16.0 (TID 503, localhost, partition 85,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO executor.Executor: Running task 85.0 in stage 16.0 (TID 503)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 82.0 in stage 16.0 (TID 500) in 59 ms on localhost (83/200)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 83.0 in stage 16.0 (TID 501) in 66 ms on localhost (84/200)16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 5 ms16/02/06 15:51:48 INFO executor.Executor: Finished task 85.0 in stage 16.0 (TID 503). 2676 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 86.0 in stage 16.0 (TID 504, localhost, partition 86,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 85.0 in stage 16.0 (TID 503) in 36 ms on localhost (85/200)16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 7 ms16/02/06 15:51:48 INFO executor.Executor: Finished task 84.0 in stage 16.0 (TID 502). 2676 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 87.0 in stage 16.0 (TID 505, localhost, partition 87,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO executor.Executor: Running task 87.0 in stage 16.0 (TID 505)16/02/06 15:51:48 INFO executor.Executor: Running task 86.0 in stage 16.0 (TID 504)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 84.0 in stage 16.0 (TID 502) in 46 ms on localhost (86/200)16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 10 ms16/02/06 15:51:48 INFO executor.Executor: Finished task 87.0 in stage 16.0 (TID 505). 2676 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 88.0 in stage 16.0 (TID 506, localhost, partition 88,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO executor.Executor: Running task 88.0 in stage 16.0 (TID 506)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 87.0 in stage 16.0 (TID 505) in 36 ms on localhost (87/200)16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 10 ms16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 22 ms16/02/06 15:51:48 INFO executor.Executor: Finished task 86.0 in stage 16.0 (TID 504). 2676 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 89.0 in stage 16.0 (TID 507, localhost, partition 89,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO executor.Executor: Running task 89.0 in stage 16.0 (TID 507)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 86.0 in stage 16.0 (TID 504) in 80 ms on localhost (88/200)16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO executor.Executor: Finished task 88.0 in stage 16.0 (TID 506). 2676 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 90.0 in stage 16.0 (TID 508, localhost, partition 90,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 88.0 in stage 16.0 (TID 506) in 60 ms on localhost (89/200)16/02/06 15:51:48 INFO executor.Executor: Running task 90.0 in stage 16.0 (TID 508)16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 37 ms16/02/06 15:51:48 INFO executor.Executor: Finished task 89.0 in stage 16.0 (TID 507). 2676 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 91.0 in stage 16.0 (TID 509, localhost, partition 91,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO executor.Executor: Running task 91.0 in stage 16.0 (TID 509)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 89.0 in stage 16.0 (TID 507) in 63 ms on localhost (90/200)16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 13 ms16/02/06 15:51:48 INFO executor.Executor: Finished task 90.0 in stage 16.0 (TID 508). 2676 bytes result sent to driver16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 7 ms16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 92.0 in stage 16.0 (TID 510, localhost, partition 92,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO executor.Executor: Running task 92.0 in stage 16.0 (TID 510)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 90.0 in stage 16.0 (TID 508) in 58 ms on localhost (91/200)16/02/06 15:51:48 INFO executor.Executor: Finished task 91.0 in stage 16.0 (TID 509). 2676 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 93.0 in stage 16.0 (TID 511, localhost, partition 93,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO executor.Executor: Running task 93.0 in stage 16.0 (TID 511)16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms16/02/06 15:51:48 INFO executor.Executor: Finished task 93.0 in stage 16.0 (TID 511). 2676 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 94.0 in stage 16.0 (TID 512, localhost, partition 94,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 93.0 in stage 16.0 (TID 511) in 14 ms on localhost (92/200)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 91.0 in stage 16.0 (TID 509) in 52 ms on localhost (93/200)16/02/06 15:51:48 INFO executor.Executor: Running task 94.0 in stage 16.0 (TID 512)16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms16/02/06 15:51:48 INFO executor.Executor: Finished task 92.0 in stage 16.0 (TID 510). 2676 bytes result sent to driver16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 95.0 in stage 16.0 (TID 513, localhost, partition 95,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO executor.Executor: Running task 95.0 in stage 16.0 (TID 513)16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 92.0 in stage 16.0 (TID 510) in 53 ms on localhost (94/200)16/02/06 15:51:48 INFO executor.Executor: Finished task 94.0 in stage 16.0 (TID 512). 2676 bytes result sent to driver16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:48 INFO scheduler.TaskSetManager: Starting task 96.0 in stage 16.0 (TID 514, localhost, partition 96,NODE_LOCAL, 1999 bytes)16/02/06 15:51:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 11 ms16/02/06 15:51:48 INFO scheduler.TaskSetManager: Finished task 94.0 in stage 16.0 (TID 512) in 43 ms on localhost (95/200)16/02/06 15:51:48 INFO executor.Executor: Running task 96.0 in stage 16.0 (TID 514)16/02/06 15:51:48 INFO executor.Executor: Finished task 95.0 in stage 16.0 (TID 513). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 97.0 in stage 16.0 (TID 515, localhost, partition 97,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 95.0 in stage 16.0 (TID 513) in 39 ms on localhost (96/200)16/02/06 15:51:49 INFO executor.Executor: Running task 97.0 in stage 16.0 (TID 515)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms16/02/06 15:51:49 INFO executor.Executor: Finished task 97.0 in stage 16.0 (TID 515). 2676 bytes result sent to driver16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms16/02/06 15:51:49 INFO executor.Executor: Finished task 96.0 in stage 16.0 (TID 514). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 98.0 in stage 16.0 (TID 516, localhost, partition 98,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 97.0 in stage 16.0 (TID 515) in 32 ms on localhost (97/200)16/02/06 15:51:49 INFO executor.Executor: Running task 98.0 in stage 16.0 (TID 516)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 99.0 in stage 16.0 (TID 517, localhost, partition 99,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO executor.Executor: Running task 99.0 in stage 16.0 (TID 517)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 8 ms16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 96.0 in stage 16.0 (TID 514) in 77 ms on localhost (98/200)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 7 ms16/02/06 15:51:49 INFO executor.Executor: Finished task 99.0 in stage 16.0 (TID 517). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 100.0 in stage 16.0 (TID 518, localhost, partition 100,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO executor.Executor: Running task 100.0 in stage 16.0 (TID 518)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 99.0 in stage 16.0 (TID 517) in 44 ms on localhost (99/200)16/02/06 15:51:49 INFO executor.Executor: Finished task 98.0 in stage 16.0 (TID 516). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 101.0 in stage 16.0 (TID 519, localhost, partition 101,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO executor.Executor: Running task 101.0 in stage 16.0 (TID 519)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 98.0 in stage 16.0 (TID 516) in 68 ms on localhost (100/200)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms16/02/06 15:51:49 INFO executor.Executor: Finished task 100.0 in stage 16.0 (TID 518). 2676 bytes result sent to driver16/02/06 15:51:49 INFO executor.Executor: Finished task 101.0 in stage 16.0 (TID 519). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 102.0 in stage 16.0 (TID 520, localhost, partition 102,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 100.0 in stage 16.0 (TID 518) in 47 ms on localhost (101/200)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 103.0 in stage 16.0 (TID 521, localhost, partition 103,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO executor.Executor: Running task 102.0 in stage 16.0 (TID 520)16/02/06 15:51:49 INFO executor.Executor: Running task 103.0 in stage 16.0 (TID 521)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 101.0 in stage 16.0 (TID 519) in 46 ms on localhost (102/200)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms16/02/06 15:51:49 INFO executor.Executor: Finished task 103.0 in stage 16.0 (TID 521). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 104.0 in stage 16.0 (TID 522, localhost, partition 104,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 103.0 in stage 16.0 (TID 521) in 29 ms on localhost (103/200)16/02/06 15:51:49 INFO executor.Executor: Running task 104.0 in stage 16.0 (TID 522)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO executor.Executor: Finished task 102.0 in stage 16.0 (TID 520). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 105.0 in stage 16.0 (TID 523, localhost, partition 105,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO executor.Executor: Running task 105.0 in stage 16.0 (TID 523)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 102.0 in stage 16.0 (TID 520) in 84 ms on localhost (104/200)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 37 ms16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 15 ms16/02/06 15:51:49 INFO executor.Executor: Finished task 104.0 in stage 16.0 (TID 522). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 106.0 in stage 16.0 (TID 524, localhost, partition 106,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 104.0 in stage 16.0 (TID 522) in 77 ms on localhost (105/200)16/02/06 15:51:49 INFO executor.Executor: Running task 106.0 in stage 16.0 (TID 524)16/02/06 15:51:49 INFO executor.Executor: Finished task 105.0 in stage 16.0 (TID 523). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 107.0 in stage 16.0 (TID 525, localhost, partition 107,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO executor.Executor: Running task 107.0 in stage 16.0 (TID 525)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 105.0 in stage 16.0 (TID 523) in 57 ms on localhost (106/200)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms16/02/06 15:51:49 INFO executor.Executor: Finished task 107.0 in stage 16.0 (TID 525). 2676 bytes result sent to driver16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 108.0 in stage 16.0 (TID 526, localhost, partition 108,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO executor.Executor: Running task 108.0 in stage 16.0 (TID 526)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 107.0 in stage 16.0 (TID 525) in 47 ms on localhost (107/200)16/02/06 15:51:49 INFO executor.Executor: Finished task 106.0 in stage 16.0 (TID 524). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 109.0 in stage 16.0 (TID 527, localhost, partition 109,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 106.0 in stage 16.0 (TID 524) in 71 ms on localhost (108/200)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 7 ms16/02/06 15:51:49 INFO executor.Executor: Running task 109.0 in stage 16.0 (TID 527)16/02/06 15:51:49 INFO executor.Executor: Finished task 108.0 in stage 16.0 (TID 526). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 110.0 in stage 16.0 (TID 528, localhost, partition 110,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO executor.Executor: Running task 110.0 in stage 16.0 (TID 528)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 108.0 in stage 16.0 (TID 526) in 54 ms on localhost (109/200)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 10 ms16/02/06 15:51:49 INFO executor.Executor: Finished task 110.0 in stage 16.0 (TID 528). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 111.0 in stage 16.0 (TID 529, localhost, partition 111,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO executor.Executor: Running task 111.0 in stage 16.0 (TID 529)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 110.0 in stage 16.0 (TID 528) in 43 ms on localhost (110/200)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 5 ms16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 23 ms16/02/06 15:51:49 INFO executor.Executor: Finished task 109.0 in stage 16.0 (TID 527). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 112.0 in stage 16.0 (TID 530, localhost, partition 112,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 109.0 in stage 16.0 (TID 527) in 109 ms on localhost (111/200)16/02/06 15:51:49 INFO executor.Executor: Running task 112.0 in stage 16.0 (TID 530)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:49 INFO executor.Executor: Finished task 112.0 in stage 16.0 (TID 530). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 113.0 in stage 16.0 (TID 531, localhost, partition 113,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO executor.Executor: Running task 113.0 in stage 16.0 (TID 531)16/02/06 15:51:49 INFO executor.Executor: Finished task 111.0 in stage 16.0 (TID 529). 2676 bytes result sent to driver16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 114.0 in stage 16.0 (TID 532, localhost, partition 114,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 112.0 in stage 16.0 (TID 530) in 36 ms on localhost (112/200)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 111.0 in stage 16.0 (TID 529) in 65 ms on localhost (113/200)16/02/06 15:51:49 INFO executor.Executor: Running task 114.0 in stage 16.0 (TID 532)16/02/06 15:51:49 INFO executor.Executor: Finished task 113.0 in stage 16.0 (TID 531). 2676 bytes result sent to driver16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 7 ms16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 115.0 in stage 16.0 (TID 533, localhost, partition 115,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 113.0 in stage 16.0 (TID 531) in 32 ms on localhost (114/200)16/02/06 15:51:49 INFO executor.Executor: Running task 115.0 in stage 16.0 (TID 533)16/02/06 15:51:49 INFO executor.Executor: Finished task 114.0 in stage 16.0 (TID 532). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 116.0 in stage 16.0 (TID 534, localhost, partition 116,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 114.0 in stage 16.0 (TID 532) in 40 ms on localhost (115/200)16/02/06 15:51:49 INFO executor.Executor: Running task 116.0 in stage 16.0 (TID 534)16/02/06 15:51:49 INFO executor.Executor: Finished task 115.0 in stage 16.0 (TID 533). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 117.0 in stage 16.0 (TID 535, localhost, partition 117,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO executor.Executor: Running task 117.0 in stage 16.0 (TID 535)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 115.0 in stage 16.0 (TID 533) in 28 ms on localhost (116/200)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:49 INFO executor.Executor: Finished task 116.0 in stage 16.0 (TID 534). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 118.0 in stage 16.0 (TID 536, localhost, partition 118,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 116.0 in stage 16.0 (TID 534) in 25 ms on localhost (117/200)16/02/06 15:51:49 INFO executor.Executor: Running task 118.0 in stage 16.0 (TID 536)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms16/02/06 15:51:49 INFO executor.Executor: Finished task 117.0 in stage 16.0 (TID 535). 2676 bytes result sent to driver16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 7 ms16/02/06 15:51:49 INFO executor.Executor: Finished task 118.0 in stage 16.0 (TID 536). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 119.0 in stage 16.0 (TID 537, localhost, partition 119,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 117.0 in stage 16.0 (TID 535) in 38 ms on localhost (118/200)16/02/06 15:51:49 INFO executor.Executor: Running task 119.0 in stage 16.0 (TID 537)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 120.0 in stage 16.0 (TID 538, localhost, partition 120,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO executor.Executor: Running task 120.0 in stage 16.0 (TID 538)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 118.0 in stage 16.0 (TID 536) in 33 ms on localhost (119/200)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 6 ms16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms16/02/06 15:51:49 INFO executor.Executor: Finished task 120.0 in stage 16.0 (TID 538). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 121.0 in stage 16.0 (TID 539, localhost, partition 121,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO executor.Executor: Running task 121.0 in stage 16.0 (TID 539)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 120.0 in stage 16.0 (TID 538) in 30 ms on localhost (120/200)16/02/06 15:51:49 INFO executor.Executor: Finished task 119.0 in stage 16.0 (TID 537). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 122.0 in stage 16.0 (TID 540, localhost, partition 122,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 119.0 in stage 16.0 (TID 537) in 42 ms on localhost (121/200)16/02/06 15:51:49 INFO executor.Executor: Running task 122.0 in stage 16.0 (TID 540)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms16/02/06 15:51:49 INFO executor.Executor: Finished task 121.0 in stage 16.0 (TID 539). 2676 bytes result sent to driver16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 123.0 in stage 16.0 (TID 541, localhost, partition 123,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO executor.Executor: Running task 123.0 in stage 16.0 (TID 541)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 121.0 in stage 16.0 (TID 539) in 16 ms on localhost (122/200)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 4 ms16/02/06 15:51:49 INFO executor.Executor: Finished task 122.0 in stage 16.0 (TID 540). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 124.0 in stage 16.0 (TID 542, localhost, partition 124,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO executor.Executor: Running task 124.0 in stage 16.0 (TID 542)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 122.0 in stage 16.0 (TID 540) in 24 ms on localhost (123/200)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 15 ms16/02/06 15:51:49 INFO executor.Executor: Finished task 124.0 in stage 16.0 (TID 542). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 125.0 in stage 16.0 (TID 543, localhost, partition 125,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 124.0 in stage 16.0 (TID 542) in 61 ms on localhost (124/200)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO executor.Executor: Running task 125.0 in stage 16.0 (TID 543)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 39 ms16/02/06 15:51:49 INFO executor.Executor: Finished task 123.0 in stage 16.0 (TID 541). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 126.0 in stage 16.0 (TID 544, localhost, partition 126,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO executor.Executor: Running task 126.0 in stage 16.0 (TID 544)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 123.0 in stage 16.0 (TID 541) in 87 ms on localhost (125/200)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms16/02/06 15:51:49 INFO executor.Executor: Finished task 126.0 in stage 16.0 (TID 544). 2676 bytes result sent to driver16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 12 ms16/02/06 15:51:49 INFO executor.Executor: Finished task 125.0 in stage 16.0 (TID 543). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 127.0 in stage 16.0 (TID 545, localhost, partition 127,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO executor.Executor: Running task 127.0 in stage 16.0 (TID 545)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 125.0 in stage 16.0 (TID 543) in 43 ms on localhost (126/200)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 128.0 in stage 16.0 (TID 546, localhost, partition 128,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 126.0 in stage 16.0 (TID 544) in 29 ms on localhost (127/200)16/02/06 15:51:49 INFO executor.Executor: Running task 128.0 in stage 16.0 (TID 546)16/02/06 15:51:49 INFO executor.Executor: Finished task 127.0 in stage 16.0 (TID 545). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 129.0 in stage 16.0 (TID 547, localhost, partition 129,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO executor.Executor: Running task 129.0 in stage 16.0 (TID 547)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 127.0 in stage 16.0 (TID 545) in 25 ms on localhost (128/200)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 4 ms16/02/06 15:51:49 INFO executor.Executor: Finished task 129.0 in stage 16.0 (TID 547). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 130.0 in stage 16.0 (TID 548, localhost, partition 130,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO executor.Executor: Running task 130.0 in stage 16.0 (TID 548)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 129.0 in stage 16.0 (TID 547) in 23 ms on localhost (129/200)16/02/06 15:51:49 INFO executor.Executor: Finished task 128.0 in stage 16.0 (TID 546). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 131.0 in stage 16.0 (TID 549, localhost, partition 131,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO executor.Executor: Running task 131.0 in stage 16.0 (TID 549)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 6 ms16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 128.0 in stage 16.0 (TID 546) in 42 ms on localhost (130/200)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:49 INFO executor.Executor: Finished task 131.0 in stage 16.0 (TID 549). 2676 bytes result sent to driver16/02/06 15:51:49 INFO executor.Executor: Finished task 130.0 in stage 16.0 (TID 548). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 132.0 in stage 16.0 (TID 550, localhost, partition 132,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO executor.Executor: Running task 132.0 in stage 16.0 (TID 550)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 130.0 in stage 16.0 (TID 548) in 26 ms on localhost (131/200)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 133.0 in stage 16.0 (TID 551, localhost, partition 133,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 131.0 in stage 16.0 (TID 549) in 27 ms on localhost (132/200)16/02/06 15:51:49 INFO executor.Executor: Running task 133.0 in stage 16.0 (TID 551)16/02/06 15:51:49 INFO executor.Executor: Finished task 132.0 in stage 16.0 (TID 550). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 134.0 in stage 16.0 (TID 552, localhost, partition 134,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO executor.Executor: Running task 134.0 in stage 16.0 (TID 552)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 132.0 in stage 16.0 (TID 550) in 29 ms on localhost (133/200)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 5 ms16/02/06 15:51:49 INFO executor.Executor: Finished task 133.0 in stage 16.0 (TID 551). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 135.0 in stage 16.0 (TID 553, localhost, partition 135,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO executor.Executor: Running task 135.0 in stage 16.0 (TID 553)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 133.0 in stage 16.0 (TID 551) in 29 ms on localhost (134/200)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 4 ms16/02/06 15:51:49 INFO executor.Executor: Finished task 135.0 in stage 16.0 (TID 553). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 136.0 in stage 16.0 (TID 554, localhost, partition 136,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 13 ms16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 135.0 in stage 16.0 (TID 553) in 25 ms on localhost (135/200)16/02/06 15:51:49 INFO executor.Executor: Running task 136.0 in stage 16.0 (TID 554)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO executor.Executor: Finished task 134.0 in stage 16.0 (TID 552). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 137.0 in stage 16.0 (TID 555, localhost, partition 137,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 25 ms16/02/06 15:51:49 INFO executor.Executor: Running task 137.0 in stage 16.0 (TID 555)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 134.0 in stage 16.0 (TID 552) in 69 ms on localhost (136/200)16/02/06 15:51:49 INFO executor.Executor: Finished task 136.0 in stage 16.0 (TID 554). 2676 bytes result sent to driver16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 7 ms16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 138.0 in stage 16.0 (TID 556, localhost, partition 138,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 136.0 in stage 16.0 (TID 554) in 53 ms on localhost (137/200)16/02/06 15:51:49 INFO executor.Executor: Finished task 137.0 in stage 16.0 (TID 555). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 139.0 in stage 16.0 (TID 557, localhost, partition 139,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO executor.Executor: Running task 139.0 in stage 16.0 (TID 557)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 137.0 in stage 16.0 (TID 555) in 24 ms on localhost (138/200)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO executor.Executor: Running task 138.0 in stage 16.0 (TID 556)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 5 ms16/02/06 15:51:49 INFO executor.Executor: Finished task 139.0 in stage 16.0 (TID 557). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 140.0 in stage 16.0 (TID 558, localhost, partition 140,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO executor.Executor: Running task 140.0 in stage 16.0 (TID 558)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 139.0 in stage 16.0 (TID 557) in 27 ms on localhost (139/200)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:49 INFO executor.Executor: Finished task 138.0 in stage 16.0 (TID 556). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 141.0 in stage 16.0 (TID 559, localhost, partition 141,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO executor.Executor: Running task 141.0 in stage 16.0 (TID 559)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 138.0 in stage 16.0 (TID 556) in 60 ms on localhost (140/200)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 20 ms16/02/06 15:51:49 INFO executor.Executor: Finished task 140.0 in stage 16.0 (TID 558). 2676 bytes result sent to driver16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 24 ms16/02/06 15:51:49 INFO executor.Executor: Finished task 141.0 in stage 16.0 (TID 559). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 142.0 in stage 16.0 (TID 560, localhost, partition 142,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 143.0 in stage 16.0 (TID 561, localhost, partition 143,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 140.0 in stage 16.0 (TID 558) in 70 ms on localhost (141/200)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 141.0 in stage 16.0 (TID 559) in 58 ms on localhost (142/200)16/02/06 15:51:49 INFO executor.Executor: Running task 143.0 in stage 16.0 (TID 561)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 6 ms16/02/06 15:51:49 INFO executor.Executor: Running task 142.0 in stage 16.0 (TID 560)16/02/06 15:51:49 INFO executor.Executor: Finished task 143.0 in stage 16.0 (TID 561). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 144.0 in stage 16.0 (TID 562, localhost, partition 144,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO executor.Executor: Running task 144.0 in stage 16.0 (TID 562)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 143.0 in stage 16.0 (TID 561) in 33 ms on localhost (143/200)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms16/02/06 15:51:49 INFO executor.Executor: Finished task 144.0 in stage 16.0 (TID 562). 2676 bytes result sent to driver16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms16/02/06 15:51:49 INFO executor.Executor: Finished task 142.0 in stage 16.0 (TID 560). 2676 bytes result sent to driver16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 145.0 in stage 16.0 (TID 563, localhost, partition 145,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO executor.Executor: Running task 145.0 in stage 16.0 (TID 563)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 144.0 in stage 16.0 (TID 562) in 18 ms on localhost (144/200)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Starting task 146.0 in stage 16.0 (TID 564, localhost, partition 146,NODE_LOCAL, 1999 bytes)16/02/06 15:51:49 INFO scheduler.TaskSetManager: Finished task 142.0 in stage 16.0 (TID 560) in 58 ms on localhost (145/200)16/02/06 15:51:49 INFO executor.Executor: Running task 146.0 in stage 16.0 (TID 564)16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:49 INFO executor.Executor: Finished task 146.0 in stage 16.0 (TID 564). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 147.0 in stage 16.0 (TID 565, localhost, partition 147,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO executor.Executor: Running task 147.0 in stage 16.0 (TID 565)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 146.0 in stage 16.0 (TID 564) in 16 ms on localhost (146/200)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 145.0 in stage 16.0 (TID 563). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 148.0 in stage 16.0 (TID 566, localhost, partition 148,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO executor.Executor: Running task 148.0 in stage 16.0 (TID 566)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 145.0 in stage 16.0 (TID 563) in 35 ms on localhost (147/200)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO executor.Executor: Finished task 147.0 in stage 16.0 (TID 565). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 149.0 in stage 16.0 (TID 567, localhost, partition 149,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO executor.Executor: Running task 149.0 in stage 16.0 (TID 567)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 147.0 in stage 16.0 (TID 565) in 29 ms on localhost (148/200)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 4 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 149.0 in stage 16.0 (TID 567). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 150.0 in stage 16.0 (TID 568, localhost, partition 150,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 7 ms16/02/06 15:51:50 INFO executor.Executor: Running task 150.0 in stage 16.0 (TID 568)16/02/06 15:51:50 INFO executor.Executor: Finished task 148.0 in stage 16.0 (TID 566). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 149.0 in stage 16.0 (TID 567) in 25 ms on localhost (149/200)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 151.0 in stage 16.0 (TID 569, localhost, partition 151,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 148.0 in stage 16.0 (TID 566) in 39 ms on localhost (150/200)16/02/06 15:51:50 INFO executor.Executor: Running task 151.0 in stage 16.0 (TID 569)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 150.0 in stage 16.0 (TID 568). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 152.0 in stage 16.0 (TID 570, localhost, partition 152,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO executor.Executor: Running task 152.0 in stage 16.0 (TID 570)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 150.0 in stage 16.0 (TID 568) in 18 ms on localhost (151/200)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 151.0 in stage 16.0 (TID 569). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 153.0 in stage 16.0 (TID 571, localhost, partition 153,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO executor.Executor: Running task 153.0 in stage 16.0 (TID 571)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 151.0 in stage 16.0 (TID 569) in 24 ms on localhost (152/200)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 10 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 153.0 in stage 16.0 (TID 571). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 154.0 in stage 16.0 (TID 572, localhost, partition 154,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO executor.Executor: Running task 154.0 in stage 16.0 (TID 572)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 153.0 in stage 16.0 (TID 571) in 32 ms on localhost (153/200)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 14 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 152.0 in stage 16.0 (TID 570). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 155.0 in stage 16.0 (TID 573, localhost, partition 155,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO executor.Executor: Running task 155.0 in stage 16.0 (TID 573)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 152.0 in stage 16.0 (TID 570) in 60 ms on localhost (154/200)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 8 ms16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 4 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 154.0 in stage 16.0 (TID 572). 2676 bytes result sent to driver16/02/06 15:51:50 INFO executor.Executor: Finished task 155.0 in stage 16.0 (TID 573). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 156.0 in stage 16.0 (TID 574, localhost, partition 156,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 154.0 in stage 16.0 (TID 572) in 40 ms on localhost (155/200)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 155.0 in stage 16.0 (TID 573) in 24 ms on localhost (156/200)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 157.0 in stage 16.0 (TID 575, localhost, partition 157,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO executor.Executor: Running task 156.0 in stage 16.0 (TID 574)16/02/06 15:51:50 INFO executor.Executor: Running task 157.0 in stage 16.0 (TID 575)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 15 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 157.0 in stage 16.0 (TID 575). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 158.0 in stage 16.0 (TID 576, localhost, partition 158,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO executor.Executor: Running task 158.0 in stage 16.0 (TID 576)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 157.0 in stage 16.0 (TID 575) in 52 ms on localhost (157/200)16/02/06 15:51:50 INFO executor.Executor: Finished task 156.0 in stage 16.0 (TID 574). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 159.0 in stage 16.0 (TID 577, localhost, partition 159,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO executor.Executor: Running task 159.0 in stage 16.0 (TID 577)16/02/06 15:51:50 INFO executor.Executor: Finished task 158.0 in stage 16.0 (TID 576). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 156.0 in stage 16.0 (TID 574) in 74 ms on localhost (158/200)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 160.0 in stage 16.0 (TID 578, localhost, partition 160,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO executor.Executor: Running task 160.0 in stage 16.0 (TID 578)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 158.0 in stage 16.0 (TID 576) in 37 ms on localhost (159/200)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 12 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 159.0 in stage 16.0 (TID 577). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 161.0 in stage 16.0 (TID 579, localhost, partition 161,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO executor.Executor: Running task 161.0 in stage 16.0 (TID 579)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 159.0 in stage 16.0 (TID 577) in 28 ms on localhost (160/200)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 6 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 160.0 in stage 16.0 (TID 578). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 162.0 in stage 16.0 (TID 580, localhost, partition 162,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO executor.Executor: Running task 162.0 in stage 16.0 (TID 580)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 160.0 in stage 16.0 (TID 578) in 33 ms on localhost (161/200)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 161.0 in stage 16.0 (TID 579). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 163.0 in stage 16.0 (TID 581, localhost, partition 163,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 161.0 in stage 16.0 (TID 579) in 27 ms on localhost (162/200)16/02/06 15:51:50 INFO executor.Executor: Running task 163.0 in stage 16.0 (TID 581)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 162.0 in stage 16.0 (TID 580). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 164.0 in stage 16.0 (TID 582, localhost, partition 164,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 11 ms16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 162.0 in stage 16.0 (TID 580) in 33 ms on localhost (163/200)16/02/06 15:51:50 INFO executor.Executor: Running task 164.0 in stage 16.0 (TID 582)16/02/06 15:51:50 INFO executor.Executor: Finished task 163.0 in stage 16.0 (TID 581). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 165.0 in stage 16.0 (TID 583, localhost, partition 165,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms16/02/06 15:51:50 INFO executor.Executor: Running task 165.0 in stage 16.0 (TID 583)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 163.0 in stage 16.0 (TID 581) in 42 ms on localhost (164/200)16/02/06 15:51:50 INFO executor.Executor: Finished task 164.0 in stage 16.0 (TID 582). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 166.0 in stage 16.0 (TID 584, localhost, partition 166,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO executor.Executor: Running task 166.0 in stage 16.0 (TID 584)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 164.0 in stage 16.0 (TID 582) in 36 ms on localhost (165/200)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 4 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 166.0 in stage 16.0 (TID 584). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 167.0 in stage 16.0 (TID 585, localhost, partition 167,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO executor.Executor: Running task 167.0 in stage 16.0 (TID 585)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 166.0 in stage 16.0 (TID 584) in 34 ms on localhost (166/200)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 165.0 in stage 16.0 (TID 583). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 168.0 in stage 16.0 (TID 586, localhost, partition 168,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO executor.Executor: Running task 168.0 in stage 16.0 (TID 586)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 165.0 in stage 16.0 (TID 583) in 54 ms on localhost (167/200)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 167.0 in stage 16.0 (TID 585). 2676 bytes result sent to driver16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 12 ms16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 169.0 in stage 16.0 (TID 587, localhost, partition 169,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO executor.Executor: Running task 169.0 in stage 16.0 (TID 587)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 167.0 in stage 16.0 (TID 585) in 47 ms on localhost (168/200)16/02/06 15:51:50 INFO executor.Executor: Finished task 168.0 in stage 16.0 (TID 586). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 170.0 in stage 16.0 (TID 588, localhost, partition 170,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 168.0 in stage 16.0 (TID 586) in 41 ms on localhost (169/200)16/02/06 15:51:50 INFO executor.Executor: Running task 170.0 in stage 16.0 (TID 588)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 14 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 170.0 in stage 16.0 (TID 588). 2676 bytes result sent to driver16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 15 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 169.0 in stage 16.0 (TID 587). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 171.0 in stage 16.0 (TID 589, localhost, partition 171,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 172.0 in stage 16.0 (TID 590, localhost, partition 172,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 169.0 in stage 16.0 (TID 587) in 64 ms on localhost (170/200)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 170.0 in stage 16.0 (TID 588) in 60 ms on localhost (171/200)16/02/06 15:51:50 INFO executor.Executor: Running task 172.0 in stage 16.0 (TID 590)16/02/06 15:51:50 INFO executor.Executor: Running task 171.0 in stage 16.0 (TID 589)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 19 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 171.0 in stage 16.0 (TID 589). 2676 bytes result sent to driver16/02/06 15:51:50 INFO executor.Executor: Finished task 172.0 in stage 16.0 (TID 590). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 173.0 in stage 16.0 (TID 591, localhost, partition 173,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 174.0 in stage 16.0 (TID 592, localhost, partition 174,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO executor.Executor: Running task 173.0 in stage 16.0 (TID 591)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 171.0 in stage 16.0 (TID 589) in 62 ms on localhost (172/200)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 172.0 in stage 16.0 (TID 590) in 64 ms on localhost (173/200)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 173.0 in stage 16.0 (TID 591). 2676 bytes result sent to driver16/02/06 15:51:50 INFO executor.Executor: Running task 174.0 in stage 16.0 (TID 592)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 175.0 in stage 16.0 (TID 593, localhost, partition 175,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO executor.Executor: Running task 175.0 in stage 16.0 (TID 593)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 173.0 in stage 16.0 (TID 591) in 25 ms on localhost (174/200)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 7 ms16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 174.0 in stage 16.0 (TID 592). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 176.0 in stage 16.0 (TID 594, localhost, partition 176,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 174.0 in stage 16.0 (TID 592) in 45 ms on localhost (175/200)16/02/06 15:51:50 INFO executor.Executor: Finished task 175.0 in stage 16.0 (TID 593). 2676 bytes result sent to driver16/02/06 15:51:50 INFO executor.Executor: Running task 176.0 in stage 16.0 (TID 594)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 177.0 in stage 16.0 (TID 595, localhost, partition 177,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 175.0 in stage 16.0 (TID 593) in 29 ms on localhost (176/200)16/02/06 15:51:50 INFO executor.Executor: Running task 177.0 in stage 16.0 (TID 595)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 176.0 in stage 16.0 (TID 594). 2676 bytes result sent to driver16/02/06 15:51:50 INFO executor.Executor: Finished task 177.0 in stage 16.0 (TID 595). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 178.0 in stage 16.0 (TID 596, localhost, partition 178,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 179.0 in stage 16.0 (TID 597, localhost, partition 179,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 176.0 in stage 16.0 (TID 594) in 44 ms on localhost (177/200)16/02/06 15:51:50 INFO executor.Executor: Running task 178.0 in stage 16.0 (TID 596)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 177.0 in stage 16.0 (TID 595) in 44 ms on localhost (178/200)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO executor.Executor: Running task 179.0 in stage 16.0 (TID 597)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 11 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 178.0 in stage 16.0 (TID 596). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 180.0 in stage 16.0 (TID 598, localhost, partition 180,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO executor.Executor: Running task 180.0 in stage 16.0 (TID 598)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 178.0 in stage 16.0 (TID 596) in 33 ms on localhost (179/200)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 180.0 in stage 16.0 (TID 598). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 181.0 in stage 16.0 (TID 599, localhost, partition 181,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO executor.Executor: Running task 181.0 in stage 16.0 (TID 599)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 180.0 in stage 16.0 (TID 598) in 19 ms on localhost (180/200)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 10 ms16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 179.0 in stage 16.0 (TID 597). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 182.0 in stage 16.0 (TID 600, localhost, partition 182,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO executor.Executor: Running task 182.0 in stage 16.0 (TID 600)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 179.0 in stage 16.0 (TID 597) in 76 ms on localhost (181/200)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO executor.Executor: Finished task 181.0 in stage 16.0 (TID 599). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 183.0 in stage 16.0 (TID 601, localhost, partition 183,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 181.0 in stage 16.0 (TID 599) in 44 ms on localhost (182/200)16/02/06 15:51:50 INFO executor.Executor: Running task 183.0 in stage 16.0 (TID 601)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 18 ms16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 9 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 182.0 in stage 16.0 (TID 600). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 184.0 in stage 16.0 (TID 602, localhost, partition 184,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO executor.Executor: Running task 184.0 in stage 16.0 (TID 602)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 182.0 in stage 16.0 (TID 600) in 47 ms on localhost (183/200)16/02/06 15:51:50 INFO executor.Executor: Finished task 183.0 in stage 16.0 (TID 601). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 185.0 in stage 16.0 (TID 603, localhost, partition 185,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO executor.Executor: Running task 185.0 in stage 16.0 (TID 603)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 183.0 in stage 16.0 (TID 601) in 37 ms on localhost (184/200)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 185.0 in stage 16.0 (TID 603). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 186.0 in stage 16.0 (TID 604, localhost, partition 186,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 185.0 in stage 16.0 (TID 603) in 23 ms on localhost (185/200)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO executor.Executor: Running task 186.0 in stage 16.0 (TID 604)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 13 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 184.0 in stage 16.0 (TID 602). 2676 bytes result sent to driver16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 187.0 in stage 16.0 (TID 605, localhost, partition 187,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 184.0 in stage 16.0 (TID 602) in 48 ms on localhost (186/200)16/02/06 15:51:50 INFO executor.Executor: Running task 187.0 in stage 16.0 (TID 605)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 186.0 in stage 16.0 (TID 604). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 188.0 in stage 16.0 (TID 606, localhost, partition 188,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO executor.Executor: Running task 188.0 in stage 16.0 (TID 606)16/02/06 15:51:50 INFO executor.Executor: Finished task 187.0 in stage 16.0 (TID 605). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 189.0 in stage 16.0 (TID 607, localhost, partition 189,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 186.0 in stage 16.0 (TID 604) in 52 ms on localhost (187/200)16/02/06 15:51:50 INFO executor.Executor: Running task 189.0 in stage 16.0 (TID 607)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 187.0 in stage 16.0 (TID 605) in 35 ms on localhost (188/200)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 188.0 in stage 16.0 (TID 606). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 190.0 in stage 16.0 (TID 608, localhost, partition 190,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO executor.Executor: Running task 190.0 in stage 16.0 (TID 608)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 188.0 in stage 16.0 (TID 606) in 40 ms on localhost (189/200)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 190.0 in stage 16.0 (TID 608). 2676 bytes result sent to driver16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 16 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 189.0 in stage 16.0 (TID 607). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 191.0 in stage 16.0 (TID 609, localhost, partition 191,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 190.0 in stage 16.0 (TID 608) in 26 ms on localhost (190/200)16/02/06 15:51:50 INFO executor.Executor: Running task 191.0 in stage 16.0 (TID 609)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 192.0 in stage 16.0 (TID 610, localhost, partition 192,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO executor.Executor: Running task 192.0 in stage 16.0 (TID 610)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 192.0 in stage 16.0 (TID 610). 2676 bytes result sent to driver16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 191.0 in stage 16.0 (TID 609). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 189.0 in stage 16.0 (TID 607) in 73 ms on localhost (191/200)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 193.0 in stage 16.0 (TID 611, localhost, partition 193,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 194.0 in stage 16.0 (TID 612, localhost, partition 194,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 192.0 in stage 16.0 (TID 610) in 30 ms on localhost (192/200)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 191.0 in stage 16.0 (TID 609) in 40 ms on localhost (193/200)16/02/06 15:51:50 INFO executor.Executor: Running task 194.0 in stage 16.0 (TID 612)16/02/06 15:51:50 INFO executor.Executor: Running task 193.0 in stage 16.0 (TID 611)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 8 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 194.0 in stage 16.0 (TID 612). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 195.0 in stage 16.0 (TID 613, localhost, partition 195,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO executor.Executor: Running task 195.0 in stage 16.0 (TID 613)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 194.0 in stage 16.0 (TID 612) in 36 ms on localhost (194/200)16/02/06 15:51:50 INFO executor.Executor: Finished task 193.0 in stage 16.0 (TID 611). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 196.0 in stage 16.0 (TID 614, localhost, partition 196,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 193.0 in stage 16.0 (TID 611) in 41 ms on localhost (195/200)16/02/06 15:51:50 INFO executor.Executor: Running task 196.0 in stage 16.0 (TID 614)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 196.0 in stage 16.0 (TID 614). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 197.0 in stage 16.0 (TID 615, localhost, partition 197,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO executor.Executor: Finished task 195.0 in stage 16.0 (TID 613). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 196.0 in stage 16.0 (TID 614) in 16 ms on localhost (196/200)16/02/06 15:51:50 INFO executor.Executor: Running task 197.0 in stage 16.0 (TID 615)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 198.0 in stage 16.0 (TID 616, localhost, partition 198,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO executor.Executor: Running task 198.0 in stage 16.0 (TID 616)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 195.0 in stage 16.0 (TID 613) in 30 ms on localhost (197/200)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 4 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 198.0 in stage 16.0 (TID 616). 2676 bytes result sent to driver16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 5 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 197.0 in stage 16.0 (TID 615). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Starting task 199.0 in stage 16.0 (TID 617, localhost, partition 199,NODE_LOCAL, 1999 bytes)16/02/06 15:51:50 INFO executor.Executor: Running task 199.0 in stage 16.0 (TID 617)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 197.0 in stage 16.0 (TID 615) in 31 ms on localhost (198/200)16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 198.0 in stage 16.0 (TID 616) in 23 ms on localhost (199/200)16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks16/02/06 15:51:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms16/02/06 15:51:50 INFO executor.Executor: Finished task 199.0 in stage 16.0 (TID 617). 2676 bytes result sent to driver16/02/06 15:51:50 INFO scheduler.TaskSetManager: Finished task 199.0 in stage 16.0 (TID 617) in 18 ms on localhost (200/200)16/02/06 15:51:50 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 16.0, whose tasks have all completed, from pool 16/02/06 15:51:50 INFO scheduler.DAGScheduler: ResultStage 16 (show at <console>:26) finished in 7.563 s16/02/06 15:51:50 INFO scheduler.DAGScheduler: Job 10 finished: show at <console>:26, took 5.666527 s+---------+-----------+| author|countNumber|+---------+-----------+| yiyou| 3||jingruhou| 1|+---------+-----------+
1 0
- Spark实战-Spark SQL（一）
- Spark实战-Spark SQL（二）
- Spark实战-Spark SQL（三）
- Spark修炼之道（进阶篇）——Spark入门到精通：第十节 Spark SQL案例实战（一）
- Spark Streaming实战(一)
- Spark SQL之案例实战（四）
- 实战Spark分布式SQL引擎
- Spark Streaming 实战案例（一)
- Spark 实战（四）
- Spark SQL 处理流程分析 （一）
- spark SQL （一）初识 ，简介
- Spark实战
- SPARK 第4期：通过案例实战掌握spark sql（dataframe）
- Spark入门实战指南——Spark SQL入门
- spark-sql(一)---cli模式
- spark SQL编程动手实战-01
- Spark SQL数据加载和保存实战
- Spark-Sql之DataFrame实战详解
- K-D树
- 杭电ACM----2019 数列有序!
- 杭电ACM----2020 绝对值排序
- nyoj733圣诞派对
- 二叉树
- Spark实战-Spark SQL（一）
- java小的误区
- 快速求幂取模
- 图（拓扑排序）2
- poj_3450 Corporate Identity（KMP+枚举）
- 图（拓扑排序）3
- ubuntu14.04中搭建qt for android环境
- 搜索（枚举） 1
- 算法学习--二叉树
