Giraph集群测试Pagerank

[liuqiang2@mu02 bin]$  giraph ../giraph-core-1.1.0.jar  org.apache.giraph.benchmark.PageRankComputation -vif  org.apache.giraph.io.formats.IntFloatNullTextInputFormat -vip /test/youTube.txt  -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /output  -w 15

16/03/18 17:32:05 INFO utils.ConfigurationUtils: No edge input format specified. Ensure your InputFormat does not require one.16/03/18 17:32:05 INFO utils.ConfigurationUtils: No edge output format specified. Ensure your OutputFormat does not require one.16/03/18 17:32:05 INFO job.GiraphJob: run: Since checkpointing is disabled (default), do not allow any task retries (setting mapred.map.max.attempts = 0, old value = 4)16/03/18 17:32:13 INFO job.GiraphJob: Tracking URL: http://mu02:50030/jobdetails.jsp?jobid=job_201603181726_000216/03/18 17:32:13 INFO job.GiraphJob: Waiting for resources... Job will start only when it gets all 16 mappers16/03/18 17:33:28 INFO job.HaltApplicationUtils$DefaultHaltInstructionsWriter: writeHaltInstructions: To halt after next superstep execute: 'bin/halt-application --zkServer c02b10:22181 --zkNode /_hadoopBsp/job_201603181726_0002/_haltComputation'16/03/18 17:33:28 INFO mapred.JobClient: Running job: job_201603181726_000216/03/18 17:33:29 INFO mapred.JobClient:  map 100% reduce 0%16/03/18 17:33:36 INFO mapred.JobClient: Job complete: job_201603181726_000216/03/18 17:33:36 INFO mapred.JobClient: Counters: 6816/03/18 17:33:36 INFO mapred.JobClient:   Zookeeper halt node16/03/18 17:33:36 INFO mapred.JobClient:     /_hadoopBsp/job_201603181726_0002/_haltComputation=016/03/18 17:33:36 INFO mapred.JobClient:   Zookeeper base path16/03/18 17:33:36 INFO mapred.JobClient:     /_hadoopBsp/job_201603181726_0002=016/03/18 17:33:36 INFO mapred.JobClient:   Job Counters 16/03/18 17:33:36 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=77055216/03/18 17:33:36 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=016/03/18 17:33:36 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=016/03/18 17:33:36 INFO mapred.JobClient:     Launched map tasks=1616/03/18 17:33:36 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=016/03/18 17:33:36 INFO mapred.JobClient:   Giraph Timers16/03/18 17:33:36 INFO mapred.JobClient:     Superstep 27 PageRankComputation (ms)=192916/03/18 17:33:36 INFO mapred.JobClient:     Superstep 3 PageRankComputation (ms)=173216/03/18 17:33:36 INFO mapred.JobClient:     Superstep 1 PageRankComputation (ms)=44516/03/18 17:33:36 INFO mapred.JobClient:     Superstep 11 PageRankComputation (ms)=21016/03/18 17:33:36 INFO mapred.JobClient:     Superstep 23 PageRankComputation (ms)=16716/03/18 17:33:36 INFO mapred.JobClient:     Superstep 20 PageRankComputation (ms)=201216/03/18 17:33:36 INFO mapred.JobClient:     Superstep 8 PageRankComputation (ms)=94116/03/18 17:33:36 INFO mapred.JobClient:     Superstep 13 PageRankComputation (ms)=193616/03/18 17:33:36 INFO mapred.JobClient:     Shutdown (ms)=921316/03/18 17:33:36 INFO mapred.JobClient:     Superstep 2 PageRankComputation (ms)=34416/03/18 17:33:36 INFO mapred.JobClient:     Superstep 25 PageRankComputation (ms)=240316/03/18 17:33:36 INFO mapred.JobClient:     Superstep 29 PageRankComputation (ms)=18016/03/18 17:33:36 INFO mapred.JobClient:     Superstep 12 PageRankComputation (ms)=163916/03/18 17:33:36 INFO mapred.JobClient:     Superstep 15 PageRankComputation (ms)=163716/03/18 17:33:36 INFO mapred.JobClient:     Superstep 26 PageRankComputation (ms)=159716/03/18 17:33:36 INFO mapred.JobClient:     Superstep 18 PageRankComputation (ms)=17216/03/18 17:33:36 INFO mapred.JobClient:     Superstep 9 PageRankComputation (ms)=166216/03/18 17:33:36 INFO mapred.JobClient:     Superstep 10 PageRankComputation (ms)=24516/03/18 17:33:36 INFO mapred.JobClient:     Superstep 14 PageRankComputation (ms)=18616/03/18 17:33:36 INFO mapred.JobClient:     Superstep 0 PageRankComputation (ms)=47316/03/18 17:33:36 INFO mapred.JobClient:     Superstep 24 PageRankComputation (ms)=162816/03/18 17:33:36 INFO mapred.JobClient:     Superstep 16 PageRankComputation (ms)=165916/03/18 17:33:36 INFO mapred.JobClient:     Input superstep (ms)=321916/03/18 17:33:36 INFO mapred.JobClient:     Superstep 4 PageRankComputation (ms)=23516/03/18 17:33:36 INFO mapred.JobClient:     Initialize (ms)=1342316/03/18 17:33:36 INFO mapred.JobClient:     Superstep 21 PageRankComputation (ms)=17116/03/18 17:33:36 INFO mapred.JobClient:     Setup (ms)=14616/03/18 17:33:36 INFO mapred.JobClient:     Superstep 5 PageRankComputation (ms)=171716/03/18 17:33:36 INFO mapred.JobClient:     Superstep 30 PageRankComputation (ms)=13216/03/18 17:33:36 INFO mapred.JobClient:     Total (ms)=4282416/03/18 17:33:36 INFO mapred.JobClient:     Superstep 7 PageRankComputation (ms)=21916/03/18 17:33:36 INFO mapred.JobClient:     Superstep 28 PageRankComputation (ms)=166416/03/18 17:33:36 INFO mapred.JobClient:     Superstep 17 PageRankComputation (ms)=15516/03/18 17:33:36 INFO mapred.JobClient:     Superstep 22 PageRankComputation (ms)=161716/03/18 17:33:36 INFO mapred.JobClient:     Superstep 19 PageRankComputation (ms)=87416/03/18 17:33:36 INFO mapred.JobClient:     Superstep 6 PageRankComputation (ms)=25616/03/18 17:33:36 INFO mapred.JobClient:   Zookeeper server:port16/03/18 17:33:36 INFO mapred.JobClient:     c02b10:22181=016/03/18 17:33:36 INFO mapred.JobClient:   Giraph Stats16/03/18 17:33:36 INFO mapred.JobClient:     Aggregate edges=298762416/03/18 17:33:36 INFO mapred.JobClient:     Sent message bytes=016/03/18 17:33:36 INFO mapred.JobClient:     Superstep=3116/03/18 17:33:36 INFO mapred.JobClient:     Last checkpointed superstep=016/03/18 17:33:36 INFO mapred.JobClient:     Current workers=1516/03/18 17:33:36 INFO mapred.JobClient:     Aggregate sent messages=8962872016/03/18 17:33:36 INFO mapred.JobClient:     Current master task partition=016/03/18 17:33:36 INFO mapred.JobClient:     Sent messages=016/03/18 17:33:36 INFO mapred.JobClient:     Aggregate finished vertices=113489016/03/18 17:33:36 INFO mapred.JobClient:     Aggregate sent message message bytes=71804901016/03/18 17:33:36 INFO mapred.JobClient:     Aggregate vertices=113489016/03/18 17:33:36 INFO mapred.JobClient:   File Output Format Counters 16/03/18 17:33:36 INFO mapred.JobClient:     Bytes Written=016/03/18 17:33:36 INFO mapred.JobClient:   FileSystemCounters16/03/18 17:33:36 INFO mapred.JobClient:     HDFS_BYTES_READ=2958503316/03/18 17:33:36 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=210307316/03/18 17:33:36 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=2252083516/03/18 17:33:36 INFO mapred.JobClient:   File Input Format Counters 16/03/18 17:33:36 INFO mapred.JobClient:     Bytes Read=016/03/18 17:33:36 INFO mapred.JobClient:   Map-Reduce Framework16/03/18 17:33:36 INFO mapred.JobClient:     Map input records=1616/03/18 17:33:36 INFO mapred.JobClient:     Physical memory (bytes) snapshot=415370444816/03/18 17:33:36 INFO mapred.JobClient:     Spilled Records=016/03/18 17:33:36 INFO mapred.JobClient:     CPU time spent (ms)=31627016/03/18 17:33:36 INFO mapred.JobClient:     Total committed heap usage (bytes)=295069286416/03/18 17:33:36 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=1360572416016/03/18 17:33:36 INFO mapred.JobClient:     Map output records=016/03/18 17:33:36 INFO mapred.JobClient:     SPLIT_RAW_BYTES=704