hadoop jar singlnode下运行的命令行输出

来源:互联网 发布:日文图片翻译软件 编辑:程序博客网 时间:2024/05/21 17:33
[hadoop@aaa ~]$ hadoop jar /opt/workspace/PGMdriver.jar GibbsSamplingDriver /output14/05/22 20:39:13 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:803214/05/22 20:39:13 WARN mapreduce.JobSubmitter: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.14/05/22 20:39:14 INFO mapreduce.JobSubmitter: number of splits:214/05/22 20:39:14 INFO Configuration.deprecation: user.name is deprecated. Instead, use mapreduce.job.user.name14/05/22 20:39:14 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar14/05/22 20:39:14 INFO Configuration.deprecation: mapred.cache.files.filesizes is deprecated. Instead, use mapreduce.job.cache.files.filesizes14/05/22 20:39:14 INFO Configuration.deprecation: mapred.cache.files is deprecated. Instead, use mapreduce.job.cache.files14/05/22 20:39:14 INFO Configuration.deprecation: mapred.mapoutput.value.class is deprecated. Instead, use mapreduce.map.output.value.class14/05/22 20:39:14 INFO Configuration.deprecation: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class14/05/22 20:39:14 INFO Configuration.deprecation: mapred.job.name is deprecated. Instead, use mapreduce.job.name14/05/22 20:39:14 INFO Configuration.deprecation: mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class14/05/22 20:39:14 INFO Configuration.deprecation: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir14/05/22 20:39:14 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps14/05/22 20:39:14 INFO Configuration.deprecation: mapred.cache.files.timestamps is deprecated. Instead, use mapreduce.job.cache.files.timestamps14/05/22 20:39:14 INFO Configuration.deprecation: mapred.mapoutput.key.class is deprecated. Instead, use mapreduce.map.output.key.class14/05/22 20:39:14 INFO Configuration.deprecation: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir14/05/22 20:39:14 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1400777095877_000314/05/22 20:39:15 INFO impl.YarnClientImpl: Submitted application application_1400777095877_0003 to ResourceManager at /0.0.0.0:803214/05/22 20:39:15 INFO mapreduce.Job: The url to track the job: http://localhost:8088/proxy/application_1400777095877_0003/14/05/22 20:39:15 INFO mapreduce.Job: Running job: job_1400777095877_000314/05/22 20:39:22 INFO mapreduce.Job: Job job_1400777095877_0003 running in uber mode : false14/05/22 20:39:22 INFO mapreduce.Job:  map 0% reduce 0%14/05/22 20:39:30 INFO mapreduce.Job:  map 100% reduce 0%14/05/22 20:39:37 INFO mapreduce.Job:  map 100% reduce 100%14/05/22 20:39:38 INFO mapreduce.Job: Job job_1400777095877_0003 completed successfully14/05/22 20:39:38 INFO mapreduce.Job: Counters: 43File System CountersFILE: Number of bytes read=34651FILE: Number of bytes written=308564FILE: Number of read operations=0FILE: Number of large read operations=0FILE: Number of write operations=0HDFS: Number of bytes read=40HDFS: Number of bytes written=33845HDFS: Number of read operations=7HDFS: Number of large read operations=0HDFS: Number of write operations=2Job Counters Launched map tasks=2Launched reduce tasks=1Other local map tasks=2Total time spent by all maps in occupied slots (ms)=10975Total time spent by all reduces in occupied slots (ms)=4275Map-Reduce FrameworkMap input records=0Map output records=200Map output bytes=34045Map output materialized bytes=34657Input split bytes=40Combine input records=0Combine output records=0Reduce input groups=176Reduce shuffle bytes=34657Reduce input records=200Reduce output records=200Spilled Records=400Shuffled Maps =2Failed Shuffles=0Merged Map outputs=2GC time elapsed (ms)=201CPU time spent (ms)=2410Physical memory (bytes) snapshot=662515712Virtual memory (bytes) snapshot=2658840576Total committed heap usage (bytes)=512229376Shuffle ErrorsBAD_ID=0CONNECTION=0IO_ERROR=0WRONG_LENGTH=0WRONG_MAP=0WRONG_REDUCE=0File Input Format Counters Bytes Read=0File Output Format Counters Bytes Written=33845

以上位运行成功

下面换了个jar,运行没错,但是结果不对,貌似没读到hdfs里的文件,命令行显示结果如下:

[hadoop@aaa~]$ hadoop jar /opt/workspace/PGM2205.jar /output14/05/22 21:23:10 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:803214/05/22 21:23:11 WARN mapreduce.JobSubmitter: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.14/05/22 21:23:11 INFO mapreduce.JobSubmitter: number of splits:314/05/22 21:23:11 INFO Configuration.deprecation: user.name is deprecated. Instead, use mapreduce.job.user.name14/05/22 21:23:11 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar14/05/22 21:23:11 INFO Configuration.deprecation: mapred.cache.files.filesizes is deprecated. Instead, use mapreduce.job.cache.files.filesizes14/05/22 21:23:11 INFO Configuration.deprecation: mapred.cache.files is deprecated. Instead, use mapreduce.job.cache.files14/05/22 21:23:11 INFO Configuration.deprecation: mapred.mapoutput.value.class is deprecated. Instead, use mapreduce.map.output.value.class14/05/22 21:23:11 INFO Configuration.deprecation: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class14/05/22 21:23:11 INFO Configuration.deprecation: mapred.job.name is deprecated. Instead, use mapreduce.job.name14/05/22 21:23:11 INFO Configuration.deprecation: mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class14/05/22 21:23:11 INFO Configuration.deprecation: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir14/05/22 21:23:11 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps14/05/22 21:23:11 INFO Configuration.deprecation: mapred.cache.files.timestamps is deprecated. Instead, use mapreduce.job.cache.files.timestamps14/05/22 21:23:11 INFO Configuration.deprecation: mapred.mapoutput.key.class is deprecated. Instead, use mapreduce.map.output.key.class14/05/22 21:23:11 INFO Configuration.deprecation: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir14/05/22 21:23:12 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1400777095877_000914/05/22 21:23:12 INFO impl.YarnClientImpl: Submitted application application_1400777095877_0009 to ResourceManager at /0.0.0.0:803214/05/22 21:23:12 INFO mapreduce.Job: The url to track the job: http://localhost:8088/proxy/application_1400777095877_0009/14/05/22 21:23:12 INFO mapreduce.Job: Running job: job_1400777095877_000914/05/22 21:23:18 INFO mapreduce.Job: Job job_1400777095877_0009 running in uber mode : false14/05/22 21:23:18 INFO mapreduce.Job:  map 0% reduce 0%14/05/22 21:23:27 INFO mapreduce.Job:  map 33% reduce 0%14/05/22 21:23:29 INFO mapreduce.Job:  map 100% reduce 0%14/05/22 21:23:34 INFO mapreduce.Job:  map 100% reduce 100%14/05/22 21:23:34 INFO mapreduce.Job: Job job_1400777095877_0009 completed successfully14/05/22 21:23:34 INFO mapreduce.Job: Counters: 43File System CountersFILE: Number of bytes read=6FILE: Number of bytes written=318973FILE: Number of read operations=0FILE: Number of large read operations=0FILE: Number of write operations=0HDFS: Number of bytes read=60HDFS: Number of bytes written=0HDFS: Number of read operations=9HDFS: Number of large read operations=0HDFS: Number of write operations=2Job Counters Launched map tasks=3Launched reduce tasks=1Other local map tasks=3Total time spent by all maps in occupied slots (ms)=21498Total time spent by all reduces in occupied slots (ms)=3932Map-Reduce FrameworkMap input records=0Map output records=0Map output bytes=0Map output materialized bytes=18Input split bytes=60Combine input records=0Combine output records=0Reduce input groups=0Reduce shuffle bytes=18Reduce input records=0Reduce output records=0Spilled Records=0Shuffled Maps =3Failed Shuffles=0Merged Map outputs=3GC time elapsed (ms)=143CPU time spent (ms)=2560Physical memory (bytes) snapshot=946880512Virtual memory (bytes) snapshot=3543068672Total committed heap usage (bytes)=717225984Shuffle ErrorsBAD_ID=0CONNECTION=0IO_ERROR=0WRONG_LENGTH=0WRONG_MAP=0WRONG_REDUCE=0File Input Format Counters Bytes Read=0File Output Format Counters Bytes Written=0
为什么呢?

0 0