hadoop-运行hadoop jar hadoop-examples-1.2.1.jar wordcount /wc/input/ /wc/output/

来源：互联网发布：头戴式蓝牙推荐知乎编辑：程序博客网时间：2024/06/05 00:11

查看结果：

在地址栏：输入主机名：50070

点击_logs 下面的进入统计单词的结果。

[root@redhat5 hadoop-1.2.1]# hadoop jar hadoop-examples-1.2.1.jar wordcount /wc/input/ /wc/output/
16/01/20 22:27:56 INFO input.FileInputFormat: Total input paths to process : 7
16/01/20 22:27:56 INFO util.NativeCodeLoader: Loaded the native-hadoop library
16/01/20 22:27:56 WARN snappy.LoadSnappy: Snappy native library not loaded
16/01/20 22:27:59 INFO mapred.JobClient: Running job: job_201601202034_0003
16/01/20 22:28:00 INFO mapred.JobClient: map 0% reduce 0%
16/01/20 22:29:27 INFO mapred.JobClient: map 28% reduce 0%
16/01/20 22:30:25 INFO mapred.JobClient: map 42% reduce 0%
16/01/20 22:30:27 INFO mapred.JobClient: map 42% reduce 9%
16/01/20 22:30:29 INFO mapred.JobClient: map 57% reduce 9%
16/01/20 22:30:34 INFO mapred.JobClient: map 57% reduce 14%
16/01/20 22:30:41 INFO mapred.JobClient: map 57% reduce 19%
16/01/20 22:31:01 INFO mapred.JobClient: map 71% reduce 19%
16/01/20 22:31:04 INFO mapred.JobClient: map 85% reduce 19%
16/01/20 22:31:10 INFO mapred.JobClient: map 85% reduce 28%
16/01/20 22:31:25 INFO mapred.JobClient: map 100% reduce 28%
16/01/20 22:31:36 INFO mapred.JobClient: map 100% reduce 92%
16/01/20 22:31:37 INFO mapred.JobClient: map 100% reduce 100%
16/01/20 22:31:52 INFO mapred.JobClient: Job complete: job_201601202034_0003
16/01/20 22:31:52 INFO mapred.JobClient: Counters: 29
16/01/20 22:31:52 INFO mapred.JobClient: Job Counters
16/01/20 22:31:53 INFO mapred.JobClient: Launched reduce tasks=1
16/01/20 22:31:53 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=372203
16/01/20 22:31:53 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0
16/01/20 22:31:53 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0
16/01/20 22:31:53 INFO mapred.JobClient: Launched map tasks=7
16/01/20 22:31:53 INFO mapred.JobClient: Data-local map tasks=7
16/01/20 22:31:53 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=127071
16/01/20 22:31:53 INFO mapred.JobClient: File Output Format Counters

16/01/20 22:31:53 INFO mapred.JobClient: Bytes Written=6549
16/01/20 22:31:53 INFO mapred.JobClient: FileSystemCounters
16/01/20 22:31:53 INFO mapred.JobClient: FILE_BYTES_READ=10600
16/01/20 22:31:53 INFO mapred.JobClient: HDFS_BYTES_READ=16263
16/01/20 22:31:53 INFO mapred.JobClient: FILE_BYTES_WRITTEN=462821
16/01/20 22:31:53 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=6549
16/01/20 22:31:53 INFO mapred.JobClient: File Input Format Counters
16/01/20 22:31:53 INFO mapred.JobClient: Bytes Read=15486
16/01/20 22:31:53 INFO mapred.JobClient: Map-Reduce Framework
16/01/20 22:31:53 INFO mapred.JobClient: Map output materialized bytes=10636
16/01/20 22:31:53 INFO mapred.JobClient: Map input records=389

16/01/20 22:31:53 INFO mapred.JobClient: Reduce shuffle bytes=10636
16/01/20 22:31:53 INFO mapred.JobClient: Spilled Records=1196
16/01/20 22:31:53 INFO mapred.JobClient: Map output bytes=21294
16/01/20 22:31:53 INFO mapred.JobClient: Total committed heap usage (bytes)=1323302912
16/01/20 22:31:53 INFO mapred.JobClient: CPU time spent (ms)=38790
16/01/20 22:31:53 INFO mapred.JobClient: Combine input records=1761
16/01/20 22:31:53 INFO mapred.JobClient: SPLIT_RAW_BYTES=777
16/01/20 22:31:53 INFO mapred.JobClient: Reduce input records=598
16/01/20 22:31:53 INFO mapred.JobClient: Reduce input groups=427
16/01/20 22:31:53 INFO mapred.JobClient: Combine output records=598
16/01/20 22:31:53 INFO mapred.JobClient: Physical memory (bytes) snapshot=1390465024
16/01/20 22:31:53 INFO mapred.JobClient: Reduce output records=427
16/01/20 22:31:53 INFO mapred.JobClient: Virtual memory (bytes) snapshot=4970704896
16/01/20 22:31:53 INFO mapred.JobClient: Map output records=1761

在命令行：查看

在命令下查看结果：

[root@redhat5 conf]# hadoop fs -lsr /wc/input/

-rw-r--r-- 1 root supergroup 7457 2016-01-20 06:20 /wc/input/capacity-scheduler.xml
-rw-r--r-- 1 root supergroup 378 2016-01-20 06:21 /wc/input/core-site.xml
-rw-r--r-- 1 root supergroup 327 2016-01-20 06:21 /wc/input/fair-scheduler.xml
-rw-r--r-- 1 root supergroup 4644 2016-01-20 06:21 /wc/input/hadoop-policy.xml
-rw-r--r-- 1 root supergroup 358 2016-01-20 06:21 /wc/input/hdfs-site.xml
-rw-r--r-- 1 root supergroup 2033 2016-01-20 06:21 /wc/input/mapred-queue-acls.xml
-rw-r--r-- 1 root supergroup 289 2016-01-20 06:21 /wc/input/mapred-site.xml

[root@redhat5 conf]# hadoop fs -lsr /wc/output/

-rw-r--r-- 1 root supergroup 0 2016-01-20 22:31 /wc/output/_SUCCESS
drwxr-xr-x - root supergroup 0 2016-01-20 22:27 /wc/output/_logs
drwxr-xr-x - root supergroup 0 2016-01-20 22:27 /wc/output/_logs/history
-rw-r--r-- 1 root supergroup 30337 2016-01-20 22:27 /wc/output/_logs/history/job_201601202034_0003_1453300078993_root_word+count
-rw-r--r-- 1 root supergroup 48088 2016-01-20 22:27 /wc/output/_logs/history/job_201601202034_0003_conf.xml
-rw-r--r-- 1 root supergroup 6549 2016-01-20 22:31 /wc/output/part-r-00000

[root@redhat5 conf]# hadoop fs -text /wc/output/part-r-00000

0 0