hadoop 命令总结

1. 查看文件

hadoop fs -lsr

hadoop fs -cat

2. 创建文件夹
    hadoop fs -mkdir /user/data
3. 删除文件夹
   hadoop fs -rmr /user/data

5. 拷贝本地文件到远程机器
  hdfs -copyFromLocal data /user/data
 
6.  steaming cli

/usr/bin/hadoop jar /usr/lib/hadoop/contrib/streaming/hadoop-streaming-0.20.2-cdh3u4.jar  -input /user/data/*  -output  /user/out  -file /mapper.php -file /reducer.php -mapper mapper.php -reducer reducer.php

-input 指定原始数据

-output 输出结果

-file 提交文件到机器

-mapper 指定 mapper

-reducer 指定 reducer

可以不指定reducer,这样输出的结果是按照mapper的Key 排序好了的

运行结果

a 9
b 8
file 17
is 17
this 17

7 不指定reducer，输出是按照key 排序的

a 3
a 3
a 3
b 4
b 4
file 4
file 4
file 3
file 3
file 3
is 3
is 3
is 4
is 4
is 3
this 3
this 3
this 3
this 4
this 4