hadoop备忘

hadoop安装目录：usr/local/hadoop
hease安装目录：usr/local/hbase

启动hadoop：start-all.sh
停止hadoop：stop-all.sh

启动hadoop后查看进程是否都启动：jps
如下正常：4784 Jps
4003 DataNode
3843 NameNode
4202 SecondaryNameNode
4346 ResourceManager
4475 NodeManager

查看HDFS文件目录（/user/hadoop/文件夹为例）：
hadoop@sun:~$ hadoop dfs -ls /user/hadoop/

查看HDFS文件输出（以/user/hadoop/output为例）：
hadoop@sun:~$ hadoop dfs -cat /user/hadoop/output

删除HDFS上的文件（/user/hadoop/output为例）：
hadoop@sun:~$ hadoop dfs -rm /user/hadoop/output
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.

17/01/13 16:16:27 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicable
17/01/13 16:16:27 INFO fs.TrashPolicyDefault: Namenode trash configuration: Deletion interval = 0 minutes, Emptier interval = 0 minutes.
Deleted /user/hadoop/output

启动hbase：
hadoop@sun:/usr/local/hbase/bin$ ./start-hbase.sh
localhost: starting zookeeper, logging to /usr/local/hbase/bin/../logs/hbase-hadoop-zookeeper-sun.out
starting master, logging to /usr/local/hbase/bin/../logs/hbase-hadoop-master-sun.out
starting regionserver, logging to /usr/local/hbase/bin/../logs/hbase-hadoop-1-regionserver-sun.out
查看hbase是否启动成功：
hadoop@sun:/usr/local/hbase/bin$ jps
4003 DataNode
3843 NameNode
25732 Jps
25446 HQuorumPeer
25640 HRegionServer
4202 SecondaryNameNode
4346 ResourceManager
4475 NodeManager
25516 HMaster

停止hbase：
hadoop@sun:/usr/local/hbase/bin$ ./stop-hbase.sh

进入 HBase Shell:
hadoop@sun:/usr/local/hbase/bin$ ./hbase shell
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/hbase/lib/slf4j-log4j12-1.7.7.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2017-01-13 15:49:44,207 WARN [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicable
HBase Shell; enter ‘help’ for list of supported commands.
Type “exit” to leave the HBase Shell
Version 1.0.3, rf1e1312f9790a7c40f6a4b5a1bab2ea1dd559890, Tue Jan 19 19:26:53 PST 2016

退出hbase shell ：
hbase(main):002:0> quit
hadoop@sun:/usr/local/hbase/bin$

启动eclipse：hadoop@sun:
/usr/local/eclipse$ sudo ./eclipse

因为经常root与hadoop用户混用导致有很多权限问题，如果是测试环境，可以取消hadoop hdfs的用户权限检查。打开conf/hdfs-site.xml，找到dfs.permissions属性修改为false（默认为true）