hadoop 上路记

已经配置的环境下进行一些简单的命令操作，熟悉一下，有亲近感。

http://hadoop.apache.org/docs/r2.7.3/hadoop-project-dist/hadoop-common/CommandsManual.html

http://hadoop.apache.org/docs/r2.7.3/hadoop-project-dist/hadoop-common/FileSystemShell.html

官方文档里对用户有详细的说明。

建立目录

$ bin/hdfs dfs -mkdir -p sunfan

$ bin/hdfs dfs -mkdir -p sunfan/input

$ bin/hdfs dfs -mkdir -p sunfan/output

$ bin/hdfs dfs -ls

drwxr-xr-x   - root supergroup          0 2017-07-28 22:23 sunfan

$ bin/hdfs dfs -ls sunfan

drwxr-xr-x   - root supergroup          0 2017-07-28 22:23 sunfan/input
drwxr-xr-x   - root supergroup          0 2017-07-28 22:23 sunfan/output

$ bin/hdfs dfs -put ~/test sunfan/input

$ bin/hadoop fs -cat sunfan/input/test

查看HDFS 概况信息

$ bin/hadoop dfsadmin -report

Configured Capacity: 18402373632 (17.14 GB)
Present Capacity: 14503063552 (13.51 GB)
DFS Remaining: 14503002112 (13.51 GB)
DFS Used: 61440 (60 KB)
DFS Used%: 0.00%
Under replicated blocks: 0
Blocks with corrupt replicas: 0
Missing blocks: 0
Missing blocks (with replication factor 1): 0


-------------------------------------------------
Live datanodes (1):


Name: 127.0.0.1:50010 (localhost)
Hostname: localhost
Decommission Status : Normal
Configured Capacity: 18402373632 (17.14 GB)
DFS Used: 61440 (60 KB)
Non DFS Used: 3899310080 (3.63 GB)
DFS Remaining: 14503002112 (13.51 GB)
DFS Used%: 0.00%
DFS Remaining%: 78.81%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Sat Jul 29 00:06:05 CST 2017

查看一下安全模式状态,刚启动时系统 会处于安全模式，为了检查datanode上数据的有效性，此时不允许写操作

$  bin/hadoop dfsadmin -safemode leave

Safe mode is OFF

必要情况下可以打开

$ bin/hadoop dfsadmin -safemode enter