hadoop commands(hadoop-2.0.0-cdh4.4.0)
来源:互联网 发布:淘宝店卖什么比较火 编辑:程序博客网 时间:2024/05/21 05:40
Overview
All hadoop commands are invoked by the bin/hadoop script. Running the hadoop script without any arguments prints the description for all commands.
Usage: hadoop [--config confdir] [COMMAND] [GENERIC_OPTIONS] [COMMAND_OPTIONS]
Hadoop has an option parsing framework that employs parsing generic options as well as running classes.
Generic Options
The following options are supported by dfsadmin, fs, fsck, job and fetchdt. Applications should implement Tool to support GenericOptions.
User Commands
Commands useful for users of a hadoop cluster.
archive
Creates a hadoop archive. More information can be found at Hadoop Archives.
Usage: hadoop archive -archiveName NAME <src>* <dest>
distcp
Copy file or directories recursively. More information can be found at Hadoop DistCp Guide.
Usage: hadoop distcp <srcurl> <desturl>
fs
Usage: hadoop fs [GENERIC_OPTIONS] [COMMAND_OPTIONS]
Deprecated, use hdfs dfs instead.
Runs a generic filesystem user client.
The various COMMAND_OPTIONS can be found at File System Shell Guide.
fsck
Runs a HDFS filesystem checking utility. See Fsck for more info.
Usage: hadoop fsck [GENERIC_OPTIONS] <path> [-move | -delete | -openforwrite] [-files [-blocks [-locations | -racks]]]
fetchdt
Gets Delegation Token from a NameNode. See fetchdt for more info.
Usage: hadoop fetchdt [GENERIC_OPTIONS] [--webservice <namenode_http_addr>] <path>
jar
Runs a jar file. Users can bundle their Map Reduce code in a jar file and execute it using this command.
Usage: hadoop jar <jar> [mainClass] args...
The streaming jobs are run via this command. Examples can be referred from Streaming examples
Word count example is also run using jar command. It can be referred from Wordcount example
job
Command to interact with Map Reduce Jobs.
Usage: hadoop job [GENERIC_OPTIONS] [-submit <job-file>] | [-status <job-id>] | [-counter <job-id> <group-name> <counter-name>] | [-kill <job-id>] | [-events <job-id> <from-event-#> <#-of-events>] | [-history [all] <jobOutputDir>] | [-list [all]] | [-kill-task <task-id>] | [-fail-task <task-id>] | [-set-priority <job-id> <priority>]
pipes
Runs a pipes job.
Usage: hadoop pipes [-conf <path>] [-jobconf <key=value>, <key=value>, ...] [-input <path>] [-output <path>] [-jar <jar file>] [-inputformat <class>] [-map <class>] [-partitioner <class>] [-reduce <class>] [-writer <class>] [-program <executable>] [-reduces <num>]
queue
command to interact and view Job Queue information
Usage: hadoop queue [-list] | [-info <job-queue-name> [-showJobs]] | [-showacls]
version
Prints the version.
Usage: hadoop version
CLASSNAME
hadoop script can be used to invoke any class.
Usage: hadoop CLASSNAME
Runs the class named CLASSNAME.
classpath
Prints the class path needed to get the Hadoop jar and the required libraries.
Usage: hadoop classpath
Administration Commands
Commands useful for administrators of a hadoop cluster.
balancer
Runs a cluster balancing utility. An administrator can simply press Ctrl-C to stop the rebalancing process. See Rebalancer for more details.
Usage: hadoop balancer [-threshold <threshold>]
daemonlog
Get/Set the log level for each daemon.
Usage: hadoop daemonlog -getlevel <host:port> <name> Usage: hadoop daemonlog -setlevel <host:port> <name> <level>
datanode
Runs a HDFS datanode.
Usage: hadoop datanode [-rollback]
dfsadmin
Runs a HDFS dfsadmin client.
Usage: hadoop dfsadmin [GENERIC_OPTIONS] [-report] [-safemode enter | leave | get | wait] [-refreshNodes] [-finalizeUpgrade] [-upgradeProgress status | details | force] [-metasave filename] [-setQuota <quota> <dirname>...<dirname>] [-clrQuota <dirname>...<dirname>] [-restoreFailedStorage true|false|check] [-help [cmd]]
1. does not accept changes to the name space (read-only)
2. does not replicate or delete blocks.
Safe mode is entered automatically at Namenode startup, and leaves safe mode automatically when the configured minimum percentage of blocks satisfies the minimum replication condition. Safe mode can also be entered manually, but then it can only be turned off manually as well.-refreshNodesRe-read the hosts and exclude files to update the set of Datanodes that are allowed to connect to the Namenode and those that should be decommissioned or recommissioned.-finalizeUpgradeFinalize upgrade of HDFS. Datanodes delete their previous version working directories, followed by Namenode doing the same. This completes the upgrade process.-upgradeProgress status / details / forceRequest current distributed upgrade status, a detailed status or force the upgrade to proceed.-metasave filenameSave Namenode's primary data structures to filename in the directory specified by hadoop.log.dir property. filename will contain one line for each of the following
1. Datanodes heart beating with Namenode
2. Blocks waiting to be replicated
3. Blocks currrently being replicated
4. Blocks waiting to be deleted -setQuota quotadirname...dirnameSet the quota quota for each directory dirname. The directory quota is a long integer that puts a hard limit on the number of names in the directory tree. Best effort for the directory, with faults reported if
1. N is not a positive integer, or
2. user is not an administrator, or
3. the directory does not exist or is a file, or
4. the directory would immediately exceed the new quota. -clrQuotadirname...dirnameClear the quota for each directory dirname. Best effort for the directory. with fault reported if
1. the directory does not exist or is a file, or
2. user is not an administrator. It does not fault if the directory has no quota.-restoreFailedStorage true / false / checkThis option will turn on/off automatic attempt to restore failed storage replicas. If a failed storage becomes available again the system will attempt to restore edits and/or fsimage during checkpoint. 'check' option will return current setting.-help [cmd]Displays help for the given command or all commands if none is specified.
mradmin
Runs MR admin client
Usage: hadoop mradmin [ GENERIC_OPTIONS ] [-refreshQueueAcls]
jobtracker
Runs the MapReduce job Tracker node.
Usage: hadoop jobtracker [-dumpConfiguration]
namenode
Runs the namenode. More info about the upgrade, rollback and finalize is at Upgrade Rollback
Usage: hadoop namenode [-format] | [-upgrade] | [-rollback] | [-finalize] | [-importCheckpoint]
secondarynamenode
Runs the HDFS secondary namenode. See Secondary Namenode for more info.
Usage: hadoop secondarynamenode [-checkpoint [force]] | [-geteditsize]
tasktracker
Runs a MapReduce task Tracker node.
Usage: hadoop tasktracker
- hadoop commands(hadoop-2.0.0-cdh4.4.0)
- hadoop-2.0.0-cdh4.4.0 doc
- CDH4.1(hadoop-2.0.0-cdh4.1.2)安装部署文档
- cdh4 ha (hadoop-2.0.0-cdh4.1.2.tar.gz)
- CDH4.1(hadoop-2.0.0-cdh4.1.2)安装部署文档
- CDH4.1(hadoop-2.0.0-cdh4.1.2)安装部署文档
- CDH4.1(hadoop-2.0.0-cdh4.1.2)安装部署文档
- hadoop-2.0.0-cdh4.2.1安装手册
- Hadoop 2.0.0-cdh4.5.0安装
- hadoop-2.0.0-cdh4.5.0安装
- Hadoop 2.0.0-cdh4.5.0安装
- Hadoop cdh4.2.0配置 ShortCircuitRead
- 搭建hadoop-dist-2.0.0-cdh4.2.0开发测试环境
- ganglia3.6.0 监控 Hadoop 2.0.0-cdh4.2.0
- hadoop-2.0.0-cdh4.1.2.tar.gzl安装配置
- hadoop-2.0.0-mr1-cdh4.2.2 eclipse插件安装
- Hadoop CDH4.4.0上HIVE安装
- hadoop cdh4 下载地址
- CSharp调用Matlab编译的dll
- (用树的遍历求解层次性问题8.1.1)POJ 1330 Nearest Common Ancestors(求解最近共同祖先)
- 【转】POJ 1009
- Qt控件中文乱码的解决办法
- linux FrameBuffer
- hadoop commands(hadoop-2.0.0-cdh4.4.0)
- 黄淮学院CSDN高校俱乐部第一次HTML网页设计培训
- JSP乱码问题
- 第十周-求1000以内所有偶数的和(for语句)。
- linux下Vim设置显示行数 tab空格数
- tcp客户/服务器回射程序之五-----用shutdown函数解决在批量方式下所引起的问题
- Android eclipse中程序调试
- oj整除和商的问题1104
- 《More Effective C++》条款26:限制某个Class所能产生的对象数量