程序博客网 > ps做淘宝主图

hadoop安装配置

来源：互联网 发布：ps做淘宝主图 编辑：程序博客网 时间：2024/04/28 11:43

 

1）JDK软件

    下载地址：http://www.oracle.com/technetwork/java/javase/index.html

    JDK版本：jdk-6u31-linux-i586.bin

2）Hadoop软件

    下载地址：http://hadoop.apache.org/common/releases.html

    Hadoop版本：hadoop-1.0.0.tar.gz

 

 

Hadoop-2.5.1集群安装配置笔记


1.环境


1.1.虚拟机

准备3台虚拟机，安装Centos 64-bit操作系统，采用最小安装。

（本来想多跑几台虚拟机，但本人笔记本电脑内存有限，最多只能同时跑3个虚拟机）

虚拟机一律配置静态IP地址，配置域名解析，各虚拟机时间同步。


192.168.17.100 nameNode

192.168.17.101 dataNode1

192.168.17.102 dataNode2



2.安装


2.1.安装前


2.1.1.安装wget和ssh

用于下载和ssh登录

[html] view plaincopyprint?在CODE上查看代码片派生到我的代码片
  1. yum -y install wget  
  2.   
  3. yum -y install openssh*  

 


2.1.2.安装JDK、配置环境变量

在etc/profile最后添加：

#hadoop variable settings
export HADOOP_HOME=/usr/hadoop


export HADOOP_COMMON_HOME=$HADOOP_HOME

export PATH=$PATH:$HADOOP_HOME/bin
export HADOOP_HDFS_HOME=$HADOOP_HOME
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_YARN_HOME=$HADOOP_HOME
export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$HADOOP_HOME/lib
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib"



2.1.3.配置ssh公钥密钥自动登录

*配置所有Slave无密码登录Master

1、首先创建"Slave1.Hadoop"自己的公钥和私钥，并把自己的公钥追加到"authorized_keys"文件中。用到的命令如下：

ssh-keygen –t rsa –P ''

cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

2、接着是用命令"scp"复制"Slave1.Hadoop"的公钥"id_rsa.pub"到"Master.Hadoop"的"/home/hadoop/"目录下，并追加到"Master.Hadoop"的"authorized_keys"中。

scp ~/.ssh/id_rsa.pub   master:~/
cat ~/id_rsa.pub >> ~/.ssh/authorized_keys

3、最后是测试从"Slave1.Hadoop"到"Master.Hadoop"无密码登录。

ssh master

【我安装的时候是在/root/.ssh/目录下】

2.2.开始安装

下载最新版本hadoop-2.5.1

[plain] view plaincopyprint?在CODE上查看代码片派生到我的代码片
  1. wget http://mirrors.hust.edu.cn/apache/hadoop/common/hadoop-2.5.1/hadoop-2.5.1.tar.gz  

 

解压

[html] view plaincopyprint?在CODE上查看代码片派生到我的代码片
  1. tar -zxf hadoop-2.5.1.tar.gz  



 

2.3.配置文件

进入配置文件目录：cd hadoop-2.5.1/etc/hadoop

2.3.1.core-site.xml

[html] view plaincopyprint?在CODE上查看代码片派生到我的代码片
  1. <configuration>  
  2.     <property>  
  3.         <name>hadoop.tmp.dir</name>  
  4.         <value>/home/hadoop/tmp</value>  
  5.         <description>Abase for other temporary directories.</description>  
  6.     </property>  
  7.     <property>  
  8.         <name>fs.defaultFS</name>  
  9.         <value>hdfs://nameNode:9000</value>  
  10.     </property>  
  11.     <property>  
  12.         <name>io.file.buffer.size</name>  
  13.         <value>4096</value>  
  14.     </property>  
  15. </configuration>  


 

2.3.2.hdfs-site.xml

[html] view plaincopyprint?在CODE上查看代码片派生到我的代码片
  1. <configuration>  
  2.     <property>  
  3.         <name>dfs.nameservices</name>  
  4.         <value>hadoop-cluster1</value>  
  5.     </property>  
  6.     <property>  
  7.         <name>dfs.namenode.secondary.http-address</name>  
  8.         <value>nameNode:50090</value>  
  9.     </property>  
  10.     <property>  
  11.         <name>dfs.namenode.name.dir</name>  
  12.         <value>file:///home/hadoop/dfs/name</value>  
  13.     </property>  
  14.     <property>  
  15.         <name>dfs.datanode.data.dir</name>  
  16.         <value>file:///home/hadoop/dfs/data</value>  
  17.     </property>  
  18.     <property>  
  19.         <name>dfs.replication</name>  
  20.         <value>2</value>  
  21.     </property>  
  22.     <property>  
  23.         <name>dfs.webhdfs.enabled</name>  
  24.         <value>true</value>  
  25.     </property>  
  26. </configuration>  



 

2.3.3.mapred-site.xml

[html] view plaincopyprint?在CODE上查看代码片派生到我的代码片
  1. <configuration>  
  2.     <property>  
  3.         <name>mapreduce.framework.name</name>  
  4.         <value>yarn</value>  
  5.     </property>  
  6.     <property>  
  7.         <name>mapreduce.jobtracker.http.address</name>  
  8.         <value>nameNode:50030</value>  
  9.     </property>  
  10.     <property>  
  11.         <name>mapreduce.jobhistory.address</name>  
  12.         <value>nameNode:10020</value>  
  13.     </property>  
  14.     <property>  
  15.         <name>mapreduce.jobhistory.webapp.address</name>  
  16.         <value>nameNode:19888</value>  
  17.     </property>  
  18. </configuration>  



 

2.3.4.yarn-site.xml

[html] view plaincopyprint?在CODE上查看代码片派生到我的代码片
  1. <configuration>  
  2.   
  3. <!-- Site specific YARN configuration properties -->  
  4.     <property>  
  5.         <name>yarn.nodemanager.aux-services</name>  
  6.         <value>mapreduce_shuffle</value>  
  7.     </property>  
  8.     <property>  
  9.         <name>yarn.resourcemanager.address</name>  
  10.         <value>nameNode:8032</value>  
  11.     </property>  
  12.     <property>  
  13.         <name>yarn.resourcemanager.scheduler.address</name>  
  14.         <value>nameNode:8030</value>  
  15.     </property>  
  16.     <property>  
  17.         <name>yarn.resourcemanager.resource-tracker.address</name>  
  18.         <value>nameNode:8031</value>  
  19.     </property>  
  20.     <property>  
  21.         <name>yarn.resourcemanager.admin.address</name>  
  22.         <value>nameNode:8033</value>  
  23.     </property>  
  24.     <property>  
  25.         <name>yarn.resourcemanager.webapp.address</name>  
  26.         <value>nameNode:8088</value>  
  27.     </property>  
  28. </configuration>  



 

2.3.5.slaves

[plain] view plaincopyprint?在CODE上查看代码片派生到我的代码片
  1. dataNode1  
  2. dataNode2  

 


2.3.6.修改JAVA_HOME

分别在文件hadoop-env.sh和yarn-env.sh中添加JAVA_HOME配置
vi hadoop-env.sh
[plain] view plaincopyprint?在CODE上查看代码片派生到我的代码片
  1. export JAVA_HOME=/usr/java/jdk1.7.0_65  
vi yarn-env.sh
[plain] view plaincopyprint?在CODE上查看代码片派生到我的代码片
  1. export JAVA_HOME=/usr/java/jdk1.7.0_65  

2.4.格式化文件系统

格式化文件系统：

[plain] view plaincopyprint?在CODE上查看代码片派生到我的代码片
  1. bin/hdfs namenode -format  

输出：

[plain] view plaincopyprint?在CODE上查看代码片派生到我的代码片
  1. 14/09/21 11:57:22 INFO namenode.NameNode: STARTUP_MSG:   
  2. /************************************************************  
  3. STARTUP_MSG: Starting NameNode  
  4. STARTUP_MSG:   host = nameNode/127.0.0.1  
  5. STARTUP_MSG:   args = [-format]  
  6. STARTUP_MSG:   version = 2.5.1  
  7. STARTUP_MSG:   classpath = /home/hadoop-2.5.1/etc/hadoop:/home/hadoop-2.5.1/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/jets3t-0.9.0.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/jersey-server-1.9.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/asm-3.2.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/guava-11.0.2.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/jetty-6.1.26.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/api-util-1.0.0-M20.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/xz-1.0.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/paranamer-2.3.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/commons-lang-2.6.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/commons-codec-1.4.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/commons-math3-3.1.1.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/commons-configuration-1.6.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/jsch-0.1.42.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/netty-3.6.2.Final.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/zookeeper-3.4.6.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/commons-cli-1.2.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/activation-1.1.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/commons-collections-3.2.1.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/hadoop-auth-2.5.1.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/hamcrest-core-1.3.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/commons-el-1.0.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/xmlenc-0.52.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/jsr305-1.3.9.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/log4j-1.2.17.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/httpclient-4.2.5.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/jersey-json-1.9.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/avro-1.7.4.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/jasper-compiler-5.5.23.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/jersey-core-1.9.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/hadoop-annotations-2.5.1.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/commons-httpclient-3.1.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/commons-io-2.4.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/commons-net-3.1.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/stax-api-1.0-2.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/commons-compress-1.4.1.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/junit-4.11.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/slf4j-api-1.7.5.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/jetty-util-6.1.26.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/jsp-api-2.1.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/mockito-all-1.8.5.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/httpcore-4.2.5.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/servlet-api-2.5.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/commons-digester-1.8.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/jettison-1.1.jar:/home/hadoop-2.5.1/share/hadoop/common/lib/commons-logging-1.1.3.jar:/home/hadoop-2.5.1/share/hadoop/common/hadoop-nfs-2.5.1.jar:/home/hadoop-2.5.1/share/hadoop/common/hadoop-common-2.5.1-tests.jar:/home/hadoop-2.5.1/share/hadoop/common/hadoop-common-2.5.1.jar:/home/hadoop-2.5.1/share/hadoop/hdfs:/home/hadoop-2.5.1/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/home/hadoop-2.5.1/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/home/hadoop-2.5.1/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/home/hadoop-2.5.1/share/hadoop/hdfs/lib/asm-3.2.jar:/home/hadoop-2.5.1/share/hadoop/hdfs/lib/guava-11.0.2.jar:/home/hadoop-2.5.1/share/hadoop/hdfs/lib/jetty-6.1.26.jar:/home/hadoop-2.5.1/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/home/hadoop-2.5.1/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/home/hadoop-2.5.1/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/home/hadoop-2.5.1/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/home/hadoop-2.5.1/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/home/hadoop-2.5.1/share/hadoop/hdfs/lib/commons-el-1.0.jar:/home/hadoop-2.5.1/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/home/hadoop-2.5.1/share/hadoop/hdfs/lib/jasper-runtime-5.5.23.jar:/home/hadoop-2.5.1/share/hadoop/hdfs/lib/jsr305-1.3.9.jar:/home/hadoop-2.5.1/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/home/hadoop-2.5.1/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/home/hadoop-2.5.1/share/hadoop/hdfs/lib/commons-io-2.4.jar:/home/hadoop-2.5.1/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/home/hadoop-2.5.1/share/hadoop/hdfs/lib/jsp-api-2.1.jar:/home/hadoop-2.5.1/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/home/hadoop-2.5.1/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/home/hadoop-2.5.1/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/home/hadoop-2.5.1/share/hadoop/hdfs/hadoop-hdfs-2.5.1.jar:/home/hadoop-2.5.1/share/hadoop/hdfs/hadoop-hdfs-nfs-2.5.1.jar:/home/hadoop-2.5.1/share/hadoop/hdfs/hadoop-hdfs-2.5.1-tests.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/jersey-server-1.9.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/asm-3.2.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/guava-11.0.2.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/jetty-6.1.26.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/xz-1.0.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/commons-lang-2.6.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/commons-codec-1.4.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/javax.inject-1.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/netty-3.6.2.Final.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/zookeeper-3.4.6.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/commons-cli-1.2.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/activation-1.1.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/commons-collections-3.2.1.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/jline-0.9.94.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/jsr305-1.3.9.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/log4j-1.2.17.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/jersey-json-1.9.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/guice-3.0.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/jersey-core-1.9.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/commons-httpclient-3.1.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/commons-io-2.4.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/jetty-util-6.1.26.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/servlet-api-2.5.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/jersey-client-1.9.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/aopalliance-1.0.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/jettison-1.1.jar:/home/hadoop-2.5.1/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/home/hadoop-2.5.1/share/hadoop/yarn/hadoop-yarn-client-2.5.1.jar:/home/hadoop-2.5.1/share/hadoop/yarn/hadoop-yarn-common-2.5.1.jar:/home/hadoop-2.5.1/share/hadoop/yarn/hadoop-yarn-server-tests-2.5.1.jar:/home/hadoop-2.5.1/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.5.1.jar:/home/hadoop-2.5.1/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.5.1.jar:/home/hadoop-2.5.1/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.5.1.jar:/home/hadoop-2.5.1/share/hadoop/yarn/hadoop-yarn-api-2.5.1.jar:/home/hadoop-2.5.1/share/hadoop/yarn/hadoop-yarn-server-common-2.5.1.jar:/home/hadoop-2.5.1/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.5.1.jar:/home/hadoop-2.5.1/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.5.1.jar:/home/hadoop-2.5.1/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.5.1.jar:/home/hadoop-2.5.1/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.9.13.jar:/home/hadoop-2.5.1/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/home/hadoop-2.5.1/share/hadoop/mapreduce/lib/asm-3.2.jar:/home/hadoop-2.5.1/share/hadoop/mapreduce/lib/xz-1.0.jar:/home/hadoop-2.5.1/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/home/hadoop-2.5.1/share/hadoop/mapreduce/lib/javax.inject-1.jar:/home/hadoop-2.5.1/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/home/hadoop-2.5.1/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/home/hadoop-2.5.1/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/home/hadoop-2.5.1/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar:/home/hadoop-2.5.1/share/hadoop/mapreduce/lib/jackson-core-asl-1.9.13.jar:/home/hadoop-2.5.1/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/home/hadoop-2.5.1/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/home/hadoop-2.5.1/share/hadoop/mapreduce/lib/avro-1.7.4.jar:/home/hadoop-2.5.1/share/hadoop/mapreduce/lib/guice-3.0.jar:/home/hadoop-2.5.1/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/home/hadoop-2.5.1/share/hadoop/mapreduce/lib/hadoop-annotations-2.5.1.jar:/home/hadoop-2.5.1/share/hadoop/mapreduce/lib/commons-io-2.4.jar:/home/hadoop-2.5.1/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/home/hadoop-2.5.1/share/hadoop/mapreduce/lib/junit-4.11.jar:/home/hadoop-2.5.1/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/home/hadoop-2.5.1/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/home/hadoop-2.5.1/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/home/hadoop-2.5.1/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.5.1.jar:/home/hadoop-2.5.1/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.5.1.jar:/home/hadoop-2.5.1/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.5.1.jar:/home/hadoop-2.5.1/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.5.1-tests.jar:/home/hadoop-2.5.1/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.5.1.jar:/home/hadoop-2.5.1/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.5.1.jar:/home/hadoop-2.5.1/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.5.1.jar:/home/hadoop-2.5.1/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.5.1.jar:/home/hadoop-2.5.1/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.5.1.jar:/home/hadoop-2.5.1/contrib/capacity-scheduler/*.jar  
  8. STARTUP_MSG:   build = https://git-wip-us.apache.org/repos/asf/hadoop.git -r 2e18d179e4a8065b6a9f29cf2de9451891265cce; compiled by 'jenkins' on 2014-09-05T23:11Z  
  9. STARTUP_MSG:   java = 1.7.0_65  
  10. ************************************************************/  
  11. 14/09/21 11:57:22 INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT]  
  12. 14/09/21 11:57:22 INFO namenode.NameNode: createNameNode [-format]  
  13. 14/09/21 11:57:26 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable  
  14. Formatting using clusterid: CID-85e34a63-6cd7-4f1e-bbb9-add72ccaa660  
  15. 14/09/21 11:57:29 INFO namenode.FSNamesystem: fsLock is fair:true  
  16. 14/09/21 11:57:30 INFO blockmanagement.DatanodeManager: dfs.block.invalidate.limit=1000  
  17. 14/09/21 11:57:30 INFO blockmanagement.DatanodeManager: dfs.namenode.datanode.registration.ip-hostname-check=true  
  18. 14/09/21 11:57:30 INFO blockmanagement.BlockManager: dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000  
  19. 14/09/21 11:57:30 INFO blockmanagement.BlockManager: The block deletion will start around 2014 九月 21 11:57:30  
  20. 14/09/21 11:57:30 INFO util.GSet: Computing capacity for map BlocksMap  
  21. 14/09/21 11:57:30 INFO util.GSet: VM type       = 64-bit  
  22. 14/09/21 11:57:30 INFO util.GSet: 2.0% max memory 966.7 MB = 19.3 MB  
  23. 14/09/21 11:57:30 INFO util.GSet: capacity      = 2^21 = 2097152 entries  
  24. 14/09/21 11:57:31 INFO blockmanagement.BlockManager: dfs.block.access.token.enable=false  
  25. 14/09/21 11:57:31 INFO blockmanagement.BlockManager: defaultReplication         = 3  
  26. 14/09/21 11:57:31 INFO blockmanagement.BlockManager: maxReplication             = 512  
  27. 14/09/21 11:57:31 INFO blockmanagement.BlockManager: minReplication             = 1  
  28. 14/09/21 11:57:31 INFO blockmanagement.BlockManager: maxReplicationStreams      = 2  
  29. 14/09/21 11:57:31 INFO blockmanagement.BlockManager: shouldCheckForEnoughRacks  = false  
  30. 14/09/21 11:57:31 INFO blockmanagement.BlockManager: replicationRecheckInterval = 3000  
  31. 14/09/21 11:57:31 INFO blockmanagement.BlockManager: encryptDataTransfer        = false  
  32. 14/09/21 11:57:31 INFO blockmanagement.BlockManager: maxNumBlocksToLog          = 1000  
  33. 14/09/21 11:57:31 INFO namenode.FSNamesystem: fsOwner             = root (auth:SIMPLE)  
  34. 14/09/21 11:57:31 INFO namenode.FSNamesystem: supergroup          = supergroup  
  35. 14/09/21 11:57:31 INFO namenode.FSNamesystem: isPermissionEnabled = true  
  36. 14/09/21 11:57:31 INFO namenode.FSNamesystem: Determined nameservice ID: hadoop-cluster1  
  37. 14/09/21 11:57:31 INFO namenode.FSNamesystem: HA Enabled: false  
  38. 14/09/21 11:57:31 INFO namenode.FSNamesystem: Append Enabled: true  
  39. 14/09/21 11:57:33 INFO util.GSet: Computing capacity for map INodeMap  
  40. 14/09/21 11:57:33 INFO util.GSet: VM type       = 64-bit  
  41. 14/09/21 11:57:33 INFO util.GSet: 1.0% max memory 966.7 MB = 9.7 MB  
  42. 14/09/21 11:57:33 INFO util.GSet: capacity      = 2^20 = 1048576 entries  
  43. 14/09/21 11:57:33 INFO namenode.NameNode: Caching file names occuring more than 10 times  
  44. 14/09/21 11:57:33 INFO util.GSet: Computing capacity for map cachedBlocks  
  45. 14/09/21 11:57:33 INFO util.GSet: VM type       = 64-bit  
  46. 14/09/21 11:57:33 INFO util.GSet: 0.25% max memory 966.7 MB = 2.4 MB  
  47. 14/09/21 11:57:33 INFO util.GSet: capacity      = 2^18 = 262144 entries  
  48. 14/09/21 11:57:33 INFO namenode.FSNamesystem: dfs.namenode.safemode.threshold-pct = 0.9990000128746033  
  49. 14/09/21 11:57:33 INFO namenode.FSNamesystem: dfs.namenode.safemode.min.datanodes = 0  
  50. 14/09/21 11:57:33 INFO namenode.FSNamesystem: dfs.namenode.safemode.extension     = 30000  
  51. 14/09/21 11:57:33 INFO namenode.FSNamesystem: Retry cache on namenode is enabled  
  52. 14/09/21 11:57:33 INFO namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis  
  53. 14/09/21 11:57:33 INFO util.GSet: Computing capacity for map NameNodeRetryCache  
  54. 14/09/21 11:57:33 INFO util.GSet: VM type       = 64-bit  
  55. 14/09/21 11:57:33 INFO util.GSet: 0.029999999329447746% max memory 966.7 MB = 297.0 KB  
  56. 14/09/21 11:57:33 INFO util.GSet: capacity      = 2^15 = 32768 entries  
  57. 14/09/21 11:57:33 INFO namenode.NNConf: ACLs enabled? false  
  58. 14/09/21 11:57:33 INFO namenode.NNConf: XAttrs enabled? true  
  59. 14/09/21 11:57:33 INFO namenode.NNConf: Maximum size of an xattr: 16384  
  60. 14/09/21 11:57:34 INFO namenode.FSImage: Allocated new BlockPoolId: BP-955896090-127.0.0.1-1411271853454  
  61. 14/09/21 11:57:34 INFO common.Storage: Storage directory /home/hadoop/dfs/name has been successfully formatted.  
  62. 14/09/21 11:57:36 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0  
  63. 14/09/21 11:57:36 INFO util.ExitUtil: Exiting with status 0  
  64. 14/09/21 11:57:36 INFO namenode.NameNode: SHUTDOWN_MSG:   
  65. /************************************************************  
  66. SHUTDOWN_MSG: Shutting down NameNode at nameNode/127.0.0.1  
  67. ************************************************************/  

 


2.5.启动、停止服务

现在可以启动服务了

2.5.1.启动


[plain] view plaincopyprint?在CODE上查看代码片派生到我的代码片
  1. [root@nameNode sbin]# ./start-dfs.sh  


 

[plain] view plaincopyprint?在CODE上查看代码片派生到我的代码片
  1. [root@nameNode sbin]# ./start-yarn.sh  

 


2.5.2.停止

[plain] view plaincopyprint?在CODE上查看代码片派生到我的代码片
  1. [root@nameNode sbin]# ./stop-dfs.sh  

[plain] view plaincopyprint?在CODE上查看代码片派生到我的代码片
  1. [root@nameNode sbin]# ./stop-yarn.sh  


3.验证


3.1.查看启动的进程

[plain] view plaincopyprint?在CODE上查看代码片派生到我的代码片
  1. [root@nameNode sbin]# jps  
[plain] view plaincopyprint?在CODE上查看代码片派生到我的代码片
  1. 7854 Jps  
  2. 7594 ResourceManager  
  3. 7357 NameNode  

3.2.通过浏览器访问

http://192.168.17.100:50070/



http://192.168.17.100:8088/


 

 

原文：http://blog.csdn.net/greensurfer/article/details/39450369

 

0 0
ps做淘宝主图 ps做淘宝主图
原创粉丝点击
热门IT博客
警用手铐淘宝能买吗警用手铐淘宝能买吗
360无线网卡网络不稳定360无线网卡网络不稳定
淘宝网男装内衣
水星网络u3驱动
mac怎么用sai
龙芯cpu能跑windows
6轴传感器imu算法
大数据技术和标准化
网络打印机怎么安装
淘宝怎么销量做上去
泉州广电网络
哔哩哔哩 mac客户端
提高优化效率的词语
win7音效增强软件
王者荣耀代打接单软件
three.js源码下载
linux的几个版本
youtube是什么软件
windows ping命令
怎样做淘宝客
热门问题 老师的惩罚 人脸识别 我在镇武司摸鱼那些年 重生之率土为王 我在大康的咸鱼生活 盘龙之生命进化 天生仙种 凡人之先天五行 春回大明朝 姑娘不必设防，我是瞎子 递推公式 递推 数列递推公式 等差数列的递推公式 递推宝i版安卓版 数列递推公式九种例题 数列递推公式13种 途中 秋浦途中 途中吉他谱 途中歌词 公交司机途中发病 上下班途中自己摔伤算工伤 凉宫凉上学途中 上班途中摔伤算工伤吗 只为途中与你相见 去黄龙景区途中4100米海拔 长征途中的故事 上班途中工伤新规定 2018年上下班途中工伤新规定 长沙女律师办案途中被撞身亡 图乐 东风日产途乐报价 途乐什么时候换代 途乐国外售价 尼桑图乐 日产 途乐 尼桑乐途 日产涂乐 日产乐途 尼桑涂乐 尼桑途乐越野车价格 途乐4.0 途乐y60 途乐越野车2017款图片 日产途乐价格 途乐汽车 途乐y61价格 尼桑途乐新款图片 尼桑途乐越野车 途乐和陆巡哪个好

程序博客网，程序员的互联网技术博客家园。csdn论坛精品 msdn技术资料都在这里