hadoopCDH4.1+zookeeperCDH4.1+hbaseCDH4.1环境搭建

来源：互联网发布：南京市软件定制开发编辑：程序博客网时间：2024/05/29 12:00

一、安装准备工作。

（1）3台或以上liunx_64机器。

（2）每台机器安装JDK1.6或以上版本。

（3）从https://ccp.cloudera.com/display/SUPPORT/CDH4+Downloadable+Tarballs+-+4.1.1下载hadoop,hbase,zookeeper的CDH4版本，下载需要注册。

（4）本地安装ftp(Winscp),xshell工具。

二、配置hadoop。

（1）core-site.xml配置

<configuration>
   <property>
       <name>fs.default.name</name>
       <value>hdfs://test01:9000</value>
       <final>true</final>
   </property>
   <property>
       <name>io.native.lib.available</name>
       <value>true</value>
   </property>
   <property>
       <name>hadoop.tmp.dir</name>
       <value>/data/hadoop/tmp</value>
   </property>
</configuration>

（2）hadoop-env.sh

export JAVA_HOME=/usr/java/jdk1.6.0_31

（3）hdfs-site.xml

<configuration>
   <property>
     <name>dfs.replication</name>
     <value>3</value>
   </property>
   <property>
     <name>dfs.support.append</name>
     <value>true</value>
   </property>
   <property>
     <name>dfs.permissions</name>
     <value>false</value>
   </property>
   <property>
     <name>dfs.datanode.max.xcievers</name>
     <value>4096</value>
   </property>
</configuration>

（4）mapred-site.xml

<configuration>
       <property>
          <name>mapreduce.framework.name</name>
          <value>yarn</value>
       </property>
       <property>
         <name>mapred.job.tracker</name>
         <value>test01:9001</value>
         <final>true</final>
       </property>
       <property>
         <name>mapred.map.java.opts</name>
         <value>-Xmx1024m</value>
       </property>
       <property>
         <name>mapred.child.java.opts</name>
         <value>-Xmx1024m</value>
       </property>
</configuration>

（5）yarn-site.xml

<configuration>
      <property>
                   <name>yarn.resourcemanager.address</name>
                   <value>test01:18080</value>
         </property>
         <property>
                   <name>yarn.resourcemanager.scheduler.address</name>
                   <value>test01:18081</value>
         </property>
         <property>
                   <name>yarn.resourcemanager.resource-tracker.address</name>
                   <value>test01:18082</value>
         </property>
         <property>
                   <name>yarn.nodemanager.aux-services</name>
                   <value>mapreduce.shuffle</value>
         </property>
         <property>
                   <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
                   <value>org.apache.hadoop.mapred.ShuffleHandler</value>
         </property>
         <property>
                   <name>yarn.nodemanager.local-dirs</name>
                   <value>file:/data/hadoop/nmdata/local</value>
                   <description>thelocal directories used by the nodemanager</description>
         </property>
         <property>
                   <name>yarn.nodemanager.log-dirs</name>
                   <value>file:/data/hadoop/nmdata/log</value>
         </property>
</configuration>

（6）yarn-env.xml

    export JAVA_HOME=/usr/java/jdk1.6.0_30
    export HADOOP_HOME=/home/fdhadoop/hadoop-2.0.0-cdh4.1.1
    export HADOOP_PREFIX=${HADOOP_HOME}
    export HADOOP_MAPRED_HOME=${HADOOP_HOME}
    export HADOOP_COMMON_HOME=${HADOOP_HOME}
    export HADOOP_HDFS_HOME=${HADOOP_HOME}
    export HADOOP_YARN_HOME=${HADOOP_HOME}
    export PATH=$PATH:${JAVA_HOME}/bin:${HADOOP_HOME}/bin:${HADOOP_HOME}/sbin
    export JAVA_HOME JAVA_BIN PATH CLASSPATH JAVA_OPTS
    export HADOOP_LIB=${HADOOP_HOME}/lib
    export HADOOP_CONF_DIR=${HADOOP_HOME}/etc/hadoop

（7）slaves

test01
test02
test03

三、配置hbase。

（1）hbase-site.xml

<configuration>
    <property>
       <name>hbase.rootdir</name>
       <value>hdfs://test01:9000/hbase</value>
    </property>
    <property>
        <name>hbase.cluster.distributed</name>
        <value>true</value>
    </property>
    <property>
        <name>hbase.master</name>
        <value>test01:60000</value>
    </property>
    <property>
        <name>hbase.zookeeper.quorum</name>
        <value>test01,test02,test03</value> #奇数个
    </property>
    <property>
        <name>hbase.zookeeper.property.clientPort</name>
        <value>2181</value>
    </property>
     <property>
         <name>hbase.regionserver.lease.period</name>
         <value>120000</value>
     </property>
     <property>
         <name>hbase.rpc.timeout</name>
         <value>120000</value>
     </property>
</configuration>

（2）hbase-env.xml

    export JAVA_HOME=/usr/java/jdk1.6.0_30
    export HBASE_OPTS="$HBASE_OPTS -XX:+HeapDumpOnOutOfMemoryError -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode"
    export HBASE_MANAGES_ZK=true
    export HADOOP_HOME=/home/fdhadoop/hadoop-2.0.0-cdh4.1.1
    export HBASE_LOG_DIR=/home/fdhadoop/hbase-0.92.1-cdh4.1.1/logs
    export HBASE_CLASSPATH=/home/fdhadoop/hadoop-2.0.0-cdh4.1.1/etc/hadoop

（3）regionservers

test01
test02
test03

（4）将zook.cfg和hdfs-site.xml拷贝到目录下。

四、配置zookeeper。
             tickTime=2000
             initLimit=10
             syncLimit=5
           dataDir=/data/hadoop/zookeeper
             clientPort=2181
             server.1=172.18.117.32:2888:3888
             server.2=172.18.117.109:2888:3888
             server.3=172.18.117.69:2888:3888

五、启动环境。

（1）启动hadoop