hadoop&hbase学习00--安装hadoop

软件 版本 系统 red hat 6.5 hadoop hadoop-2.7.3 hbase hbase-1.2.4

一、hadoop的配置

1.1、环境变量配置

vi /etc/profile 添加Java， hadoop, hbase的配置#java 环境变量export JAVA_HOME=/usr/java/export JAVA_BIN=$JAVA_HOME/binexport PATH=$PATH:$JAVA_BINexport CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar#hadoop 环境变量export HADOOP_HOME=/var/www/html/hadoop-2.7.3export PATH=$PATH:$HADOOP_HOME/bin#hbase 环境变量export HBASE_HOME=/var/www/html/hbase-1.2.4export PATH=$PATH:$HBASE_HOME/bin

注意使用source /etc/profile使环境变量生效

1.2、配置主从节点文件

#配置主节点vi /var/www/html/hadoop-2.7.3/etc/hadoop/mastermaster#配置从节点vi /var/www/html/hadoop-2.7.3/etc/hadoop/slavesmasterslave1slave2

1.3、配置host文件

vi /etc/hosts

1.3、core-site.xml的配置

<!-- Put site-specific property overrides in this file. --><configuration>    <property>        <name>fs.default.name</name>        <value>hdfs://master:9000</value>    </property>    <property>        <name>fs.defaultFS</name>        <value>hdfs://master:9000</value>    </property>    <property>        <name>io.file.buffer.size</name>        <value>131072</value>    </property>    <property>        <name>hadoop.tmp.dir</name>        <value>file:/usr/local/hadoop/tmp</value>        <description>A base for other temporary directories.</description>    </property>    <property>        <name>hadoop.proxyuser.hduser.hosts</name>        <value>*</value>    </property>    <property>        <name>hadoop.proxyuser.hduser.groups</name>        <value>*</value>    </property></configuration>

1.4、hdfs-site.xml

<configuration>    <property>        <name>dfs.namenode.name.dir</name>        <value>file:/usr/local/hadoop/hdfs/name</value>        </property>    <property>        <name>dfs.datanode.data.dir</name>        <value>file:/usr/local/hadoop/hdfs/data</value>        </property>    <property>        <name>dfs.replication</name>        <value>3</value>        </property>    <property>        <name>dfs.webhdfs.enabled</name>        <value>true</value>        </property></configuration>

1.5、yarn-site.xml配置

<configuration><property>    <name>yarn.resourcemanager.hostname</name>    <value>master</value></property><property>    <name>yarn.nodemanager.aux-services</name>    <value>mapreduce_shuffle</value></property></configuration>

1.6、mapred-site.xml

<configuration>    <property>        <name>mapred.job.tracker</name>        <value>master:9001</value>    </property>    <property>    <name>mapreduce.framework.name</name>    <value>yarn</value>    </property>    <property>          <name>mapreduce.jobhistory.address</name>          <value>master:10020</value>      </property></configuration>