最近重新在Hadoop集群上安装apache-hive操作记录

安装hive

安装在Hadoop的namenode上，拷贝安装文件到Linux中/usr/hadoop/apache-Hive-2.1.0-bin.tar.gz

解压：

tar –zxvf apache-hive-2.1.0-bin.tar.gz

添加到环境变量

vi /etc/profile

编辑

#hive

export HIVE_HOME=/usr/hadoop/apache-hive-2.1.0-bin

export PATH=$PATH:$HIVE_HOME/bin

保存后使其生效：

source /etc/profile

 

安装mysql作为hive的Metastore

首先检查MySQL是否已安装：

rpm -qa | grep -i mysql

结果：

mysql-libs-5.1.71-1.el6.x86_64

删除已安装的mysql

yum -y remove mysql-libs*

                                       

tar xvf MySQL-5.5.49-1.linux2.6.x86_64.rpm-bundle.tar

 

rpm -ivhMySQL-server-5.5.49-1.linux2.6.x86_64.rpm

rpm -ivhMySQL-devel-5.5.49-1.linux2.6.x86_64.rpm

rpm -ivhMySQL-client-5.5.49-1.linux2.6.x86_64.rpm

 

启动mysql

service mysqlstart

首次安装时，默认密码为空，可以使用如下命令修改root密码

mysqladmin -uroot  password mypassword

mypassword 为你设定的新密码

登录mysql

mysql -u root –p

 

rpm包安装的MySQL是不会安装/etc/my.cnf文件的，解决方法，只需要复制/usr/share/mysql目录下的my-huge.cnf 文件到/etc目录，并改名为my.cnf即可

cp/usr/share/mysql/my-huge.cnf /etc/my.cnf

 

mysql默认不可以远程访问，设置远程访问

--GRANT ALL PRIVILEGES ON *.* TO 'root'@'%'WITH GRANT OPTION;

上面这句远程访问不需要密码，如果需要密码使用下面这句

GRANT ALLPRIVILEGES ON *.* TO 'root'@'%'IDENTIFIED BY 'sa' WITH GRANT OPTION;

 

使权限生效：

FLUSH PRIVILEGES;

 

设置etc/my.cnf文件，使binlog_format=mixed

vi etc/my.cnf

将注释掉的binlog_format=mixed这一行前面的注释去掉然后保存，重启mysql即可

service mysqlrestart

配置hive

在hdfs中新建目录/user/hive/warehouse

hdfs dfs –mkdir /tmp

hdfs dfs –mkdir /user

hdfs dfs –mkdir /user/hive

hdfs dfs –mkdir -p /user/hive/warehouse

 

hdfs dfs -chmod g+w /tmp

hdfs dfs -chmod g+w /user/hive/warehouse

将mysql的驱动jar包mysql-connector-Java-5.1.7-bin.jar拷入hive的lib目录下面

进入hive的conf目录下面复制一下hive-default.xml.template名子命名为：hive-site.xml

cp hive-default.xml.template hive-site.xml

 

<property>

   <name>javax.jdo.option.ConnectionURL</name>

    <value>jdbc:mysql://127.0.0.1:3306/hive?createDatabaseIfNotExist=true&amp;useSSL=false</value>

   <description>JDBC connect string for a JDBCmetastore</description>

 </property>

 <property>

   <name>javax.jdo.option.ConnectionDriverName</name>

    <value>com.mysql.jdbc.Driver</value>

   <description>Driver class name for a JDBCmetastore</description>

 </property>

 <property>

   <name>javax.jdo.option.ConnectionUserName</name>

   <value>root</value>

   <description>Username to use against metastoredatabase</description>

 </property>

 <property>

   <name>javax.jdo.option.ConnectionPassword</name>

   <value>123456</value>

   <description>password to use against metastoredatabase</description>

 </property>

 

  <property>

   <name>hive.exec.local.scratchdir</name>

   <value>/usr/hadoop/apache-hive-2.1.0-bin/tmp</value>

   <description>Local scratch space for Hive jobs</description>

 </property>

  <property>

   <name>hive.downloaded.resources.dir</name>

   <value>/usr/hadoop/apache-hive-2.1.0-bin/tmp/resources</value>

   <description>Temporary local directory for added resources in theremote file system.</description>

 </property>

  <property>

   <name>hive.querylog.location</name>

   <value>/usr/hadoop/apache-hive-2.1.0-bin/tmp/querylog</value>

   <description>Location of Hive run time structured logfile</description>

 </property>

  <property>

   <name>hive.server2.logging.operation.log.location</name>

   <value>/usr/hadoop/apache-hive-2.1.0-bin/tmp/operation_logs</value>

    <description>Toplevel directory where operation logs are stored if logging functionality isenabled</description>

 </property>

使用schematool 初始化metastore的schema：

schematool -initSchema -dbType mysql

 

启动hive命令：

在第一个窗口中输入：hive --service hiveserver2 或者是hiveserver2

再打开一个窗口输入：beeline -u jdbc:hive2://localhost:10000 -n root

 

 

 

alter database hive character set latin1;

setcharacter_set_client=latin1;

 

hive配置遇到的问题（ Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D）

创建

/usr/local/hadoop-2.7.3/apache-hive-2.1.1-bin/tmp

/usr/local/hadoop-2.7.3/apache-hive-2.1.1-bin/tmp/resources
/usr/local/hadoop-2.7.3/apache-hive-2.1.1-bin/tmp/querylog
/usr/local/hadoop-2.7.3/apache-hive-2.1.1-bin/tmp/operation_logs

修改配置节

<property>
   <name>hive.exec.local.scratchdir</name>
   <value>/usr/local/hadoop-2.7.3/apache-hive-2.1.1-bin/tmp</value>
   <description>Local scratch space for Hive jobs</description>
 </property>
<property>
   <name>hive.downloaded.resources.dir</name>
   <value>/usr/local/hadoop-2.7.3/apache-hive-2.1.1-bin/tmp/resources</value>
   <description>Temporary local directory for added resources in theremote file system.</description>
 </property>
<property>
   <name>hive.querylog.location</name>
   <value>/usr/local/hadoop-2.7.3/apache-hive-2.1.1-bin/tmp/querylog</value>
   <description>Location of Hive run time structured logfile</description>
 </property>
<property>
   <name>hive.server2.logging.operation.log.location</name>
   <value>/usr/local/hadoop-2.7.3/apache-hive-2.1.1-bin/tmp/operation_logs</value>
    <description>Toplevel directory where operation logs are stored if logging functionality isenabled</description>
 </property>