linux中hive安装和部署详解

来源：互联网发布：as3 数组删除指定元素编辑：程序博客网时间：2024/05/17 00:59

一.前期准备

1.1 hadoop

版本：Hadoop 2.6.5

安装：http://blog.csdn.net/a123demi/article/details/70652959

1.2 mysql

版本：5.6.33 MySQL Community Server (GPL)

1.3 mysql驱动包

版本：mysql-connector-java-5.1.40-bin.jar

1.4 hive安装包

官网下载：apache-hive-2.1.1-bin.tar.gz

二.hive安装

2.1 hive部署

#定位cd /opt/software#解压tar -zxvf apache-hive-2.1.1-bin.tar.gz #复制hive解压到/usr/local/hivecp -r apache-hive-2.1.1-bin /usr/local/hive

2.2 mysql驱动包导入

把1.3中的mysql驱动包放置到$HIVE_HOME\lib目录

三.hadoop配置

3.1 启动dfs+yarn

#定位cd /usr/local/hadoop#启动dfs+yarnsbin/start-all.sh

3.2 创建HDFS目录并赋予权限

hdfs dfs -mkdir -p /usr/hive/warehousehdfs dfs -mkdir -p /usr/hive/tmphdfs dfs -mkdir -p /usr/hive/loghdfs dfs -chmod g+w /usr/hive/warehousehdfs dfs -chmod g+w /usr/hive/tmphdfs dfs -chmod g+w /usr/hive/log

四.hive配置

4.1 配置环境变量

vi /etc/profile--------------------------------------------#hiveexport HIVE_HOME=/usr/local/hive#pathexport PATH=$PATH:$HIVE_HOME/bin--------------------------------------------重新编译profilesource /etc/profile

4.2 hive-env.sh配置

#定位cd /usr/local/hive/conf#生成hive-env.sh文件cp -r hive-env.sh.template hive-env.sh#配置vi hive-env.sh--------------------------------------------# Folder containing extra ibraries required for hive compilation/execution can be controlled by:# export HIVE_AUX_JARS_PATH=export JAVA_HOME=/usr/local/jdkexport HADOOP_HOME=/usr/local/hadoopexport HIVE_HOME=/usr/local/hive# HADOOP_HOME=${bin}/../../hadoop# Hive Configuration Directory can be controlled by:export HIVE_CONF_DIR=$HIVE_HOME/conf# Folder containing extra ibraries required for hive compilation/execution can be controlled by:export HIVE_AUX_JARS_PATH=/usr/local/hive/lib/*

4.3 hive-site.xml配置

#生成hive-site.xml文件cp -r hive-default.xml.template hive-site.xml #配置vi hive-site.xml--------------------------------------------<configuration>  <!-- WARNING!!! This file is auto generated for documentation purposes ONLY! -->  <!-- WARNING!!! Any changes you make to this file will be ignored by Hive.   -->  <!-- WARNING!!! You must make your changes in hive-site.xml instead.         -->  <!-- Hive Execution Parameters -->  <property>    <name>hive.exec.scratchdir</name>    <value>/usr/hive/tmp</value>    <description>HDFS root scratch dir for Hive jobs which gets created with write all (733) permission. For each connecting user, an HDFS scratch dir: ${hive.exec.scratchdir}/<username> is created, with ${hive.scratch.dir.permission}.</description>  </property>  <property>    <name>hive.metastore.warehouse.dir</name>    <value>/usr/hive/warehouse</value>    <description>location of default database for the warehouse</description>  </property><property>    <name>hive.querylog.location</name>    <value>/usr/hive/log</value>    <description>Location of Hive run time structured log file</description>  </property> <property>    <name>javax.jdo.option.ConnectionURL</name>    <value>jdbc:mysql://192.168.32.128:3306/hive?createDatabaseIfNotExist=true&characterEncoding=UTF-8&useSSL=false</value>  </property>  <property>    <name>javax.jdo.option.ConnectionDriverName</name>    <value>com.mysql.jdbc.Driver</value>  </property>  <property>    <name>javax.jdo.option.ConnectionUserName</name>    <value>root</value>  </property>  <property>    <name>javax.jdo.option.ConnectionPassword</name>    <value>root</value>  </property></configuration>

4.4 注意

本文mysql用户使用的是root用户，生产环境可以自建一个用户，

详情可参看：https://my.oschina.net/jackieyeah/blog/735424

五.测试实现

5.1schematool 命令来执行初始化操作

从 Hive 2.1 版本开始, 我们需要先运行 schematool 命令来执行初始化操作，

schematool -dbType mysql -initSchema

5.2 Hive客户端启动

#命令hive

hive操作的命令基本与mysql一致。

阅读全文

0 0