hadoop上hive安装配置及报错处理

参考网址：http://nunknown.com/study/282/#3
1. 在master上安装hive和mysql，mysql作为元数据存储位置：
hive下载： wget http://mirror.bit.edu.cn/apache/hive/hive-2.1.1/apache-hive-2.1.1-bin.tar.gz
解压： tar -zxf apache-hive-2.1.1-bin.tar.gz
配置环境变量：
HIVE_HOME=/home/hadoop/apache-hive-1.2.1-bin
PATH=$PATH:$HIVE_HOME/bin
2. Metastore：
Ubuntu上安装MySQL：

         apt-getinstall mysql-server         apt-getisntall mysql-client

执行：apt-get isntall mysql-client

     E: Invalid operation isntall

原因：
apt-getinstall mysql-server命令会安装以下包：
apparmor
mysql-client
mysql-common
mysql-server
mysql-server
mysql-server-core
验证：netstat -tap | grep mysql
进入mysql： mysql -u root -p 密码：111111
创建Hive用户：

CREATEUSER 'hive' IDENTIFIED BY '111111';GRANTALL PRIVILEGES ON *.* TO 'hive'@'master' WITH GRANT OPTION;//分配权限flushprivileges;//跟新配置

报错：
connection refuse （[mysqld]）
/etc/mysql/my.cnf中设置了
bind_address=127.0.0.1
说明：这种情况可以TCP/IP连接，不能使用hostname连接
注释掉就可以了

建立 Hive 专用的元数据库：
create database hive;
3. hive启动：
启动前：
schematool -initSchema -dbType mysql createDatabaseIfNotExist=true
这里使用mysql数据库而不是默认数据库
报错处理：
[Fatal Error] hive-site.xml:1:2: The markup in the documentpreceding the root element must be well-formed.
<1?xmlversion=”1.0” encoding=”UTF-8”standalone=”no”?>
多添加了一个“1”

Caused by: MetaException(message:Version information notfound in metastore. )

修改conf/hive-site.xml 中的“hive.metastore.schema.verification” 值为 false

Exception in thread "main"java.lang.RuntimeException: java.net.ConnectException: Call Frommaster/172.16.*.* to master:9000 failed on connection exception:java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused

解决：namenode未启动，重新启动即可

Exception in thread "main"java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative pathin absolute URI: ${system:java.io.tmpdir%7D/$%7Bhive.session.id%7D_resources    Caused by: java.net.URISyntaxException: Relative pathin absolute URI: ${system:java.io.tmpdir%7D/$%7Bhive.session.id%7D_resources

解决：路径设置问题

<property>    <name>hive.exec.local.scratchdir</name>    <value>/usr/local/apache-hive-2.1.1-bin/iotmp</value>    <!--value>${system:java.io.tmpdir}/${system:user.name}</value-->    <description>Localscratch space for Hive jobs</description></property><property>    <name>hive.downloaded.resources.dir</name>    <value>/tmp/hive/resources</value>    <!--value>${system:java.io.tmpdir}/${hive.session.id}_resources</value-->    <description>Temporarylocal directory for added resources in the remote filesystem.</description></property>```4. 移动到其它节点：<div class="se-preview-section-delimiter"></div>

scp -r /usr/local/apache-hive-2.1.1-bin node1:/usr/local

报错：<div class="se-preview-section-delimiter"></div>

Caused by: java.sql.SQLException: null, message from server: “Host’116.62..’ is not allowed to connect to this MySQL server”

mysql没有授权该IP：可以在创建数据库是授权不限制IP登陆<div class="se-preview-section-delimiter"></div>

grantall PRIVILEGES on . to hive@’%’ identified by ‘111111’;
flushprivileges;
“`

ACID and Transactions in Hive参见网址：
https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions
https://cwiki.apache.org/confluence/display/Hive/Streaming+Data+Ingest#StreamingDataIngest-TransactionBatch