Hadoop单机模式中Hive安装配置
来源:互联网 发布:浮雕设计软件 编辑:程序博客网 时间:2024/06/09 12:05
本文主要是演示在Hadoop单机模式中Hive默认(嵌入式derby 模式)安装配置过程,目录结构如下:
- 基础环境
- Hive安装配置
- 启动及演示
[一]、基础环境
- Mac OSX 10.9.1
- Java 1.6+
- Hadoop 2.2.0 (安装配置详见: http://www.micmiu.com/opensource/hadoop/hadoop2x-single-node-setup/ )
- Hive 0.12.0 (截止2014-02-09最新的发布版本)
[二]、Hive安装配置
1、下载发布包
到官方下载最近发布包以 0.12.0为例:
$ tar -zxf hive-0.12.0-bin.tar.gz -C /usr/local/share $ cd /usr/local/share$ ln -s hive-0.12.0-bin hive
本文中 HIVE_HOME = “/usr/local/share/”
2、设置环境变量
执行 vi ~/.profile
,添加如下内容:
#Hive @micmiu.comexport HIVE_HOME="/usr/local/share/hive"export PATH=$HIVE_HOME/bin:$PATH
3、配置文件
在目录 <HIVE_HOME>/conf
目录下有4个模板文件:
hive-default.xml.templatehive-env.sh.templatehive-exec-log4j.properties.templatehive-log4j.properties.template
copy 生成四个配置文件然后既可自定义相关属性:
$ cd /usr/local/share/hive/conf$ copy hive-default.xml.template hive-site.xml$ copy hive-env.sh.template hive-env.sh$ copy hive-exec-log4j.properties.template hive-exec-log4j.properties$ copy hive-log4j.properties.template hive-log4j.properties
ps:注意文件名称: hive-site.xml
,本文以嵌入式derby 模式做演示,故以默认配置即可无效修改相关参数。
不过官方0.12.0的发布版本中的 hive-default.xml.template 中有 bug,在 2000行:
<value>auth</auth>
修改为: <value>auth</value>
有关 hive.metastore.schema.verification 版本检查的问题,有两个解决办法
方法一: 修改配置文件
第一次运行前先将 hive.metastore.schema.verification 设为false
......<!-- 设为false 不做验证--><name>hive.metastore.schema.verification</name><value>false</value>......
方法二: 不改配置,先初始化好数据
执行初始化命令: schematool -dbType derby -initSchema
micmiu-mbp:~ micmiu$ schematool -dbType derby -initSchemaMetastore connection URL: jdbc:derby:;databaseName=metastore_db;create=trueMetastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriverMetastore connection User: APPStarting metastore schema initialization to 0.12.0Initialization script hive-schema-0.12.0.derby.sqlInitialization script completedschemaTool completeted
查看初始化后的信息: schematool -dbType derby -info
micmiu-mbp:~ micmiu$ schematool -dbType derby -infoMetastore connection URL: jdbc:derby:;databaseName=metastore_db;create=trueMetastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriverMetastore connection User: APPHive distribution version: 0.12.0Metastore schema version: 0.12.0schemaTool completeted
详见: https://cwiki.apache.org/confluence/display/Hive/Hive+Schema+Tool
以上方法都可以,否则第一次运行时会类似如下的报错信息:
ERROR exec.DDLTask (DDLTask.java:execute(435)) - org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient at org.apache.hadoop.hive.ql.metadata.Hive.getDatabase(Hive.java:1143) ......Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1212) ......Caused by: java.lang.reflect.InvocationTargetException ......Caused by: MetaException(message:Version information not found in metastore. ) at org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:5638) at org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java:5622) ......
4、配置dfs中得目录和权限
$ hdfs dfs -mkdir /tmp$ hdfs dfs -mkdir /user/hive/warehouse$ hdfs dfs -chmod g+w /tmp$ hdfs dfs -chmod g+w /user/hive/warehouse
[三]、运行和测试
确保 HADOOP_HOME 在环境变量中配置好,然后以CLI(command line interface)方式下运行,直接执行命令 hive
即可,然后执行一些测试命令如下:
hive> show databases;OKdefaultTime taken: 4.966 seconds, Fetched: 1 row(s)hive> show tables;OKTime taken: 0.186 secondshive> CREATE TABLE micmiu_blog (id INT, siteurl STRING);OKTime taken: 0.359 secondshive> SHOW TABLES;OKmicmiu_blogTime taken: 0.023 seconds, Fetched: 1 row(s)hive>
到此嵌入式derby 模式下的Hive安装配置已经成功。
参考:
- https://cwiki.apache.org/confluence/display/Hive/GettingStarted
- https://cwiki.apache.org/confluence/display/Hive/Hive+Schema+Tool
- https://cwiki.apache.org/confluence/display/Hive/AdminManual+MetastoreAdmin
Metastore connection URL: jdbc:derby:;databaseName=metastore_db;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 1.2.0
Initialization script hive-schema-1.2.0.derby.sql
[ERROR] Terminal initialization failed; falling back to unsupported
java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but interface was expected
at jline.TerminalFactory.create(TerminalFactory.java:101)
at org.apache.hive.beeline.BeeLineOpts.<init>(BeeLineOpts.java:102)
at org.apache.hive.beeline.BeeLine.<init>(BeeLine.java:117)
条件:hive1.2.1
hadoop2.6.0
原因:
hadoop目录下存在老版本jline:
/hadoop-2.6.0/share/hadoop/yarn/lib:
-rw-r--r-- 1 root root 87325 Mar 10 18:10 jline-0.9.94.jar
解决:
cp /hive/apache-hive-1.1.0-bin/lib/jline-2.12.jar /hadoop-2.5.2/share/hadoop/yarn/lib
错误二
Exception in thread "main"java.lang.RuntimeException: java.lang.IllegalArgumentException:java.net.URISyntaxException: Relative path in absolute URI:${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D
atorg.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:444)
atorg.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:672)
解决办法:
1.查看hive-site.xml配置,会看到配置值含有"system:java.io.tmpdir"的配置项
2.新建文件夹/home/grid/hive-0.14.0-bin/iotmp
3.将含有"system:java.io.tmpdir"的配置项的值修改为如上地址
启动hive,成功!
—————– EOF @Michael Sun —————–
- Hadoop单机模式中Hive安装配置
- Hadoop单机模式的配置与安装
- Hadoop环境安装初探-单机模式配置
- Hadoop环境安装初探-单机模式配置
- hadoop单机模式配置
- Hadoop单机模式配置
- hadoop单机模式安装
- Hadoop单机模式安装
- Hadoop单机模式安装
- HADOOP单机模式安装
- Hadoop单机模式安装
- hadoop单机安装配置
- Hadoop初识--Hadoop单机模式安装和环境配置
- CDH5 hadoop-hive-habse单机版配置
- Hadoop单机(独立)模式的安装与配置
- Hadoop配置之单机模式
- 单机模式hadoop的配置
- Ubuntu14.04.2中配置单机模式的Hadoop
- 网易云音乐地址解析含官方接口分享
- aws的rds导出问题
- Android中Drawable适配介绍
- Periodic Signal BeiJing 2016 区域赛
- vue.js下载
- Hadoop单机模式中Hive安装配置
- 浅析 Linux poll 机制
- android 程序根据重力感应切换程序的方向
- IOS9App Transport Security has blocked a cleartext HTTP (http://) resource load since it is
- mac卸载mysql
- HTTP状态码总结
- 设计模式(三)建造者模式(创建型)
- 好的文档收集
- Report Studio 分组 分组表头