load数据到hive分区表报错: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTas
来源:互联网 发布:java中main函数的参数 编辑:程序博客网 时间:2024/05/16 05:21
在启动hive创建表的时候,兴许你们回碰到这个错误:http://blog.csdn.net/qq_35732963/article/details/54139581;百度一下,说是字符编码的问题,那个好解决,但是下面这个错,却不一样,归根到底是数据库出了问题!
1、上传数据到hive表时报错:
hive> load data local inpath '/home/hadoop/data1.txt' into table partition_tablespartition(gender='M');
Loading data to table hive1.partition_tables partition (gender=M)
Failed with exception MetaException(message:javax.jdo.JDODataStoreException: Error(s) were found while auto-creating/validating the datastore for classes. The errors are printed in the log, and are attached to this exception.
at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)
at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:732)
at org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:752)
at org.apache.hadoop.hive.metastore.ObjectStore.addPartition(ObjectStore.java:1431)
。。。。
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask
2、说到底还是改下数据库方面的:删除原先自动创建的hive数据库,新建metastore数据库
mysql> create database metastore;
Query OK, 1 row affected (0.01 sec)
mysql> show create database metastore;
+-----------+--------------------------------------------------------------------+
| Database | Create Database |
+-----------+--------------------------------------------------------------------+
| metastore | CREATE DATABASE `metastore` /*!40100 DEFAULT CHARACTER SET utf8 */ |
+-----------+--------------------------------------------------------------------+
1 row in set (0.00 sec)
mysql> alter database metastore character set latin1;
Query OK, 1 row affected (0.00 sec)
3、去hadoop用户下修改hive/conf里面的hive-site.xml
这里原本是hive的数据库,但由于报错,把原来的hive'删了,新建一个metastore
[hadoop@master conf]$ vi hive-site.xml
<!-- metadata database connection configuration -->
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://localhost:3306/metastore?createDatabaseIfNotExist=true</value> ----这个参数使用来设置元数据连接字串
<description>JDBC connect string for a JDBC metastore</description>
</property>
4、 ---给创建的2个文件 权限
[root@master hadoop]# ll
total 76
drwxrwxr-x. 4 hadoop hadoop 4096 Dec 26 19:30 app
-rw-r--r-- 1 hadoop hadoop 55 Jan 9 19:12 data1.txt
-rw-r--r-- 1 root root 44 Jan 9 19:13 data2.txt
[root@master hadoop]# chgrp hadoop data2.txt
[root@master hadoop]# chown hadoop data2.txt
5、启动hive
[hadoop@master conf]$ hive
Logging initialized using configuration in file:/home/hadoop/hive-0.14.0/conf/hive-log4j.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hadoop/hadoop-2.6.2/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hadoop/hive-0.14.0/lib/hive-jdbc-0.14.0-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
hive> create database hive1;
OK
Time taken: 0.415 seconds
hive> create table partition_tables
> (tid int,tname string, age int)
> partitioned by (gender string)
> row format delimited fields terminated by ',';
OK
Time taken: 0.328 seconds
6、再次上传文件到分区表,显示上传成功
hive> load data local inpath '/home/hadoop/data1.txt' into table partition_tables partition(gender='M');
Loading data to table default.partition_tables partition (gender=M)
Partition default.partition_tables{gender=M} stats: [numFiles=1, numRows=0, totalSize=55, rawDataSize=0]
OK
Time taken: 1.138 seconds
hive> load data local inpath '/home/hadoop/data2.txt' into table partition_tables partition(gender='F');
Loading data to table default.partition_tables partition (gender=F)
Partition default.partition_tables{gender=F} stats: [numFiles=1, numRows=0, totalSize=44, rawDataSize=0]
OK
Time taken: 0.661 seconds
7、进入http://master:50070/查看partition_table分区是否成功
- load数据到hive分区表报错: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTas
- FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask
- FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(me
- FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask.
- FAILED: Execution Error, return code 3 from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
- FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
- FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask
- FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
- FAILED: Execution Error, return code 3 from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
- Hive报错:FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
- 程序中执行hive sql出错[FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask]
- Hive连接Mysql---FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
- hive创建表失败:FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
- Hive连接Mysql---FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
- hive Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException
- FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org/bson/convers
- FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.Runtim
- Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
- fragment+viewpager+tab(PagerSlidingTabStrip)
- 使用Hudson进行持续集成
- 数据结构:平衡二叉树(AVL树)
- 剑指offer——左旋转字符串
- 舒服
- load数据到hive分区表报错: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTas
- 极大似然估计值的标准差
- HttpCore
- leetcode7. Reverse Integer
- 日常训练 巨神兵 obelisk
- [git] submodule
- *args是非关键字参数,用于元组,**kw是关键字参数,用于字典
- PAT甲级1027
- Opencv卷积滤波cvFilter2D-高通与低通