hive异常MetaException-Metastore contains multiple versions

来源：互联网发布：淘宝宝贝怎么上架编辑：程序博客网时间：2024/05/17 02:06

在执行hive运行脚本时，出现了MetaException Metastore contains multiple versions异常错误

Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClientat org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:522)at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:497)at org.apache.hadoop.util.RunJar.run(RunJar.java:221)at org.apache.hadoop.util.RunJar.main(RunJar.java:136)Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClientat org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1523)at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)... 8 moreCaused by: java.lang.reflect.InvocationTargetExceptionat sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)at java.lang.reflect.Constructor.newInstance(Constructor.java:422)at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)... 14 moreCaused by: MetaException(message:Metastore contains multiple versions (2) [ version = 1.2.0, comment = Set by MetaStore spark@10.50.70.120 ] [ version = 1.2.0, comment = Set by MetaStore spark@10.50.70.121 ])at org.apache.hadoop.hive.metastore.ObjectStore.getMSchemaVersion(ObjectStore.java:6745)at org.apache.hadoop.hive.metastore.ObjectStore.getMetaStoreSchemaVersion(ObjectStore.java:6703)at org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:6661)at org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java:6645)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:497)at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)at com.sun.proxy.$Proxy6.verifySchema(Unknown Source)at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:572)at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)... 19 more

在spark中使用hivecontext时也会遇到这个异常，原因在于hive的metastore表VERSION中存在重复的记录，从而报出存在多条hive版本的异常信息

VER_IDSCHEMA_VERSIONVERSION_COMMENTID主键Hive版本版本说明10.13.0Set by MetaStore解决办法：

1. 删除VERSION表中多余的版本信息即可解决

2.修改hive-site.xml中配置，取消metastore版本信息校验

<property>  <name>hive.metastore.schema.verification</name>  <value>false</value><property>

这个异常信息多次出现，每次都是由spark的一台work插入进去，正好这台机器是虚拟机节点非物理机，猜测是在这台虚拟机网络环境不好的时候，检查不到hive metastore中version版本信息再插入一条新的记录导致重复异常的

参考：https://issues.apache.org/jira/browse/HIVE-9543

0 0