hive 虚拟机下执行 HQL 被killed退出

来源:互联网 发布:目前淘宝卖什么最火 编辑:程序博客网 时间:2024/06/04 19:26

背景:学习大数据,装了个虚拟机,搭载centos7的迷你,内存分配的512MB

hive> select A.name,A.time
    > from
    > (select name,time,sum(count) from sale group by name, time) A join
    > (select name,time,sum(count) from sale group by name, time) B
    > on A.name=B.name
    > where A.time >= B.time
    > group by A.name,A.time
    > order by A.name,A.time;
WARNING: Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
Query ID = root_20170227013835_5b837518-4c01-4dd5-8097-99d9dfdd39d7
Total jobs = 7
Launching Job 1 out of 7
Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapreduce.job.reduces=<number>
Starting Job = job_1488173761905_0005, Tracking URL = http://hadoop.lsd1.com:8088/proxy/application_1488173761905_0005/
Kill Command = /bigdata/hadoop-2.6.5/bin/hadoop job  -kill job_1488173761905_0005
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2017-02-27 01:39:10,866 Stage-1 map = 0%,  reduce = 0%
Killed

[root@hadoop apache-hive-2.1.1-bin]# 

可以看出,hive退出了,但是没有打印出什么错误信息,上yarn的web端查看日志也没发现map或者reduce任务有报什么错,log就不贴了;另外这里补充一下,要查询的表的数据也都是有的,而且可以看出来hql本身也没有语法问题;咨询前辈也说看日志没什么问题,最后考虑是不是虚拟机内存太小的缘故,把虚拟机内存分配成1G(我用的是virtualbox,这个好设置,在对应虚拟机的设置--》系统 中就可以分配),然后再执行该语句正常完成,再改回原来的512MB又有时候会出现同样的问题,所以应该是内存分配不够导致的。

故而,下回如果hive出现无故退出又看不出哪里有什么问题的情况,可以考虑一下是否虚拟机内存分配得不够。


0 0
原创粉丝点击