Kylin下构建Cube第一步出错:shell-init: error retrieving current directory

来源:互联网 发布:数据库用什么语言写的 编辑:程序博客网 时间:2024/05/16 09:56

问题背景:
生产环境部署的Kylin-2.1,官方发布的最新安装包并不支持更改hbase存储的namespace,修改源码后重新打包部署过程中,build cube第一步出错

大概错误信息是:

OS command error exit with 5 – hive -e "USE default;DROP TABLE IF EXISTS kylin_intermediate_kylin_sales_cube_desc_20120101000000_20160502000000;CREATE EXTERNAL TABLE IF NOT EXISTS kylin_intermediate_kylin_sales_cube_desc_20120101000000_20160502000000(DEFAULT_KYLIN_SALES_PART_DT date,DEFAULT_KYLIN_SALES_LEAF_CATEG_ID bigint,DEFAULT_KYLIN_SALES_LSTG_SITE_ID int,DEFAULT_KYLIN_CATEGORY_GROUPINGS_META_CATEG_NAME string,DEFAULT_KYLIN_CATEGORY_GROUPINGS_CATEG_LVL2_NAME string,DEFAULT_KYLIN_CATEGORY_GROUPINGS_CATEG_LVL3_NAME string,DEFAULT_KYLIN_SALES_LSTG_FORMAT_NAME string,DEFAULT_KYLIN_SALES_PRICE decimal(19,4),DEFAULT_KYLIN_SALES_SELLER_ID bigint)ROW FORMAT DELIMITED FIELDS TERMINATED BY '\177'STORED AS SEQUENCEFILELOCATION '/kylin/kylin_metadata/kylin-5020d91f-3114-421b-929b-f733d438411a/kylin_intermediate_kylin_sales_cube_desc_20120101000000_20160502000000';SET dfs.replication=2;SET dfs.block.size=33554432;SET hive.exec.compress.output=true;SET hive.auto.convert.join.noconditionaltask=true;SET hive.auto.convert.join.noconditionaltask.size=300000000;SET mapreduce.map.output.compress.codec=org.apache.hadoop.io.compress.SnappyCodec;SET mapreduce.output.fileoutputformat.compress.codec=org.apache.hadoop.io.compress.SnappyCodec;SET hive.merge.mapfiles=true;SET hive.merge.mapredfiles=true;SET mapred.output.compression.type=BLOCK;SET hive.merge.size.per.task=256000000;SET hive.support.concurrency=false;SET mapreduce.job.split.metainfo.maxsize=-1;INSERT OVERWRITE TABLE kylin_intermediate_kylin_sales_cube_desc_20120101000000_20160502000000 SELECTKYLIN_SALES.PART_DT,KYLIN_SALES.LEAF_CATEG_ID,KYLIN_SALES.LSTG_SITE_ID,KYLIN_CATEGORY_GROUPINGS.META_CATEG_NAME,KYLIN_CATEGORY_GROUPINGS.CATEG_LVL2_NAME,KYLIN_CATEGORY_GROUPINGS.CATEG_LVL3_NAME,KYLIN_SALES.LSTG_FORMAT_NAME,KYLIN_SALES.PRICE,KYLIN_SALES.SELLER_IDFROM DEFAULT.KYLIN_SALES as KYLIN_SALES INNER JOIN DEFAULT.KYLIN_CAL_DT as KYLIN_CAL_DTON KYLIN_SALES.PART_DT = KYLIN_CAL_DT.CAL_DTINNER JOIN DEFAULT.KYLIN_CATEGORY_GROUPINGS as KYLIN_CATEGORY_GROUPINGSON KYLIN_SALES.LEAF_CATEG_ID = KYLIN_CATEGORY_GROUPINGS.LEAF_CATEG_ID AND KYLIN_SALES.LSTG_SITE_ID = KYLIN_CATEGORY_GROUPINGS.SITE_IDWHERE (KYLIN_SALES.PART_DT >= '2012-01-01' AND KYLIN_SALES.PART_DT < '2016-05-02');"shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directoryshell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directorychdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directorychdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directorychdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directoryshell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directorychdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directorychdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directorychdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directoryshell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directorychdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directorychdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directorychdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directorychdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directoryUnable to determine Hadoop version information.'hadoop version' returned:shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directorychdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directorychdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directorychdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directorychdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directoryError occurred during initialization of VM java.lang.Error: Properties init: Could not determine current working directory. at java.lang.System.initProperties(Native Method) at java.lang.System.initializeSystemClass(System.java:1119)

Kylin之前是没有问题的,构建了很多cube都没有问题,即便是出错按理讲也不会出现在第一步,第一步的动作是创建一个宽表,放到hive作为临时表。
解决思路:
第一感觉是权限问题,修改了sh脚本的执行权限,修改了hdfs上 kylin 工作目录的权限后,问题仍然存在。
重新找原因,其中有报工作目录的问题,看上去是本地目录并不是Hdfs上的目录,这才想起可能是上次重新部署Kylin的时候把旧版本的直接删除 ,但并没有停掉进程。

ps aux | grep kylin

查看进程查然有两个Kylin进程在跑,于是Kill掉旧的进程

突然想起之前有人误删了kylin的文件夹,所以去查看了一下,果然某一台机器下面存在两个kylin进程。kill 后重启就好了。

猜测的原因是原进程的安装目录已经不存在了,并行着两个Kylin
进程产生冲突。

阅读全文
0 0
原创粉丝点击