Linux7 下Hadoop集群用户管理方案之五 安装Hadoop集群遇到的坑

自己在安装时候遇到的一部分的坑。

密码都不能用了。。

难道是因为。。改了kdc.conf?? 从ip改成端口了

PS。这个还没有解决。。Kerberos创建用户，创建的用户密码过期之后不可用的问题。Kerberos还没有完全吃透，还没解决。

/opt/cm-5.9.0/share/cmf/bin/gen_credentials.sh failed with exit code 1 and output of <<+ export PATH=/usr/kerberos/bin:/usr/kerberos/sbin:/usr/lib/mit/sbin:/usr/sbin:/usr/lib/mit/bin:/usr/bin:/sbin:/usr/sbin:/bin:/usr/bin+ PATH=/usr/kerberos/bin:/usr/kerberos/sbin:/usr/lib/mit/sbin:/usr/sbin:/usr/lib/mit/bin:/usr/bin:/sbin:/usr/sbin:/bin:/usr/bin+ CMF_REALM=HADOOP.COM+ KEYTAB_OUT=/tmp/cmf1986556804032057570.keytab+ PRINC=yarn/[e-mail]+ MAX_RENEW_LIFE=432000+ KADMIN='kadmin -k -t /tmp/cmf817148388354671948.keytab -p cloudera-scm/[e-mail] -r HADOOP.COM'+ RENEW_ARG=+ '[' 432000 -gt 0 ']'+ RENEW_ARG='-maxrenewlife "432000 sec"'+ '[' -z /tmp/krb59017674432558873467.conf ']'+ echo 'Using custom config path '\''/tmp/krb59017674432558873467.conf'\'', contents below:'+ cat /tmp/krb59017674432558873467.conf+ kadmin -k -t /tmp/cmf817148388354671948.keytab -p cloudera-scm/[e-mail] -r HADOOP.COM -q 'addprinc -maxrenewlife "432000 sec" -randkey yarn/[e-mail]'kadmin: Incorrect password while initializing kadmin interface>>

就是因为密码不行了。。

怎么弄一个永久的密码呢

kinit: Cannot determine realm for host (principal host/6accda1c7f91@)

 Failed on local exception: java.io.IOException: java.lang.IllegalArgumentException: Server has invalid Kerberos prin

cipal: hdfs/[e-mail]; Host Details : local host is: "6accda1c7f91/172.19.0.2"; destination host is: "10.20.66.212":8020;

解决方案：需要在/etc/hosts 中配置  10.20.66.212 master

/opt/cm-5.9.0/share/cmf/bin/gen_credentials.sh failed with exit code 1 and output of <<
+ export PATH=/usr/kerberos/bin:/usr/kerberos/sbin:/usr/lib/mit/sbin:/usr/sbin:/usr/lib/mit/bin:/usr/bin:/sbin:/usr/sbin:/bin:/usr/bin
+ PATH=/usr/kerberos/bin:/usr/kerberos/sbin:/usr/lib/mit/sbin:/usr/sbin:/usr/lib/mit/bin:/usr/bin:/sbin:/usr/sbin:/bin:/usr/bin
+ CMF_REALM=HADOOP.COM
+ KEYTAB_OUT=/tmp/cmf2885609246198664122.keytab
+ PRINC=yarn/[e-mail]
+ MAX_RENEW_LIFE=432000
+ KADMIN='kadmin -k -t /tmp/cmf8365314439897372734.keytab -p cloudera-scm/[e-mail] -r HADOOP.COM'
+ RENEW_ARG=
+ '[' 432000 -gt 0 ']'
+ RENEW_ARG='-maxrenewlife "432000 sec"'
+ '[' -z /tmp/krb54503148931985644829.conf ']'
+ echo 'Using custom config path '\''/tmp/krb54503148931985644829.conf'\'', contents below:'
+ cat /tmp/krb54503148931985644829.conf
+ kadmin -k -t /tmp/cmf8365314439897372734.keytab -p cloudera-scm/[e-mail] -r HADOOP.COM -q 'addprinc -maxrenewlife "432000 sec" -randkey yarn/[e-mail]'
kadmin: Cannot contact any KDC for realm 'HADOOP.COM' while initializing kadmin interface

>>

kinit: Cannot contact any KDC for realm 'HADOOP.COM' while getting initial credentials

解决方案  1：查看/etc/krb5.conf

2: 查看/etc/hosts

3. :查看Kerberos服务有没有启动

启动Agent报错

.tail -200f /opt/cm-5.9.0/log/cloudera-scm-agent/cloudera-scm-agent.log

[25/Apr/2017 13:15:31 +0000] 15953 MainThread agent        INFO     CM server guid: 8dd2c1ca-ef6c-434f-93bf-267c0fd763c7

[25/Apr/2017 13:15:31 +0000] 15953 MainThread agent        INFO     Using parcels directory from server provided value: /opt/cloudera/parcels

[25/Apr/2017 13:15:31 +0000] 15953 MainThread parcel       INFO     Agent does create users/groups and apply file permissions

[25/Apr/2017 13:15:31 +0000] 15953 MainThread parcel_cache INFO     Using /opt/cloudera/parcel-cache for parcel cache

[25/Apr/2017 13:15:31 +0000] 15953 MainThread agent        ERROR    Caught unexpected exception in main loop.

Traceback (most recent call last):

  File "/opt/cm-5.9.0/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.9.0-py2.7.egg/cmf/agent.py", line 758, in start

    self._init_after_first_heartbeat_response(resp_data)

  File "/opt/cm-5.9.0/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.9.0-py2.7.egg/cmf/agent.py", line 938, in _init_after_first_heartbeat_response

    self.client_configs.load()

  File "/opt/cm-5.9.0/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.9.0-py2.7.egg/cmf/client_configs.py", line 682, in load

    new_deployed.update(self._lookup_alternatives(fname))

  File "/opt/cm-5.9.0/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.9.0-py2.7.egg/cmf/client_configs.py", line 432, in _lookup_alternatives

    return self._parse_alternatives(alt_name, out)

  File "/opt/cm-5.9.0/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.9.0-py2.7.egg/cmf/client_configs.py", line 444, in _parse_alternatives

    path, _, _, priority_str = line.rstrip().split(" ")

ValueError: too many values to unpack

解决方案：

http://community.cloudera.com/t5/Cloudera-Manager-Installation/Freeze-while-activating-the-CDH-Parcels/td-p/47771

好像是JDK的版本问题？

https://community.cloudera.com/t5/Cloudera-Manager-Installation/Freeze-while-activating-the-CDH-Parcels/m-p/47805#M8795

需要删除OpenJDK。使用Oracle JDK。

注意source 

另外。。注意  export JAVA_HOME  ……犯了低级错误

无法使用pstree命令

1. #On Mac OS  
2. brew install pstree  
3.   
4. #On Fedora/Red Hat/CentOS  
5. yum install psmisc #using psmisc package for pstree  
6.   
7. #On Ubuntu/Debian APT  
8. apt-get install psmisc  

又一个问题。

Failed to format NameNode

/usr/lib/hadoop-hdfs/bin/hdfs: 没有那个文件或目录

一直听在添加节点的页面。。显示delete CDH

难道因为没有用户？？

useradd --system --home=/opt/cm-5.6.0/run/cloudera-scm-server --shell=/bin/false--comment "Cloudera SCM User" cloudera-scm

chown cloudera-scm:cloudera-scm /opt/cloudera/parcels 

mkdir -p /opt/cloudera/parcel-repo 

chown cloudera-scm:cloudera-scm /opt/cloudera/parcel-repo 

mkdir -p /opt/cloudera/parcels 

chown cloudera-scm:cloudera-scm /opt/cloudera/parcels 

解决方案：

果然是没有用户，还有就是免密登陆

另外就是把agent节点上 parcel-repo 目录删掉。

把parcles目录中的残留数据清空

启动Hive的时候 message:Version information not found in metastore

Parcel not distributed but have active state ACTIVATING

可以尝试把 /opt/cloudera/parcels 下的残留文件删除

Failed to format namenode

在每个节点可以尝试创建cloudera-scm用户   另外设置对应的权限。

新的一个Hadoop的机器。访问总是跟我说

 Server has invalid Kerberos principal: hdfs/[e-mail]; Host Details : local host is: "shihys-MacBook-Pro.local/10.20.5.172"; destination host is: "10.10.106.156":8020;

想起来没有设置 /etc/hosts

设置上之后。。

又出新错。

Failed on local exception: java.io.IOException: Couldn't setup connection for [e-mail] to edu-bigdata-01.novalocal/10.10.106.156:8020; Host Details : local host is: "shihys-MacBook-Pro.local/10.20.5.174"; destination host is: "edu-bigdata-01.novalocal":8020;

 [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]

kinit: krb5_get_init_creds: time skew (43202) larger than max (300)

时间不对。相差太多

我去了。。。竟然是因为Kerberos 时间跟本地时间相差太多。。

终于找到原因了。。晕死了。。

主要需要看一下时区。。

EDU  CST。。。

date  -U 看UTC时间。。最准了。