Cloudera 5.7集群部署（二）--CM、CDH安装

1 Cloudera安装配置

1.1 安装前注意事项

前期在fengkongj上操作，再将配置好的安装包拷给fengkongb,fengkongc,fengkongd,fengkonge, fengkongf, fengkongg, fengkongh, fengkongi,基本思路是：先在后台配置好cm(cloudera management)、parcels，然后在页面上操作cm，安装zookeeper、yarn、spark、hbase等。参考官方文档：http://www.cloudera.com/documentation/enterprise/latest/topics/cm_ig_install_path_c.html

1.2 安装配置cm
1） 在http://archive.cloudera.com/cm5/cm/5/下载：cloudera-manager-el6-cm5.7.0_x86_64.tar.gz
2） 创建/opt/cloudera-manager目录,上传并将tar包解压到该目录下
3） 创建用户

 useradd --system --home=/opt/cloudera-manager/cm-5.7.0/run/cloudera-scm-server --no-create-home --comment "Cloudera SCM User" cloudera-scm

4） 修改拥有者用户

 chown -R cloudera-scm:cloudera-scm /opt/cloudera-manager

1.3 安装配置parcels
1） cloudera-cdh5 parcels下载地址: http://archive.cloudera.com/cdh5/parcels/5.7.0/

2） 按照官网的文件格式下载源文件, 上传到linux.
目录为/opt/cloudera/parcel-repo,文件夹拥有者要改为cloudera-scm：

 chown -R cloudera-scm:cloudera-scm /opt/cloudera

3） 并在/var/www/html上配置链接：

 ln -s /opt/cloudera/parcel-repo /var/www/html/CDH5.7.0

打开浏览器，输入192.168.89.10/CDH5.7.3，查看效果：

4） 修改manifest.json,删除剩下以下parcelName的字符簇:
…
“parcelName”: “CDH-5.4.0-1.cdh5.4.0.p0.27-el7.parcel”,
….
5） 修改sha1为sha:

mv CDH-5.5.2-1.cdh5.5.2.p0.4-el7.parcel.sha1 CDH-5.5.2-1.cdh5.5.2.p0.4-el7.parcel.sha

1.4 配置cdh库和用户
注：可选
create database amon DEFAULT CHARACTER SETutf8;
grant all on amon.* TO ’ amon’@’%’ IDENTIFIEDBY ‘amon_123’;
create database rman DEFAULT CHARACTER SETutf8;
grant all on rman.* TO ‘rman’@’%’ IDENTIFIEDBY ‘rman_123’;
create database metastore DEFAULT CHARACTERSET utf8;
grant all on metastore.* TO ‘hive’@’%’IDENTIFIED BY ‘hive_123’;
create database sentry DEFAULT CHARACTER SETutf8;
grant all on sentry.* TO ‘sentry’@’%’IDENTIFIED BY ‘sentry_123’;
create database nav DEFAULT CHARACTER SETutf8;
grant all on nav.* TO ‘nav’@’%’ IDENTIFIED BY’nav_123’;
create database navms DEFAULT CHARACTER SETutf8;
grant all on navms.* TO ‘navms’@’%’ IDENTIFIEDBY ‘navms_123’;
create database hue DEFAULT CHARACTER SETutf8;
grant all on hue.* TO ‘hue’@’%’ IDENTIFIED BY’hue_123’;
create database hive DEFAULT CHARACTER SETutf8;
grant all on hive.* TO ‘hive’@’%’ IDENTIFIEDBY ‘hive_123’;
create database oozie DEFAULT CHARACTER SETutf8;
grant all on oozie.* TO ‘oozie’@’%’ IDENTIFIEDBY ‘oozie_123’;
create database cdhreport DEFAULT CHARACTERSET utf8;
grant all on cdhreport.* TO ‘report’@’%’IDENTIFIED BY ‘report_123’;
create database cdhmonitor DEFAULT CHARACTERSET utf8;
grant all on cdhmonitor.* TO ‘monitor’@’%’IDENTIFIED BY ‘monitor_123’;

mysql> flush privileges;

1.5 mysql客户端驱动

1） 下载http://dev.mysql.com/downloads/connector/j/5.1.html
2） 上传mysql-connector-java-5.1.38.tar.gz
3） 解压后将mysql-connector-java-5.1.38-bin.jar放入/opt/cloudera-manager/cm-5.7.0/share/cmf/lib
4） 也放一份在下面目录

cp mysql-connector-java-5.1.38-bin.jar /usr/share/java/mysql-connector-java.jar

1.6 创建scm数据库

vi /opt/cloudera-manager/cm-5.7.0/share/cmf/schema/scm_prepare_database.sh

JAVA_HOME=/opt/jdk1.8.0_60

 /opt/cloudera-manager/cm-5.7.0/share/cmf/schema/scm_prepare_database.sh mysql -h fengkonga -P 3306 -uroot -p123456 --scm-host fengkonga scm scm scm_123

注：输出一下内容，说明成功：
…
Executing: /opt/jdk1.8.0_60/bin/java -cp/usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/opt/cloudera-manager/cm-5.7.3/share/cmf/schema/.
./lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /opt/cloudera-manager/cm-5.7.0/etc/cloudera-scm-server/db.propertiescom.cloudera.cmf.db.[main] DbCommandExecutor INFOSuccessfullyconnected to database.
All done,your SCM database is configured correctly!

1.7 Cloudera agent安装

fengkongb,fengkongc,fengkongd, fengkonge, fengkongf, fengkongg, fengkongh, fengkongi:   

vi /opt/cloudera-manager/cm-5.7.0/etc/cloudera-scm-agent/config.ini

…
server_host=192.168.89.10
…

fengkongj:

sudo scp -r cloudera fengkongb:/optsudo scp -rcloudera-manager fengkongb:/optsudo scp -r cloudera fengkongc:/optsudo scp -rcloudera-manager fengkongc:/optsudo scp -r cloudera fengkongd:/optsudo scp -rcloudera-manager fengkongd:/optsudo scp -r cloudera fengkonge:/optsudo scp -rcloudera-manager fengkonge:/optsudo scp -r cloudera fengkongf:/optsudo scp -rcloudera-manager fengkongf:/optsudo scp -r cloudera fengkongg:/optsudo scp -rcloudera-manager fengkongg:/optsudo scp -r cloudera fengkongh:/optsudo scp -rcloudera-manager fengkongh:/optsudo scp -r cloudera fengkongi:/optsudo scp -r cloudera-managerfengkongi:/opt

fengkongb,fengkongc,fengkongd, fengkonge, fengkongf, fengkongg, fengkongh, fengkongi:

useradd --system--home=/opt/cloudera-manager/cm-5.7.0/run/cloudera-scm-server --no-create-home --comment "Cloudera SCM User" cloudera-scmchown -R cloudera-scm:cloudera-scm /opt/cloudera-managerchown -R cloudera-scm:cloudera-scm /opt/cloudera

1.8 Cloudera server启动

fengkongj：

/opt/cloudera-manager/cm-5.7.0/etc/init.d/cloudera-scm-server starttail -f /opt/cloudera-manager/cm-5.7.0/log/cloudera-scm-server/cloudera-scm-server.log

注：server端在初始化，看日志知道出现以下字样表明初始化完成：
。。。
2016-03-23 11:17:50,333 INFOWebServerImpl:org.mortbay.log: jetty-6.1.26.cloudera.4
2016-03-23 11:17:50,334 INFOWebServerImpl:org.mortbay.log: Started SelectChannelConnector@0.0.0.0:7180
2016-03-23 11:17:50,334 INFOWebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.

1.9 Cloudera agent启动

/opt/cloudera-manager/cm-5.7.0/etc/init.d/cloudera-scm-agent start tail -f /opt/cloudera-manager/cm-5.7.0/log/cloudera-scm-agent/cloudera-scm-agent.log

注：如果启动失败，检查/opt/cloudera-manager/cm-5.7.0/run目录下是否有cloudera-scm-agent文件夹，没有则新建：

mkdir cloudera-scm-agentchown cloudera-scm:cloudera-scm cloudera-scm-agent/

注：
When you start the Cloudera Manager Server and Agents,Cloudera Manager assumes you are not already running HDFS and MapReduce.
参考
http://www.cloudera.com/documentation/enterprise/5-5-x/topics/cm_ig_install_path_c.html?scroll=cmig_topic_6_7

1.10 Cdh倚赖包安装

yum -y install bind-utils psmisc libxsltzlib sqlite cyrus-sasl-plain cyrus-sasl-gssapi fuse portmap fuse-libsredhat-lsb

1.11 访问Cloudera管理台WEB
浏览器输入：
192.168.89.10:7180
默认：
用户：admin
密码：admin
出现登录界面则表示cm安装成功。