安装Cloudera Manager

来源:互联网 发布:mac 怎么使用语音朗读 编辑:程序博客网 时间:2024/05/16 14:18

cloudera manager有四大功能:

(1)管理:对集群进行管理,如添加、删除节点等操作。(2)监控:监控集群的健康情况,对设置的各种指标和系统运行情况进行全面监控。(3)诊断:对集群出现的问题进行诊断,对出现的问题给出建议解决方案。(4)集成:对hadoop的多组件进行整合。

一、环境准备

搭建好yum源
(1)设置IP

    #vim /etc/sysconfig/network-scripts/ifcfg-eth0  

(2)修改主机名

    #vim /etc/sysconfig/network     NETWORKING=yes     HOSTNAME=hadoop1

(3)关闭seliunx

    #setenforce 0 (临时生效)      #vi /etc/selinux/config       SELINUX=disabled (重启后永久生效)

(4)关闭防火墙

    service iptables stop     chkconfig iptables off 

(5)同步时间

   #ntpdate 192.168.1.68

(6)修改hosts文件

   #vim /etc/hosts    192.168.3.68 hadoop1    192.168.3.69 hadoop2    192.168.3.70 hadoop3

(7)实现免密登录(root)

   安装scp (yum install openssh-clients -y#ssh-keygen   #ls    authorized_keys:存放远程免密登录的公钥,主要通过这个文件记录多台机器的公钥  id_rsa : 生成的私钥文件  id_rsa.pub : 生成的公钥文件  know_hosts : 已知的主机公钥清单  如果希望ssh公钥生效需满足至少下面两个条件:     1) .ssh目录的权限必须是700      2) .ssh/authorized_keys文件权限必须是600

(8)安装JDK7

    #yum install java-1.7.0 -y

二、开始安装

(1)cdh下载地址
http://archive.cloudera.com/cdh5/parcels/5.10.0.41/CDH-5.10.0-1.cdh5.10.0.p0.41-el6.parcel
http://archive.cloudera.com/cdh5/parcels/5.10.0.41/CDH-5.10.0-1.cdh5.10.0.p0.41-el6.parcel.sha1
http://archive.cloudera.com/cdh5/parcels/5.10.0.41/manifest.json

(2)cdm下载地址
http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5.10.0/RPMS/x86_64/cloudera-manager-agent-5.10.0-1.cm5100.p0.85.el6.x86_64.rpm
http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5.10.0/RPMS/x86_64/cloudera-manager-daemons-5.10.0-1.cm5100.p0.85.el6.x86_64.rpm
http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5.10.0/RPMS/x86_64/cloudera-manager-server-5.10.0-1.cm5100.p0.85.el6.x86_64.rpm

(3)安装依赖包

  1、下载gdbm-devel包(因为lsb依赖gdbm-devel包)  #wget http://ie.archive.ubuntu.com/pld-linux/pool/g/gdbm-devel/gdbm-devel-1.8.0-16.i686.rpm  #rpm -ivh gdbm-devel-1.8.0-16.i686.rpm  2、安装各种依赖  #yum install bind-utils libxslt cyrus-sasl-plain cyrus-sasl-gssapi fuse portmap  perl fuse-libs httpd  mod_ssl openssl-devel python-psycopg2  MySQL-python lsb -y  

(4)安装cloudera-manager包

#rpm -ivh cloudera-manager-daemons-5.10.0-1.cm5100.p0.85.el6.x86_64.rpm #rpm -ivh cloudera-manager-agent-5.10.0-1.cm5100.p0.85.el6.x86_64.rpm  #rpm -ivh cloudera-manager-server-5.10.0-1.cm5100.p0.85.el6.x86_64.rpm 

三、Server主节点设置
(1)安装配置MySQL数据库

     1、 yum install mysql-server -y     2、vim /etc/my.cnf (CDH要求引擎为INNODB)        [mysqld]         default-storage-engine=INNODB     3、启动mysql         service mysqld start         chkconfig mysqld off     4、修改密码       mysqladmin -uroot password "123456"

(2)安装mysql-jdk的驱动器
下载JDBC驱动,http://dev.mysql.com/downloads/connector/j/,

  #mv mysql-connector-java-5.1.40-bin.jar /usr/share/cmf/lib/

(3)初始化数据

#/usr/share/cmf/schema/scm_prepare_database.sh mysql -h localhost -uroot -p'123456' --scm-host localhost scm scm scm初始化数据库后,可查看/etc/cloudera-scm-server/ db.properties       

(4)、CDH的parcel相关的包放到/opt/cloudera/parcel-repo/目录下
(注意 .sha1改为.sha)

# cd /opt/cloudera/parcel-repo/#ls   CDH-5.10.0-1.cdh5.10.0.p0.41-el6.parcel  CDH-5.10.0-1.cdh5.10.0.p0.41-el6.parcel.sha  manifest.json

(5)搭建http服务

#mkdir /var/www/html/parcel-repo/#ln -s /opt/cloudera/parcel-repo/ /var/www/html/parcel-repo/    

(6)启动server

  #/etc/init.d/cloudera-scm-server start  #lsof -i:7180

四、agent从节点配置

1、修改agent配置

  #vim /etc/cloudera-scm-agent/config.ini  server_host=hadoop1  #主节点

2、从节点启动agent

    #/etc/init.d/cloudera-scm-agent start

查看错误日志目录

   #/var/log/cloudera-scm-server/cloudera-scm-server.log

五、登录安装服务

浏览器:主节点IP:7180
登录web界面:admin和admin

1.首先启动cloudera manager服务
2.修改Parcel仓库的远程URL为:http://IP/parcel-repo/

六、授权agent能登录数据库

#mysql -uroot -p123456>grant  all on scm.* to 'scm'@'192.168.1.165' identified by 'scm';

参考:
http://192.168.1.118:7180/static/help/topics/cm_ig_installing_configuring_dbs.html
http://www.cnblogs.com/jasondan/p/4011153.html

0 0
原创粉丝点击