安装Cloudera Manager
来源:互联网 发布:mac 怎么使用语音朗读 编辑:程序博客网 时间:2024/05/16 14:18
cloudera manager有四大功能:
(1)管理:对集群进行管理,如添加、删除节点等操作。(2)监控:监控集群的健康情况,对设置的各种指标和系统运行情况进行全面监控。(3)诊断:对集群出现的问题进行诊断,对出现的问题给出建议解决方案。(4)集成:对hadoop的多组件进行整合。
一、环境准备
搭建好yum源
(1)设置IP
#vim /etc/sysconfig/network-scripts/ifcfg-eth0
(2)修改主机名
#vim /etc/sysconfig/network NETWORKING=yes HOSTNAME=hadoop1
(3)关闭seliunx
#setenforce 0 (临时生效) #vi /etc/selinux/config SELINUX=disabled (重启后永久生效)
(4)关闭防火墙
service iptables stop chkconfig iptables off
(5)同步时间
#ntpdate 192.168.1.68
(6)修改hosts文件
#vim /etc/hosts 192.168.3.68 hadoop1 192.168.3.69 hadoop2 192.168.3.70 hadoop3
(7)实现免密登录(root)
安装scp (yum install openssh-clients -y) #ssh-keygen #ls authorized_keys:存放远程免密登录的公钥,主要通过这个文件记录多台机器的公钥 id_rsa : 生成的私钥文件 id_rsa.pub : 生成的公钥文件 know_hosts : 已知的主机公钥清单 如果希望ssh公钥生效需满足至少下面两个条件: 1) .ssh目录的权限必须是700 2) .ssh/authorized_keys文件权限必须是600
(8)安装JDK7
#yum install java-1.7.0 -y
二、开始安装
(1)cdh下载地址
http://archive.cloudera.com/cdh5/parcels/5.10.0.41/CDH-5.10.0-1.cdh5.10.0.p0.41-el6.parcel
http://archive.cloudera.com/cdh5/parcels/5.10.0.41/CDH-5.10.0-1.cdh5.10.0.p0.41-el6.parcel.sha1
http://archive.cloudera.com/cdh5/parcels/5.10.0.41/manifest.json
(2)cdm下载地址
http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5.10.0/RPMS/x86_64/cloudera-manager-agent-5.10.0-1.cm5100.p0.85.el6.x86_64.rpm
http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5.10.0/RPMS/x86_64/cloudera-manager-daemons-5.10.0-1.cm5100.p0.85.el6.x86_64.rpm
http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5.10.0/RPMS/x86_64/cloudera-manager-server-5.10.0-1.cm5100.p0.85.el6.x86_64.rpm
(3)安装依赖包
1、下载gdbm-devel包(因为lsb依赖gdbm-devel包) #wget http://ie.archive.ubuntu.com/pld-linux/pool/g/gdbm-devel/gdbm-devel-1.8.0-16.i686.rpm #rpm -ivh gdbm-devel-1.8.0-16.i686.rpm 2、安装各种依赖 #yum install bind-utils libxslt cyrus-sasl-plain cyrus-sasl-gssapi fuse portmap perl fuse-libs httpd mod_ssl openssl-devel python-psycopg2 MySQL-python lsb -y
(4)安装cloudera-manager包
#rpm -ivh cloudera-manager-daemons-5.10.0-1.cm5100.p0.85.el6.x86_64.rpm #rpm -ivh cloudera-manager-agent-5.10.0-1.cm5100.p0.85.el6.x86_64.rpm #rpm -ivh cloudera-manager-server-5.10.0-1.cm5100.p0.85.el6.x86_64.rpm
三、Server主节点设置
(1)安装配置MySQL数据库
1、 yum install mysql-server -y 2、vim /etc/my.cnf (CDH要求引擎为INNODB) [mysqld] default-storage-engine=INNODB 3、启动mysql service mysqld start chkconfig mysqld off 4、修改密码 mysqladmin -uroot password "123456"
(2)安装mysql-jdk的驱动器
下载JDBC驱动,http://dev.mysql.com/downloads/connector/j/,
#mv mysql-connector-java-5.1.40-bin.jar /usr/share/cmf/lib/
(3)初始化数据
#/usr/share/cmf/schema/scm_prepare_database.sh mysql -h localhost -uroot -p'123456' --scm-host localhost scm scm scm初始化数据库后,可查看/etc/cloudera-scm-server/ db.properties
(4)、CDH的parcel相关的包放到/opt/cloudera/parcel-repo/目录下
(注意 .sha1改为.sha)
# cd /opt/cloudera/parcel-repo/#ls CDH-5.10.0-1.cdh5.10.0.p0.41-el6.parcel CDH-5.10.0-1.cdh5.10.0.p0.41-el6.parcel.sha manifest.json
(5)搭建http服务
#mkdir /var/www/html/parcel-repo/#ln -s /opt/cloudera/parcel-repo/ /var/www/html/parcel-repo/
(6)启动server
#/etc/init.d/cloudera-scm-server start #lsof -i:7180
四、agent从节点配置
1、修改agent配置
#vim /etc/cloudera-scm-agent/config.ini server_host=hadoop1 #主节点
2、从节点启动agent
#/etc/init.d/cloudera-scm-agent start
查看错误日志目录
#/var/log/cloudera-scm-server/cloudera-scm-server.log
五、登录安装服务
浏览器:主节点IP:7180
登录web界面:admin和admin
1.首先启动cloudera manager服务
2.修改Parcel仓库的远程URL为:http://IP/parcel-repo/
六、授权agent能登录数据库
#mysql -uroot -p123456>grant all on scm.* to 'scm'@'192.168.1.165' identified by 'scm';
参考:
http://192.168.1.118:7180/static/help/topics/cm_ig_installing_configuring_dbs.html
http://www.cnblogs.com/jasondan/p/4011153.html
- Cloudera Manager安装
- 手动安装Cloudera Manager
- 安装cloudera-manager
- Cloudera Manager安装
- 安装Cloudera Manager 5
- Cloudera Manager 5安装
- cloudera manager安装
- cloudera-manager安装
- 安装Cloudera Manager
- 离线安装cloudera Manager
- cloudera-manager安装
- Cloudera学习(1)------安装Cloudera Manager
- Cloudera Manager安装部署手册
- Cloudera Manager CDH4的安装
- Cloudera Manager安装部署手册
- cloudera manager 安装时候出现
- Cloudera Manager下安装kafka
- Cloudera Manager及CDH安装
- codeforces781A Andryusha and Colored Balloons
- 57. Insert Interval
- OutputStreamWriter和InputStreamReader以及Writer和Reader和PrintWriter
- Java学习笔记(Java程序设计实用教程4th——叶核亚)
- Android系统源代码情景分析读书笔记
- 安装Cloudera Manager
- 玲珑杯round11-E:萌萌哒的第五题
- 文章标题
- 深度学习数据集
- 如何安全地监控远程JVM/Tomcat
- atoi函数源代码
- pureftp默认支持匿名用户的
- 关于js原型链 原型的个人理解
- ROS service入门到应用