Deploy_Cluster_CDH
来源:互联网 发布:床上用品知乎 编辑:程序博客网 时间:2024/04/27 14:08
1.卸载java:rpm -qa | grep java
卸载:rpm -e --nodeps 查询出来的包名
2.配置ssh: cd ~/.ssh/
ssh-keygen -t rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
chmod 644 id_rsa.pub
6.修改主机名: vi /etc/sysconfig/network 通过重启网络服务生效。
7.修改主机名和ip的对应关系:vi /etc/hosts
2.java环境:rpm -ivh jdk-7u60-linux-x64.rpm
2.解压缩CM:tar -zxvf cloudera-manager-el6-cm5.2.0_x86_64.tar.gz
3.将安装包放到cloudera/parcel-repo/下:mv CDH-5.2.0-1.cdh5.2.0.p0.36-el6.parcel* cloudera/parcel-repo/ mv manifest.json cloudera/parcel-repo/
4.将连接mysql的jar放到指定目录下:cp mysql-connector-java-5.1.16-bin.jar /opt/cm-5.2.0/share/cmf/lib/
10.agent的配置主节点的ip:vi /opt/cm-5.2.0/etc/cloudera-scm-agent/config.ini server_host:10.0.4.15
5.安装mysql:yum install -y mysql-server mysql mysql-deve
查看mysql的版本: rpm -qi mysql-server
启动mysql:service mysqld start
设置mysql开机启动:chkconfig mysqld on
root账号设置密码为 root:mysqladmin -u root password 'root'
查看端口:netstat -ntpl |grep 3306
查看端口:netstat -ntpl |grep 3306
进入mysql:mysql -uroot -proot
配置文件所在地:vi /etc/my.cnf
11.在所有节点创建cloudera-scm用户:useradd --system --home=/opt/cm-5.2.0/run/cloudera-scm-server/ --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm
9.在主节点初始化CM5的数据库:/opt/
cm
-5.2.0/share/cmf/schema/scm_prepare_database.
sh
mysql
cm
-hlocalhost -uroot -proot --scm-host localhost scm scm scm
6.mysql -uroot -pxxxx进入mysql命令行,创建以下数据库:
#hive
create database hive DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
#activity monitorcreate database amon DEFAULT CHARSET utf8 COLLATE utf8_general_ci;设置root授权访问以上所有的数据库:#授权root用户在主节点拥有所有数据库的访问权限 grant all privileges on *.* to 'root'@'%' identified by 'jDqzzb*u}8-C!0' with grant option;
grant all privileges on *.* to 'root'@'lewa1' identified by 'root' with grant option;
flush privileges;grant all privileges on *.* to 'root'@'n1' identified by 'xxxx' with grant option; flush privileges;
8.配置ssh
[root@10 opt]# cm-5.2.0/share/cmf/schema/scm_prepare_database.sh mysql cm -hlocalhost -uroot -proot --scm-host localhost scm scm scm
Verifying that we can write to /opt/cm-5.2.0/etc/cloudera-scm-server log4j:ERROR Could not find value for key log4j.appender.A
log4j:ERROR Could not instantiate appender named "A".
Creating SCM configuration file in /opt/cm-5.2.0/etc/cloudera-scm-server
groups: cloudera-scm: No such user
Executing: /home/h1/cloud/jdk1.7.0_51/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/opt/cm-5.2.0/share/cmf/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /opt/cm-5.2.0/etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.
log4j:ERROR Could not find value for key log4j.appender.A
log4j:ERROR Could not instantiate appender named "A".
[2015-01-12 21:08:44,064] INFO 0[main] - com.cloudera.enterprise.dbutil.DbCommandExecutor.testDbConnection(DbCommandExecutor.java:237) - Successfully connected to database.
All done, your SCM database is configured correctly! 谁知道这是成功了还是失败了呀
Verifying that we can write to /opt/cm-5.2.0/etc/cloudera-scm-server
Creating SCM configuration file in /opt/cm-5.2.0/etc/cloudera-scm-server
groups: cloudera-scm: No such user
Executing: /usr/java/jdk1.7.0_60/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/opt/cm-5.2.0/share/cmf/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /opt/cm-5.2.0/etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.
2015-01-15 13:43:01,210 [main] INFO com.cloudera.enterprise.dbutil.DbCommandExecutor - Successfully connected to database.
All done, your SCM database is configured correctly!
[root@lewa02 opt]#
10.关闭防火墙:service iptables stop
chkconfig iptables --list
chkconfig iptables off
11.所有节点配置NTP服务:yum install ntp
配置开机启动:
chkconfig ntpd on
检查是否设置成功:chkconfig --list ntpd
其中2-5为on状态就代表成功。 对时:ntpdate -u 65.55.56.206
配置文件:vi /var/lib/ntp/drift
配置文件:vi /var/lib/ntp/drift
启动服务:service ntpd start
查看同步状态:ntpstat
server
driftfile /var/lib/ntp/drift
restrict 127.0.0.1
restrict -6 ::1
restrict
default
nomodify notrap
server 10.1.0.202 prefer
includefile /etc/ntp/crypto/pw
keys /etc/ntp/keys
agent
driftfile /var/lib/ntp/drift
restrict
127.0.0.1
restrict
-6 ::1
restrict
default
kod nomodify notrap nopeer noquery
restrict
-6
default
kod nomodify notrap nopeer noquery
server 10.0.4.13
includefile /etc/ntp/crypto/pw
keys /etc/ntp/keys
2015013
16.清除之前启动的痕迹在agent:rm -rf /opt/CDH_HOME/cm-5.2.0/lib/cloudera-scm-agent/*
12.启动服务端:/opt/cm-5.2.0/etc/init.d/cloudera-scm-server start
13.启动Agent服务:/opt/
cm-5.2.0/etc/init.d/cloudera-scm-agent start
14.查看相应的端口是否已经开启:ps aux|grep 7181
15.hive的驱动安装
:
cp /opt/cm-5.2.0/share/cmf/lib/mysql-connector-java-5.1.16-bin.jar /opt/cloudera/parcels/CDH/lib/hive/lib/drwxr-xr-x. 8 h1 h1 4096 Sep 25 11:47 jdk1.7.0_51
drwxr-xr-x 8 root root 4096 Dec 19 2013 java
http://www.tuicool.com/articles/ENjmeaY
http://www.cnblogs.com/jasondan/p/4011153.html
0 0
- Deploy_Cluster_CDH
- 【BLE】CC2541之看门狗
- Othello
- 第9周项目1 - 猴子选大王(数组版)
- 第9周项目2 - 对称矩阵压缩存储的实现与应用(1)
- 第9周项目2 - 对称矩阵压缩存储的实现与应用(2)
- Deploy_Cluster_CDH
- entity framework增删改查简单操作
- C语言结构体(struct)常见使用方法
- Android导入项目时Android jar包丢失的解决
- [Canvas绘图] 第01节 画布准备
- 第9周实践项目3-稀疏矩阵的三元组表示的实现(3.1)
- 男人应该关注的是自己现在有多少钱,不应该关注自己以后有多少钱
- 第十三周 项目1-prim算法的验证
- 第13周 项目1、2- 验证算法