ubuntu 12.04 安装 Cloudera Manager 4.8.2

Cloudera可以简化Hadoop的安装配置过程，自动在集群节点上安装hadoop相关组件，创建用户。Cloudera是Hadoop生态系统中，规模最大、知名度最高的公司

1.准备

系统版本

1iflab@cdh1:~$uname-a

2Linux cdh1 3.11.0-15-generic #25~precise1-Ubuntu SMP Thu Jan 30 17:39:31 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

3iflab@cdh1:~$

角色分配(需要修改hostname、hosts)

1IP               主机名

2222.249.250.211  cdh1    manager master 

3222.249.250.212  cdh2    slave1

4222.249.250.213  cdh3    slave2

5222.249.250.214  cdh4    slave3

确保所有机器root密码一致或者sudo免输入密码用户的用户名、密码一致！

2.添加Cloudera源

1sudosudo nano /etc/apt/sources.list.d/cloudera.list

写入

1deb [arch=amd64] http://archive.cloudera.com/cdh4/ubuntu/precise/amd64/cdh precise-cdh4 contrib

2deb-srchttp://archive.cloudera.com/cdh4/ubuntu/precise/amd64/cdh precise-cdh4 contrib

3deb [arch=amd64] http://archive.cloudera.com/cm4/ubuntu/precise/amd64/cm precise-cm4 contrib

4deb-srchttp://archive.cloudera.com/cm4/ubuntu/precise/amd64/cm precise-cm4 contrib

获取apt key

1curl -s http://archive.cloudera.com/cdh4/ubuntu/precise/amd64/cdh/archive.key | sudoapt-key add -

刷新源列表

1sudoapt-get update

3.安装java环境

1sudoapt-get -o Dpkg::Options::=--force-confdef -o Dpkg::Options::=--force-confold -y installoracle-j2sdk1.6

4.安装mysql以及JDBC驱动

1sudoapt-get installmysql-server libmysql-java

5.配置mysql

1sudonano /etc/mysql/conf.d/mysql_cloudera_manager.cnf

写入

1[mysqld]

2transaction-isolation=READ-COMMITTED

3# Disabling symbolic-links is recommended to prevent assorted security risks;

4# to do so, uncomment this line:

5# symbolic-links=0

6 

7key_buffer              = 16M

8key_buffer_size         = 32M

9max_allowed_packet      = 16M

10thread_stack            = 256K

11thread_cache_size       = 64

12query_cache_limit       = 8M

13query_cache_size        = 64M

14query_cache_type        = 1

15# Important: see Configuring the Databases and Setting max_connections

16max_connections         = 550

17 

18# log-bin should be on a disk with enough free space

19log-bin=/var/log/mysql/mysql_binary_log

20 

21# For MySQL version 5.1.8 or later. Comment out binlog_format for older versions.

22binlog_format           = mixed

23 

24read_buffer_size = 2M

25read_rnd_buffer_size = 16M

26sort_buffer_size = 8M

27join_buffer_size = 8M

28 

29# InnoDB settings

30innodb_file_per_table = 1

31innodb_flush_log_at_trx_commit  = 2

32innodb_log_buffer_size          = 64M

33innodb_buffer_pool_size         = 4G

34innodb_thread_concurrency       = 8

35innodb_flush_method             = O_DIRECT

36innodb_log_file_size = 512M

37 

38[mysqld_safe]

39log-error=/var/log/mysqld.log

40pid-file=/var/run/mysqld/mysqld.pid

编辑my.cnf
sudo nano /etc/mysql/my.cnf
把下面这一行注释掉

1#bind-address           = 127.0.0.1

6.配置innodb

1sudosu

2mv/var/lib/mysql/ib_logfile* /var/tmp/

3logout

7.初始化数据库

1sudoservice mysql restart

2mysql -uroot -p

3 

4create database amon DEFAULT CHARACTER SET utf8;

5grant all on amon.* TO 'amon'@'%'IDENTIFIED BY 'amon_password';

6grant all on amon.* TO 'amon'@'cdh1'IDENTIFIED BY 'amon_password';

7create database smon DEFAULT CHARACTER SET utf8;

8grant all on smon.* TO 'smon'@'%'IDENTIFIED BY 'smon_password';

9grant all on smon.* TO 'smon'@'cdh1'IDENTIFIED BY 'smon_password';

10create database rman DEFAULT CHARACTER SET utf8;

11grant all on rman.* TO 'rman'@'%'IDENTIFIED BY 'rman_password';

12grant all on rman.* TO 'rman'@'cdh1'IDENTIFIED BY 'rman_password';

13create database hmon DEFAULT CHARACTER SET utf8;

14grant all on hmon.* TO 'hmon'@'%'IDENTIFIED BY 'hmon_password';

15grant all on hmon.* TO 'hmon'@'cdh1'IDENTIFIED BY 'hmon_password';

16create database hive DEFAULT CHARACTER SET utf8;

17grant all on hive.* TO 'hive'@'%'IDENTIFIED BY 'hive_password';

18grant all on hive.* TO 'hive'@'cdh1'IDENTIFIED BY 'hive_password';

8.安装 cloudera-manager以及agent(因为master也是一个节点)

1sudoapt-get installcloudera-manager-daemons cloudera-manager-server  cloudera-manager-agent

9.配置cloudera-manager-server数据库

1sudo/usr/share/cmf/schema/scm_prepare_database.sh mysql  -uroot -p --scm-host localhost scm scm scm_password

10.配置其余节点（cdh2、cdh3、cdh4）

配置其余节点的源

1sudoscp iflab@222.249.250.211:/etc/apt/sources.list.d/cloudera.list /etc/apt/sources.list.d/cloudera.list

获取apt key

1curl -s http://archive.cloudera.com/cdh4/ubuntu/precise/amd64/cdh/archive.key | sudoapt-key add -

刷新源列表

1sudoapt-get update

在其余节点安装jdk

view source

print?

1sudoapt-get -o Dpkg::Options::=--force-confdef -o Dpkg::Options::=--force-confold -y installoracle-j2sdk1.6

在其余节点安装cloudera-manager-agent 和cloudera-manager-daemons

1sudoapt-get installcloudera-manager-agent cloudera-manager-daemons

修改agent的配置文件
sudo nano /etc/cloudera-scm-agent/config.ini
修改server_host＝cdh1

重启每个节点的agent

1sudoservice cloudera-scm-agent  restart

11.重启控制节点cloudera-manager以及agent

1sudoservice cloudera-scm-server restart

2sudoservice cloudera-scm-agent  restart

12.访问web控制台

http://222.249.250.211:7180/ 用户名admin密码admin