zookeeper集群的安装和配置

来源:互联网 发布:淘宝减肥产品哪个好 编辑:程序博客网 时间:2024/05/16 18:34

Zookeeper的目的是封装好复杂易出错的关键服务,将简单易用的接口和性能高效、功能稳定的系统提供给用户。Zookeeper有两种运行模式,单机模式(Standalone)和集群模式(Distributed),但是Zookeeper的长处在于可以搭建分布式的Zookeeper集群(一个Leader,多个Follower),从而可以保证集群的高可用性、高稳定性、高可靠性。

下面说明一下Zookeeper两种模式的配置:
在配置Zookeeper之前,下载Zookeeper的安装包:http://zookeeper.apache.org/releases.html#download,我使用的是zookeeper-3.3.6版本的。

Zookeeper的单机模式(standalone)


下载安装包之后,使用Winscp放到Linux的环境中,然后解压安装包:

tar zxvf zookeeper-3.3.6.tar.gz

解压后开始配置Zookeeper,进入到配置文件的目录:

cd  zookeeper-3.3.6/conf/

将目录下面的 zoo_sample.cfg修改为zoo.cfg,然后配置内容为:

# The number of milliseconds of each ticktickTime=2000# The number of ticks that the initial # synchronization phase can takeinitLimit=10# The number of ticks that can pass between # sending a request and getting an acknowledgementsyncLimit=5# the directory where the snapshot is stored.dataDir=/tmp/zookeeper# the port at which the clients will connectclientPort=2181

上面各个配置参数的含义:

tickTime:这个时间是作为 Zookeeper 服务器之间或客户端与服务器之间维持心跳的时间间隔,也就是每个 tickTime 时间就会发送一个心跳。dataDir:顾名思义就是 Zookeeper 保存数据的目录,默认情况下,Zookeeper 将写数据的日志文件也保存在这个目录里。clientPort:这个端口就是客户端连接 Zookeeper 服务器的端口,Zookeeper 会监听这个端口,接受客户端的访问请求。

dataLogDir: log目录, 同样可以是任意目录. 如果没有设置该参数, 将使用和dataDir相同的设置,这个参数我没有设置。

配置好后,启动Zookeeper服务器进程:

cd  zookeeper-3.3.6/bin/zkServer.sh start

启动成功后,用jps -m的命令查询,Zookeeper的进程为QuorumPeerMain

用客户端进程连接服务器,执行如下命令:

bin/zkCli.sh  -server logsrv03:2181

上面的logsrv是我的主机名,后面的2181是连接的端口号

连接成功后,出现以下的连接信息:

[root@logsrv03 zookeeper-3.3.6]# bin/zkCli.sh -server logsrv03:2181Connecting to logsrv03:21812015-08-18 10:23:48,966 - INFO  [main:Environment@97] - Client environment:zookeeper.version=3.3.6-1366786, built on 07/29/2012 06:22 GMT2015-08-18 10:23:48,969 - INFO  [main:Environment@97] - Client environment:host.name=logsrv032015-08-18 10:23:48,970 - INFO  [main:Environment@97] - Client environment:java.version=1.7.0_712015-08-18 10:23:48,970 - INFO  [main:Environment@97] - Client environment:java.vendor=Oracle Corporation2015-08-18 10:23:48,971 - INFO  [main:Environment@97] - Client environment:java.home=/usr/local/jdk1.7.0_71/jre2015-08-18 10:23:48,971 - INFO  [main:Environment@97] - Client environment:java.class.path=/usr/local/jiang/zookeeper-3.3.6/bin/../build/classes:/usr/local/jiang/zookeeper-3.3.6/bin/../build/lib/*.jar:/usr/local/jiang/zookeeper-3.3.6/bin/../zookeeper-3.3.6.jar:/usr/local/jiang/zookeeper-3.3.6/bin/../lib/log4j-1.2.15.jar:/usr/local/jiang/zookeeper-3.3.6/bin/../lib/jline-0.9.94.jar:/usr/local/jiang/zookeeper-3.3.6/bin/../src/java/lib/*.jar:/usr/local/jiang/zookeeper-3.3.6/bin/../conf:.:/usr/local/jdk1.7.0_71/lib:/usr/local/jdk1.7.0_71/lib/dt.jar:/usr/local/jdk1.7.0_71/lib/tools.jar2015-08-18 10:23:48,972 - INFO  [main:Environment@97] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib2015-08-18 10:23:48,972 - INFO  [main:Environment@97] - Client environment:java.io.tmpdir=/tmp2015-08-18 10:23:48,973 - INFO  [main:Environment@97] - Client environment:java.compiler=<NA>2015-08-18 10:23:48,973 - INFO  [main:Environment@97] - Client environment:os.name=Linux2015-08-18 10:23:48,974 - INFO  [main:Environment@97] - Client environment:os.arch=amd642015-08-18 10:23:48,974 - INFO  [main:Environment@97] - Client environment:os.version=2.6.32-431.el6.x86_642015-08-18 10:23:48,975 - INFO  [main:Environment@97] - Client environment:user.name=root2015-08-18 10:23:48,975 - INFO  [main:Environment@97] - Client environment:user.home=/root2015-08-18 10:23:48,976 - INFO  [main:Environment@97] - Client environment:user.dir=/usr/local/jiang/zookeeper-3.3.62015-08-18 10:23:48,977 - INFO  [main:ZooKeeper@379] - Initiating client connection, connectString=logsrv03:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@1698ab04Welcome to ZooKeeper!2015-08-18 10:23:48,995 - INFO  [main-SendThread():ClientCnxn$SendThread@1058] - Opening socket connection to server logsrv03/172.17.6.148:21812015-08-18 10:23:49,002 - INFO  [main-SendThread(logsrv03:2181):ClientCnxn$SendThread@947] - Socket connection established to logsrv03/172.17.6.148:2181, initiating sessionJLine support is enabled2015-08-18 10:23:49,020 - INFO  [main-SendThread(logsrv03:2181):ClientCnxn$SendThread@736] - Session establishment complete on server logsrv03/172.17.6.148:2181, sessionid = 0x24f3b0a17ea0002, negotiated timeout = 30000WATCHER::WatchedEvent state:SyncConnected type:None path:null[zk: logsrv03:2181(CONNECTED) 0] 

到这,Zookeeper Standalone模式就配置完成了。


Zookeeper的集群模式(Distributed


Zookeeper服务自身组成一个集群(2n+1个服务允许n个失效)。Zookeeper服务有两个角色,一个是leader,负责写服务和数据同步,剩下的是follower,提供读服务,leader失效后会在follower中重新选举新的leader。在整个Zookeeper集群模式下,整个集群是一个整体,在集群中的任何一个Zookeeper节点共享整个集群的服务。详情可以参考http://zookeeper.apache.org/doc/trunk/zookeeperInternals.html

下面介绍集群模式的配置过程:

第一步:配置主机名和ip的映射


在Zookeeper集群运行的过程中,每一个Zookeeper节点是需要相互通信的,并且是通过主机名来相互通信的,集群每个节点需要相互识别,所以,在配置一个Zookeeper节点的时候需要配置其他节点主机名到ip的映射

例如配置我的Zookeeper集群,集群中一共有三台机子:logsrv02、logsrv03、logsrv04,以logsrv03为例子,进入/etc/hosts,内容如下所示:

172.17.6.142 logsrv02172.17.6.148 logsrv03172.17.6.149 logsrv04

第二步:配置Zookeeper的配置文件


# The number of milliseconds of each ticktickTime=2000# The number of ticks that the initial # synchronization phase can takeinitLimit=10# The number of ticks that can pass between # sending a request and getting an acknowledgementsyncLimit=5# the directory where the snapshot is stored.dataDir=/tmp/zookeeper# the port at which the clients will connectclientPort=2181server.1=logsrv02:2888:3888server.2=logsrv03:2888:3888server.3=logsrv04:2888:3888
以上配置文件与单机模式相比其实也就多了集群中server.id和主机名、端口号的映射信息,配置文件的详细说明可以参考官方说明:
http://zookeeper.apache.org/doc/trunk/zookeeperStarted.html#sc_RunningReplicatedZooKeeper

第三步:在logsrv02和logsrv04添加安装文件


我们在logsrv03中配置好了Zookeeper安装文件,在一个集群中,每一个Zookeeper节点只需要使用相同的安装文件就行了,所以需要将配置好的文件复制到另外两台机子上,这里介绍两种方法:

第一种:使用Winscp将文件直接复制出去,然后再复制到另外两台机子上,这种方式需要安装Winscp(用于在本地与远程计算机间安全的复制文件,支持scp协议

下载地址:http://winscp.net/eng/download.php

第二种:远程命令的方式直接复制到logsrv02和logsrv04上,如下面所示:

cd /usr/local/scp -r zookeeper-3.3.6/ root@logsrv02:/usr/local/scp -r zookeeper-3.3.4/ root@logsrv04:/usr/local/

第四步:设置myid


在我们配置的dataDir指定的目录下面,创建一个myid文件,里面内容为一个数字,用来标识当前主机,在conf/zoo.cfg中配置的server.x中x是什么数字就设置id为这个数字,如下面所示:

[root@logsrv02 zookeeper]# echo "1" > /tmp/zookeeper/myid[root@logsrv03 zookeeper]# echo "2" > /tmp/zookeeper/myid[root@logsrv04 zookeeper]# echo "3" > /tmp/zookeeper/myid

第五步:配置好后,启动集群


配置好后,进入/usr/local/zookeeper-3.3.6/目录下,然后执行如下命令启动zookeeper:

[root@logsrv02 zookeeper-3.3.6]# bin/zkServer start[root@logsrv03 zookeeper-3.3.6]# bin/zkServer start[root@logsrv04 zookeeper-3.3.6]# bin/zkServer start

第六步:安装验证


安装完成后,可以通过zookeeper的脚本来查看zookeeper的启动状态,以及每个Zookeeper节点在集群中的角色(Leader和Follower),下面是我的Zookeeper集群各节点的查询结果:

[root@logsrv02 zookeeper-3.3.6]# bin/zkServer.sh statusJMX enabled by defaultUsing config: /usr/local/jiang/zookeeper-3.3.6/bin/../conf/zoo.cfgMode: follower[root@logsrv03 zookeeper-3.3.6]# bin/zkServer.sh statusJMX enabled by defaultUsing config: /usr/local/jiang/zookeeper-3.3.6/bin/../conf/zoo.cfgMode: follower[root@logsrv04 zookeeper-3.3.6]# bin/zkServer.sh statusJMX enabled by defaultUsing config: /usr/local/zookeeper-3.3.6/bin/../conf/zoo.cfgMode: leader

通过上面的信息可以得知,logsrv04是Leader,其余两个都是Follower。

集群配置好后,可以通过其中一个Zookeeper节点连接点集群上,而且通过一个节点可以共享整个集群的服务,例如:当你配置好Zookeeper集群后,再安装中间件KAFKA后,通过zookeeper启动KAFKA,然后通过脚本创建topic,topic的创建时需要指定zookeeper的节点,当许多节点都对应了topic,这个时候在zookeeper集群中,查询一个节点对应的topic列表,可以查询到整个集群的所有节点的topic列表。


参考链接


下面是我在学习Zookeeper的时候,搜集的与Zookeeper相关的链接,可以参考学习:

http://zookeeper.apache.org/doc/trunk/zookeeperStarted.html#sc_RunningReplicatedZooKeeper

http://zookeeper.apache.org/doc/trunk/

http://zookeeper.apache.org/doc/trunk/zookeeperInternals.html

http://zookeeper.apache.org/

http://blog.csdn.net/shirdrn/article/details/7183503

http://jm-blog.aliapp.com/?p=1384

http://www.ibm.com/developerworks/cn/opensource/os-cn-zookeeper/

http://cailin.iteye.com/blog/2014486

http://blog.jpush.cn/push_zookeeper_study_usage/

http://coolxing.iteye.com/blog/1871009

http://blog.yemou.net/article/query/info/tytfjhfascvhzxcyt192

0 0