zookeeper集群搭建

zookeeper

zookeeper集群搭建

1.安装jdk

略

2.设置java堆栈大小。

差不多4g的机器设置heap大小为3g。为了避免内存swap。

3.安装zookeeper

http://hadoop.apache.org/zookeeper/releases.html

4.创建一个配置文件zoo.cfg

tickTime=2000dataDir=/var/zookeeper/clientPort=2181initLimit=5syncLimit=2server.1=zoo1:2888:3888server.2=zoo2:2888:3888server.3=zoo3:2888:3888

集群配置格式，server.id=host:port:port

5. myid文件

id号为1到255，必须配置在datadir下面，文件名为myid。

6. 启动zookeeper

bin/zkServer.sh start

7. zookeeper 命令

ZooKeeper Commands: The Four Letter WordsZooKeeper responds to a small set of commands. Each command is composed of four letters. You issue the commands to ZooKeeper via telnet or nc, at the client port.Three of the more interesting commands: "stat" gives some general information about the server and connected clients, while "srvr" and "cons" give extended details on server and connections respectively.confNew in 3.3.0: Print details about serving configuration.consNew in 3.3.0: List full connection/session details for all clients connected to this server. Includes information on numbers of packets received/sent, session id, operation latencies, last operation performed, etc...crstNew in 3.3.0: Reset connection/session statistics for all connections.dumpLists the outstanding sessions and ephemeral nodes. This only works on the leader.enviPrint details about serving environmentruokTests if server is running in a non-error state. The server will respond with imok if it is running. Otherwise it will not respond at all.A response of "imok" does not necessarily indicate that the server has joined the quorum, just that the server process is active and bound to the specified client port. Use "stat" for details on state wrt quorum and client connection information.srstReset server statistics.srvrNew in 3.3.0: Lists full details for the server.statLists brief details for the server and connected clients.wchsNew in 3.3.0: Lists brief information on watches for the server.wchcNew in 3.3.0: Lists detailed information on watches for the server, by session. This outputs a list of sessions(connections) with associated watches (paths). Note, depending on the number of watches this operation may be expensive (ie impact server performance), use it carefully.wchpNew in 3.3.0: Lists detailed information on watches for the server, by path. This outputs a list of paths (znodes) with associated sessions. Note, depending on the number of watches this operation may be expensive (ie impact server performance), use it carefully.Here's an example of the ruok command:$ echo ruok | nc 127.0.0.1 5111imok

很简单，可以用nc来发命令取zookeeper的信息。

8. data 目录

data目录下其实只有2个文件，

• myid ，上面介绍过了，集群id，1~255。
• snapshot. ,zookeeper 持久化的数据。

9. kazoo python库

# coding=utf-8# kazoo库# kazoo的一些特性# A wide range of recipe implementations, like Lock, Election or Queue# Data and Children Watchers# Simplified Zookeeper connection state tracking# Unified asynchronous API for use with greenlets or threads# Support for gevent 0.13 and gevent 1.0# Support for eventlet# Support for Zookeeper 3.3 and 3.4 servers# Integrated testing helpers for Zookeeper clusters# Pure-Python based implementation of the wire protocol, avoiding all the memory leaks, lacking features, and debugging madness of the C libraryfrom kazoo.client import KazooClientimport loggingfrom kazoo.client import KazooStatelogging.basicConfig()zk = KazooClient(hosts='127.0.0.1:2181')def my_listener(state):    if state == KazooState.LOST:        # Register somewhere that the session was lost        print "byebye"    elif state == KazooState.SUSPENDED:        # Handle being disconnected from Zookeeper        pass    else:        pass        # Handle being connected/reconnected to Zookeeperzk.add_listener(my_listener)zk.start()# 创建节点# ensure_path和create的区别，create可以创建值。zk.ensure_path("/my/aca")zk.create('/my/test', b'a value' )# 读节点# exists()# get# get_children()if zk.exists("/my/test"):    data, stat = zk.get("/my/test")    print("Version: %s, data: %s" % (stat.version, data.decode("utf-8")))# update node# 调用set就可以了。zk.set("/my/test", b"some data")# 删除节点zk.delete("/my/aca")zk.stop()