Zookeeper和zkpython 客户端测试

简介：

　　Zookeeper (http://zookeeper.apache.org/)分布式服务框架是 Apache Hadoop 的一个子项目，它主要是用来解决分布式应用中经常遇到的一些数据管理问题，如：统一命名服务、状态同步服务、集群管理、分布式应用配置项的管理等。

具体简介可以参照这篇文章。

zkpython的安装：

python中有一个zkpython的包，是基于zookeeper的c-client开发的，所以安装的时候需要先安装zookeeper的c客户端。安装步骤如下：

 

1

2

3

4

5

6

7

8

9

10

11

12

13

# 首先下载zookeeper

wget http://labs.renren.com/apache-mirror//zookeeper/zookeeper-3.3.3/zookeeper-3.3.3.tar.gz

tar xzvf zookeeper-3.3.3.tar.gz

cd zookeeper-3.3.3/src/c/

./configure

make

make install

 

# 然后下载zkpython

wget http://pypi.python.org/packages/source/z/zkpython/zkpython-0.4.tar.gz#md5=3de220615aaddf57f1462b78d32477f9

tar xzvf zkpython-0.4.tar.gz

cd zkpython-0.4

python setup.pyinstall

这样就完成了zkpython的安装。

一个简单的demo：

之后让我们来写一个简单的demo吧。（demo中用到的zkclient.py:https://github.com/piglei/zkpython_example/blob/master/zkclient.py）

 

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

# coding: utf-8

import logging

from os.path import basename, join

 

from zkclient import ZKClient, zookeeper, watchmethod

 

logging.basicConfig(

    level= logging.DEBUG,

    format= "[%(asctime)s] %(levelname)-8s %(message)s"

)

 

log =logging

 

classGJZookeeper(object):

 

    ZK_HOST= "localhost:2181"

    ROOT= "/app"

    WORKERS_PATH= join(ROOT,"workers")

    MASTERS_NUM= 1

    TIMEOUT= 10000

 

    def__init__(self, verbose= True):

        self.VERBOSE= verbose

        self.masters= []

        self.is_master= False

        self.path= None

 

        self.zk= ZKClient(self.ZK_HOST, timeout= self.TIMEOUT)

        self.say("login ok!")

        # init

        self.__init_zk()

        # register

        self.register()

 

    def__init_zk(self):

        """

        create the zookeeper node if not exist

        """

        nodes= (self.ROOT,self.WORKERS_PATH)

        fornode in nodes:

            ifnot self.zk.exists(node):

                try:

                    self.zk.create(node, "")

                except:

                    pass

 

    @property

    defis_slave(self):

        returnnot self.is_master

 

    defregister(self):

        """

        register a node for this worker

        """

        self.path= self.zk.create(self.WORKERS_PATH+ "/worker","1", flags=zookeeper.EPHEMERAL | zookeeper.SEQUENCE)

        self.path= basename(self.path)

        self.say("register ok! I'm %s"% self.path)

        # check who is the master

        self.get_master()

 

    defget_master(self):

        """

        get children, and check who is the smallest child

        """

        @watchmethod

        defwatcher(event):

            self.say("child changed, try to get master again.")

            self.get_master()

 

        children= self.zk.get_children(self.WORKERS_PATH, watcher)

        children.sort()

        self.say("%s's children: %s"% (self.WORKERS_PATH, children))

 

        # check if I'm master

        self.masters= children[:self.MASTERS_NUM]

        ifself.path inself.masters:

            self.is_master= True

            self.say("I've become master!")

        else:

            self.say("%s is masters, I'm slave"% self.masters)

 

 

    defsay(self, msg):

        """

        print messages to screen

        """

        ifself.VERBOSE:

            ifself.path:

                log.info("[ %s(%s) ] %s"% (self.path,"master" ifself.is_master else "slave", msg))

            else:

                log.info(msg)

 

def main():

    gj_zookeeper= GJZookeeper()

 

if __name__ =="__main__":

    main()

    importtime

    time.sleep(1000)

　　这个简单的demo所做的事情，就是通过在zookeeper的/app/workers节点下建立临时的子节点( flags=zookeeper.EPHEMERAL | zookeeper.SEQUENCE )，每次create完成之后检查自己是不是在最小的MASTERS_NUM(例子中为1，即单master)里。如果是的话，作为master运行，否则的话，作为slave运行。

　　这样的话，当我们的master挂掉以后，与zookeeper之间的连接也会中断，过了指定的TIMEOUT以后，master之前在worker下的子节点就会被删除，于是slave节点之前设置的watcher会被触发，再次检查自己是否为master，如果是的话则完成切换。

demo运行结果：

 

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

# 第一个实例

Connected in 20 ms, handle is 0

[2011-09-09 12:40:43,702] INFO     login ok!

Node /app/workers/worker created in 4 ms

[2011-09-09 12:40:43,708] INFO     [ worker0000000022(slave) ] register ok! I'm worker0000000022

[2011-09-09 12:40:43,709] INFO     [ worker0000000022(slave) ] /app/workers's children: ['worker0000000022']

[2011-09-09 12:40:43,709] INFO     [ worker0000000022(master) ] I've become master!

 

# 这时再起第二个实例

Connected in 64 ms, handle is 0

[2011-09-09 12:43:08,334] INFO     login ok!

Node /app/workers/worker created in 11 ms

[2011-09-09 12:43:08,346] INFO     [ worker0000000023(slave) ] register ok! I'm worker0000000023

[2011-09-09 12:43:08,347] INFO     [ worker0000000023(slave) ] /app/workers's children: ['worker0000000022', 'worker0000000023']

[2011-09-09 12:43:08,347] INFO     [ worker0000000023(slave) ] ['worker0000000022'] is masters, I'm slave

 

# 杀掉master，第二个实例发生的变化

[2011-09-09 12:44:06,016] INFO     [ worker0000000023(slave) ] child changed, try to get master again.

[2011-09-09 12:44:06,017] INFO     [ worker0000000023(slave) ] /app/workers's children: ['worker0000000023']

[2011-09-09 12:44:06,017] INFO     [ worker0000000023(master) ] I've become master!

 

demo代码地址： https://github.com/piglei/zkpython_example