Ubuntu下安装和使用zookeeper和kafka

1.在清华镜像站下载kafka_2.10-0.10.0.0.tgz 和 zookeeper-3.4.10.tar.gz

分别解压到/usr/local目录下

2.进入zookeeper目录,在conf目录下将zoo_sample.cfg文件拷贝,并更名为zoo.cfg

参考 https://my.oschina.net/phoebus789/blog/730787

zoo.cfg文件的内容

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

# The number of ticks that the initial

# synchronization phase can take

initLimit=10

# The number of ticks that can pass between

# sending a request and getting an acknowledgement

syncLimit=5

# the directory where the snapshot is stored.

# do not use /tmp for storage, /tmp here is just

# example sakes.

dataDir=/home/common/zookeeper/zookeeperdir/zookeeper-data

dataLogDir=/home/common/zookeeper/zookeeperdir/logs

# the port at which the clients will connect

clientPort=2181

server.1=10.10.100.10:2888:3888

# the maximum number of client connections.

# increase this if you need to handle more clients

#maxClientCnxns=60

#

# Be sure to read the maintenance section of the

# administrator guide before turning on autopurge.

#

# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance

#

# The number of snapshots to retain in dataDir

#autopurge.snapRetainCount=3

# Purge task interval in hours

# Set to "0" to disable auto purge feature

#autopurge.purgeInterval=1

新建下面这两个目录

1

2

/home/common/zookeeper/zookeeperdir/zookeeper-data

/home/common/zookeeper/zookeeperdir/logs

在zookeeper-data目录下新建一个myid文件,内容为1,代表这个服务器的编号是1,具体参考上面网址中的内容

最后在/etc/profile中添加环境变量,并source

1

2

export ZOOKEEPER_HOME=/usr/local/zookeeper

export PATH=${ZOOKEEPER_HOME}/bin:$PATH

现在zookeeper就安装好了,现在启动zookeeper

1

bin/zkServer.sh start

查看状态

1

bin/zkServer.sh status

启动客户端脚本

1

bin/zkCli.sh -server zookeeper:2181

停止zookeeper

1

bin/zkServer.sh stop

 

 

1.现在安装kafka,同样是解压之后就安装好了

参考 http://www.jianshu.com/p/efc8b9dbd3bd

2.进入kafka目录下

kafka需要使用Zookeeper,首先需要启动Zookeeper服务,上面的操作就已经启动了Zookeeper服务

如果没有的话,可以使用kafka自带的脚本启动一个简单的单一节点Zookeeper实例

1

bin/zookeeper-server-start.sh config/zookeeper.properties

启动 Kafka服务

1

bin/kafka-server-start.sh config/server.properties

停止 Kafka服务

1

bin/kafka-server-stop.sh config/server.properties

 

3.创建一个主题

首先创建一个名为test的topic，只使用单个分区和一个复本

1

bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topictest

 现在可以运行list topic命令看到我们的主题

1

bin/kafka-topics.sh --list --zookeeper localhost:2181

 4.发送消息

1

2

3

bin/kafka-console-producer.sh --broker-list localhost:9092 --topictest

This is a message

This is another message

 5.启动一个消费者,消费者会接收到消息

1

bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topictest --from-beginning 2>/dev/null