Setting up Kafka
来源:互联网 发布:小约翰 震惊 知乎 编辑:程序博客网 时间:2024/06/04 18:02
Download the latest release package from kafka site. Un-tar it and move extracted files to where you want to put them, like
$HOME/usr
or$HOME/bin
.Change the config.
For some scenario, we only need to updateserver.properties
, reference to: brokerconfigs:- broker.id: the unique broker id in integer.
- port: default is 9092. The port the socket server listens on.
- log.dirs: a comma-separated list of one or more directories in which Kafka data is stored. It's better to change this directory to local dir, such as
${HOME}/data/kafka
. Default dir is/tmp/kafka-logs
. - zookeeper.connect: zookeeper connection string, a comma separated host:port pairs, each corresponding to a zk server. ZooKeeper also allows you to add a "chroot" path which will make all kafka data for this cluster appear under a particular path. For example,
${zk_host1}:2181,${zk_host2}:2181,${zk_host3}:2181/kafka
Start kafka server.
After setting config, in kafka install dir, run following command to start kafka cluster server:bin/kafka-server-start.sh config/server.properties
Kafka Utility
Create topic
bin/kafka-topics.sh --create --zookeeper ${zk_host}:${zk_port}[${zk_chroot}] --replication-factor ${replication_factor} --partitions ${partition_count} --topic ${topic_name}
Note: for zookeeper connect string
${zk_host}:${zk_port}[${zk_chroot}]
,${zk_chroot}
is optionalDescribe topic
- Describe topic all topics
bin/kafka-topics.sh --describe --zookeeper ${zk_host}:${zk_port}[${zk_chroot}]
- Describe specified topic
bin/kafka-topics.sh --describe --zookeeper ${zk_host}:${zk_port}[${zk_chroot}] --topic ${topic_name}
- Describe topic all topics
Publish messages. You can use this utility to test you topic.
bin/kafka-console-producer.sh --broker-list ${kafka_server}:{kafka_port} --topic ${topic_name}
Note: if you local kafka server is started, you can use
localhost:9092
asbroker-list
.Consume messages
bin/kafka-console-consumer.sh --zookeeper ${zk_host}:${zk_port}[${zk_chroot}] [--from-beginning] --topic ${topic_name}
Modifying topics
Currently, support increase partitions well:bin/kafka-topics.sh --zookeeper ${zk_host}:${zk_port}[${zk_chroot}] --alter --topic ${topic_name} --partitions ${partition_count}
Kafka does not currently support reducing the number of partitions for a topic or changing the replication factor.
Delete topic
bin/kafka-topics.sh --zookeeper ${zk_host}:${zk_port}[${zk_chroot}] --delete --topic ${topic_name}
Topic deletion option is disabled by default. To enable it set the server config:
delete.topic.enable=true
- Setting up Kafka
- Setting Up Cygwin
- Setting up Zend debugger.
- Setting up USBnet
- Setting Up Subversion
- Setting up MaNGOS
- Setting up CEDET
- Virtual Machine setting up
- Setting up Pricing Agreements
- Setting up Relays
- Setting up Eclipse CDT
- Setting up QCustomPlot
- Setting Up Your Environment
- Setting Up Redis Cluster
- setting up the centos7
- Setting Up File Sharing
- Setting Up File Sharing
- Setting Up a RequestQueue
- bzoj3529: [Sdoi2014]数表
- win10在vbox虚拟机下运行
- 排序—基数排序(js实现)
- 2015Daejeon UVALive 7236 K - Tree Edit
- SOA的浅析
- Setting up Kafka
- 栈---链式存储
- 梵高
- JavaScript没有块级作用域
- MySQL系统函数
- C++11新特性应用--介绍几个新增的便利算法(stl中的heap使用,最大堆)
- iOS MD5加密 —— HERO博客
- 面试经历---网易(2016年01月19日下午面试)
- 精度丢失&const&scanf函数诸多问题