Kafka试用记录

为了初步了解Kafka功能和性能，通过实验简单试用。（1）kafka_2.9.1-0.8.2.1演示：zookeeper server：192.168.1.66:2181kafka server：192.168.1.66:9092kafka consumer：192.168.1.55wget  http://mirror.bit.edu.cn/apache/kafka/0.8.2.1/kafka_2.9.1-0.8.2.1.tgztar xzvf kafka_2.9.1-0.8.2.1.tgzcd kafka_2.9.1-0.8.2.1前提zookeeper已经启动：在zookeeper的路径下执行：bin/zkServer.sh start启动Kafka server：$ bin/kafka-server-start.sh config/server.properties $ bin/kafka-topics.sh --create --zookeeper 192.168.1.66:2181 --replication-factor 1 --partitions 1 --topic test Created topic "test". $ bin/kafka-topics.sh --list --zookeeper 192.168.1.66:2181 test-------kafka producer：host：192.168.1.66 #consle演示bin/kafka-console-producer.sh --broker-list 192.168.1.66:9092 --topic test #性能测试bin/kafka-producer-perf-test.sh --topic test --batch-size=1000   --broker-list=192.168.1.66:9092  --message-size 100  --messages=1000000start.time, end.time, compression, message.size, batch.size, total.data.sent.in.MB, MB.sec, total.data.sent.in.nMsg, nMsg.sec2016-01-22 13:11:02:308, 2016-01-22 13:11:13:177, 0, 100, 1000, 95.37, 8.7743, 1000000, 92004.7842-------kafka consumer：host：192.168.1.55#console演示bin/kafka-console-consumer.sh --zookeeper 192.168.1.66:2181 --topic test --from-beginning#性能测试bin/kafka-consumer-perf-test.sh --topic test --message-size 100 --messages 1000000 --zookeeper 192.168.1.66:2181 --threads 1start.time, end.time, fetch.size, data.consumed.in.MB, MB.sec, data.consumed.in.nMsg, nMsg.sec2016-01-22 13:13:37:857, 2016-01-22 13:13:44:046, 1048576, 122.0698, 102.6660, 1000000, 841042.8932（2）kafka_2.11-0.9.0.0演示全部在localhost进行演示wget http://mirror.bit.edu.cn/apache/kafka/0.9.0.0/kafka_2.11-0.9.0.0.tgztar xzvf kafka_2.11-0.9.0.0.tgzcd kafka_2.11-0.9.0.0  前提zookeeper已经启动：在zookeeper的路径下执行：bin/zkServer.sh start启动Kafka server：bin/kafka-server-start.sh config/server.properties bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test #console演示bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test #性能测试bin/kafka-producer-perf-test.sh --topic test --num-records=400000 --record-size 128 --throughput 200000 --producer-props bootstrap.servers=localhost:9092bin/kafka-producer-perf-test.sh --topic test --num-records=4000000 --record-size 128 --throughput 400000 --producer-props bootstrap.servers=localhost:9092 producer.type=sync compression.codec=none serializer.class=kafka.serializer.DefaultEncoder#console演示bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test --from-beginning#性能测试(default: 10 threads)bin/kafka-consumer-perf-test.sh --broker-list localhost:9092 --topic test --message-size 128 --messages 400000 --zookeeper localhost:2181#性能测试(1 thread)bin/kafka-consumer-perf-test.sh --broker-list localhost:9092 --topic test --message-size 128 --messages 4000000 --zookeeper localhost:2181 --threads 1（3）参考http://kafka.apache.org/documentation.html#quickstart