Kafka系列2-producer和consumer报错
来源:互联网 发布:开淘宝店怎么找客户 编辑:程序博客网 时间:2024/06/06 03:12
1. 使用127.0.0.1启动生产和消费进程:
1)启动生产者进程:
bin/kafka-console-producer.sh --broker-list 127.0.0.1:9092 --topic test
输入消息:
this is msg
生产者进程报错:
[2016-06-03 11:33:47,934] WARN Bootstrap broker 127.0.0.1:9092 disconnected (org.apache.kafka.clients.NetworkClient)[2016-06-03 11:33:49,554] WARN Bootstrap broker 127.0.0.1:9092 disconnected (org.apache.kafka.clients.NetworkClient)[2016-06-03 11:33:51,177] WARN Bootstrap broker 127.0.0.1:9092 disconnected (org.apache.kafka.clients.NetworkClient)[2016-06-03 11:33:53,398] WARN Bootstrap broker 127.0.0.1:9092 disconnected (org.apache.kafka.clients.NetworkClient)
2)启动消费者进程:
bin/kafka-console-consumer.sh --zookeeper 127.0.0.1:2181 --topic test --from-beginning
消费者进程报错:
[2016-06-03 11:34:53,574] WARN Fetching topic metadata with correlation id 0 for topics [Set(test)] from broker [BrokerEndPoint(0,218.30.64.194,9092)] failed (kafka.client.ClientUtils$)java.nio.channels.ClosedChannelExceptionat kafka.network.BlockingChannel.send(BlockingChannel.scala:110)at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:80)at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:79)at kafka.producer.SyncProducer.send(SyncProducer.scala:124)at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:59)at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:94)at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66)at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)[2016-06-03 11:34:53,651] WARN [console-consumer-72675_zzs-1464924871670-2192d80a-leader-finder-thread], Failed to find leader for Set([test,0]) (kafka.consumer.ConsumerFetcherManager$LeaderFinderThread)kafka.common.KafkaException: fetching topic metadata for topics [Set(test)] from broker [ArrayBuffer(BrokerEndPoint(0,218.30.64.194,9092))] failedat kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:73)at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:94)at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66)at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)Caused by: java.nio.channels.ClosedChannelExceptionat kafka.network.BlockingChannel.send(BlockingChannel.scala:110)at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:80)at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:79)at kafka.producer.SyncProducer.send(SyncProducer.scala:124)at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:59)... 3 more[2016-06-03 11:35:14,916] WARN Fetching topic metadata with correlation id 1 for topics [Set(test)] from broker [BrokerEndPoint(0,218.30.64.194,9092)] failed (kafka.client.ClientUtils$)java.nio.channels.ClosedChannelExceptionat kafka.network.BlockingChannel.send(BlockingChannel.scala:110)at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:80)at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:79)at kafka.producer.SyncProducer.send(SyncProducer.scala:124)at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:59)at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:94)at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66)at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)[2016-06-03 11:35:14,918] WARN [console-consumer-72675_zzs-1464924871670-2192d80a-leader-finder-thread], Failed to find leader for Set([test,0]) (kafka.consumer.ConsumerFetcherManager$LeaderFinderThread)kafka.common.KafkaException: fetching topic metadata for topics [Set(test)] from broker [ArrayBuffer(BrokerEndPoint(0,218.30.64.194,9092))] failedat kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:73)at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:94)at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66)at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)Caused by: java.nio.channels.ClosedChannelExceptionat kafka.network.BlockingChannel.send(BlockingChannel.scala:110)at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:80)at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:79)at kafka.producer.SyncProducer.send(SyncProducer.scala:124)at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:59)... 3 more
2 使用localhost启动生产和消费进程:
[root@zzs kafka_2.11-0.10.0.0]# bin/kafka-console-producer.sh --broker-list localhost:9092 --topic testthis is msg[2016-06-03 11:44:16,932] WARN Error while fetching metadata with correlation id 0 : {test=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)[2016-06-03 11:44:18,255] WARN Error while fetching metadata with correlation id 1 : {test=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)[2016-06-03 11:44:18,648] WARN Error while fetching metadata with correlation id 2 : {test=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)[2016-06-03 11:44:18,801] WARN Error while fetching metadata with correlation id 3 : {test=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)[2016-06-03 11:44:18,928] WARN Error while fetching metadata with correlation id 4 : {test=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)[2016-06-03 11:44:19,035] WARN Error while fetching metadata with correlation id 5 : {test=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)[2016-06-03 11:44:19,180] WARN Error while fetching metadata with correlation id 6 : {test=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)[2016-06-03 11:44:19,308] WARN Error while fetching metadata with correlation id 7 : {test=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[root@zzs kafka_2.11-0.10.0.0]# bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test --from-beginning[2016-06-03 11:45:18,330] WARN Fetching topic metadata with correlation id 0 for topics [Set(test)] from broker [BrokerEndPoint(0,218.30.64.194,9092)] failed (kafka.client.ClientUtils$)java.nio.channels.ClosedChannelExceptionat kafka.network.BlockingChannel.send(BlockingChannel.scala:110)at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:80)at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:79)at kafka.producer.SyncProducer.send(SyncProducer.scala:124)at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:59)at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:94)at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66)at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)[2016-06-03 11:45:18,541] WARN [console-consumer-42554_zzs-1464925496436-ae2ee9c7-leader-finder-thread], Failed to find leader for Set([test,0]) (kafka.consumer.ConsumerFetcherManager$LeaderFinderThread)kafka.common.KafkaException: fetching topic metadata for topics [Set(test)] from broker [ArrayBuffer(BrokerEndPoint(0,218.30.64.194,9092))] failedat kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:73)at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:94)at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66)at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)Caused by: java.nio.channels.ClosedChannelExceptionat kafka.network.BlockingChannel.send(BlockingChannel.scala:110)at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:80)at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:79)at kafka.producer.SyncProducer.send(SyncProducer.scala:124)at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:59)... 3 more
3.解决问题
1)查看Kafka的配置文件,cat config/server.properties
zookeeper.connect=localhost:2181
连接的zookeeper的为localhost,所以需要用localhost启动生产和消费进程
2)查看kafka启动的日志,发现
Registered broker 0 at path /brokers/ids/0 with addresses: PLAINTEXT -> EndPoint(218.30.64.194,9092,PLAINTEXT) (kafka.utils.ZkUtils)为什么启动的broker的ip是 218.30.64.194
==> 没有绑定Kafka启动监听的host信息
vi config/server.properties
listeners=PLAINTEXT://localhost:9092
3)重新启动zookeeper、kafka、consumer、producer
./bin/zookeeper-server-start.sh ./config/zookeeper.properties./bin/kafka-server-start.sh ./config/server.properties
bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test --from-beginning
bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test
在producer中输入消息,可以在producer中消费:
[root@zzs kafka_2.11-0.10.0.0]# bin/kafka-console-producer.sh --broker-list localhost:9092 --topic testthis is msgthis is msg2
[root@zzs kafka_2.11-0.10.0.0]# bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test --from-beginningthis is msgthis is msg2this is msg3this is mgs4
问题解决!
- Kafka系列2-producer和consumer报错
- Kafka的 Consumer和Producer
- Kafka的Producer和Consumer源码学习
- Kafka 学习笔记(4) - Consumer和Producer
- spring集成kafka实现producer和consumer
- Kafka(producer和consumer编程实践)
- java--kafka编程实例--producer和consumer
- spring集成kafka实现producer和consumer
- Avro kafka(Producer-Consumer)
- kafka--producer&consumer
- Kafka系列3-python版本producer生产者和consumer消费者实例
- kafka学习笔记 --- Scala实现Kafka producer 和 consumer
- 手动初始化kafka consumer、producer
- Kafka的Producer和Consumer的示例(使用java语言)
- 漫游Kafka设计篇之Producer和Consumer
- 漫游Kafka设计篇之Producer和Consumer
- 漫游Kafka设计篇之Producer和Consumer
- Kafka学习整理七(producer和consumer编程实践)
- 决策树
- dataview的RowFilter的用法
- 解决在Windows server 2008 安装Python2.7版本
- HTTP状态码中301与302的区别
- Linux常用命令:chgrp命令
- Kafka系列2-producer和consumer报错
- 在UIAlterView的代理方法中 点击确定时 添加一个UIView到[[UIApplication sharedApplication].keyWindow上只出现了0.5秒左右就消失的问题
- React native Image组件
- JAVA二叉树遍历(递归和非递归)
- RTK设置ok
- Android手机分辨率基础知识(DPI,DIP计算)
- 一个简单的小棋牌游[代码]
- 北航面试之数据结构试题
- Android使用命令获取sha1证书指纹数据的方法