kafka.common.ConsumerRebalanceFailedException异常解决
来源:互联网 发布:专业淘宝刷销量 编辑:程序博客网 时间:2024/04/28 11:14
kafka.common.ConsumerRebalanceFailedException: group_dd-1446432618163-2746a209 can't rebalance after 10 retries
at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:432)
at kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$$reinitializeConsumer(ZookeeperConsumerConnector.scala:722)
at kafka.consumer.ZookeeperConsumerConnector.consume(ZookeeperConsumerConnector.scala:212)
at kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreams(ZookeeperConsumerConnector.scala:80)
at kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreams(ZookeeperConsumerConnector.scala:92)
at com.symboltech.mine.ConsumerUtil.start(ConsumerUtil.java:40)
at com.symboltech.mine.KafkaConsumer.sdf(KafkaConsumer.java:58)
解决办法:
1.配置zk问题(kafka的consumer配置)
zookeeper.session.timeout.ms=5000
zookeeper.connection.timeout.ms=10000
rebalance.backoff.ms=2000
rebalance.max.retries=10
官方解释:
consumer rebalancing fails (you will see ConsumerRebalanceFailedException): This is due to conflicts when two consumers are trying to own the same topic partition. The log will show you what caused the conflict (search for "conflict in ").
If your consumer subscribes to many topics and your ZK server is busy, this could be caused by consumers not having enough time to see a consistent view of all consumers in the same group. If this is the case, try Increasing rebalance.max.retries and rebalance.backoff.ms.
Another reason could be that one of the consumers is hard killed. Other consumers during rebalancing won't realize that consumer is gone after zookeeper.session.timeout.ms time. In the case, make sure that rebalance.max.retries * rebalance.backoff.ms > zookeeper.session.timeout.ms.
at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:432)
at kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$$reinitializeConsumer(ZookeeperConsumerConnector.scala:722)
at kafka.consumer.ZookeeperConsumerConnector.consume(ZookeeperConsumerConnector.scala:212)
at kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreams(ZookeeperConsumerConnector.scala:80)
at kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreams(ZookeeperConsumerConnector.scala:92)
at com.symboltech.mine.ConsumerUtil.start(ConsumerUtil.java:40)
at com.symboltech.mine.KafkaConsumer.sdf(KafkaConsumer.java:58)
解决办法:
1.配置zk问题(kafka的consumer配置)
zookeeper.session.timeout.ms=5000
zookeeper.connection.timeout.ms=10000
rebalance.backoff.ms=2000
rebalance.max.retries=10
官方解释:
consumer rebalancing fails (you will see ConsumerRebalanceFailedException): This is due to conflicts when two consumers are trying to own the same topic partition. The log will show you what caused the conflict (search for "conflict in ").
If your consumer subscribes to many topics and your ZK server is busy, this could be caused by consumers not having enough time to see a consistent view of all consumers in the same group. If this is the case, try Increasing rebalance.max.retries and rebalance.backoff.ms.
Another reason could be that one of the consumers is hard killed. Other consumers during rebalancing won't realize that consumer is gone after zookeeper.session.timeout.ms time. In the case, make sure that rebalance.max.retries * rebalance.backoff.ms > zookeeper.session.timeout.ms.
0 0
- kafka.common.ConsumerRebalanceFailedException异常解决
- 【疑难杂症】记录 kafka.common.ConsumerRebalanceFailedException:异常
- apache kafka系列之kafka.common.ConsumerRebalanceFailedException异常解决办法
- apache kafka系列之kafka.common.ConsumerRebalanceFailedException异常解决办法
- apache kafka系列之kafka.common.ConsumerRebalanceFailedException异常解决办法
- apache kafka系列之kafka.common.ConsumerRebalanceFailedException异常解决办法
- apache kafka系列之kafka.common.ConsumerRebalanceFailedException异常解决办法
- apache kafka系列之kafka.common.ConsumerRebalanceFailedException异常解决办法
- apache kafka系列之kafka.common.ConsumerRebalanceFailedException异常解决办法
- kafka.common.ConsumerRebalanceFailedException
- Kafka--Caused by: kafka.common.ConsumerRebalanceFailedException
- kafka消费者报错:kafka.common.ConsumerRebalanceFailedException
- 大数据_Kafka_kafka.common.ConsumerRebalanceFailedException异常解决办法
- kafka.common.ConsumerRebalanceFailedException :log-push-record-consumer-group_mobile-pushremind02.lf
- kafka.common.ConsumerRebalanceFailedException-org.I0Itec.zkclient.exception.ZkNoNodeException
- [异常处理]class kafka.common.UnknownTopicOrPartitionException (kafka.server.ReplicaFetcherThread)
- 修改kafka源码,编译kafka 0.8.2.2,解决bug kafka.common.InvalidTopicException
- 求助,kafka运行异常,该怎么解决
- OC为什么以及什么时候要用到block?
- 版面排版快捷键Ctrl + Alt + L
- jQuery-easyUI的使用:tree的使用
- 数组的一点心得
- Documentation/driver-model/device.txt 翻译学习 -- 需完善
- kafka.common.ConsumerRebalanceFailedException异常解决
- iOS实现简单的关灯游戏
- Android手势识别
- Nginx开启404自定义页面
- 15241052
- android的eclipse快捷键汇总中
- 利用idea环境进行Spring框架的使用和体验
- x86/i386/i686/i86_64
- 从校园到工作的路(五)————java中的类型转换