Error UNKNOWN_MEMBER_ID occurred while committing offsets for group ***
来源:互联网 发布:老男孩linux培训课件 编辑:程序博客网 时间:2024/06/07 03:04
遇到这个问题,一般会造成kafka offset不断变换,导致重复消费,实际这个方法是因为提交事务超时导致,解决的办法也非常简单,就是增加超时时长控制
先看看stackoverflow上的说法:
I just faced the same issue. I have been investigating, and in this thread and in this wiki you can find the solution. The issue seems to be that the processing of a batch takes longer than the session timeout. Either increase the session timeout or the polling frequency or limit the number of bytes received. What worked for me was changing max.partition.fetch.bytes. But you can also modify session.timeout.ms or the value you pass to your consumer.poll(TIMEOUT)
调整session.timeout.ms需要注意,kafkay有相关连配置,request.timeout.ms必须大于session.timeout.ms和fetch.max.wait.ms
fetch.max.wait.ms默认为500,request.timeout.ms默认为30000
org.apache.kafka.common.config.ConfigException: request.timeout.ms should be greater than session.timeout.ms and fetch.max.wait.ms
经调整,flume 1.7.0 kafka_channel增加如下配置:
agent1.channels.c2.kafka.consumer.session.timeout.ms=100000agent1.channels.c2.kafka.consumer.request.timeout.ms=110000agent1.channels.c2.kafka.consumer.fetch.max.wait.ms=1000
阅读全文
0 0
- kafka : Error UNKNOWN_MEMBER_ID occurred while committing offsets for group alert
- kafka : Error UNKNOWN_MEMBER_ID occurred while committing offsets for group alert
- Error UNKNOWN_MEMBER_ID occurred while committing offsets for group ***
- Unexpected error occurred while trying to check for group existence.
- IAR FOR STM8S 错误 An error occurred while retrieving GDI features: gdi-error [40201]解决方法
- Error 25003. Error occurred while initializing fusion
- [DeploymentService:290066]Error occurred while downloading files from admin server for deployment re
- Android Studio 中 An error occurred while creating the AVD.See idea.log for detail
- An error occurred while updating the entries. See the inner exception for details.解决方案
- Error occurred while downloading files from Administration Server for deployment request
- an error occurred while mounting /home 解决方法
- Error occurred while loading modules from classpath
- The error occurred while processing mapper_resultMap[DrugConditionResult]
- Eclipse : An error occurred while filtering resources
- An error occurred while installing system .....github
- The error occurred while setting parameters
- eclipse:an error occurred while filtering resources
- an error occurred while processing the directive
- 【如何校验邮件地址的有效性】第一篇 :为什么要校验邮箱地址有效性?使用场景在哪里?
- 24.返回栈
- Linux环境下通过ODBC访问MSSql Server
- 第8章节_进程控制
- Java编程入门学习开发的20个必备技术点
- Error UNKNOWN_MEMBER_ID occurred while committing offsets for group ***
- binder 1 kernel开篇
- ubuntu 16.10 安装 openjdk 7
- Java 链式调用
- socket服务器
- hihocoder 字符消除
- AES加密解密
- Java学习笔记---反射
- Android-times-square自定义样式