Failing to produce to kafka brokers java.nio.BufferUnderflowException
来源:互联网 发布:电视墙控制软件 编辑:程序博客网 时间:2024/04/27 21:57
storm程序在虚拟机的测试环境没有问题,部署上生产环境后出现如下问题:
2017-04-17 09:43:14 k.p.a.DefaultEventHandler [INFO] Back off for 100 ms before retrying send. Remaining retries = 12017-04-17 09:43:14 k.c.ClientUtils$ [INFO] Fetching metadata from broker BrokerEndPoint(3,node12,6667) with correlation id 312494 for 1 topic(s) Set(zyq)2017-04-17 09:43:14 k.p.SyncProducer [INFO] Connected to node12:6667 for producing2017-04-17 09:43:14 k.p.SyncProducer [INFO] Disconnecting from node12:66672017-04-17 09:43:14 k.p.SyncProducer [INFO] Disconnecting from node11:66672017-04-17 09:43:14 k.p.SyncProducer [INFO] Disconnecting from node10:66672017-04-17 09:43:14 k.p.SyncProducer [INFO] Disconnecting from node9:66672017-04-17 09:43:14 k.p.SyncProducer [INFO] Connected to node11:6667 for producing2017-04-17 09:43:14 k.p.a.DefaultEventHandler [WARN] Failed to send producer request with correlation id 312695 to broker 1 with data for partitions [zyq,2]java.nio.BufferUnderflowException: nullat java.nio.Buffer.nextGetIndex(Buffer.java:498) ~[na:1.7.0_79]at java.nio.HeapByteBuffer.getLong(HeapByteBuffer.java:406) ~[na:1.7.0_79]at kafka.api.ProducerResponse$$anonfun$1$$anonfun$apply$1.apply(ProducerResponse.scala:40) ~[stormjar.jar:na]at kafka.api.ProducerResponse$$anonfun$1$$anonfun$apply$1.apply(ProducerResponse.scala:36) ~[stormjar.jar:na]at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) ~[stormjar.jar:na]at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) ~[stormjar.jar:na]at scala.collection.immutable.Range.foreach(Range.scala:141) ~[stormjar.jar:na]at scala.collection.TraversableLike$class.map(TraversableLike.scala:244) ~[stormjar.jar:na]at scala.collection.AbstractTraversable.map(Traversable.scala:105) ~[stormjar.jar:na]at kafka.api.ProducerResponse$$anonfun$1.apply(ProducerResponse.scala:36) ~[stormjar.jar:na]at kafka.api.ProducerResponse$$anonfun$1.apply(ProducerResponse.scala:33) ~[stormjar.jar:na]at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:251) ~[stormjar.jar:na]at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:251) ~[stormjar.jar:na]at scala.collection.immutable.Range.foreach(Range.scala:141) ~[stormjar.jar:na]at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:251) ~[stormjar.jar:na]at scala.collection.AbstractTraversable.flatMap(Traversable.scala:105) ~[stormjar.jar:na]at kafka.api.ProducerResponse$.readFrom(ProducerResponse.scala:33) ~[stormjar.jar:na]at kafka.producer.SyncProducer.send(SyncProducer.scala:114) ~[stormjar.jar:na]at kafka.producer.async.DefaultEventHandler.kafka$producer$async$DefaultEventHandler$$send(DefaultEventHandler.scala:275) [stormjar.jar:na]at kafka.producer.async.DefaultEventHandler$$anonfun$dispatchSerializedData$2.apply(DefaultEventHandler.scala:113) [stormjar.jar:na]at kafka.producer.async.DefaultEventHandler$$anonfun$dispatchSerializedData$2.apply(DefaultEventHandler.scala:105) [stormjar.jar:na]at scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:772) [stormjar.jar:na]at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98) [stormjar.jar:na]at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98) [stormjar.jar:na]at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:226) [stormjar.jar:na]at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39) [stormjar.jar:na]at scala.collection.mutable.HashMap.foreach(HashMap.scala:98) [stormjar.jar:na]at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:771) [stormjar.jar:na]at kafka.producer.async.DefaultEventHandler.dispatchSerializedData(DefaultEventHandler.scala:105) [stormjar.jar:na]at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:78) [stormjar.jar:na]at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:106) [stormjar.jar:na]at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:89) [stormjar.jar:na]at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:69) [stormjar.jar:na]at scala.collection.Iterator$class.foreach(Iterator.scala:727) [stormjar.jar:na]at scala.collection.AbstractIterator.foreach(Iterator.scala:1157) [stormjar.jar:na]at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:68) [stormjar.jar:na]at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:46) [stormjar.jar:na]2017-04-17 09:43:14 k.p.a.DefaultEventHandler [INFO] Back off for 100 ms before retrying send. Remaining retries = 02017-04-17 09:43:14 k.c.ClientUtils$ [INFO] Fetching metadata from broker BrokerEndPoint(0,node9,6667) with correlation id 312696 for 1 topic(s) Set(zyq)2017-04-17 09:43:14 k.p.SyncProducer [INFO] Connected to node9:6667 for producing2017-04-17 09:43:14 k.p.SyncProducer [INFO] Disconnecting from node9:66672017-04-17 09:43:14 k.p.SyncProducer [INFO] Disconnecting from node11:66672017-04-17 09:43:14 k.p.SyncProducer [INFO] Disconnecting from node10:66672017-04-17 09:43:14 k.p.SyncProducer [INFO] Disconnecting from node9:66672017-04-17 09:43:14 k.p.a.DefaultEventHandler [ERROR] Failed to send requests for topics zyq with correlation ids in [311889,312696]2017-04-17 09:43:14 k.p.a.ProducerSendThread [ERROR] Error in handling batch of 200 eventskafka.common.FailedToSendMessageException: Failed to send messages after 3 tries.at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:96) ~[stormjar.jar:na]at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:106) [stormjar.jar:na]at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:89) [stormjar.jar:na]at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:69) [stormjar.jar:na]at scala.collection.Iterator$class.foreach(Iterator.scala:727) [stormjar.jar:na]at scala.collection.AbstractIterator.foreach(Iterator.scala:1157) [stormjar.jar:na]at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:68) [stormjar.jar:na]at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:46) [stormjar.jar:na]
之后发现生产环境和测试环境中的kafka的jar包版本不一样,生产环境是0.8.1,eclipse中的环境是0.10.0,将topology中的kafka的jar包的版本换成0.8.1就解决了。
2 0
- Failing to produce to kafka brokers java.nio.BufferUnderflowException
- java.nio.BufferUnderflowException
- 关于mina的java.nio.BufferUnderflowException
- Use Autotools to Produce Makefile
- Javadoc in Eclipse failing to recognize packages
- kafka produce源码研究
- Resoution for Failing To Agree With New iPhone Agreement
- Cocos2d v2.x, failing to install templates.. permission denied.
- Failing over to RSL framework_4.6.0.23201.swz
- 【JAVA】【NIO】6、Java NIO Channel to Channel Transfers
- Java NIO Channel to Channel Transfers
- Java NIO(6-Channel to Channel Transfers)
- Unable to execute dex: java.nio.BufferOverflowException
- Java NIO Channel to Channel Transfers
- Unable to execute dex: java.nio.BufferOverflowException
- Unable to execute dex: java.nio.BufferOverflowException
- Unable to execute dex: java.nio.BufferOverflowException
- Unable to execute dex: java.nio.BufferOverflowException.
- Redis 和 I/O 多路复用
- 用RxJava实现Rxbus替换EventBus事件总线
- [BZOJ1737][Usaco2005 jan]Naptime 午睡时间(dp)
- Java流实例-文件复制
- Windows/Linux下的Python安装指导
- Failing to produce to kafka brokers java.nio.BufferUnderflowException
- java多线程并发(一)Semaphore,volatile,synchronized ,Lock, CyclicBarrier和CountDownLatch
- TensorFlow25: 使用深度学习做阅读理解+完形填空
- 图像横纵填充算法
- 使用if和goto语句构造循环
- 史上最全的Javascript面试题总结(内附答案)
- Android 基础工具类 摘自quantan.liu 易读项目
- html rel="external nofollow"
- myEclipse 非正常关闭