Flume开发问题记录

一 、Flume 启动异常：

2017-11-24 17:24:39,115 (kafka-producer-network-thread | producer-1) [DEBUG - org.apache.kafka.common.network.Selector.poll(Selector.java:307)] Connection with /xxxx disconnectedjava.io.EOFException    at org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:83)    at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:71)    at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:153)    at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:134)    at org.apache.kafka.common.network.Selector.poll(Selector.java:286)    at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:256)    at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:216)    at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:128)    at java.lang.Thread.run(Thread.java:745)

因为当时还迁移了zookeeper节点和kafka节点。以为是zookeeper或者kafka的问题。查看zookeeper发现如下异常：

 WARN  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@362] - Exception causing close of session 0x0 due to java.io.IOException: Unreasonable length = 1467246

意思是报文长度过长，不合理。最后发现是批量修改flume节点kafka配置出了问题。kafka.bootstrap.servers的端口配置成了zookeeper端口2181。应该是kafka端口9092。
总之这个问题，是Flume kafka配置错误，导致的连接异常。