Flume配置

来源：互联网发布：an软件编辑：程序博客网时间：2024/06/06 03:04

cp flume-env.sh.template flume-env.sh

加入JAVA_HOME=/home/cdh/java

在conf/avro-flume.conf

agent3.channels = ch1

agent3.sources = avro-source1

agent3.sinks = log-sink1

agent3.sources.avro-source1.channels = ch1

agent3.sources.avro-source1.type = avro

agent3.sources.avro-source1.bind = 0.0.0.0

agent3.sources.avro-source1.port = 12343

agent3.channels.ch1.type = memory

agent3.sinks.log-sink1.channel = ch1

agent3.sinks.log-sink1.type = logger

先启动agent

# bin/flume-ng agent --conf ./conf/ -f conf/avro-flume.conf -Dflume.root.logger=DEBUG,console -n agent3

再启动avro-client

bin/flume-ng avro-client --conf conf -H localhost -p 12343 -F test-data/data1.txt -Dflume.root.logger=DEBUG,console

启动完avro后面的信息是

… …

2015-04-15 10:56:58,933 (main) [DEBUG - org.apache.flume.client.avro.AvroCLIClient.run(AvroCLIClient.java:234)] Finished

2015-04-15 10:56:58,933 (main) [DEBUG - org.apache.flume.client.avro.AvroCLIClient.run(AvroCLIClient.java:237)] Closing reader

2015-04-15 10:56:58,938 (main) [DEBUG - org.apache.flume.client.avro.AvroCLIClient.run(AvroCLIClient.java:241)] Closing RPC client

2015-04-15 10:56:58,949 (main) [DEBUG - org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:84)] Exiting

再看agent端的信息

2015-04-15 11:03:43,026 (conf-file-poller-0) [DEBUG - org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:126)] Checking file:conf/avro-flume.conf for changes

2015-04-15 11:03:52,313 (New I/O server boss #1 ([id: 0xaf76f29e, /0:0:0:0:0:0:0:0:12343])) [INFO - org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream(NettyServer.java:171)] [id: 0xca765c3f, /127.0.0.1:34023 => /127.0.0.1:12343] OPEN

2015-04-15 11:03:52,314 (New I/O worker #3) [INFO - org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream(NettyServer.java:171)] [id: 0xca765c3f, /127.0.0.1:34023 => /127.0.0.1:12343] BOUND: /127.0.0.1:12343

2015-04-15 11:03:53,192 (New I/O worker #3) [DEBUG - org.apache.flume.source.AvroSource.appendBatch(AvroSource.java:371)] Avro source avro-source1: Received avro event batch of 1 events.

2015-04-15 11:03:53,201 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:70)]Event: { headers:{} body: 31 32 33 123 }

2015-04-15 11:03:53,262 (New I/O worker #3) [INFO - org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream(NettyServer.java:171)] [id: 0xca765c3f, /127.0.0.1:34023 :> /127.0.0.1:12343] DISCONNECTED

2015-04-15 11:03:53,263 (New I/O worker #3) [INFO - org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream(NettyServer.java:171)] [id: 0xca765c3f, /127.0.0.1:34023 :> /127.0.0.1:12343] UNBOUND

2015-04-15 11:03:53,264 (New I/O worker #3) [INFO - org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream(NettyServer.java:171)] [id: 0xca765c3f, /127.0.0.1:34023 :> /127.0.0.1:12343] CLOSED

2015-04-15 11:03:53,264 (New I/O worker #3) [INFO - org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.channelClosed(NettyServer.java:209)] Connection to /127.0.0.1:34023 disconnected.

这样就表示部署成功

0 0