Flume把事件写入多个信宿

1.创建agent配置文件
把下列内容存入agent7.conf,并保存到Flume的工作目录/opt/flume/bin下面

agent7.sources = netsource
agent7.sinks = hdfssink filesink
agent7.channels = memorychannel1 memorychannel2

agent7.sources.netsource.type = netcat
agent7.sources.netsource.bind = localhost
agent7.sources.netsource.port = 3000
agent7.sources.netsource.interceptors = ts

agent7.sources.netsource.interceptors.ts.type = org.apache.flume.interceptor.TimestampInterceptor$Builder

agent7.sinks.hdfssink.type = hdfs
agent7.sinks.hdfssink.hdfs.path = /flume2sink/ts-%Y-%m-%d
agent7.sinks.hdfssink.hdfs.filePrefix = log
agent7.sinks.hdfssink.hdfs.rollInterval = 0
agent7.sinks.hdfssink.hdfs.rollCount = 5
agent7.sinks.hdfssink.hdfs.fileType = DataStream

agent7.sinks.filesink.type = FILE_ROLL
agent7.sinks.filesink.sink.directory = /opt/flume/files2sink
agent7.sinks.filesink.sink.rollInterval = 0

agent7.channels.memorychannel1.type = memory
agent7.channels.memorychannel1.capacity = 1000
agent7.channels.memorychannel1.transactionCapacity = 100

agent7.channels.memorychannel2.type = memory
agent7.channels.memorychannel2.capacity = 1000
agent7.channels.memorychannel2.transactionCapacity = 100

agent7.sources.netsource.channels = memorychannel1 memorychannel2
agent7.sinks.hdfssink.channel = memorychannel1
agent7.sinks.filesink.channel = memorychannel2

agent7.sources.netsource.selector.type = replicating

(需要创建信宿目录：caiyong@caiyong:/opt/flume$ mkdir files2sink)

2.启动Flume代理
caiyong@caiyong:/opt/flume/bin$ flume-ng agent --conf conf --conf-file agent7.conf --name agent7

3.在另一个窗口中开启一个远程连接并发送几个事件
caiyong@caiyong:~$ telnet localhost 3000
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
one
OK
source
OK
two
OK
sinks
OK
this
OK
is   
OK
a
OK
test
OK
OK,byebye.
OK

4.检查结果
（1）检查file信宿内容
caiyong@caiyong:/opt/flume/files2sink$ ls -alh
总用量 12K
drwxrwxr-x  2 caiyong caiyong 4.0K  3月 14 21:54 .
drwxr-xr-x 10 caiyong root    4.0K  3月 14 15:55 ..
-rw-rw-r--  1 caiyong caiyong   56  3月 14 21:54 1426341241611-1
caiyong@caiyong:/opt/flume/files2sink$ cat *
one
source
two
sinks
this
is
a
test
OK,byebye.

（2）检查HDFS信宿内容
caiyong@caiyong:/opt/hadoop$ bin/hadoop fs -ls /flume2sink/
Found 1 items
drwxr-xr-x   - caiyong supergroup          0 2015-03-14 21:54 /flume2sink/ts-2015-03-14
caiyong@caiyong:/opt/hadoop$ bin/hadoop fs -ls /flume2sink/*
Found 2 items
-rw-r--r--   1 caiyong supergroup         31 2015-03-14 21:54 /flume2sink/ts-2015-03-14/log.1426341252881
-rw-r--r--   1 caiyong supergroup          0 2015-03-14 21:54 /flume2sink/ts-2015-03-14/log.1426341252882.tmp
caiyong@caiyong:/opt/hadoop$ bin/hadoop fs -cat /flume2sink/ts*/*2881
one
source
two
sinks
this
caiyong@caiyong:/opt/hadoop$ bin/hadoop fs -cat /flume2sink/ts*/*mp
is
a
test
OK,byebye.

分析：创建的配置文件设置了一个Netcat信源和两个信宿（file信宿和HDFS信宿），设置了两个memory信道分别连接信源和两个信宿。设置信源选择器类型为replicating，这样所有事件都会发送给两个信道。