9.Flume Sinks

来源：互联网发布：股票辅助软件编辑：程序博客网时间：2024/06/08 16:42

1.Hdfs Sink

a1.channels = c1a1.sinks = k1a1.sinks.k1.type = hdfsa1.sinks.k1.channel = c1a1.sinks.k1.hdfs.path = /flume/events/%y-%m-%d/%H%M/%Sa1.sinks.k1.hdfs.filePrefix = events-a1.sinks.k1.hdfs.round = truea1.sinks.k1.hdfs.roundValue = 10a1.sinks.k1.hdfs.roundUnit = minute

2.Hive Sink

a1.channels = c1a1.channels.c1.type = memorya1.sinks = k1a1.sinks.k1.type = hivea1.sinks.k1.channel = c1a1.sinks.k1.hive.metastore = thrift://127.0.0.1:9083a1.sinks.k1.hive.database = logsdba1.sinks.k1.hive.table = weblogsa1.sinks.k1.hive.partition = asia,%{country},%y-%m-%d-%H-%Ma1.sinks.k1.useLocalTimeStamp = falsea1.sinks.k1.round = truea1.sinks.k1.roundValue = 10a1.sinks.k1.roundUnit = minutea1.sinks.k1.serializer = DELIMITEDa1.sinks.k1.serializer.delimiter = "\t"a1.sinks.k1.serializer.serdeSeparator = '\t'a1.sinks.k1.serializer.fieldnames =id,,msg

3.Logger Sink

a1.channels = c1a1.sinks = k1a1.sinks.k1.type = loggera1.sinks.k1.channel = c1

4.Avro Sink

a1.channels = c1a1.sinks = k1a1.sinks.k1.type = avroa1.sinks.k1.channel = c1a1.sinks.k1.hostname = 10.10.10.10a1.sinks.k1.port = 4545

5.Thrift Sink

a1.channels = c1a1.sinks = k1a1.sinks.k1.type = thrifta1.sinks.k1.channel = c1a1.sinks.k1.hostname = 10.10.10.10a1.sinks.k1.port = 4545

6.IRC Sink

a1.channels = c1a1.sinks = k1a1.sinks.k1.type = irca1.sinks.k1.channel = c1a1.sinks.k1.hostname = irc.yourdomain.coma1.sinks.k1.nick = flumea1.sinks.k1.chan = #flume

7.File Roll Sink

a1.channels = c1a1.sinks = k1a1.sinks.k1.type = file_rolla1.sinks.k1.channel = c1a1.sinks.k1.sink.directory = /var/log/flume

8.NULL Sink

丢弃它从通道接收的所有事件。

a1.channels = c1a1.sinks = k1a1.sinks.k1.type = nulla1.sinks.k1.channel = c1

9.Hbase Sink

a1.channels = c1a1.sinks = k1a1.sinks.k1.type = hbasea1.sinks.k1.table = foo_tablea1.sinks.k1.columnFamily = bar_cfa1.sinks.k1.serializer = org.apache.flume.sink.hbase.RegexHbaseEventSerializera1.sinks.k1.channel = c1

10.AsyncHbase Sink

a1.channels = c1a1.sinks = k1a1.sinks.k1.type = asynchbasea1.sinks.k1.table = foo_tablea1.sinks.k1.columnFamily = bar_cfa1.sinks.k1.serializer = org.apache.flume.sink.hbase.SimpleAsyncHbaseEventSerializera1.sinks.k1.channel = c1

11.MorphlineSolrSink

a1.channels = c1a1.sinks = k1a1.sinks.k1.type = org.apache.flume.sink.solr.morphline.MorphlineSolrSinka1.sinks.k1.channel = c1a1.sinks.k1.morphlineFile = /etc/flume-ng/conf/morphline.conf# a1.sinks.k1.morphlineId = morphline1# a1.sinks.k1.batchSize = 1000# a1.sinks.k1.batchDurationMillis = 1000

11.ElasticSearchSink

a1.channels = c1a1.sinks = k1a1.sinks.k1.type = elasticsearcha1.sinks.k1.hostNames = 127.0.0.1:9200,127.0.0.2:9300a1.sinks.k1.indexName = foo_indexa1.sinks.k1.indexType = bar_typea1.sinks.k1.clusterName = foobar_clustera1.sinks.k1.batchSize = 500a1.sinks.k1.ttl = 5da1.sinks.k1.serializer = org.apache.flume.sink.elasticsearch.ElasticSearchDynamicSerializera1.sinks.k1.channel = c1

12.Kite Dataset Sink

13.Kafka Sink

a1.sinks.k1.channel = c1a1.sinks.k1.type = org.apache.flume.sink.kafka.KafkaSinka1.sinks.k1.kafka.topic = mytopica1.sinks.k1.kafka.bootstrap.servers = localhost:9092a1.sinks.k1.kafka.flumeBatchSize = 20a1.sinks.k1.kafka.producer.acks = 1a1.sinks.k1.kafka.producer.linger.ms = 1a1.sinks.ki.kafka.producer.compression.type = snappy

14.Custom Sink

a1.channels = c1a1.sinks = k1a1.sinks.k1.type = org.example.MySinka1.sinks.k1.channel = c1

阅读全文

1 0