Flume对接Kafka
来源:互联网 发布:vk聊天软件 编辑:程序博客网 时间:2024/05/16 15:46
1.配置文件:s1kkafka.conf
# s1kkafka.conf: A single-source and kafka-sink configuration
# Name the components on this agent
a3.sources = r1
a3.sinks = k1
a3.channels = c1
# Describe/configure the source
a3.sources.r1.type = netcat
a3.sources.r1.bind = localhost
a3.sources.r1.port = 44444
# Describe the sink
a3.sinks.k1.type = org.apache.flume.sink.kafka.KafkaSink
a3.sinks.k1.brokerList = bigdata-SZTS-Slave01:9091,bigdata-SZTS-Slave02:9093
a3.sinks.k1.topic = lwglscan
a3.sinks.k1.requiredAcks = 1
a3.sinks.k1.batchSize = 2000
# Use a channel which buffers events in memory
a3.channels.c1.type = memory
a3.channels.c1.capacity = 1000
a3.channels.c1.transactionCapacity = 100
# Bind the source and sink to the channel
a3.sources.r1.channels = c1
a3.sinks.k1.channel = c1
2.测试
telnet 127.0.0.1 44444,启动一个console作为消费者,应该能看到telnet输入的回显
# s1kkafka.conf: A single-source and kafka-sink configuration
# Name the components on this agent
a3.sources = r1
a3.sinks = k1
a3.channels = c1
# Describe/configure the source
a3.sources.r1.type = netcat
a3.sources.r1.bind = localhost
a3.sources.r1.port = 44444
# Describe the sink
a3.sinks.k1.type = org.apache.flume.sink.kafka.KafkaSink
a3.sinks.k1.brokerList = bigdata-SZTS-Slave01:9091,bigdata-SZTS-Slave02:9093
a3.sinks.k1.topic = lwglscan
a3.sinks.k1.requiredAcks = 1
a3.sinks.k1.batchSize = 2000
# Use a channel which buffers events in memory
a3.channels.c1.type = memory
a3.channels.c1.capacity = 1000
a3.channels.c1.transactionCapacity = 100
# Bind the source and sink to the channel
a3.sources.r1.channels = c1
a3.sinks.k1.channel = c1
2.测试
telnet 127.0.0.1 44444,启动一个console作为消费者,应该能看到telnet输入的回显
0 0
- Flume对接Kafka
- flume和kafka对接
- Flume对接Kafka的实例
- flume同kafka的对接
- Kafka对接flume实现(一)
- flume+kafka
- flume+kafka
- flume kafka
- Spark Streaming+Flume对接实验
- flume+kafka+storm
- flume+kafka+storm调研
- flume 读取kafka 数据
- Flume+kafka+storm整合
- Flume与Kafka整合
- flume+kafka+storm+mysql
- Kafka flume 整合
- Flume+storm+kafka
- Kafka flume 整合
- C#中List<object>之间的转换
- Java读取XLS文件-JXL的使用
- Linux下yum安装MySQL yum安装MySQL指定版本
- Undefined symbols for architecture x86_64: "_OBJC_CLASS_$_JSContext", referenced from: ...........
- 关于光源色温与标准光源的讨论
- Flume对接Kafka
- C++ set 详解
- 详细解析Linux /etc/passwd文件
- 【STM32】STM32之霍尔检测电机归位时掉电
- 同步、异步方式对SEO搜索引擎优化的影响
- Ionic Datepicker的使用
- request,response,session,cookie,application jun转载
- Spring入门笔记(二)AOP及AspectJ
- Hive 执行时磁盘空间不足的问题