flume ng 和flume og的区别
来源:互联网 发布:大数据专家委员会竞赛 编辑:程序博客网 时间:2024/04/27 15:50
摘自: http://blog.csdn.net/rzhzhz/article/details/8778266
Flume OG:Flume original generation 即Flume 0.9.x版本
Flume NG:Flume next generation ,即Flume 1.x版本
对于Flume OG ,可以说他是一个分布式日志收集系统,有Mater概念,依赖于zookeeper,以下是其架构图
Agent用于采集数据,agent是flume中产生数据流的地方,同时,agent会将产生的数据流传输到collector。对应的,collector用于对数据进行聚合,往往会产生一个更大的流。
而对于Flume NG,它摒弃了Master和zookeeper,collector也没有了,web配置台也没有了,只剩下source,sink和channel,此时一个Agent的概念包括source,channel和sink,完全由一个分布式系统变成了传输工具。不同机器之间的数据传输不再是OG那样由agent->collector,而是由一个Agent端的sink流向另一个agent的source。其新的架构如下
Flume NG is a huge departure from Flume OG (original generation, or "original gangsta," if you prefer) in its implementation although many of the original concepts are the same. If you're already familiar with Flume, here's what you need to know.
- You still have sources and sinks and they still do the same thing. They are now connected by channels.
- Channels are pluggable and dictate durability. Flume NG ships with an in-memory channel for fast, but non-durable event delivery and a JDBC-based channel for durable event delivery. We have recently added a file-based durable channel too.
- There's no more logical or physical nodes. We call all physical nodes agents and agents can run zero or more sources and sinks.
- There's no master and no ZooKeeper dependency anymore. At this time, Flume runs with a simple file-based configuration system.
- Just about everything is a plugin, some end user facing, some for tool and system developers. (Specifically, sources, sinks, channels, configuration providers, lifecycle management policies, input and output formats, compression, source and sink channel adapters, and the kitchen sink.)
- Tons of things are not yet implemented. Please file JIRAs and / or vote for features you deem important.
- flume ng 和flume og的区别
- Flume OG和Flume NG的区别
- flume-ng与flume-og的区别
- Flume-og和Flume-ng的变化
- Flume OG & Flume NG
- Flume NG:Flume OG 区别对比
- Flume OG 与 Flume NG 的对比
- flume OG VS flume NG
- flume-ng与flume-og对比
- flume-ng与flume-og对比
- Flume OG 介绍和配置
- Spark Streaming 和 Flume-NG的整合
- Flume-ng的原理和使用
- Flume-ng的原理和使用
- Flume-ng的原理和使用
- Flume-ng的原理和使用
- Flume-NG 的原理和使用
- Spark和Flume-ng的整合
- 静态局部变量初始化问题
- ACM训练方案-POJ题目分类
- 关于python二维数组的声明
- 尊重原创,请保证您的文章为原创作品
- Insertion Sort List -- LeetCode
- flume ng 和flume og的区别
- ZigZag Conversion -- LeetCode
- flume的原理和使用
- String to Integer (atoi) -- LeetCode
- Palindrome Number -- LeetCode
- kafka初步认识
- Container With Most Water -- LeetCode
- Regular Expression Matching -- LeetCode
- Integer to Roman -- LeetCode