Flume OG 与 Flume NG 的对比
来源:互联网 发布:出题软件 编辑:程序博客网 时间:2024/04/27 18:15
很久没接触flume了,刚掀开官网一看,发现flume已然不是以前的那个flume了,其实早在flume技术群就听到NG这个字眼,以前没特注意,今天做了些对比,发现flume确实有了投胎换骨般的改变。首先介绍下Flume OG & Flume NG这两个概念
Flume OG:Flume original generation 即Flume 0.9.x版本
Flume NG:Flume next generation ,即Flume 1.x版本
对于Flume OG ,可以说他是一个分布式日志收集系统,有Mater概念,依赖于zookeeper,以下是其架构图
Agent用于采集数据,agent是flume中产生数据流的地方,同时,agent会将产生的数据流传输到collector。对应的,collector用于对数据进行聚合,往往会产生一个更大的流。
而对于Flume NG,它摒弃了Master和zookeeper,collector也没有了,web配置台也没有了,只剩下source,sink和channel,此时一个Agent的概念包括source,channel和sink,完全由一个分布式系统变成了传输工具。不同机器之间的数据传输不再是OG那样由agent->collector,而是由一个Agent端的sink流向另一个agent的source。其新的架构如下
Flume NG is a huge departure from Flume OG (original generation, or "original gangsta," if you prefer) in its implementation although many of the original concepts are the same. If you're already familiar with Flume, here's what you need to know.
- You still have sources and sinks and they still do the same thing. They are now connected by channels.
- Channels are pluggable and dictate durability. Flume NG ships with an in-memory channel for fast, but non-durable event delivery and a JDBC-based channel for durable event delivery. We have recently added a file-based durable channel too.
- There's no more logical or physical nodes. We call all physical nodes agents and agents can run zero or more sources and sinks.
- There's no master and no ZooKeeper dependency anymore. At this time, Flume runs with a simple file-based configuration system.
- Just about everything is a plugin, some end user facing, some for tool and system developers. (Specifically, sources, sinks, channels, configuration providers, lifecycle management policies, input and output formats, compression, source and sink channel adapters, and the kitchen sink.)
- Tons of things are not yet implemented. Please file JIRAs and / or vote for features you deem important.
- Flume OG 与 Flume NG 的对比
- flume-ng与flume-og对比
- flume-ng与flume-og对比
- flume-ng与flume-og的区别
- Flume NG:Flume OG 区别对比
- Flume OG & Flume NG
- flume ng 和flume og的区别
- Flume-og和Flume-ng的变化
- Flume OG和Flume NG的区别
- flume OG VS flume NG
- Flume OG 编译
- Kafka 与 Flume 对比
- flume-ng
- Flume NG
- flume ng
- Flume-ng
- flume-ng与kafka整合
- flume-ng 与elasticsearch整合
- 第十章 信号及信号处理
- VB.Net程序设计:动态添加右键菜单和子菜单
- Ubuntu10.10修改环境变量
- 使用 Boost 的 IPC 和 MPI 库进行并发编程
- Moravec角点检测算子和Harris角点检测算子
- Flume OG 与 Flume NG 的对比
- php如何实现定时任务 php定时任务方法 最佳解决方案 php自动任务处理
- #pragma pack( [ n] )
- 测试1
- 编程之美 2.18数组分割问题
- matlab中m文件的命名规则
- Apache如何开启Mod_rewrite模块以及Wordpress伪静态
- WPF-14:绑定中数据模型必须为public问题
- js操作android-sqlite数据库