flume 安装
来源:互联网 发布:mysql查看所有表空间 编辑:程序博客网 时间:2024/05/19 08:42
转自:http://blog.csdn.net/desilting/article/details/22811593
flume下载:http://www.apache.org/dyn/closer.cgi/flume/1.4.0/apache-flume-1.4.0-bin.tar.gz
flume安装:tar zvxf apache-flume-1.4.0-bin.tar.gz
修改conf/flume-conf.properties.template文件名为flume-conf
修改文件内容:
- a1.sources = r1
- a1.sinks = k1
- a1.channels = c1
- # Describe/configure the source
- a1.sources.r1.type = netcat
- a1.sources.r1.bind = localhost
- a1.sources.r1.port = 5858
- # Describe the sink
- a1.sinks.k1.type = logger
- # Use a channel which buffers events in memory
- a1.channels.c1.type = memory
- a1.channels.c1.capacity = 1000
- a1.channels.c1.transactionCapacity = 100
- # Bind the source and sink to the channel
- a1.sources.r1.channels = c1
- a1.sinks.k1.channel = c1
启动服务:
bin/flume-ng agent --conf conf --conf-file conf/flume-conf --name a1 -Dflume.root.logger=INFO,console
输出日志:
- + exec /etc/local/jdk1.6.0_37/bin/java -Xmx20m -Dflume.root.logger=INFO,console -cp '/usr/local/apache-flume-1.4.0-bin/conf:/usr/local/apache-flume-1.4.0-bin/lib/*' -Djava.library.path= org.apache.flume.node.Application --conf-file conf/flume-conf --name a1
- 2014-04-02 02:08:42,373 (lifecycleSupervisor-1-0) [INFO - org.apache.flume.node.PollingPropertiesFileConfigurationProvider.start(PollingPropertiesFileConfigurationProvider.java:61)] Configuration provider starting
- 2014-04-02 02:08:42,381 (conf-file-poller-0) [INFO - org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:133)] Reloading configuration file:conf/flume-conf
- 2014-04-02 02:08:42,389 (conf-file-poller-0) [INFO - org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:930)] Added sinks: k1 Agent: a1
- 2014-04-02 02:08:42,390 (conf-file-poller-0) [INFO - org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:1016)] Processing:k1
- 2014-04-02 02:08:42,390 (conf-file-poller-0) [INFO - org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:1016)] Processing:k1
- 2014-04-02 02:08:42,424 (conf-file-poller-0) [INFO - org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:140)] Post-validation flume configuration contains configuration for agents: [a1]
- 2014-04-02 02:08:42,424 (conf-file-poller-0) [INFO - org.apache.flume.node.AbstractConfigurationProvider.loadChannels(AbstractConfigurationProvider.java:150)] Creating channels
- 2014-04-02 02:08:42,443 (conf-file-poller-0) [INFO - org.apache.flume.channel.DefaultChannelFactory.create(DefaultChannelFactory.java:40)] Creating instance of channel c1 type memory
- 2014-04-02 02:08:42,451 (conf-file-poller-0) [INFO - org.apache.flume.node.AbstractConfigurationProvider.loadChannels(AbstractConfigurationProvider.java:205)] Created channel c1
- 2014-04-02 02:08:42,454 (conf-file-poller-0) [INFO - org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:39)] Creating instance of source r1, type netcat
- 2014-04-02 02:08:42,490 (conf-file-poller-0) [INFO - org.apache.flume.sink.DefaultSinkFactory.create(DefaultSinkFactory.java:40)] Creating instance of sink: k1, type: logger
- 2014-04-02 02:08:42,505 (conf-file-poller-0) [INFO - org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:119)] Channel c1 connected to [r1, k1]
- 2014-04-02 02:08:42,531 (conf-file-poller-0) [INFO - org.apache.flume.node.Application.startAllComponents(Application.java:138)] Starting new configuration:{ sourceRunners:{r1=EventDrivenSourceRunner: { source:org.apache.flume.source.NetcatSource{name:r1,state:IDLE} }} sinkRunners:{k1=SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@380e28b9 counterGroup:{ name:null counters:{} } }} channels:{c1=org.apache.flume.channel.MemoryChannel{name: c1}} }
- 2014-04-02 02:08:42,570 (conf-file-poller-0) [INFO - org.apache.flume.node.Application.startAllComponents(Application.java:145)] Starting Channel c1
- 2014-04-02 02:08:42,650 (lifecycleSupervisor-1-1) [INFO - org.apache.flume.instrumentation.MonitoredCounterGroup.register(MonitoredCounterGroup.java:110)] Monitoried counter group for type: CHANNEL, name: c1, registered successfully.
- 2014-04-02 02:08:42,650 (lifecycleSupervisor-1-1) [INFO - org.apache.flume.instrumentation.MonitoredCounterGroup.start(MonitoredCounterGroup.java:94)] Component type: CHANNEL, name: c1 started
- 2014-04-02 02:08:42,653 (conf-file-poller-0) [INFO - org.apache.flume.node.Application.startAllComponents(Application.java:173)] Starting Sink k1
- 2014-04-02 02:08:42,656 (lifecycleSupervisor-1-1) [INFO - org.apache.flume.source.NetcatSource.start(NetcatSource.java:150)] Source starting
- 2014-04-02 02:08:42,684 (lifecycleSupervisor-1-1) [INFO - org.apache.flume.source.NetcatSource.start(NetcatSource.java:164)] Created serverSocket:sun.nio.ch.ServerSocketChannelImpl[/127.0.0.1:5858]
再打开一个终端,输入“flume success!”:
- lz@ubuntu:~$ telnet localhost 5858
- Trying 127.0.0.1...
- Connected to localhost.
- Escape character is '^]'.
- flume success!
- OK
退出telnet: 输入ctrl+] 然后 q
- 2014-04-02 02:10:56,707 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:70)] Event: { headers:{} body: 66 6C 75 6D 65 20 73 75 63 63 65 73 73 21 0D flume success!. }
0 0
- Flume-安装Flume
- flume 安装
- flume安装
- flume 安装
- flume 安装
- Flume安装
- flume安装
- flume安装
- Flume安装
- flume 安装
- Flume安装
- flume安装
- Flume安装
- flume安装
- Flume 安装
- flume安装及配置
- flume安装及配置
- flume安装配置
- ajax post data 获取不到数据,注意content-type的设置post/get
- 关于文件在内存中读写的问题
- DELPHI把一个字符串中的某个子串,用另一个子串去替换
- Task Affinity
- 几种分布式调用技术的比较 -- RPC VS REST
- flume 安装
- POJ3208 Apocalypse Someday Solution
- 【悠然】对虚拟目录 虚拟主机的理解 2014.08.19 20;40;20
- Delphi更新本地时间
- AutoCompleteTextView中的弹出框实现模糊查询
- Fragment上下文菜单示例
- FreeMarker
- 模板方法设计模式
- 面向对象