RedHat6.5安装单机flume1.6
来源:互联网 发布:淘宝好看的男鞋店铺 编辑:程序博客网 时间:2024/06/05 09:38
版本号:
RedHat6.5 JDK1.8 apache-flume-1.6.0
1.apache-flume-1.6.0-bin.tar.gz 下载
官网下载地址:http://archive.apache.org/dist/flume/1.6.0/apache-flume-1.6.0-bin.tar.gz
百度云盘下载地址:链接:http://pan.baidu.com/s/1pKPah0f 密码:ovbe
2.安装
新建一个/usr/local/flume目录,上传apache-flume-1.6.0-bin.tar.gz,解压:
mkdir /usr/local/flume
tar -xvf /usr/local/flume/apache-flume-1.6.0-bin.tar.gz
如图:
3.配置 /etc/profile
sudo gedit /etc/profile
- #set flume environment
- export FLUME_HOME=/usr/local/flume/apache-flume-1.6.0-bin
- export FLUME_CONF_DIR=$FLUME_HOME/conf
- export PATH=$FLUME_HOME/bin:$PATH
source /etc/profile
4.配置flume JAVA_HOME
把flume-env.sh.template重命名为flume-env.sh,往里添加
cd /usr/local/flume/apache-flume-1.6.0-bin/conf
cp flume-env.sh.template flume-env.sh
sudo gedit flume-env.sh
- export JAVA_HOME=/usr/local/java/jdk1.8
检验
/usr/local/flume/apache-flume-1.6.0-bin/bin/flume-ng version
- [root@master conf]# /usr/local/flume/apache-flume-1.6.0-bin/bin/flume-ng version
- Flume 1.6.0
- Source code repository: https://git-wip-us.apache.org/repos/asf/flume.git
- Revision: 2561a23240a71ba20bf288c7c2cda88f443c2080
- Compiled by hshreedharan on Mon May 11 11:15:44 PDT 2015
- From source with checksum b29e416802ce9ece3269d34233baf43f
- [root@master conf]#
5. 简单测试示例
5.1 添加配置文件
- touch /usr/local/flume/apache-flume-1.6.0-bin/conf/flume.conf
- sudo gedit /usr/local/flume/apache-flume-1.6.0-bin/conf/flume.conf
flume.conf输入以下内容:
- # 指定Agent的组件名称
- agent1.sources = source1
- agent1.sinks = sink1
- agent1.channels = channel1
- # 指定Flume source(要监听的路径)
- agent1.sources.source1.type = spooldir
- agent1.sources.source1.spoolDir= /usr/local/flume/logtest
- # 指定Flume sink
- agent1.sinks.sink1.type = logger
- # 指定Flume channel
- agent1.channels.channel1.type = memory
- agent1.channels.channel1.capacity = 1000
- agent1.channels.channel1.transactionCapacity = 100
- # 绑定source和sink到channel上
- agent1.sources.source1.channels = channel1
- agent1.sinks.sink1.channel = channel1
在/usr/local/flume目录下新建一个logtest目录:
mkdir /usr/local/flume/logtest
5.2 启动flume agent
- cd /usr/local/flume/apache-flume-1.6.0-bin
- bin/flume-ng agent --conf conf --conf-file conf/flume.conf --name agent1 -Dflume.root.logger=INFO,console
参数作用举例
–conf 或 -c 指定配置文件夹,包含flume-env.sh和log4j的配置文件–conf conf
–conf-file 或 -f 配置文件地址–conf-file conf/flume.conf
–name 或 -n agent名称 –name agent1
-z zookeeper连接字符串-z zkhost:2181,zkhost1:2181
-p zookeeper中的存储路径前缀-p /flume
5.3 写入日志内容
在/usr/local/flume目录下面新建一个testFlume.log日志文件,写入Hello Flume.作为测试内容:
- touch /usr/local/flume/testFlume.log
- sudo gedit /usr/local/flume/testFlume.log
然后拷贝到flume监听路径/usr/local/flume/logtest下:
- cp /usr/local/flume/testFlume.log /usr/local/flume/logtest
接着就可以在前一个终端看到刚刚采集的内容了,如下:
- 2017-07-07 21:15:59,063 (pool-3-thread-1) [INFO - org.apache.flume.client.avro.ReliableSpoolingFileEventReader.rollCurrentFile(ReliableSpoolingFileEventReader.java:348)] Preparing to move file /usr/local/flume/logtest/testFlume.log to /usr/local/flume/logtest/testFlume.log.COMPLETED
- 2017-07-07 21:16:00,841 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:94)] Event: { headers:{} body: 48 65 6C 6C 6F 20 46 6C 75 6D 65 2E Hello Flume. }
Flume成功安装,666!
阅读全文
0 0
- RedHat6.5安装单机flume1.6
- 单机RedHat6.5+JDK1.8+Hadoop2.7.3+Spark2.1.1+zookeeper3.4.6+kafka2.11+flume1.6环境搭建步骤
- RedHat6.5安装kafka单机
- RedHat6.5安装Spark单机
- RedHat6.5安装zookeeper单机
- RedHat6.5上安装Hadoop单机
- 安装flume1.5
- hello flume (Ubuntu 下 flume1.5单机版安装以及简单入门示例)
- centos(vm)下单机安装flume1.6.0(x86)
- 集群RedHat6.5+JDK1.8+Hadoop2.7.3+Spark2.1.1+zookeeper3.4.6+kafka2.11+flume1.6环境搭建步骤
- flume1.5部署安装deploy
- Redhat6.5 安装MySQL5.6集群
- Redhat6.5安装MySQL5.6 RPM包
- redhat6.5 安装 yum
- Redhat6.5安装Docker
- redhat6.5安装oracle11g
- RedHat6.5安装Oracle
- CentOS6.5下安装Flume1.5
- 笔试题2/5如何查看进程的信息(线程数)?
- hdu 2546 饭卡 动态规划 01背包
- 透过《我的前半生》悟出职场规则
- PAT乙级 1024. 科学计数法 (20)--字符串的分割和移位
- Qt 学习之路 2
- RedHat6.5安装单机flume1.6
- 阿里云linux服务器登录失败,Connection closed
- 在发送ajax请求时加时间戳或者随机数去除js缓存
- 详解在android中使用通知
- 记一次断电引起的mongodb彻底奔溃
- Opencv 创建图像时,CV_8UC1,CV_32FC3等参数的含义
- xcxcxxcx
- 全栈工程师学习
- 【HDU】6043