mac 环境安装日志收集系统搭建(Fluent,ES,Kibana)

1.安装fluent：

https://www.fluentd.org/download

选择 Mac环境安装

安装成功后，测试一下

$ sudo launchctl load /Library/LaunchDaemons/td-agent.plist$ less /var/log/td-agent/td-agent.log

$ curl -X POST -d 'json={"json":"message"}' http://localhost:8888/debug.test$ tail -n 1 /var/log/td-agent/td-agent.log

2013-04-19 16:51:47 -0700 debug.test: {"json":"message"}

配置文件

suo vim /etc/td-agent/td-agent.conf 

＃数据源通过tcp协议

<source>

  @type forward

  port 24224

</source>

＃匹配到tag,将数据存储到elasticsearch里

<match myapp.*>

  @type elasticsearch

  host 127.0.0.1

  port 9200

  index fluent_log

  type fluent

  logstash_format true

  flush_interval 10s

</match>

下载Elasticsearch 地址:https://www.elastic.co/downloads/elasticsearch

cd elasticsearch-5.4.1

./bin/elasticsearch

测试一下

curl 127.0.0.1:9200

返回结果：

{

  "name" : "Ryb91qD",

  "cluster_name" : "elasticsearch",

  "cluster_uuid" : "G3UFMzGiTzmR1QUp4viR_A",

  "version" : {

    "number" : "5.4.1",

    "build_hash" : "2cfe0df",

    "build_date" : "2017-05-29T16:05:51.443Z",

    "build_snapshot" : false,

    "lucene_version" : "6.5.1"

  },

  "tagline" : "You Know, for Search"

}

修改es配置文件

添加

http.cors.enabled: true  #是否支持跨域，默认为falsehttp.cors.allow-origin: "*"   #当设置允许跨域，默认为*,表示支持所有域名

插件下载地址：https://github.com/mobz/elasticsearch-head

git clone git://github.com/mobz/elasticsearch-head.git

cd elasticsearch-head

npm install（电脑提前得安装node）

npm run start

open http://localhost:9100/

安装fluent 操作es的插件:/opt/td-agent/embedded/bin/gem install fluent-plugin-elasticsearch-1.0.0.gem

依赖包

• elasticsearch-1.0.12.gem
• elasticsearch-api-1.0.12.gem
• elasticsearch-transport-1.0.12.gem
• excon-0.45.4.gem
• faraday-0.9.1.gem
• fluent-plugin-elasticsearch-1.0.0.gem
• multi_json-1.11.2.gem
• multipart-post-2.0.0.gem

安装kibana

https://www.elastic.co/downloads/kibana

编辑kibana/config/kibana.yml，把elasticsearch.url注释打开

bin/kibana

curl 127.0.0.1:5601