elasticsearch+filebeat日志分析

来源：互联网发布：python time sleep参数编辑：程序博客网时间：2024/05/16 12:24

起先使用logback+logstash将数据收录到日志仓库中，但是考虑到该方案会消耗java运行内存（如果日志数据量太大可能会影响系统的正常运行）

最后在官网找到了一个可以分析日志文件的日志收集器并收录到elasticsearch仓库中 (filebeat)

1:安装filebeat,有三种方式

deb:
curl -L -O https://download.elastic.co/beats/filebeat/filebeat_1.2.3_amd64.deb
sudo dpkg -i filebeat_1.2.3_amd64.deb

rpm:
curl -L -O https://download.elastic.co/beats/filebeat/filebeat-1.2.3-x86_64.rpm
sudo rpm -vi filebeat-1.2.3-x86_64.rpm

mac:
curl -L -O https://download.elastic.co/beats/filebeat/filebeat-1.2.3-darwin.tgz
tar xzvf filebeat-1.2.3-darwin.tgz

2:配置filebeat（撇弃logstash,直接将数据扔给es）

output:
elasticsearch:
hosts: ["localhost:9200"]
# A template is used to set the mapping in Elasticsearch
# By default template loading is disabled and no template is loaded.
# These settings can be adjusted to load your own template or overwrite existing ones
template:
# Template name. By default the template name is filebeat.
#name: "filebeat"
# Path to template file
path: "filebeat.template.json"
# Overwrite existing template
#overwrite: false

创建/加载模板

deb or rpm:
curl -XPUT 'http://localhost:9200/_template/filebeat' -d@/etc/filebeat/filebeat.template.json

mac:
cd filebeat-1.2.3-darwin
curl -XPUT 'http://localhost:9200/_template/filebeat' -d@filebeat.template.json

win:
PS C:\Program Files\Filebeat> Invoke-WebRequest -Method Put -InFile filebeat.template.json -Uri http://localhost:9200/_template/filebeat?pretty

启动filebeat

deb:
sudo /etc/init.d/filebeat start

rpm:
sudo /etc/init.d/filebeat start

mac:
sudo ./filebeat -e -c filebeat.yml -d "publish"

win:

PS C:\Program Files\Filebeat> Start-Service filebeat

完成,有点乱，将就看吧 elk详细搭建我博客里有,可以参照一下

curl -XDELETE http://192.168.161.92:9200/logstash-2016.07.* 删除es索引

0 0