ELK centos7

原著文章地址：https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-elk-stack-on-centos-7

转载文章2：http://blog.sina.com.cn/s/blog_6f2d2e310102wa41.html

ELK官方地址：https://www.elastic.co/products

官方的下载源在国外比较缓慢本人下载好的安装包：http://pan.baidu.com/s/1o7EIZv8 密码：5zme  最新版本5.1系列

@@@@@@@@@@@@@@@@@@@@@首先安装elasticsearch //@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 

groupadd elk && useradd -g elk elk  // elasticsearch 是不能直接用root用户启动新加elk用户启动elasticsearchchown -R elk.elk  /opt/elasticsearch  // "/opt/elasticsearch " 是我elasticsearch的安装目录（安装包请在百度云下载或者在官方下载）

修改conf下的配置文件elasticsearch.yml   使只能本机访问 network.host: localhost

nohup sh bin/elasticsearch & //后台运行 且终端关闭后仍然运行

2017-01-05T02:28:28,702][INFO ][o.e.n.Node               ] version[5.1.1], pid[13420], build[5395e21/2016-12-06T12:36:15.409Z], OS[Linux/3.10.0-229.el7.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_92/25.92-b14]

[2017-01-05T02:28:29,756][INFO ][o.e.p.PluginsService     ] [ZYem2PN] loaded module [aggs-matrix-stats]
[2017-01-05T02:28:29,756][INFO ][o.e.p.PluginsService     ] [ZYem2PN] loaded module [ingest-common]
[2017-01-05T02:28:29,756][INFO ][o.e.p.PluginsService     ] [ZYem2PN] loaded module [lang-expression]
[2017-01-05T02:28:29,757][INFO ][o.e.p.PluginsService     ] [ZYem2PN] loaded module [lang-groovy]
[2017-01-05T02:28:29,757][INFO ][o.e.p.PluginsService     ] [ZYem2PN] loaded module [lang-mustache]
[2017-01-05T02:28:29,757][INFO ][o.e.p.PluginsService     ] [ZYem2PN] loaded module [lang-painless]
[2017-01-05T02:28:29,757][INFO ][o.e.p.PluginsService     ] [ZYem2PN] loaded module [percolator]

curl下本机看下返回结果：

curl 127.0.0.1:9200
结果：

{  "name" : "ZYem2PN",  "cluster_name" : "elasticsearch",  "cluster_uuid" : "Kpt3lcQDRl-7rq8oQEGZ6Q",  "version" : {    "number" : "5.1.1",    "build_hash" : "5395e21",    "build_date" : "2016-12-06T12:36:15.409Z",    "build_snapshot" : false,    "lucene_version" : "6.3.0"  },  "tagline" : "You Know, for Search"}

&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&安装kibana&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&

sudo rpm -ivh kibana-5.1.1-x86_64.rpm  
rpm -qc kibana //查看kibbaba的配置文件
返回结果：/etc/kibana/kibana.yml

更改配置文件：server.host: "localhost"

systemctl enable kibana.service  //开机启动kibana systemctl start kibana.service   //启动kibana

%%%%%%%%%%%%%%%%%%%%%安装nginx代理本机kibbaba，elasticsearch ，并增加认证%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

将上面提供下载的epel源添加到/etc/yum.repos.d/  更新yum源：

yum makecache fastyum install nginx httpd-tools -y  //yum安装nginx和认证工具：htpasswd -c /etc/nginx/htpasswd.users admin  //生成密码认证

vim /etc/nginx/conf.d/kibana.conf //创建server 用nginx代理本地访问 记得把nginx.conf 的server注释掉

 

         server {listen 80;server_name localhost;auth_basic "Restricted Access";auth_basic_user_file /etc/nginx/htpasswd.users;            location / {proxy_pass http://localhost:5601;proxy_http_version 1.1;proxy_set_header Upgrade $http_upgrade;proxy_set_header Connection 'upgrade';proxy_set_header Host $host;proxy_cache_bypass $http_upgrade;   }}

nginx -t 检查nginx语法 没有报错就可以启动nginx          systemctl start nginx && systemctl enable nginx 

同理可以代理elasticsearch服务的9200端口 // 然后就可以通过web访问kibana

*****************************************************安装logstash*****************************

ln -s /opt/jdk1.8.0_92/bin/java /usr/bin/java  //增加java的软连接 不然安装要报错 我这里jdk的路径是/opt/jdk1.8.0_92/

[root@localhost elk]# rpm -ivh logstash-5.1.1.rpm 

Preparing...                          ################################# [100%]
Updating / installing...
   1:logstash-1:5.1.1-1               ################################# [100%]
Using provided startup.options file: /etc/logstash/startup.options
Successfully created system startup script for Logstash

Logstash配置文件使用JSON格式，路径为 /etc/logstash/conf.d/

包含 inputs | filters | outputs 三部分

# vim /etc/logstash/conf.d/02-filebeat-input.conf

input {  beats {    port => 5044    type => "logs"    ssl => true    ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"    ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"  }}

# vim /etc/logstash/conf.d/10-syslog.conf

filter {  if [type] == "syslog" {    grok {      match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }      add_field => [ "received_at", "%{@timestamp}" ]      add_field => [ "received_from", "%{host}" ]    }    syslog_pri { }    date {      match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]    }  }}

将日志存储到Elasticsearch [ 跑在本机9200端口 ]

# vim /etc/logstash/conf.d/30-elasticsearch-output.conf

output {    elasticsearch { hosts => ["localhost:9200"] }    stdout { codec => rubydebug }}

chmod 777 /var/log/logstash/logstash.logsystemctl start logstashsystemctl enable logstash

！！！！！！！！！！！！！！！！！！！！！！创建通信证书！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！

请先修改/etc/hosts 文件增加服务器和客服端的解析

cd /etc/pki/tls  openssl req -subj '/CN=yoursername/' -x509 -days 3650 -batch -nodes -newkey rsa:2048  -keyout private/logstash-forwarder.key -out certs/logstash-forwarder.crtscp /etc/pki/tls/certs/logstash-forwarder.crt root@youagent:/tmp/    //将证书拷贝到你的agent(agent需要配置证书通信)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^客户端配置^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

rpm --import http://packages.elastic.co/GPG-KEY-elasticsearch  //导入证书rpm -ivh filebeat-5.1.1-x86_64.rpm     //安装filebeatcp /tmp/logstash-forwarder.crt /etc/pki/tls/certs/     //复制server创建的证书

grpe -v“^#” /etc/filebeat/filebeat.yml   

filebeat.prospectors:- input_type: log  paths:    - /var/log/*.logoutput.logstash:  hosts: ["elkserver:5044"]  ssl.certificate_authorities: ["/etc/pki/tls/certs/logstash-forwarder.crt"]

systemctl start filebeat.service systemctl enable filebeat.servicefilebeat.sh -e -c filebeat.yml -d "Publish"  查看是否能够通向server 请根据日志进行排查

如果不能通信server 

@@@@@@@@@@@@@@@@@@@@@@@@@配置kibana信息@@@@@@@@@@@@@@@@@@@@@@@@@