ELK中利用elastalert监控日志中的异常，发送邮件警告

1. 准备工作

 sudo yum update

python版本最好2.7版本以上（现在centos7好像都能满足）

2.安装elastalert（按照官网的操作走）

先切换到自己的工作目录
我这里是 /usr/local/dev/
git 克隆源码

 git clone https://github.com/Yelp/elastalert.git cd elastalert

安装

sudo python setup.py install

出错了
Running blist-1.3.6/setup.py -q bdist_egg –dist-dir /tmp/easy_install-Gc6gbe/blist-1.3.6/egg-dist-tmp-Ik7LL2
The required version of setuptools (>=1.1.6) is not available,
and can’t be installed while this script is running. Please
install a more recent version first, using
‘easy_install -U setuptools’.

(Currently using setuptools 0.9.8 (/usr/lib/python2.7/site-packages))

ok按照它提示的，，，执行：

sudo easy_install -U setuptools

再执行安装（sudo python setup.py install），又出错了

warning: no files found matching 'blist.rst'blist/_blist.c:38:20: 致命错误：Python.h：没有那个文件或目录 #include<Python.h>

这其实是缺少开发包，执行命令：

sudo yum install python-devel

再执行安装（sudo python setup.py install）
到这里基本能安装成功。
有时候会报一些包找不到的错误，可能是网络问题，重复执行安装命令即可（我在阿里云上遇到了）。

继续执行安装命令

pip install -r requirements.txt

找不到pip ，先安装pip并更新

sudo yum -y install python-pipsudo pip install --upgrade pip

安装完继续

sudo pip install -r requirements.txt

在elasticsearch中创建elastalert的日志索引

sudo elastalert-create-index

根据自己的情况，填入elasticsearch的相关信息，关于
elastalert_status部分直接回车默认的即可。
如下所示：

Enter elasticsearch host: log.example.comEnter elasticsearch port: 9200Use SSL? t/f: tEnter optional basic-auth username (or leave blank): es_adminEnter optional basic-auth password (or leave blank):Enter optional Elasticsearch URL prefix (prepends a string to the URL of every request):New index name? (Default elastalert_status)Name of existing index to copy? (Default None)/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html  InsecureRequestWarning)/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html  InsecureRequestWarning)/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html  InsecureRequestWarning)/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html  InsecureRequestWarning)/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html  InsecureRequestWarning)/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html  InsecureRequestWarning)/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html  InsecureRequestWarning)New index elastalert_status createdDone!

3.创建配置文件

sudo cp config.yaml.example config.yamlsudo vi config.yaml

根据自己的具体情况进行修改

比如我这里的elasticsearch是ssl 并需要用户登陆的，配置大致如下：

run_every:  minutes: 1buffer_time:  minutes: 15es_host: log.example.comes_port: 9200use_ssl: Truees_send_get_body_as: GETes_username: es_admines_password: es_passwordwriteback_index: elastalert_statusalert_time_limit:  days: 2

以同样的方式配置规则

cd example_rules/sudo cp example_frequency.yaml my_rule.yamlsudo vi my_rule.yaml

这里就是根据具体的elasticsearch的信息进行配置

es_host: log.example.comes_port: 9200use_ssl: Truees_username: es_admines_password: es_password#name属性要求唯一，这里最好能标示自己的产品name: My-Product Exception Alert#类型，我选择任何匹配的条件都发送邮件警告type: any#需要监控的索引，支持通配index: logstash-*#下面两个随意配置num_events: 50timeframe:  hours: 4#根据条件进行过滤查询（这里我只要出现异常的日志，并且排除业务异常（自定义异常））filter:- query:    query_string:      query: "message: *exception* AND message: (!*BusinessException*) AND message: (!*ServiceException*)"#email的警告方式alert:- "email"#增加邮件内容，这里我附加一个日志访问路径alert_text: "Ref Log https://log.example.com:5601/app/kibana"#SMTP协议的邮件服务器相关配置（我这里是腾讯企业邮箱）smtp_host: smtp.exmail.qq.comsmtp_port: 25#用户认证文件，需要user和password两个属性smtp_auth_file: smtp_auth_file.yamlemail_reply_to: no-reply@example.comfrom_addr: [e-mail] #需要接受邮件的邮箱地址列表email:- "[e-mail]"- "[e-mail]"

接下来创建smtp_auth_file.yaml

sudo touch smtp_auth_file.yamlsudo vi smtp_auth_file.yaml##配置文件内容user: "[e-mail]"password: "password"

现在可以简单测试一下配置文件是否正确

sudo elastalert-test-rule ./my_rule.yaml

如果有问题，就检查配置文件。
然后启动运行一下试试：

sudo python -m elastalert.elastalert --verbose --rule my_rule.yaml

然后故意产生一下异常试试，比如

if(true){    throw new NullPointerException("测试日志异常监控服务");}

执行一下，一般没啥问题，邮件应该就能收到了，有问题就检查配置文件。

4.我是要监控服务，所以要让elastalert以服务的形式运行
首先创建elastalert的配置目录和相关文件

sudo mkdir /etc/elastalertcd /etc/elastalert-- 复制配置文件sudo cp /usr/local/dev/elastalert/config.yaml config.yamlsudo mkdir rulescd rules-- 复制规则文件sudo cp /usr/local/dev/elastalert/example_rules/my_rule.yaml my_rule.yaml-- 复制邮件用户认证文件sudo cp /usr/local/dev/elastalert/example_rules/smtp_auth_file.yaml smtp_auth_file.yaml

接下来修改配置文件
修改 config.yaml 中

rules_folder: /etc/elastalert/rules

修改 my_rule.yaml中

smtp_auth_file: /etc/elastalert/rules/umu_smtp_auth_file.yaml

接下来就是创建systemd服务了

cd /etc/systemd/systemsudo touch elastalert.servicesudo vi elastalert.service

elastalert.service 内容

[Unit]Description=elastalertAfter=elasticsearch.service[Service]Type=simpleUser=rootGroup=rootRestart=on-failureWorkingDirectory=/usr/local/dev/elastalertExecStart=/usr/bin/elastalert --config /etc/elastalert/config.yaml --rule /etc/elastalert/rules/my_rule.yaml[Install]WantedBy=multi-user.target

保存退出。
启动服务

sudo systemctl start elastalert

查看状态

sudo systemctl status elastalert

不出意外的话，你将看到绿色的Active: active (running)

至此，日志的异常监控服务就搭建好了，好的，收工！