使用AWStats自动分析Nginx日志

环境说明：

Nginx安装目录：/usr/local/nginx

Nginx配置文件存放目录：/usr/local/nginx/conf/nginx.conf

Nginx日志目录：/usr/local/nginx/logs

Awstats安装目录：/usr/local/awstats

Awstats配置文件存放目录：/etc/awstats

日志切割脚本存放目录：/usr/local/nginx/sbin/cut_nginx_log.sh

一、Nginx日志切割

1、修改nginx.conf配置文件，使AWSTATS支持分析日志格式

# vim /usr/local/nginx/conf/nginx.conf

log_format main '$remote_addr - $remote_user [$time_local] "$request" '

'$status $body_bytes_sent "$http_referer" '

'"$http_user_agent" "$http_x_forwarded_for"';

access_log logs/access.log main;

 

2、编辑Nginx日志切割脚本

# vim /usr/local/nginx/sbin/cut_nginx_log.sh

#!/bin/bash

mv /usr/local/nginx/logs/access.log /usr/local/nginx/logs/access_`date +%Y%m%d`.log

mv /usr/local/nginx/logs/error.log /usr/local/nginx/logs/error_`date +%Y%m%d`.log

killall -s USER1 nginx

 

# chmod +x /usr/local/nginx/sbin/cut_nginx_log.sh

3、

二、下载安装Awstats

1、下载安装

# wget http://awstats.sourceforge.net/files/awstats-7.0.tar.gz

# tar -zxvf awstats-7.0.tar.gz

# mv awstats-7.0 /usr/local/awstats

# mkdir -p /var/lib/awstats

 

2、配置

# mkdir - p /tmp/src/wwwroot/cgi-bin/

# cp

/usr/local/awstats/wwwroot/cgi-bin/awstats.model.conf /tmp/src/wwwroot/cgi-bin/

# /usr/local/awstats/tools/awstats_configure.pl

-----> Running OS detected: Linux, BSD or Unix

Warning: AWStats standard directory on Linux OS is '/usr/local/awstats'.

If you want to use standard directory, you should first move all content

of AWStats distribution from current directory:

/tmp/src

to standard directory:

/usr/local/awstats

And then, run configure.pl from this location.

Do you want to continue setup from this NON standard directory [yN] ?输入y

 

回车

-----> Check for web server install

Enter full config file path of your Web server.

Example: /etc/httpd/httpd.conf

Example: /usr/local/apache2/conf/httpd.conf

Example: c:\Program files\apache group\apache\conf\httpd.conf

Config file path ('none' to skip web server setup):

> none 这里不是apache，写none，跳过#

 

回车

-----> Update model config file '/tmp/src/wwwroot/cgi-bin/awstats.model.conf'

File awstats.model.conf updated.

-----> Need to create a new config file ?

Do you want me to build a new AWStats config/profile

file (required if first install) [y/N] ?输入 y #创建一个全新的统计

 

回车

-----> Define config file name to create

What is the name of your web site or profile analysis ?

Example: www.mysite.com

Example: demo

Your web site, virtual server or profile name:

> 192.168.2.143被统计网站的域名 #

 

回车

-----> Define config file path

In which directory do you plan to store your config file(s) ?

Default: /etc/awstats

Directory path to store config file(s) (Enter for default):

> #回车继续

-----> Create config file '/etc/awstats/awstats.192.168.2.143.conf'

Config file /etc/awstats/awstats.192.168.2.143.conf created.

-----> Add update process inside a scheduler

Sorry, configure.pl does not support automatic add to cron yet.

You can do it manually by adding the following command to your cron:

/tmp/src/wwwroot/cgi-bin/awstats.pl -update -config=192.168.2.143

Or if you have several config files and prefer having only one command:

/tmp/src/tools/awstats_updateall.pl now

Press ENTER to continue...

A SIMPLE config file has been created: /etc/awstats/awstats.192.168.2.143.conf

You should have a look inside to check and change manually main parameters.

You can then manually update your statistics for '192.168.2.143' with command:

> perl awstats.pl -update -config=192.168.2.143

You can also build static report pages for '192.168.2.143' with command:

> perl awstats.pl -output=pagetype -config=192.168.2.143

Press ENTER to finish...

三、修改统计日志文件的路径

# vim /etc/awstats/awstats.192.168.2.143.conf -

LogFile="/var/log/httpd/mylog.log"修改成--->

LogFile="/usr/local/nginx/logs/access_%YYYY-0%MM-0%DD-0.log"

 

这里是对应上面Nginx日志切割所生成的目录存放位置，注意awstats的年月日格式，分析的执行顺序是：

Nginx 产生日志 –> 日志切割 –> Nginx 继续产生日志 –> 另存切割日志 –> 交由Awstats统计 –> 生成结果

四、执行Awstats 日志更新程序开始统计分析

#/usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=192.168.2.143

Create/Update database for config "/etc/awstats/awstats.192.168.2.143.conf" by AWStats version 7.0 (build 1.971)

From data in log file "/usr/local/nginx/logs/access_20121207.log"...

Phase 1 : First bypass old records, searching new record...

Searching new records from beginning of log file...

Phase 2 : Now process new records (Flush history on disk after 20000 hosts)...

Jumped lines in file: 0

Parsed lines in file: 66466

Found 0 dropped records,

Found 0 comments,

Found 1 blank records,

Found 58472 corrupted records,

Found 0 old records,

Found 7993 new qualified records.

 

看到以上显示，证明日志切割和 Awstats 都已经运行无误了。统计分析完成后，结果还在 Awstats 的数据库中。在 Apache 上，可以直接打开 Perl 程序的网页查看统计。但本文开始时已经提到，Nginx 对 Perl 支持并不好，所以我们要换个方法，利用 awstats 的工具将统计的结果生成静态文件。

五、Awstats生成静态文件

# cp -rf /usr/local/awstats/wwwroot/ /usr/local/nginx/logs/

# mkdir /usr/local/nginx/logs/wwwroot/awstats

# /usr/local/awstats/tools/awstats_buildstaticpages.pl -update -config=192.168.2.143 -lang=cn -dir=/usr/local/nginx/logs/wwwroot/awstats/ -awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl

 

/usr/local/awstats/tools/awstats_buildstaticpages.pl #Awstats 静态页面生成脚本

-update -config=192.168.2.143 #更新配置选项

-lang=cn #语音中文

-dir=/usr/local/nginx/logs/wwwroot/awstats/ #统计结果输出目录

-awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl #日志更新程序路径

六、修改nginx配置文件

# vim /usr/local/nginx/conf/nginx.conf

server {

listen 8080;

server_name localhost;

location ~ ^/awstats/ { # html 静态页面目录

root /usr/local/nginx/logs/wwwroot/awstats;

index index.html;

access_log off;

error_log off;

charset gb2312; }

location ~ ^/icon/ { # 图标目录

root /usr/local/nginx/logs/wwwroot;

index index.html;

access_log off;

error_log off;

charset gb2312;

}

}

 

七、配置Awstats自动运行

# vim /usr/local/nginx/sbin/cut_nginx_log.sh

23 59 * * * root /usr/local/nginx/sbin/cut_nginx_log.sh

00 01 * * * /usr/local/awstats/tools/awstats_buildstaticpages.pl -update -config=192.168.2.143 -lang=cn -dir=/usr/local/nginx/logs/wwwroot/awstats/ -awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl

 

八、结果验证，URL输入http://192.168.2.143:8080/awstats/awstats.192.168.2.143.html