nginx和apache的LVS负载均衡

Varnish与Apache

安装静态缓存模块的机器IP如下： 
192.168.85.91 
192.168.85.96 

1. 安装repo源及软件 
# rpm --nosignature -i http://repo.varnish-cache.org/RedHat/varnish-3.0/el6/noarch/varnish-\ 
release-3.0-1.el6.noarch.rpm 
# yum install varnish 
# yum install httpd 

2. 修改配置文件 
(1) Apache的默认服务端口更改为8080

…
#Listen 12.34.56.78:80
Listen 8080
…

启动Apache服务： 
# service httpd start

 

(2) Varnish需要配置好后端服务器Apache的位置，以便找到需要缓存加速的文件。 
# vim /etc/varnish/default.vcl

backend default {
.host = "127.0.0.1";
.port = "8080";
}
3. 启动Varnish 
# varnishd -f /etc/varnish/default.vcl -s malloc,1G -T 127.0.0.1:2000 -a 0.0.0.0:80 
其中，-f指定的是Varnish配置文件位置，-s指定分配的缓存空间大小，-T为Varnish管理接口的监听地址，-a指定对外接收HTTP请求的监听地址。 

 

2.3 Nginx反向代理

安装反向代理模块的机器IP如下： 
192.168.85.90 
192.168.85.95 

1. 安装repo源及软件 
# rpm -Uvh http://dl.Fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm 
# yum install nginx 

2. 修改配置文件 
主配置文件/etc/nginx/nginx.conf无需做太大改动，只需将worker_processes设置成与机器CPU核数相等即可（如CPU数为1，则worker_processes 1;），以zone A为例： 
# vim /etc/nginx/conf.d/virtual.conf

upstream backend {
server 192.168.85.90;
}
server {
listen 80;
server_name 192.168.85.91;
# 静态缓存
location / {
proxy_pass http://backend;
}
# 动态请求
location ~ \.(php)?$ {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http:// 192.168.85.93;
}
}
3. 启动nginx服务 
# service nginx start 

 

2.4 LVS负载均衡

安装负载均衡模块的机器IP如下： 
192.168.85.80 
192.168.85.81 
LVS有四种常见的工作模式：NAT、DR、TUNNEL、FNAT。本文选用的工作模式为DR（Direct Route），负载均衡器接收用户的请求，然后根据负载均衡算法选取一台realserver，将包转发过去，最后由realserver直接回复给用户。 

1. 下载所需软件 
#cd /opt/ 
# wget http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.24.tar.gz 
# wget http://www.keepalived.org/software/keepalived-1.1.18.tar.gz 
# yum install -y openssl openssl-devel* popt* 

2. 安装ipvsadm（master和backup都需要安装） 
首先，建立内核快捷方式 
# ln -s /usr/src/kernels/2.6.32-358.14.1.el6.x86_64/ /usr/src/linux 
如果没有找到相关路径或文件，则需要手动安装内核 
# yum -y install kernel-devel 
然后，解压并编译ipvsadm 
# cd /opt/ 
# tar zxvf ipvsadm-1.24.tar.gz 
# cd ipvsadm-1.24 
# make 
# make install 
接下来，测试ipvsadm是否安装成功 
# /sbin/ipvsadm –v 
ipvsadm v1.24 2005/12/10 (compiled with getopt_long and IPVS v1.2.1) 
显示上述信息则表示安装成功。 

3. 安装keepalived（master和backup都需要安装） 
# cd /opt/ 
# tar zxvf keepalived-1.1.18.tar.gz 
# cd keepalived-1.1.18 
# ./configure --prefix=/usr/local/keepalived 
# make 
# make install 
# cd /usr/local/keepalived/ 
# cp etc/rc.d/init.d/keepalived /etc/rc.d/init.d/ 
# cp etc/sysconfig/keepalived /etc/sysconfig/ 
# mkdir /etc/keepalived 
# cp etc/keepalived/keepalived.conf /etc/keepalived/ 
# cp sbin/keepalived /usr/sbin/ 
启动keepalived 
# service keepalived start 

4. 修改master和backup的配置文件 
首先，开启ip_forward转发功能 
# sysctl -w net.ipv4.ip_forward=1 
然后，配置keepalived.conf文件 
# vim /etc/keepalived/keepalived.conf

! Configuration File for keepalived

global_defs {
notification_email {
[e-mail]
[e-mail]
}
notification_email_from [e-mail]
smtp_server 192.168.200.1
smtp_connect_timeout 30
router_id LVS_DEVEL
}

vrrp_instance VI_1 {
state MASTER # 备份服务器上将MASTER改为BACKUP
interface eth0 # HA监测网络接口
virtual_router_id 51 # 主、备机的virtual_router_id必须相同
priority 30 # 主、备机取不同的优先级，主机值较大，备份机值较小
advert_int 1 # VRRP Multicast广播周期秒数
authentication {
auth_type PASS # VRRP认证方式
auth_pass 1111 # VRRP口令字
}
virtual_ipaddress {
192.168.85.10 # 如果有多个VIP，继续换行填写
}
}

virtual_server 192.168.85.10 80 {
delay_loop 2 # 每隔2秒查询realserver状态
lb_algo rr # lvs 算法
lb_kind DR # Direct Route

persistence_timeout 50 # 同一IP的连接50秒内被分配到同一台realserver
protocol TCP # 用TCP协议检查realserver状态


real_server 192.168.85.90 80 {
weight 80
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}

real_server 192.168.85.95 80 {
weight 20
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}

BACKUP服务器同上配置，先安装lvs再按装keepalived,仍后配置/etc/keepalived/keepalived.conf，只需将红色标示的部分改一下即可。 5. 配置VIP 
# vim /etc/sysconfig/network-scripts/ifcfg-lo:0

DEVICE=lo:0
IPADDR=192.168.85.10
NETMASK=255.255.255.255
ONBOOT=yes
NAME=loopback

重启网卡和keepalived服务即可。

6. 配置ipvsad m（master和backup都需要设置好） 
添加负载均衡器 
# /sbin/ipvsadm -A -t 192.168.85.10:80 -s rr 
增加realserver 
# /sbin/ipvsadm -a -t 192.168.85.10:80 -r 192.168.85.90:80 -g # 192.168.85.90是真实机的ip 
# /sbin/ipvsadm -a -t 192.168.85.10:80 -r 192.168.85.95:80 -g # 192.168.85.95是真实机的ip 
# route add -host 192.168.85.10 dev lo:0 

7．配置realserver（所有的Proxy Server都要设置好） 
配置vip地址 
# /sbin/ifconfig lo:0 192.168.85.10 netmask 255.255.255.255 broadcast 192.168.85.10 up 
# route add -host 192.168.85.10 dev lo:0