KeepAlive+LVS 实现高可用负载均衡

环境介绍:

http 服务: node1(10.11.8.222), node2(10.11.8.158)
KeepAlive 服务: keep1(10.11.8.219), keep2(10.11.8.216)

keep1 & keep2 # yum -y install ipvsadm keepalived

配置 real_server(node1, node2) :

编辑/etc/sysctl.conf 添加以下记录

net.ipv4.ip_forward = 1net.ipv4.conf.eth0.arp_ignore = 1net.ipv4.conf.eth0.arp_announce = 2net.ipv4.conf.all.arp_ignore = 1net.ipv4.conf.all.arp_announce = 2

重读配置并添加 lo:0 别名:

root@node1:~# sysctl -pnet.ipv4.ip_forward = 1net.ipv4.conf.eth0.arp_ignore = 1net.ipv4.conf.eth0.arp_announce = 2net.ipv4.conf.all.arp_ignore = 1net.ipv4.conf.all.arp_announce = 2root@node1:~# ifconfig lo:0 10.11.8.100root@node2:~# sysctl -pnet.ipv4.ip_forward = 1net.ipv4.conf.eth0.arp_ignore = 1net.ipv4.conf.eth0.arp_announce = 2net.ipv4.conf.all.arp_ignore = 1net.ipv4.conf.all.arp_announce = 2root@node2:~# ifconfig lo:0 10.11.8.100

给http服务添加测试页面并启动服务:

root@node1:~# echo "real1" > /var/www/html/index.htmlroot@node1:~# service apache2 startroot@node2:~# echo "real2" > /var/www/html/index.htmlroot@node2:~# service apache2 start

配置 KeepAlive :

MASTER:

! Configuration File for keepalivedglobal_defs {   notification_email {    root@localhost #通知email的地址   }   notification_email_from keepalived@localhost #发件人email地址   smtp_server 127.0.0.1   smtp_connect_timeout 30   router_id LVS_DEVEL}vrrp_instance VI_1 {    state MASTER #MASTER,BACKUP节点的不同配置用红色标记    interface eth0    virtual_router_id 51    priority 101 #优先级值, MASTER节点的值要高于BACKUP    advert_int 1    authentication { #认证配置        auth_type PASS        auth_pass shiina    }    virtual_ipaddress {    10.11.8.100/16 dev eth0 label eth0:0 #虚拟IP地址    }}virtual_server 10.11.8.100 80 {    delay_loop 6    lb_algo rr #lb使用的调度算法    lb_kind DR #lb所使用的模型    nat_mask 255.255.0.0    #persistence_timeout 50    protocol TCP    sorry_server 127.0.0.1 80 #当所有real_server宕机后的错误提示服务器    real_server 10.11.8.222 80 { #node1节点        weight 1        HTTP_GET { #验证real_server方式            url {              path /              status_code 200            }            connect_timeout 3 #超时时间            nb_get_retry 3 #重试次数            delay_before_retry 3 #重试前的等待时间        }    }    real_server 10.11.8.158 80 { #node2节点        weight 1        HTTP_GET {            url {              path /              status_code 200            }            connect_timeout 3            nb_get_retry 3            delay_before_retry 3        }    }}

BACKUP:

! Configuration File for keepalivedglobal_defs {   notification_email {    root@localhost   }   notification_email_from keepalived@localhost   smtp_server 127.0.0.1   smtp_connect_timeout 30   router_id LVS_DEVEL}vrrp_instance VI_1 {    state BACKUP    interface eth0    virtual_router_id 51    priority 100    advert_int 1    authentication {        auth_type PASS        auth_pass shiina    }    virtual_ipaddress {    10.11.8.100/16 dev eth0 label eth0:0    }}virtual_server 10.11.8.100 80 {    delay_loop 6    lb_algo rr    lb_kind DR    nat_mask 255.255.0.0    #persistence_timeout 50    protocol TCP    sorry_server 127.0.0.1 80    real_server 10.11.8.222 80 {        weight 1        HTTP_GET {            url {              path /              status_code 200            }            connect_timeout 3            nb_get_retry 3            delay_before_retry 3        }    }    real_server 10.11.8.158 80 {        weight 1        HTTP_GET {            url {              path /              status_code 200            }            connect_timeout 3            nb_get_retry 3            delay_before_retry 3        }    }}

启动服务, 查看状态:

[root@keep1 ~]# service keepalived start Starting keepalived:                                       [  OK  ][root@keep2 ~]# service keepalived start Starting keepalived:                                       [  OK  ][root@keep1 ~]# tail /var/log/messages #查看日志Jun  1 06:22:37 keep1 kernel: IPVS: [rr] scheduler registered.Jun  1 06:22:37 keep1 Keepalived_vrrp[1600]: VRRP_Instance(VI_1) Transition to MASTER STATEJun  1 06:22:37 keep1 Keepalived_vrrp[1600]: VRRP_Instance(VI_1) Received lower prio advert, forcing new electionJun  1 06:22:37 keep1 Keepalived_vrrp[1600]: VRRP_Instance(VI_1) Received lower prio advert, forcing new electionJun  1 06:22:37 keep1 Keepalived_vrrp[1600]: VRRP_Instance(VI_1) Received lower prio advert, forcing new electionJun  1 06:22:38 keep1 Keepalived_vrrp[1600]: VRRP_Instance(VI_1) Entering MASTER STATEJun  1 06:22:38 keep1 Keepalived_vrrp[1600]: VRRP_Instance(VI_1) setting protocol VIPs.Jun  1 06:22:38 keep1 Keepalived_vrrp[1600]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 10.11.8.100Jun  1 06:22:38 keep1 Keepalived_healthcheckers[1599]: Netlink reflector reports IP 10.11.8.100 addedJun  1 06:22:43 keep1 Keepalived_vrrp[1600]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 10.11.8.100

查看运行状态:

[root@keep1 keepalived]# ifconfig eth0      Link encap:Ethernet  HWaddr 08:00:27:4F:81:C9            inet addr:10.11.8.219  Bcast:10.11.9.255  Mask:255.255.254.0          inet6 addr: fe80::a00:27ff:fe4f:81c9/64 Scope:Link          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1          RX packets:5025 errors:0 dropped:0 overruns:0 frame:0          TX packets:2800 errors:0 dropped:0 overruns:0 carrier:0          collisions:0 txqueuelen:1000           RX bytes:389646 (380.5 KiB)  TX bytes:239786 (234.1 KiB)eth0:0    Link encap:Ethernet  HWaddr 08:00:27:4F:81:C9            inet addr:10.11.8.100  Bcast:0.0.0.0  Mask:255.255.0.0          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1lo        Link encap:Local Loopback            inet addr:127.0.0.1  Mask:255.0.0.0          inet6 addr: ::1/128 Scope:Host          UP LOOPBACK RUNNING  MTU:65536  Metric:1          RX packets:261 errors:0 dropped:0 overruns:0 frame:0          TX packets:261 errors:0 dropped:0 overruns:0 carrier:0          collisions:0 txqueuelen:0           RX bytes:21926 (21.4 KiB)  TX bytes:21926 (21.4 KiB)[root@keep1 keepalived]# ipvsadm -LnIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags  -> RemoteAddress:Port           Forward Weight ActiveConn InActConnTCP  10.11.8.100:80 rr  -> 10.11.8.158:80               Route   1      1          0           -> 10.11.8.222:80               Route   1      0          0

测试服务:

[root@keep1 keepalived]# ipvsadm -LncIPVS connection entriespro expire state       source             virtual            destinationTCP 14:52  ESTABLISHED 10.11.8.138:58173  10.11.8.100:80     10.11.8.158:80root@node2:~# service apache2 stop * Stopping web server apache2                                                                      * [root@keep1 keepalived]# ipvsadm -LnIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags  -> RemoteAddress:Port           Forward Weight ActiveConn InActConnTCP  10.11.8.100:80 rr  -> 10.11.8.222:80               Route   1      0          0root@node1:~# service apache2 stop * Stopping web server apache2                                                                      * [root@keep1 keepalived]# ipvsadm -LnIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags  -> RemoteAddress:Port           Forward Weight ActiveConn InActConnTCP  10.11.8.100:80 rr  -> 127.0.0.1:80                 Local   1      0          0  [root@keep1 keepalived]# curl 10.11.8.100error this is keep1

此时所有real_server 全部停止, 启动了sorry_server

自写监测脚本，完成维护模式切换

编辑keepalived.conf 添加以下内容:

vrrp_script chk_schedown {    script "[ -e /etc/keepalived/down ] && exit 1 || exit 0" #执行的脚本    interval 1 #check间隔    weight -10 #失败时优先级的变化    fall 2 #失败几次后真正确认为失败    rise 1 #成功几次后确认为成功}

在实例中进行调用, 添加在 vrrp_instance 中, 注意vrrp_script必须定义在实例的前面

vrrp_instance VI_1 {    state MASTER    interface eth0    virtual_router_id 51    priority 101    advert_int 1    authentication {        auth_type PASS        auth_pass shiina    }    virtual_ipaddress {        10.11.8.100/16 dev eth0 label eth0:0    }    track_script {        chk_schedown    }}

当 touch /etc/keepalived down 后, MASTER 的优先级会减少10, 服务会迁移到 BACKUP 上