文章标题

用Keepalived高可用lvs负载均衡nginx：

• 拓扑图

  

• 目的：

  使用 keepalived双主模型实现 lvs负载均衡；

• 双主模型keepalived配置文件：

! Configuration File for keepalivedglobal_defs {   notification_email {        root@localhost   }   notification_email_from [e-mail]   smtp_server 127.0.0.1   smtp_connect_timeout 30   router_id r1   vrrp_mcast_group4 224.0.100.33}vrrp_instance lvs1 {    state MASTER    interface ens33    virtual_router_id 21    priority 100    advert_int 1    authentication {        auth_type PASS        auth_pass abcdef    }    virtual_ipaddress {        172.18.54.10/16    }}vrrp_instance lvs2 {    state BACKUP    interface ens33    virtual_router_id 22    priority 98    advert_int 1    authentication {        auth_type PASS        auth_pass qwerty    }    virtual_ipaddress {            172.18.54.11/16    }}virtual_server 172.18.54.10 80 {    delay_loop 6    lb_algo wrr    lb_kind DR#    persistence_timeout 3    protocol TCP    real_server 172.18.54.4 80 {        weight 1        HTTP_GET {            url {              path /              status_code 200            }            connect_timeout 3            nb_get_retry 3            delay_before_retry 3        }    }    real_server 172.18.54.5 80 {        weight 1        HTTP_GET {            url {              path /              status_code 200            }            connect_timeout 3            nb_get_retry 3            delay_before_retry 3        }    }}virtual_server 172.18.54.11 80 {    delay_loop 6    lb_algo wrr    lb_kind DR#    persistence_timeout 3    protocol TCP    real_server 172.18.54.4 80 {        weight 1        HTTP_GET {            url {              path /              status_code 200            }            connect_timeout 3            nb_get_retry 3            delay_before_retry 3        }    }    real_server 172.18.54.5 80 {        weight 1        HTTP_GET {            url {              path /              status_code 200            }            connect_timeout 3            nb_get_retry 3            delay_before_retry 3        }    }

• 另一Director的keepalived配置文件

! Configuration File for keepalivedglobal_defs {   notification_email {        root@localhost   }   notification_email_from [e-mail]   smtp_server 127.0.0.1   smtp_connect_timeout 30   router_id r1   vrrp_mcast_group4 224.0.100.33}vrrp_instance lvs1 {    state BACKUP    interface ens33    virtual_router_id 21    priority 98     advert_int 1    authentication {        auth_type PASS        auth_pass abcdef    }    virtual_ipaddress {        172.18.54.10/16    }}vrrp_instance lvs2 {    state MASTER    interface ens33    virtual_router_id 22    priority 100    advert_int 1    authentication {        auth_type PASS        auth_pass qwerty    }    virtual_ipaddress {            172.18.54.11/16    }}virtual_server 172.18.54.10 80 {    delay_loop 6    lb_algo wrr    lb_kind DR#    persistence_timeout 3    protocol TCP    real_server 172.18.54.4 80 {        weight 1        HTTP_GET {            url {              path /              status_code 200            }            connect_timeout 3            nb_get_retry 3            delay_before_retry 3        }    }    real_server 172.18.54.5 80 {        weight 1        HTTP_GET {            url {              path /              status_code 200            }            connect_timeout 3            nb_get_retry 3            delay_before_retry 3        }    }}virtual_server 172.18.54.11 80 {    delay_loop 6    lb_algo wrr    lb_kind DR#    persistence_timeout 3    protocol TCP    real_server 172.18.54.4 80 {        weight 1        HTTP_GET {            url {              path /              status_code 200            }            connect_timeout 3            nb_get_retry 3            delay_before_retry 3        }    }    real_server 172.18.54.5 80 {        weight 1        HTTP_GET {            url {              path /              status_code 200            }            connect_timeout 3            nb_get_retry 3            delay_before_retry 3        }    }

• 后端real server的配置

  
  
  • 通用配置脚本文件：

vim /root/lvs-dr.sh

• 脚本功能：
  script_name start #启动脚本，按照给定的值设置VIP地址，路由等参数；
  script_name stop #停止服务，删除所有的VIP与路由并将修改过的arp内核参数置0;

#!/bin/bash#vip='172.18.54.10'      #定义第一个VIPvip2='172.18.54.11'     #定义第二个VIPiface='lo:0'            #定义第一个接口，用来设置VIPiface2='lo:1'           #定义第二个接口，用来设置VIPnetmask='255.255.255.255' #定义掩码se=`getenforce`  case $1 instart)        systemctl stop firewalld.service        iptables -F             #避免iptalbes干扰实验，清空iptalbes规则，如不需要，注释或删除此行；        [[ $se == 'Enforcing' ]] && setenforce 0   #为了避免selinux的干扰，判断selinux的状态，如果不需要，注释即可        echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore        echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce        echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore        echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce        ifconfig $iface $vip  netmask $netmask broadcast $vip up        ifconfig $iface2 $vip2  netmask $netmask broadcast $vip2 up        route add -host $vip dev $iface        route add -host $vip2 dev $iface2        ;;stop)        echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore        echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce        echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore        echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce        ifconfig $iface down        ifconfig $iface2 down        ;;*)        echo "Usage: $(basename $0) {start|stop}"esac

• 在后端服务器上运行脚本：

./lvs-dr.sh start

• 启动前端keepalived服务：
  * systemctl start keepalived.service *

• 启动后端web服务：
  * systemctl start nginx.service *

• 使用ipvsadm命令查看lvs集群服务：
  * ipvsadm -ln　*

[root@localhost]#ipvsadm -lnIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags  -> RemoteAddress:Port           Forward Weight ActiveConn InActConnTCP  172.18.54.10:80 wrr  -> 172.18.54.4:80               Route   1      0          0           -> 172.18.54.5:80               Route   1      0          0         TCP  172.18.54.11:80 wrr  -> 172.18.54.4:80               Route   1      0          0           -> 172.18.54.5:80               Route   1      0          0  

• 测试：

  
  
  • 客户端请，使用客户端请求检测是负载均衡是否正常工作：

[root@centos7-3 ~]# for i in {1..5};do curl http://172.18.54.10 ;doneRS2:54.5:80RS1:54.4:80RS2:54.5:80RS1:54.4:80RS2:54.5:80[root@centos7-3 ~]# for i in {1..5};do curl http://172.18.54.11 ;doneRS2:54.5:80RS1:54.4:80RS2:54.5:80RS1:54.4:80RS2:54.5:80

- 负载均衡正常；- 尝试宕掉一台服务器，查看双主模式下的IP地址是否转移，负载均衡是否正常；    ```

停止172.18.54.3上的keepalived服务进程；
[root@localhost]#systemctl stop keepalived.service
在另一台Director上查看IP是否添加状态：
[root@centos7-3 ~]#ip a l
…
2: ens33:

[root@centos7-3 ~]# for i in {1..5};do curl http://172.18.54.10 ;doneRS2:54.5:80RS1:54.4:80RS2:54.5:80RS1:54.4:80RS2:54.5:80[root@centos7-3 ~]# for i in {1..5};do curl http://172.18.54.11 ;doneRS1:54.4:80RS2:54.5:80RS1:54.4:80RS2:54.5:80RS1:54.4:80

• 服务依旧正常提供；

• 尝试宕掉后端服务器：
  [root@RS1]#systemctl stop nginx.service #此rs的ip为172.18.54.4;

• 再次使用客户端访问：

      [root@centos7-3 ~]# for i in {1..5};do curl http://172.18.54.10 ;done    RS2:54.5:80    RS2:54.5:80    RS2:54.5:80    RS2:54.5:80    RS2:54.5:80    [root@centos7-3 ~]# for i in {1..5};do curl http://172.18.54.11 ;done    RS2:54.5:80    RS2:54.5:80    RS2:54.5:80    RS2:54.5:80    RS2:54.5:80

• 请求依旧能正确的调度到正常的real server上，不会被调度到宕掉的服务器上；

  • keepalived的健康状态检测，可以在四层或者7层进行检测，上面使用的就是在7层进行检测，用HTTP_GET去请求后端各real server的 / 判断响应状态码是否为200来监测后端real server是否正常。