haproxy+keepalived 实现双主配置高可用负载均衡

两个VIP地址：192.168.23.98
192.168.23.99
首先我们的拓扑图，由于双主模型，则最少需要四台服务器：
1.Haproxy特别适用于那些访问量很大，但又需要会话保持或七层应用的业务。Haproxy运行在普通的服务器硬件上，仅仅进行简单的配置就可以支持数以万计的连接。并且他的运行模式使得它可以很简单安全的整合到各种网站的架构中（可以代替lvs,nginx等负载均衡设备），同时使得应用服务器不会暴露到网络上。（NAT模式），因此，我们来拿haproxy来具体说明一下：

第一步配置后端的web服务器：web-01，web-02

[root@centos6 ~]# ech0 "web-01 test page" > /var/www/html/index.html[root@centos6 html]# cat index.html web-01 test page[root@localhost html]# echo "web-02 test page" > /var/www/html/index.html[root@localhost html]# cat index.html web-02 test page

然后启动服务 service httpd restart
haproxy测试一下网页的设置,访问后端的web服务

[root@centos7 keepalived]# curl 192.168.23.100web-01 test page[root@centos7 keepalived]# curl 192.168.23.101web-02 test page

2.在hk两个节点上都要安装haproxy和keepalived
例如：yum install haproxy keepalived -y
其次，修改内核参数设置，设置haproxy启动的时候不管有没有vip地址都可以启动
此选项为集群中关键选项，不然VIP地址没有在Haproxy服务器的时候，服务器无法正常启动

[root@centos7 keepalived]#  echo "net.ipv4.ip_nonlocal_bind = 1" >> /etc/sysctl.conf[root@centos7 keepalived]# sysctl -pnet.ipv4.ip_nonlocal_bind = 1[root@cento7 ~]# echo "net.ipv4.ip_nonlocal_bind= 1" >> /etc/sysctl.conf[root@cento7 ~]# sysctl -pnet.ipv4.ip_nonlocal_bind = 1

设置haproxy

既然haproxy要实现双主，就必须要做到分别监听两个vip地址，并且两个示例都能得到用户请求负载均衡转发给后端web服务器，使用户不论访问那个节点都可以实现负载均衡。
两个节点需要配置一样的所以用一个haproxy代替
Vim /etc/haproxy/haproxy.cfg

#---------------------------------------------------------------------# Example configuration for a possible web application.  See the# full configuration options online.##   http://haproxy.1wt.eu/download/1.4/doc/configuration.txt##---------------------------------------------------------------------#---------------------------------------------------------------------# Global settings#---------------------------------------------------------------------global    # to have these messages end up in /var/log/haproxy.log you will    # need to:    #    # 1) configure syslog to accept network log events.  This is done    #    by adding the '-r' option to the SYSLOGD_OPTIONS in    #    /etc/sysconfig/syslog    #    # 2) configure local2 events to go to the /var/log/haproxy.log    #   file. A line like the following can be added to    #   /etc/sysconfig/syslog    #    #    local2.*                       /var/log/haproxy.log    #    log         127.0.0.1 local2    chroot      /var/lib/haproxy    pidfile     /var/run/haproxy.pid    maxconn     4000    user        haproxy    group       haproxy    daemon    # turn on stats unix socket    stats socket /var/lib/haproxy/stats#---------------------------------------------------------------------# common defaults that all the 'listen' and 'backend' sections will# use if not designated in their block#---------------------------------------------------------------------defaults    mode                    http    log                     global    option                  httplog    option                  dontlognull    option http-server-close    option forwardfor       except 127.0.0.0/8    option                  redispatch    retries                 3    timeout http-request    10s    timeout queue           1m    timeout connect         10s    timeout client          1m    timeout server          1m    timeout http-keep-alive 10s    timeout check           10s    maxconn                 3000#---------------------------------------------------------------------# main frontend which proxys to the backends#---------------------------------------------------------------------#frontend  main *:5000#    acl url_static       path_beg       -i /static /images /javascript /stylesheets#    acl url_static       path_end       -i .jpg .gif .png .css .js#    use_backend static          if url_static#    default_backend             app#---------------------------------------------------------------------# static backend for serving up images, stylesheets and such#---------------------------------------------------------------------#backend static#    balance     roundrobin#    server      static 127.0.0.1:4331 check#---------------------------------------------------------------------# round robin balancing between the various backends#---------------------------------------------------------------------#backend app#    balance     roundrobin#    server  app1 127.0.0.1:5001 check#    server  app2 127.0.0.1:5002 check#    server  app3 127.0.0.1:5003 check#    server  app4 127.0.0.1:5004 checklisten stats     bind :9009     stats enable         #启用Haproxy的状态页面     stats uri /admin?stats    #设置Haproxy状态页面的访问URL     stats auth proxy:proxy     stats admin if TRUE     listen www1                   #定义一个实例       bind :80   #监听地址为VIP地址       mode tcp       #设置转发模式为TCP       option forwardfor   #允许在发往服务器的请求首部中插入“X-Forwarded-For”首部       server www01  192.168.23.100:80  check  #定义后端服务器的，并启用健康检查       server www02  192.168.23.101:80  checklisten www2                     #定义第二个实例        bind :80   #除了绑定的VIP地址和第一个实例不同之外，其他均相同       mode tcp       option forwardfor       server www01  192.168.23.100:80  check       server www02  192.168.23.101:80  check

keepalived的配置（HK-O1）

vim /etc/keepalived/keepalived.conf

! Configuration File for keepalivedglobal_defs {   smtp_server 127.0.0.1   smtp_connect_timeout 30   router_id LVS_DEVEL   vrrp_mcast_group4 224.40.100.19}vrrp_script chk_mt_down {  script "[[ -f /etc/keepalived/down ]] && exit 1 || exit 0"  interval 1           #1秒检测一次  weight -5           #优先级减五}vrrp_instance VI_1 {    state MASTER    interface ens39    virtual_router_id 51    priority 100    advert_int 1    authentication {        auth_type PASS               auth_pass 1111    }    virtual_ipaddress {         #指定漂移地址       192.168.23.98    }} track_script {   chk_mt_down   #调用上面定义的脚本，如果这里没有调用，那么上面定义的脚本是无法生效的   }vrrp_instance VI_2 {         #定义实例为HK-02的备份节点   state BACKUP           #BACKUP表示备份节点   interface ens39   virtual_router_id 52   priority 99           #优先级，低于主服务器   advert_int 1   authentication {       auth_type PASS       auth_pass qwerty    }   virtual_ipaddress {       192.168.23.99    }} track_script {   chk_mt_down   }

keepalived（HK-02）

vim /etc/keepalived/keepalived.conf

! Configuration File for keepalivedglobal_defs {   notification_email {     [e-mail]     [e-mail]     [e-mail]   }   notification_email_from [e-mail]   smtp_server 192.168.200.1   smtp_connect_timeout 30   router_id LVS_DEVEL   vrrp_mcast_group4 224.40.100.19}vrrp_script chk_down {  script "[[ -f /etc/keepalived/down ]] && exit 1 || exit 0"  interval 1           #1秒检测一次  weight -5           #优先级减五}}vrrp_instance VI_1 {    state BACKUP    interface eth1    virtual_router_id 51    priority 99    advert_int 1    authentication {        auth_type PASS        auth_pass 1111    }    virtual_ipaddress {         #指定漂移地址       192.168.23.98/32 brd 192.168.23.98    } }track_script {   chk_down   }vrrp_instance VI_2 {   state MASTER      interface eth1   virtual_router_id 52   priority 100   advert_int 1   authentication {       auth_type PASS       auth_pass qwerty    }   virtual_ipaddress {       192.168.23.99    }  track_script {   chk_down   }}

r然后分别启动haproxy和keepalived

验证

WWW1和WWW2的状态页面


两个VIP 也都启动到双主模型：

hk-01 ens39: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000    link/ether 00:0c:29:47:18:31 brd ff:ff:ff:ff:ff:ff    inet 192.168.23.148/24 brd 192.168.23.255 scope global ens39       valid_lft forever preferred_lft forever    inet 192.168.23.98/32 scope global ens39       valid_lft forever preferred_lft forever    inet6 fe80::e679:1a79:44ee:8733/64 scope link        valid_lft forever preferred_lft foreverhk-02[root@cento7 keepalived]# ip a1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00    inet 127.0.0.1/8 scope host lo       valid_lft forever preferred_lft forever    inet6 ::1/128 scope host        valid_lft forever preferred_lft forever2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000    link/ether 00:0c:29:c6:20:3d brd ff:ff:ff:ff:ff:ff    inet 172.16.250.240/16 brd 172.16.255.255 scope global dynamic eth0       valid_lft 81320sec preferred_lft 81320sec    inet6 fe80::20c:29ff:fec6:203d/64 scope link        valid_lft forever preferred_lft forever3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000    link/ether 00:0c:29:c6:20:47 brd ff:ff:ff:ff:ff:ff    inet 192.168.23.149/24 brd 192.168.23.255 scope global eth1       valid_lft forever preferred_lft forever    inet 192.168.23.99/32 scope global eth1       valid_lft forever preferred_lft forever    inet6 fe80::20c:29ff:fec6:2047/64 scope link        valid_lft forever preferred_lft forever

分别查看两vip能否负载均衡

[root@centos7 keepalived]# curl 192.168.23.98web-01 test page[root@centos7 keepalived]# curl 192.168.23.98web-02 test page[root@centos7 keepalived]# curl 192.168.23.99web-01 test page[root@centos7 keepalived]# curl 192.168.23.99web-02 test page

验证是否会漂移地址

关闭hk-02

[root@cento7 keepalived]#  touch /etc/keepalived/down[root@cento7 keepalived]# ip a1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00    inet 127.0.0.1/8 scope host lo       valid_lft forever preferred_lft forever    inet6 ::1/128 scope host        valid_lft forever preferred_lft forever2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000    link/ether 00:0c:29:c6:20:3d brd ff:ff:ff:ff:ff:ff    inet 172.16.250.240/16 brd 172.16.255.255 scope global dynamic eth0       valid_lft 80637sec preferred_lft 80637sec    inet6 fe80::20c:29ff:fec6:203d/64 scope link        valid_lft forever preferred_lft forever3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000    link/ether 00:0c:29:c6:20:47 brd ff:ff:ff:ff:ff:ff    inet 192.168.23.149/24 brd 192.168.23.255 scope global eth1       valid_lft forever preferred_lft forever    inet6 fe80::20c:29ff:fec6:2047/64 scope link        valid_lft forever preferred_lft forever

两个地址均漂移到hk-01

[root@centos7 keepalived]# ip a1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00    inet 127.0.0.1/8 scope host lo       valid_lft forever preferred_lft forever    inet6 ::1/128 scope host        valid_lft forever preferred_lft forever2: ens38: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000    link/ether 00:0c:29:47:18:27 brd ff:ff:ff:ff:ff:ff    inet 172.16.253.224/16 brd 172.16.255.255 scope global dynamic ens38       valid_lft 80584sec preferred_lft 80584sec    inet6 fe80::e528:d692:e718:3a5d/64 scope link        valid_lft forever preferred_lft forever3: ens39: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000    link/ether 00:0c:29:47:18:31 brd ff:ff:ff:ff:ff:ff    inet 192.168.23.148/24 brd 192.168.23.255 scope global ens39       valid_lft forever preferred_lft forever    inet 192.168.23.98/32 scope global ens39       valid_lft forever preferred_lft forever    inet 192.168.23.99/32 scope global ens39       valid_lft forever preferred_lft forever    inet6 fe80::e679:1a79:44ee:8733/64 scope link        valid_lft forever preferred_lft forever4: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN qlen 1000    link/ether 52:54:00:3b:81:ab brd ff:ff:ff:ff:ff:ff    inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0       valid_lft forever preferred_lft forever

健康状态检查

手动关闭 web-01

[root@centos6 ~]# service httpd stop Stopping httpd:                                            [  OK  ][root@centos6 ~]# 

web状态页面web-01已经下线

验证下不会访问web-01

[root@cento7 keepalived]# curl 192.168.23.98web-02 test page[root@cento7 keepalived]# curl 192.168.23.98web-02 test page[root@cento7 keepalived]# curl 192.168.23.98web-02 test page[root@cento7 keepalived]# curl 192.168.23.99web-02 test page[root@cento7 keepalived]# curl 192.168.23.99web-02 test page[root@cento7 keepalived]# curl 192.168.23.99web-02 test page