Haproxy、Keepalived双主高可用负载均衡

来源：互联网发布：对外经济贸易大学知乎编辑：程序博客网时间：2024/05/24 05:21

通过Haproxy+Keepalived实现双主模式，以便充分利用了服务器资源。测试环境如下：
vip1=192.168.36.210（a.3evip.cn）
vip2=192.168.36.200（b.3evip.cn）
haproxy01=192.168.36.102
haproxy02=192.168.36.207
web1：192.168.36.54 部署a.3evip.cn和b.3evip.cn

web2：192.168.36.189 部署a.3evip.cn和b.3evip.cn

架构图：

一、安装配置web服务

为了模拟web服务，使用apache搭建web服务

1.安装apache

yum -y install httpd

2.配置：

vi /etc/httpd/conf/httpd.conf

#增加监听端口

Listen 8080
Listen 8081

#配置虚拟主机

<VirtualHost *:8080>
ServerAdmin root@slave-a
DocumentRoot /var/www/html/8080/
ServerName a.3evip.cn
ErrorLog logs/dummy-host.example.com-error_log
CustomLog logs/dummy-host.example.com-access_log common
</VirtualHost>
<VirtualHost *:8081>
ServerAdmin root@slave-a
DocumentRoot /var/www/html/8081/
ServerName b.3evip.cn
ErrorLog logs/dummy-host.example.com-error_log
CustomLog logs/dummy-host.example.com-access_log common
</VirtualHost>

在 /var/www/html/8080/和/var/www/html/8081/创建访问页面

echo "server:192.168.36.54:8080 ==> a.3evip.cn" > index.html

echo "server:192.168.36.54:8081 ==> b.3evip.cn" > index.html

另一台web：192.168.36.189配置类似

3.启动服务

service httpd start

若出现如下错误：

Starting httpd: httpd: Could not reliably determine the server's fully qualified domain name, using 192.168.36.54 for ServerName(13)Permission denied: make_sock: could not bind to address [::]:8081(13)Permission denied: make_sock: could not bind to address 0.0.0.0:8081no listening sockets available, shutting downUnable to open logs[FAILED]

由于权限不足导致，selinux有3个模式
Enforcing：强制模式
Permissive：警告模式
Disabled：关闭模式

vi /etc/selinux/config
#SELINUX=enforcing #注释掉
#SELINUXTYPE=targeted #注释掉
SELINUX=disabled #增加
:wq! #保存退出
setenforce 0 #使配置立即生效

关闭防火墙

启动后，访问看看是否正常：http://192.168.36.54:8080/

显示：server:192.168.36.54:8080 ==>a.3evip.cn

二、安装配置haproxy（haproxy用户）

1.安装忽略可以参考：http://blog.csdn.net/zhu_tianwei/article/details/41117323

2,.配置

global        log 127.0.0.1   local0        log 127.0.0.1   local1 notice        maxconn 1000        daemondefaults        log     global        mode    http        option  httplog        option  dontlognull        retries 3        maxconn 1000        timeout connect 5000        timeout client  50000        timeout  server 50000        balance roundrobinlisten admin_stats          bind 0.0.0.0:1080               #设置Frontend和Backend的组合体，监控组的名称，按需要自定义名称          mode http                       #http的7层模式          option httplog                  #采用http日志格式          maxconn 10                                              #默认的最大连接数          stats refresh 30s               #统计页面自动刷新时间          stats uri /stats                #统计页面url          stats realm XingCloud\ Haproxy  #统计页面密码框上提示文本          stats auth admin:admin     #设置监控页面的用户和密码:admin,可以设置多个用户名          stats auth  Frank:Frank   #设置监控页面的用户和密码：Frank          stats hide-version              #隐藏统计页面上HAProxy的版本信息          stats  admin if TRUE       #设置手工启动/禁用，后端服务器(haproxy-1.4.9以后版本)  frontend www        bind *:8888        acl aweb hdr_beg(host) -i a.3evip.cn #ACL规则若配置hdr(host) 则访问非80端口为503错误        acl bweb hdr_beg(host) -i b.3evip.cn        use_backend awebserver if aweb        use_backend bwebserver if bwebbackend awebserver        mode http        balance   roundrobin        option  httpchk /index.html        server     web01 192.168.36.54:8080  check inter 2000 fall 3 weight 10        server     web02 192.168.36.189:8080  check inter 2000 fall 3 weight 10backend bwebserver        mode http        balance   roundrobin        option  httpchk /index.html        server     web01 192.168.36.54:8081  check inter 2000 fall 3 weight 10        server     web02 192.168.36.189:8081  check inter 2000 fall 3 weight 10

2台机器配置一样

3.启动

./sbin/haproxy -f ./conf/haproxy.cfg

可以通关控制台查看配置。

三、安装配置keepalived(root用户)

1.下载keepalived，解压

wget http://www.keepalived.org/software/keepalived-1.2.13.tar.gz

tar -zxvf keepalived-1.2.13.tar.gz

2.安装

cd keepalived-1.2.13

./configure #配置，必须看到以下提示，说明配置正确，才能继续安装
Use IPVS Framework : Yes
IPVS sync daemon support : Yes
Use VRRP Framework : Yes

make

make install

3.配置启动

cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/
mkdir /etc/keepalived
cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/
cp /usr/local/sbin/keepalived /usr/sbin/
cp /usr/local/etc/rc.d/init.d/keepalived /etc/rc.d/init.d/
chmod +x /etc/rc.d/init.d/keepalived
chkconfig keepalived on

4.配置keepalived

haproxy01配置：

vi /etc/keepalived/keepalived.conf

global_defs {   notification_email {     tianwei7518@163.com   }   notification_email_from keepalived@master-c   smtp_server 127.0.0.1   smtp_connect_timeout 30   router_id LVS_DEVEL}vrrp_script chk_http_port {   script "/opt/check_haproxy.sh"   interval 2   weight 2}vrrp_instance VI_1 {    state MASTER    interface eth0    virtual_router_id 51    priority 100    advert_int 1    authentication {        auth_type PASS        auth_pass 1111    }    track_script {        chk_http_port #执行监控服务    }    virtual_ipaddress {        192.168.36.210    }notify_master "/opt/clean_arp.sh  192.168.36.210"  #更新虚拟服务器（VIP）地址的arp记录到网关}vrrp_instance VI_2 {    state BACKUP    interface eth0    virtual_router_id 52    priority 99    advert_int 1    authentication {        auth_type PASS        auth_pass 1111    }       virtual_ipaddress {        192.168.36.200    }notify_master "/opt/clean_arp.sh  192.168.36.200"  #更新虚拟服务器（VIP）地址的arp记录到网关}

haproxy02配置：

global_defs {   notification_email {     tianwei7518@163.com   }   notification_email_from keepalived@master-c   smtp_server 127.0.0.1   smtp_connect_timeout 30   router_id LVS_DEVEL}vrrp_script chk_http_port {   script "/opt/check_haproxy.sh"   interval 2   weight 2}vrrp_instance VI_1 {    state BACKUP    interface eth0    virtual_router_id 51    priority 99    advert_int 1    authentication {        auth_type PASS        auth_pass 1111    }    track_script {        chk_http_port #执行监控服务    }    virtual_ipaddress {        192.168.36.210    }notify_master "/opt/clean_arp.sh  192.168.36.210"  #更新虚拟服务器（VIP）地址的arp记录到网关}vrrp_instance VI_2 {    state MASTER    interface eth0    virtual_router_id 52    priority 100    advert_int 1    authentication {        auth_type PASS        auth_pass 1111    }       virtual_ipaddress {        192.168.36.200    }notify_master "/opt/clean_arp.sh  192.168.36.200"  #更新虚拟服务器（VIP）地址的arp记录到网关}

5.检查脚本，为了防止haproxy服务关闭导致keepalived不自动切换。

vi /opt/check_haproxy.sh

#!/bin/bashif [ $(ps -C haproxy --no-header | wc -l) -eq 0 ]; then     su - haproxy -c "/home/haproxy/haproxy/sbin/haproxy -f /home/haproxy/haproxy/conf/haproxy.cfg"fisleep 2if [ $(ps -C haproxy --no-header | wc -l) -eq 0 ]; then       /etc/init.d/keepalived stopfi

chmod +x /opt/check_haproxy.sh

6.设置更新虚拟服务器（VIP）地址的arp记录到网关脚本

vi /opt/clean_arp.sh

#!/bin/shVIP=$1GATEWAY=192.168.21.2 #网关地址/sbin/arping -I eth0 -c 5 -s $VIP $GATEWAY &>/dev/null

7.系统内核优化

在两台HAProxy服务器上分别执行以下命令

echo 1024 60999 > /proc/sys/net/ipv4/ip_local_port_rangeecho 30 > /proc/sys/net/ipv4/tcp_fin_timeoutecho 4096 > /proc/sys/net/ipv4/tcp_max_syn_backlogecho 262144 > /proc/sys/net/ipv4/tcp_max_tw_bucketsecho 262144 > /proc/sys/net/ipv4/tcp_max_orphansecho 300 > /proc/sys/net/ipv4/tcp_keepalive_timeecho 1 > /proc/sys/net/ipv4/tcp_tw_recycleecho 0 > /proc/sys/net/ipv4/tcp_timestampsecho 0 > /proc/sys/net/ipv4/tcp_ecnecho 1 > /proc/sys/net/ipv4/tcp_sackecho 0 > /proc/sys/net/ipv4/tcp_dsack

8，启动keepalived服务

service keepalived start (stop/status)

9.查看vip生效情况

在两台HAProxy服务器：192.168.36.102、192.168.36.217上执行命令：ip addr

192.168.36.102：

[root@master-c keepalived]# ip addr1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00    inet 127.0.0.1/8 scope host lo    inet6 ::1/128 scope host        valid_lft forever preferred_lft forever2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000    link/ether 00:0c:29:f2:66:46 brd ff:ff:ff:ff:ff:ff    inet 192.168.36.102/24 brd 192.168.36.255 scope global eth0    inet 192.168.36.210/32 scope global eth0    inet6 fe80::20c:29ff:fef2:6646/64 scope link        valid_lft forever preferred_lft forever3: pan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN     link/ether 76:ad:3f:58:42:ce brd ff:ff:ff:ff:ff:ff

可以看出现在VIP：192.168.36.210指向192.168.36.102。

192.168.36.217：

[root@master-a home]# ip addr1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00    inet 127.0.0.1/8 scope host lo    inet6 ::1/128 scope host        valid_lft forever preferred_lft forever2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000    link/ether 00:0c:29:31:ca:70 brd ff:ff:ff:ff:ff:ff    inet 192.168.36.217/24 brd 192.168.36.255 scope global eth0    inet 192.168.36.200/32 scope global eth0    inet6 fe80::20c:29ff:fe31:ca70/64 scope link        valid_lft forever preferred_lft forever3: pan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN     link/ether 1e:0e:40:13:89:7b brd ff:ff:ff:ff:ff:ff

可以看出现在VIP：192.168.36.200指向192.168.36.217。

四、查看结果

访问http://a.3evip.cn:8888/ 和http://b.3evip.cn:8888/

关闭一台apache或haproxy测试一下。

关闭192.168.36.217的haproxy可以看出VIP都指向192.168.36.102

[root@master-c keepalived]# ip addr1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00    inet 127.0.0.1/8 scope host lo    inet6 ::1/128 scope host        valid_lft forever preferred_lft forever2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000    link/ether 00:0c:29:f2:66:46 brd ff:ff:ff:ff:ff:ff   inet 192.168.36.102/24 brd 192.168.36.255 scope global eth0    inet 192.168.36.210/32 scope global eth0    inet 192.168.36.200/32 scope global eth0    inet6 fe80::20c:29ff:fef2:6646/64 scope link        valid_lft forever preferred_lft forever3: pan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN     link/ether 76:ad:3f:58:42:ce brd ff:ff:ff:ff:ff:ff

0 0