HAProxy + KeepAlived构建高可用的反向代理系统
来源:互联网 发布:网络教育统考成绩时间 编辑:程序博客网 时间:2024/05/21 07:15
http://weizhifeng.net/tagged/%E9%AB%98%E5%8F%AF%E7%94%A8
前言
对于访问量较大的网站来说,随着流量的增加单台服务器已经无法处理所有的请求,这时候需要多台服务器对大量的请求进行分流处理,即负载均衡。而如果实现负载均衡,必须在网站的入口部署服务器(不只是一台)对这些请求进行分发,这台服务器即反向代理。由于反向代理服务器是网站的入口,其负载压力大且易遭到攻击,存在单点故障的风险,所以我们需要一个高可用的方案来实现当一台反向代理服务器宕机的时候,另一台服务器会自动接管服务。基于以上要求,我们使用HAProxy,KeepAlived来构建高可用的反向代理系统。
介绍
HAProxy是高性能的代理服务器,其可以提供7层和4层代理,具有healthcheck,负载均衡等多种特性,性能卓越,包括Twitter,Reddit,StackOverflow,GitHub在内的多家知名互联网公司在使用。
KeepAlived是一个高可用方案,通过VIP(即虚拟IP)和心跳检测来实现高可用。其原理是存在一组(两台)服务器,分别赋予Master,Backup两个角色,默认情况下Master会绑定VIP到自己的网卡上,对外提供服务。Master,Backup会在一定的时间间隔向对方发送心跳数据包来检测对方的状态,这个时间间隔一般为2秒钟,如果Backup发现Master宕机,那么Backup会发送ARP包到网关,把VIP绑定到自己的网卡,此时Backup对外提供服务,实现自动化的故障转移,当Master恢复的时候会重新接管服务。
环境
OS: CentOS Linux release 6.0 (Final) 2.6.32-71.29.1.el6.x86_64
HAProxy: 1.4.18
KeepAlived: 1.2.2
VIP: 192.168.1.99
M: 192.168.1.222
S: 192.168.1.189
架构
192.168.1.99 +-----------VIP----------+ | | | | Master Backup 192.168.1.189 192.168.1.222 +----------+ +----------+ | HAProxy | | HAProxy | |keepalived| |keepalived| +----------+ +----------+ | v +--------+---------+ | | | | | | v v v+------+ +------+ +------+| WEB1 | | WEB2 | | WEB3 |+------+ +------+ +------+
安装HAProxy
# 安装pcreyum install pcrewget http://haproxy.1wt.eu/download/1.4/src/haproxy-1.4.18.tar.gztar -zxvf haproxy-1.4.18.tar.gzcd haproxy-1.4.18# 注意编译参数:# TARGET是指自己系统的内核版本 ARCH指定系统是32位还是64位# CPU=native: use the build machine's specific processor optimizations# 更多编译参数内容见源码中的READMEmake TARGET=linux26 ARCH=x86_64 USE_PCRE=1 CPU=nativemake install #配置文件 /etc/haproxy.cfgglobal log 127.0.0.1 local3 maxconn 20000 uid 535 #uid和gid按照实际情况进行配置 gid 520 chroot /var/chroot/haproxy daemon nbproc 1 defaults log 127.0.0.1 local3 mode http option httplog option httpclose option dontlognull option forwardfor retries 2 balance roundrobin stats uri /haproxy-stats contimeout 5000 clitimeout 50000 srvtimeout 50000frontend http-in bind *:80 default_backend pool1backend pool1 option httpchk HEAD / HTTP/1.0 stats refresh 2 server WEB1 192.168.1.189:81 weight 3 maxconn 10000 check server WEB2 192.168.1.222:81 weight 3 maxconn 10000 check
查看HAProxy的状态:http://192.168.1.99/haproxy-stats,这个页面会显示HAProxy本身以及后端服务器的状态。
日志
haproxy会把日志记录发送到syslog server(CentOS6下是rsyslogd,UDP514端口), 编辑/etc/rsyslog.conf文件,添加如下内容:
$ModLoad imudp$UDPServerRun 514$UDPServerAddress 127.0.0.1local3.* /var/log/haproxy.log
/etc/init.d/rsyslog restart
自动轮转日志,编辑/etc/logrotate.d/haproxy.cfg,添加如下内容:
/var/log/haproxy.log{ rotate 4 daily missingok notifempty compress delaycompress sharedscripts postrotate reload rsyslog > /dev/null 2>&1 || true endscript}
启动脚本: 拷贝以下内容到/etc/init.d/haproxy,并chmod +x /etc/init.d/haproxy,使用方式:/etc/init.d/haproxy start|stop|restart
#!/bin/bash### BEGIN INIT INFO# Author: weizhifeng# Date: 2011.10.19# Description: Manage the HAProxy# version HAProxy 1.4.18### END INIT INFObin=/usr/local/sbin/haproxyconfig=/etc/haproxy.cfgpid=/tmp/haproxy.pidopts=" -f ${config} -p ${pid} -D -V "sleep_time=1start() {echo -e "Starting HAProxy......"${bin} ${opts}if [ "$?" != "0" ] ; thensleep ${sleep_time}echo " failed"exit 1elsesleep ${sleep_time}echo " done"fi}stop() {if [ ! -e ${pid} ] ; thenecho -e "HAProxy is not running"exit 0fiecho -e "Shutting down HAProxy......"kill $(cat ${pid})if [ -e ${pid} ] ; thenrm -f ${pid}fiif [ "$?" != "0" ] ; thensleep ${sleep_time}echo " failed"exit 1elsesleep ${sleep_time}echo " done"fi}restart() {stopstart}case "$1" instart)start;;stop)stop;;restart)restart;;*)echo "Usage: $0 {start|stop|restart}"exit 1;;esac
安装KeepAlived
# 依赖库yum install popt popt-develwget http://www.keepalived.org/software/keepalived-1.2.2.tar.gztar -zxvf keepalived-1.2.2.tar.gzcd keepalived-1.2.2./configure --prefix=/usr/local/keepalivedmakemake installcp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/keepalivedcp /usr/local/keepalived/sbin/keepalived /usr/sbin/cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/mkdir -p /etc/keepalived/cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf chmod +x /etc/init.d/keepalived#Master服务器上的配置 /etc/keepalived/keepalived.confglobal_defs { notification_email { user@example.com } notification_email_from mail@example.org smtp_server 192.168.x.x smtp_connect_timeout 30 router_id LVS_DEVEL}#监测haproxy进程状态,每2秒执行一次vrrp_script chk_haproxy { script "/usr/local/keepalived/chk_haproxy.sh" interval 2 weight 2}vrrp_instance VI_1 { state MASTER #标示状态为MASTER interface eth0 virtual_router_id 51 priority 101 #MASTER权重要高于BACKUP advert_int 1 mcast_src_ip 192.168.1.189 #Master服务器IP authentication { auth_type PASS #主从服务器验证方式 auth_pass 1111 } track_script { chk_haproxy #监测haproxy进程状态 } #VIP virtual_ipaddress { 192.168.1.99 #虚拟IP }}#Bakcup服务器上的配置 /etc/keepalived/keepalived.confglobal_defs { notification_email { user@example.com } notification_email_from mail@example.org smtp_server 192.168.x.x smtp_connect_timeout 30 router_id LVS_DEVEL}#监测haproxy进程状态,每2秒执行一次vrrp_script chk_haproxy { script "/usr/local/keepalived/chk_haproxy.sh" interval 2 weight 2}vrrp_instance VI_1 { state BACKUP #状态为BACKUP interface eth0 virtual_router_id 51 priority 100 #权重要低于MASTER advert_int 1 mcast_src_ip 192.168.1.222 #Backup服务器的IP authentication { auth_type PASS auth_pass 1111 } track_script { chk_haproxy #监测haproxy进程状态 } #VIP virtual_ipaddress { 192.168.1.99 #虚拟IP }}#使用方式/etc/init.d/keepalived start|stop|restart
chk_haproxy.sh内容
#!/bin/bash## author: weizhifeng# description: # 定时查看haproxy是否存在,如果不存在则启动haproxy,# 如果启动失败,则停止keepalived# status=$(ps aux|grep haproxy | grep -v grep | grep -v bash | wc -l)if [ "${status}" = "0" ]; then /etc/init.d/haproxy start status2=$(ps aux|grep haproxy | grep -v grep | grep -v bash |wc -l) if [ "${status2}" = "0" ]; then /etc/init.d/keepalived stop fifi
高可用测试
1. 在Master上停止keepalived,查看系统日志,发现MASTER释放了VIP
/etc/init.d/keepalived stoptail -f /var/log/messageKeepalived: Terminating on signal Keepalived: Stopping Keepalived v1.2.2 (11/03,2011) Keepalived_vrrp: Terminating VRRP child process on signal Keepalived_vrrp: VRRP_Instance(VI_1) removing protocol VIPs.
2. 在Backup上查看系统日志,发现Backup已经进入MASTER角色,并且绑定了VIP 192.168.1.99
tail -f /var/log/messageKeepalived_vrrp: VRRP_Instance(VI_1) Entering MASTER STATEKeepalived_vrrp: VRRP_Instance(VI_1) setting protocol VIPsKeepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.1.99 #在Backup上查看VIP是否已经绑定
3. 在Master上重新启动keepalived,查看系统日志,发现重新获得MASTER角色,并且绑定VIP 192.168.1.99
/etc/init.d/keepalived starttail -f /var/log/messageKeepalived_vrrp: VRRP_Instance(VI_1) Transition to MASTER STATEKeepalived_vrrp: VRRP_Instance(VI_1) Entering MASTER STATEKeepalived_vrrp: VRRP_Instance(VI_1) setting protocol VIPs.Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.1.99
4. 在Backup上查看系统日志,发现其重新回到BACKUP角色,并且释放VIP
tail -f /var/log/messageKeepalived_vrrp: VRRP_Instance(VI_1) Received higher prio advertKeepalived_vrrp: VRRP_Instance(VI_1) Entering BACKUP STATEKeepalived_vrrp: VRRP_Instance(VI_1) removing protocol VIPs.
并发测试
我们使用webbench来对HAProxy进行并发测试
yum install ctagswget http://home.tiscali.cz/~cz210552/distfiles/webbench-1.5.tar.gztar -zxvf webbench-1.5.tar.gzcd webbench-1.5make mkdir -p /usr/local/man && make install
测试环境: Dell PC Mem:2G CPU:Intel 双核 x86_64 主频3191MHZ 修改php-fpm.conf,设置PHP-FPM spawn的进程数量为100:
pm.start_servers = 100pm.max_spare_servers = 100
测试方法:
webbench -c 100 -t 3000 http://192.168.1.99/check.txtwebbench -c 100 -t 3000 http://192.168.1.99/test.php
测试结果: 并发访问txt文件,HAProxy的session数量为10000左右,这说明HAProxy能够hold住10000个并发连接;并发访问php文件,HAProxy的session峰值为200左右,接近于后端PHP的并发处理能力(100x2)。
参考:
http://haproxy.1wt.eu/download/1.4/doc/configuration.txt
http://kevin.vanzonneveld.net/techblog/article/haproxy_logging/
- HAProxy + KeepAlived构建高可用的反向代理系统
- 使用Haproxy,Keepalived,Tproxy实现高可用透明反向代理
- CentOS7,keepalived搭建双机热备(高可用、keepalived、Nginx、Haproxy、反向代理)
- haproxy+keepalived构建高可用负载均衡
- HAProxy和KeepAlived反向代理系统
- 如何快速构建高可用集群(Keepalived+Haproxy+Nginx)
- 构建负载均衡和高可用的PXC环境(HAProxy+Keepalived+PXC)问题补充
- 使用Nginx+KeepAlived构建高可用的负载均衡系统
- haproxy+keepalived实现高可用
- Haproxy+keepalived 高可用负载
- keepalived实现高可用nginx反向代理(Web集群)
- 利用keepalived构建高可用的MySQL
- 高可用的负载均衡配置方法(Haproxy+KeepAlived)
- keepalived实现haproxy负载均衡器的高可用
- Nginx反向代理,负载均衡,Redis集群 Twemproxy,redis session共享,keepalived高可用-搭建千万级网站系统
- haproxy+keepalived实现高可用负载均衡
- haproxy+keepalived实现高可用负载均衡
- haproxy+keepalived实现高可用负载均衡
- xp不能安装NET Framework4.0解决方法
- gcc 编译优化选项
- C# .net 子类序列化时无法识别的解决办法
- VS2003——BULID编译(之前的页面显示不出来,郁闷,分开来写!)
- SCNetworkReachability 说明使用
- HAProxy + KeepAlived构建高可用的反向代理系统
- iso8583协议说明
- 使用DbProviderFactories操作SQLite
- iOS开发之NSNotification
- LINUX设置网卡
- H264探测过程用到的几个函数原型
- 编写简单的网卡配置的脚本文件和相关的批处理文件
- Openstack 相关资源 转载
- AdaBoost中利用Haar特征进行人脸识别算法分析与总结1——Haar特征与积分图