Mycat生产实践---生产环境部署

来源:互联网 发布:c语言地址符号 编辑:程序博客网 时间:2024/03/29 09:00

单节点mycat部署

单节点mycat的部署指的是只部署一台mycat服务器,它与mycat集群部署是相对的,如果这台mycat服务器宕机了,mycat就不可用了。

mycat的高可用与负载均衡

1 什么是高可用?
高可用通常也叫HA(High Available)。指的是,一台服务器宕机了,照样能对外提供服务。常用的高可用软件方案有:LVS、keepalived、Heartbeat、roseHA(roseHA为收费软件)等。
Mycat本身是无状态的,可以用HAProxy或四层交换机等设备组成Mycat的高可用集群,后端MySQL则配置为主从同步,此时整个系统就是高可用的,下图是一个典型的Mycat系统高可用的方案:
这里写图片描述
2 haproxy + keepalived + mycat高可用与负载均衡集群配置
部署图:
这里写图片描述
集群部署图的理解:
1、keepalived和haproxy必须装在同一台机器上(如172.17.210.210.83机器上,keepalived和haproxy都要安装),keepalived负责为该服务器抢占vip(虚拟ip),抢占到vip后,对该主机的访问可以通过原来的ip(172.17.210.210.83)访问,也可以直接通过vip(172.17.210.210.103)访问。
2、172.17.210.64上的keepalived也会去抢占vip,抢占vip时有优先级,配置keepalived.conf中的(priority 150 #数值愈大,优先级越高,172.17.210.64上改为120,master和slave上该值配置不同)决 定。但是一般哪台主机上的keepalived服务先启动就会抢占到vip,即使是slave,只要先启动也能抢到。
3、haproxy负责将对vip的请求分发到mycat上。起到负载均衡的作用,同时haproxy也能检测到mycat是否存活,haproxy只会将请求转发到存活的mycat上。
4、如果一台服务器(keepalived+haproxy服务器)宕机,另外一台上的keepalived会立刻抢占vip并接管服务。
如果一台mycat服务器宕机,haporxy转发时不会转发到宕机的mycat上,所以mycat依然可用。
3 haproxy安装

useradd haproxy #wget http://haproxy.1wt.eu/download/1.4/src/haproxy-1.4.25.tar.gz # tar zxvf haproxy-1.4.25.tar.gz # cd haproxy-1.4.25# make TARGET=linux26 PREFIX=/usr/local/haproxy ARCH=x86_64 # make install PREFIX=/usr/local/haproxy #cd /usr/local/haproxy#chown -R haproxy.haproxy *

haproxy.cfg

#cd /usr/local/haproxy #touch haproxy.cfg #vi/usr/local/haproxy/haproxy.cfg global log 127.0.0.1 local0 ##记日志的功能 maxconn 4096 chroot/usr/local/haproxy user haproxy group haproxy daemon defaults log global option dontlognull retries 3 option redispatch maxconn 2000 contimeout 5000 clitimeout 50000 srvtimeout 50000 listen admin_status 172.17.210.103:48800 ##VIP stats uri/admin-status ##统计页面 stats auth admin:adminmode http option httplog listen allmycat_service 172.17.210.103:8096 ##转发到mycat的8066端口,即mycat的服务端口 mode tcp option tcplog option httpchk OPTIONS * HTTP/1.1\r\nHost:\ www balance roundrobin server mycat_64 172.17.210.64:8066 check port 48700 inter 5s rise 2 fall 3 server mycat_83 172.17.210.83:8066 check port 48700 inter 5s rise 2 fall 3 srvtimeout 20000 listen allmycat_admin 172.17.210.103:8097 ##转发到mycat的9066端口,及mycat的管理控制台端口 mode tcp option tcplog option httpchk OPTIONS * HTTP/1.1\r\nHost:\ www balance roundrobin server mycat_64 172.17.210.64:9066 check port 48700 inter 5s rise 2 fall 3 server mycat_83 172.17.210.83:9066 check port 48700 inter 5s rise 2 fall 3 srvtimeout 20000

haproxy记录日志
默认haproxy是不记录日志的,为了记录日志还需要配置syslog模块,在linux下是rsyslogd服务, 先安装

rsyslog yum –y install rsyslog

然后
记录haproxy日志的配置

 cd /etc/rsyslog.d/

如果没有这个目录,新建

cd /etc mkdir rsyslog.dcd /etc/rsyslog.d/ touch haproxy.conf

vi /etc/rsyslog.d/haproxy.conf
内容如下:

$ModLoad imudp $UDPServerRun 514 local0.* /var/log/haproxy.log

vi /etc/rsyslog.conf
1、在#### RULES ####上面一行的地方加入以下内容:

# Include all config files in /etc/rsyslog.d/ $IncludeConfig /etc/rsyslog.d/*.conf #### RULES ####

2、在local7.* /var/log/boot.log的下面加入以下内容(增加后的效果如下):

# Save boot messages also to boot.log local7.* /var/log/boot.log local0.* /var/log/haproxy.log

保存,重启rsyslog服务

service rsyslog restart

现在你就可以看到日志(/var/log/haproxy.log)了
4 配置监听mycat是否存活
在Mycat server1 Mycat server2上都需要添加检测端口48700的脚本,为此需要用到xinetd,xinetd为linux系统的基础服务。
首先在xinetd目录下面增加脚本与端口的映射配置文件
1、如果xinetd没有安装,使用如下命令安装:

yum install xinetd -y

2、检查/etc/xinetd.conf的末尾是否有这一句:

includedir /etc/xinetd.d

没有就加上
3、检查 /etc/xinetd.d文件夹是否存在,不存在也加上

cd /etc mkdir xinetd.d

4、增加 /etc/xinetd.d/mycat_status
监听mycat是否存活的配置,执行以下命令:

cd /etc mkdir xinetd.d cd /etc/xinetd.d/ touch mycat_status

vim /etc/xinetd.d/mycat_status
内容如下:

service mycat_status { flags = REUSE socket_type = stream port = 48700 wait = no user = root server =/usr/local/bin/mycat_status log_on_failure += USERID disable = no }

5、/usr/local/bin/mycat_status脚本
内容如下:

#!/bin/bash #/usr/local/bin/mycat_status.sh # This script checks if a mycat server is healthy running on localhost. It will # return:# # "HTTP/1.x 200 OK\r" (if mycat is running smoothly) # # "HTTP/1.x 503 Internal Server Error\r" (else) mycat=`/usr/local/mycat/bin/mycatstatus |grep'not running'| wc -l` if [ "$mycat" = "0" ]; then /bin/echo-e"HTTP/1.1 200 OK\r\n" else /bin/echo-e"HTTP/1.1 503 Service Unavailable\r\n" fi

6、/etc/services中加入mycat_status服务
加入mycat_status服务,

cd /etc vi services

在末尾加入以下内容:

mycat_status 48700/tcp # mycat_status

保存
重启xinetd服务

service xinetd restart

7、验证mycat_status服务是否启动成功

netstat -antup|grep 48700

如果成功会现实如下内容:

[root@localhost log]# netstat -antup|grep 48700tcp 0 0 :::48700 :::* LISTEN 12609/xinetd

5 启动haproxy

启动haproxy前必须先启动keepalived,否则启动不了。
启动命令:

/usr/local/haproxy/sbin/haproxy -f /usr/local/haproxy/haproxy.cfg

6 启动haproxy异常情况

如果报以下错误:
[root@localhost bin]# /usr/local/haproxy/sbin/haproxy -f /usr/local/haproxy/haproxy.cfg
[ALERT] 183/115915 (12890) :Starting proxy admin_status: cannot bind socket
[ALERT] 183/115915 (12890) :Starting proxy allmycat_service: cannot bind socket
[ALERT] 183/115915 (12890) :Starting proxy allmycat_admin: cannot bind socket
原因为:该机器没有抢占到vip,如果另一台服务启动正常,这个错误可以忽略不管,如果另一台也一样,使用ping vip命令看看vip是否生效,如果没有生效,说明keepalived没有启动成功,回去检查keepalived的异常再说。
为了使用方便可以增加一个启动,停止haproxy的脚本

touch /usr/local/haproxy/sbin/starthaproxy chmod +x /usr/local/haproxy/sbin/starthaproxy touch /usr/local/haproxy/sbin/stophaproxy chmod +x /usr/local/haproxy/sbin/stophaproxy

启动脚本starthap内容如下:

#!/bin/sh /usr/local/haproxy/sbin/haproxy -f /usr/local/haproxy/haproxy.cfg &

停止脚本stophap内容如下

!/bin/sh

ps -ef | grep sbin/haproxy | grep -v grep |awk '{print $2}'|xargs kill -s 9

启动后可以通过http://172.17.210.103:48800/admin-status (用户名密码都是admin,haproxy.cfg中配置的)

7 openssl安装

openssl必须安装,否则安装keepalived时无法编译,keepalived依赖openssl。

tar zxvf openssl-1.0.1g.tar.gz ./config--prefix=/usr/local/openssl ./config-t make depend make make test make install ln -s /usr/local/openssl /usr/local/ssl

8 openssl配置

vi /etc/ld.so.conf

在/etc/ld.so.conf文件的最后面,添加如下内容:

 /usr/local/openssl/lib

vi /etc/profile
内容如下:

export OPENSSL=/usr/local/openssl/bin export PATH=$PATH:$OPENSSL

执行以下语句是环境变量生效:

source /etc/profile

安装openssl-devel

yum install openssl-devel -y #如无法yum下载安装,请修改yum配置文件

测试:

ldd /usr/local/openssl/bin/openssl linux-vdso.so.1 => (0x00007fff996b9000) libdl.so.2 =>/lib64/libdl.so.2 (0x00000030efc00000) libc.so.6 =>/lib64/libc.so.6 (0x00000030f0000000) /lib64/ld-linux-x86-64.so.2 (0x00000030ef800000) which openssl /usr/bin/opensslopenssl version OpenSSL 1.0.0-fips 29 Mar 2010

9 keepalived安装

本文在172.17.30.64、172.17.30.83两台机器进行keepalived安装
安装

tar zxvf keepalived-1.2.13.tar.gz cd keepalived-1.2.13 ./configure--prefix=/usr/local/keepalived make make install cp /usr/local/keepalived/sbin/keepalived /usr/sbin/ cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/ cp /usr/local/keepalived/etc/rc.d/init.d/keepalived/etc/init.d/ mkdir /etc/keepalived cd /etc/keepalived/ cp /usr/local/keepalived/etc/keepalived/keepalived.conf/etc/keepalived mkdir-p/usr/local/keepalived/var/log

10 keepalived配置

建检查haproxy是否存活的脚本

mkdir /etc/keepalived/scripts cd /etc/keepalived/scripts

keepalived.conf:

vi /etc/keepalived/keepalived.conf

Master:

! Configuration Fileforkeepalived vrrp_script chk_http_port { script"/etc/keepalived/scripts/check_haproxy.sh" interval 2weight 2 } vrrp_instance VI_1 { state MASTER #172.17.210.64上改为BACKUP interface eth0 #对外提供服务的网络接口 virtual_router_id 51 #VRRP组名,两个节点的设置必须一样,以指明各个节点属于同一VRRP组 priority 150 #数值愈大,优先级越高,172.17.210.64上改为120 advert_int 1 #同步通知间隔 authentication { #包含验证类型和验证密码。类型主要有PASS、AH两种,通常使用的类型为PASS,据说AH使用时有问题 auth_type PASS auth_pass 1111 } track_script { chk_http_port #调用脚本check_haproxy.sh检查haproxy是否存活 } virtual_ipaddress { #vip地址,这个ip必须与我们在lvs客户端设定的vip相一致 172.17.210.103 dev eth0 scope global } notify_master/etc/keepalived/scripts/haproxy_master.sh notify_backup/etc/keepalived/scripts/haproxy_backup.sh notify_fault /etc/keepalived/scripts/haproxy_fault.sh notify_stop /etc/keepalived/scripts/haproxy_stop.sh }

slave:

! Configuration Fileforkeepalived vrrp_script chk_http_port {script"/etc/keepalived/scripts/check_haproxy.sh" interval 2 weight 2 } vrrp_instance VI_1 { state BACKUP #172.17.210.83上改为MASTER  interface eth0 #对外提供服务的网络接口  virtual_router_id 51 #VRRP组名,两个节点的设置必须一样,以指明各个节点属于同一VRRP组  priority 120 #数值愈大,优先级越高,172.17.210.83上改为150  advert_int 1 #同步通知间隔  authentication { #包含验证类型和验证密码。类型主要有PASS、AH两种,通常使用的类型为PASS,据说AH使用时有问题 auth_type PASS auth_pass 1111 }  track_script { chk_http_port #调用脚本check_haproxy.sh检查haproxy是否存活 }  virtual_ipaddress { #vip地址,这个ip必须与我们在lvs客户端设定的vip相一致 172.17.210.103 dev eth0 scope global }  notify_master /etc/keepalived/scripts/haproxy_master.sh  notify_backup /etc/keepalived/scripts/haproxy_backup.sh  notify_fault /etc/keepalived/scripts/haproxy_fault.sh  notify_stop /etc/keepalived/scripts/haproxy_stop.sh }

注意:
1.virtual_router_id 51 这个代表一个集群组,如果同一个网段还有另一组集群,请使用不同的组编号区分。如换成52、53等。
2.interface eth1 和172.17.210.103 dev eth1 scope global中的eth1指的是网卡,如果是多网卡,可能会有eth0,eth1,eth2…,可以使用ifconfig命令查看,确保eth0是本机存在的网卡地址。有些服务器如果只有一个网卡,但被人为把eth0改成eth1了,你再写eth0就找不到了的。
check_haproxy.sh
vi /etc/keepalived/scripts/check_haproxy.sh
脚本含义:如果没有haproxy进程存在,就启动haproxy,停止keepalived
check_haproxy.sh

#!/bin/bash STARTHAPROXY="/usr/local/haproxy/sbin/haproxy -f /usr/local/haproxy/haproxy.cfg" STOPKEEPALIVED="/etc/init.d/keepalived stop" LOGFILE="/usr/local/keepalived/var/log/keepalived-haproxy-state.log" echo "[check_haproxy status]" >> $LOGFILE A=`ps-C haproxy --no-header |wc-l` echo "[check_haproxy status]" >> $LOGFILE date >> $LOGFILE if [ $A -eq 0 ];then echo $STARTHAPROXY >> $LOGFILE $STARTHAPROXY >> $LOGFILE 2>&1 sleep5 fi if [ `ps -C haproxy --no-header |wc-l` -eq0 ];then exit 0 else exit 1 fi

haproxy_master.sh(master和slave一样)
/etc/keepalived/scripts/haproxy_master.sh

#!/bin/bash STARTHAPROXY=`/usr/local/haproxy/sbin/haproxy-f/usr/local/haproxy/haproxy.cfg` STOPHAPROXY=`ps-ef |grep sbin/haproxy| grep -vgrep|awk'{print $2}'|xargskill-s 9` LOGFILE="/usr/local/keepalived/var/log/keepalived-haproxy-state.log" echo "[master]" >> $LOGFILE date >> $LOGFILE echo "Being master...." >> $LOGFILE 2>&1 echo "stop haproxy...." >> $LOGFILE 2>&1 $STOPHAPROXY >> $LOGFILE 2>&1 echo "start haproxy...." >> $LOGFILE 2>&1 $STARTHAPROXY >> $LOGFILE 2>&1 echo "haproxy stared ..." >> $LOGFILE

haproxy_backup.sh(master和slave一样)
/etc/keepalived/scripts/haproxy_backup.sh

#!/bin/bash STARTHAPROXY=`/usr/local/haproxy/sbin/haproxy-f/usr/local/haproxy/haproxy.cfg` STOPHAPROXY=`ps-ef |grep sbin/haproxy| grep -vgrep|awk'{print $2}'|xargskill-s 9` LOGFILE="/usr/local/keepalived/var/log/keepalived-haproxy-state.log" echo "[backup]" >> $LOGFILE date >> $LOGFILE echo "Being backup...." >> $LOGFILE 2>&1 echo "stop haproxy...." >> $LOGFILE 2>&1 $STOPHAPROXY >> $LOGFILE 2>&1 echo "start haproxy...." >> $LOGFILE 2>&1 $STARTHAPROXY >> $LOGFILE 2>&1 echo "haproxy stared ..." >> $LOGFILE

haproxy_fault.sh(master和slave一样)
/etc/keepalived/scripts/haproxy_fault.sh

#!/bin/bash LOGFILE=/usr/local/keepalived/var/log/keepalived-haproxy-state.log echo "[fault]" >> $LOGFILE date >> $LOGFILE

haproxy_stop.sh(master和slave一样)
/etc/keepalived/scripts/haproxy_stop.sh

#!/bin/bash LOGFILE=/usr/local/keepalived/var/log/keepalived-haproxy-state.log echo "[stop]" >> $LOGFILE date >> $LOGFILE

启用服务

service keepalived start
0 0