mysql MHA
来源:互联网 发布:论文数据库 怎么查 编辑:程序博客网 时间:2024/04/20 08:39
1,用到的工具
(1)、 Manager工具:
- masterha_check_ssh : 检查MHA的SSH配置。
- masterha_check_repl : 检查MySQL复制。
- masterha_manager : 启动MHA。
- masterha_check_status : 检测当前MHA运行状态。
- masterha_master_monitor : 监测master是否宕机。
- masterha_master_switch : 控制故障转移(自动或手动)。
- masterha_conf_host : 添加或删除配置的server信息。
- masterha_stop : 停止MHA
(2)、 Node工具(这些工具通常由MHAManager的脚本触发,无需人手操作)。
- save_binary_logs : 保存和复制master的二进制日志。
- apply_diff_relay_logs : 识别差异的中继日志事件并应用于其它slave。
- filter_mysqlbinlog : 去除不必要的ROLLBACK事件(MHA已不再使用这个工具)。
- purge_relay_logs : 清除中继日志(不会阻塞SQL线程)
2,主机:
ip MHA Mysql
192.168.97.118 Node Master
192.168.97.119 Node Slave
192.168.97.120 Manager Slave 软件node也安装
3,ssh
3.1 各个节点上产生密钥
cd ~/.ssh
ssh-keygen -t dsa -P '' -f id_dsa
3.2 在118上操作,将各个节点上的公钥copy到本机,再操作
scp 192.168.97.119:/root/.ssh/id_dsa.pub ./id_dsa.pub.119
scp 192.168.97.120:/root/.ssh/id_dsa.pub ./id_dsa.pub.120
cat ./id_dsa.pub.119 >> authorized_keys
cat ./id_dsa.pub.120 >> authorized_keys
3.3 查看生成的通用密钥,并copy其他节点
cat authorized_keys
scp authorized_keys 192.168.97.119:/root/.ssh/
scp authorized_keys 192.168.97.120:/root/.ssh/
3.4,验证任意两台节点无密钥登陆
ssh 192.168.118
ssh 192.168.119
ssh 192.168.120
3.5,主机名登陆
vim /etc/hosts 编辑追加
192.168.97.118 m1
192.168.97.119 s1
192.168.97.120 s2
验证主机名无密钥登陆:
ssh m1
ssh s1
ssh s2
4,搭建主从环境、配置账户
复制环境参考:http://blog.csdn.net/jc_benben/article/details/52836135
创建用户mha管理的账号,在所有mysql服务器上都需要执行:
GRANT SUPER,RELOAD,REPLICATION CLIENT,SELECT ON *.* TO manager@'192.168.97.%' IDENTIFIED BY 'manager_1234';
GRANT CREATE,INSERT,UPDATE,DELETE,DROP ON *.* TO manager@'192.168.97.%';
创建主从账号,在所有mysql服务器上都需要执行,注意如果主从上执行了,这里就不用了,这里也可以执行ip限制:
GRANT RELOAD, SUPER, REPLICATION SLAVE ON *.* TO 'repl'@'%' IDENTIFIED BY 'repl_1234';
5,安装MHA
mha包括manager节点和data节点,
data节点:包括原有的MySQL复制结构中的主机,至少3台,即1主2从,当masterfailover后,还能保证主从结构;只需安装node包。
manager server:运行监控脚本,负责monitoring 和 auto-failover;需要安装node包和manager包。
为了节省机器,可以从现有复制架构中选一台“闲置”从库作为manager server,比如:某台从库不对外提供读的服务,
只是作为候选主库,或是专门用于备份
5.1,mha4mysql-node 所有节点
tar -zxvpf mha4mysql-node-0.56.tar.gz
perl Makefile.PL
make && make install
需要首先安装一些依赖包,yum install perl-DBI perl-DBD-mysql perl-ExtUtils-CBuilder perl-ExtUtils-MakeMaker cpan -y
5.2,mha4mysql-manager (只管理节点)
yum install -y perl-DBD-MySQL
yum install -y perl-Config-Tiny
yum install -y perl-Log-Dispatch
yum install -y perl-Parallel-ForkManager
yum install -y perl-Config-IniFiles
这里有一个地址可以查找rpm包很不错的:https://pkgs.org/
安装:
tar -xvf mha4mysql-manager-0.56.tar.gz
cd mha4mysql-manager-0.56
perl Makefile.PL
make && make install
如果最后还是安装不好,可以尝试一下perl CPAN的方式:
perl -MCPAN -e shell
cpan[1]> install Log::Dispatch
6,编辑配置
mkdir /etc/masterha/
mkdir /var/log/masterha/app1
vim /etc/masterha/app1.cnf 添加内容
[server default]
user=manager #ssh免密钥登录的帐号名
password=manager_1234
manager_workdir=/var/log/masterha/app1
manager_log=/var/log/masterha/app1/manager.log
ssh_user=root
repl_user=repl #mysql复制帐号,用来在主从机之间同步二进制日志等
repl_password=123456
ping_interval=1 #间隔,用来检测master是否正常
[server1]
hostname=192.168.97.118
candidate_master=1 #master机宕掉后,优先启用这台作为新master
master_binlog_dir=/usr/local/mysql/data/
[server2]
hostname=192.168.97.119
#candidate_master=1
master_binlog_dir=/usr/local/mysql/data/
[server3]
hostname=192.168.97.120
candidate_master=1
master_binlog_dir=/usr/local/mysql/data/
#[server4]
#hostname=host4
#no_master=1 #永远不作为master
#以下是vip切换使用内容,可以暂不考虑
#master_ip_failover_script=/etc/masterha/master_ip_failover
#master_ip_online_change_script=/etc/masterha/master_ip_online_change
7,检测(2个就够了)
检查节点连接性:
[root@s2 .ssh]# masterha_check_ssh --conf=/etc/masterha/app1.cnf
检查repl环境:
[root@s2 ~]# masterha_check_repl --conf=/etc/masterha/app1.cnf
Fri Oct 21 11:43:37 2016 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Fri Oct 21 11:43:37 2016 - [info] Reading application default configuration from /etc/masterha/app1.cnf..
Fri Oct 21 11:43:37 2016 - [info] Reading server configuration from /etc/masterha/app1.cnf..
Fri Oct 21 11:43:37 2016 - [info] MHA::MasterMonitor version 0.56.
Fri Oct 21 11:43:37 2016 - [info] GTID failover mode = 0
Fri Oct 21 11:43:37 2016 - [info] Dead Servers:
Fri Oct 21 11:43:37 2016 - [info] Alive Servers:
Fri Oct 21 11:43:37 2016 - [info] 192.168.97.118(192.168.97.118:3306)
Fri Oct 21 11:43:37 2016 - [info] 192.168.97.119(192.168.97.119:3306)
Fri Oct 21 11:43:37 2016 - [info] 192.168.97.120(192.168.97.120:3306)
Fri Oct 21 11:43:37 2016 - [info] Alive Slaves:
Fri Oct 21 11:43:37 2016 - [info] 192.168.97.119(192.168.97.119:3306) Version=5.7.14-log (oldest major version between slaves) log-bin:enabled
Fri Oct 21 11:43:37 2016 - [info] Replicating from 192.168.97.118(192.168.97.118:3306)
Fri Oct 21 11:43:37 2016 - [info] 192.168.97.120(192.168.97.120:3306) Version=5.7.14-log (oldest major version between slaves) log-bin:enabled
Fri Oct 21 11:43:37 2016 - [info] Replicating from 192.168.97.118(192.168.97.118:3306)
Fri Oct 21 11:43:37 2016 - [info] Primary candidate for the new Master (candidate_master is set)
Fri Oct 21 11:43:37 2016 - [info] Current Alive Master: 192.168.97.118(192.168.97.118:3306)
Fri Oct 21 11:43:37 2016 - [info] Checking slave configurations..
Fri Oct 21 11:43:37 2016 - [info] Checking replication filtering settings..
Fri Oct 21 11:43:37 2016 - [info] binlog_do_db= t1, binlog_ignore_db= information_schema,mysql,performance_schema,t,test
Fri Oct 21 11:43:37 2016 - [info] Replication filtering check ok.
Fri Oct 21 11:43:37 2016 - [info] GTID (with auto-pos) is not supported
Fri Oct 21 11:43:37 2016 - [info] Starting SSH connection tests..
Fri Oct 21 11:43:39 2016 - [info] All SSH connection tests passed successfully.
Fri Oct 21 11:43:39 2016 - [info] Checking MHA Node version..
Fri Oct 21 11:43:40 2016 - [info] Version check ok.
Fri Oct 21 11:43:40 2016 - [info] Checking SSH publickey authentication settings on the current master..
Fri Oct 21 11:43:40 2016 - [info] HealthCheck: SSH to 192.168.97.118 is reachable.
Fri Oct 21 11:43:41 2016 - [info] Master MHA Node version is 0.56.
Fri Oct 21 11:43:41 2016 - [info] Checking recovery script configurations on 192.168.97.118(192.168.97.118:3306)..
Fri Oct 21 11:43:41 2016 - [info] Executing command: save_binary_logs --command=test --start_pos=4 --binlog_dir=/usr/local/mysql/data/ --output_file=/var/tmp/save_binary_logs_test --manager_version=0.56 --start_file=mysql-bin.000006
Fri Oct 21 11:43:41 2016 - [info] Connecting to root@192.168.97.118(192.168.97.118:22)..
Creating /var/tmp if not exists.. ok.
Checking output directory is accessible or not..
ok.
Binlog found at /usr/local/mysql/data/, up to mysql-bin.000006
Fri Oct 21 11:43:41 2016 - [info] Binlog setting check done.
Fri Oct 21 11:43:41 2016 - [info] Checking SSH publickey authentication and checking recovery script configurations on all alive slave servers..
Fri Oct 21 11:43:41 2016 - [info] Executing command : apply_diff_relay_logs --command=test --slave_user='manager' --slave_host=192.168.97.119 --slave_ip=192.168.97.119 --slave_port=3306 --workdir=/var/tmp --target_version=5.7.14-log --manager_version=0.56 --relay_log_info=/usr/local/mysql/data/relay-log.info --relay_dir=/usr/local/mysql/data/ --slave_pass=xxx
Fri Oct 21 11:43:41 2016 - [info] Connecting to root@192.168.97.119(192.168.97.119:22)..
Checking slave recovery environment settings..
Opening /usr/local/mysql/data/relay-log.info ... ok.
Relay log found at /usr/local/mysql/data, up to relay-bin.000034
Temporary relay log file is /usr/local/mysql/data/relay-bin.000034
Testing mysql connection and privileges..mysql: [Warning] Using a password on the command line interface can be insecure.
done.
Testing mysqlbinlog output.. done.
Cleaning up test file(s).. done.
Fri Oct 21 11:43:41 2016 - [info] Executing command : apply_diff_relay_logs --command=test --slave_user='manager' --slave_host=192.168.97.120 --slave_ip=192.168.97.120 --slave_port=3306 --workdir=/var/tmp --target_version=5.7.14-log --manager_version=0.56 --relay_log_info=/usr/local/mysql/data/relay-log.info --relay_dir=/usr/local/mysql/data/ --slave_pass=xxx
Fri Oct 21 11:43:41 2016 - [info] Connecting to root@192.168.97.120(192.168.97.120:22)..
Checking slave recovery environment settings..
Opening /usr/local/mysql/data/relay-log.info ... ok.
Relay log found at /usr/local/mysql/data, up to relay-bin.000038
Temporary relay log file is /usr/local/mysql/data/relay-bin.000038
Testing mysql connection and privileges..mysql: [Warning] Using a password on the command line interface can be insecure.
done.
Testing mysqlbinlog output.. done.
Cleaning up test file(s).. done.
Fri Oct 21 11:43:42 2016 - [info] Slaves settings check done.
Fri Oct 21 11:43:42 2016 - [info]
192.168.97.118(192.168.97.118:3306) (current master)
+--192.168.97.119(192.168.97.119:3306)
+--192.168.97.120(192.168.97.120:3306)
Fri Oct 21 11:43:42 2016 - [info] Checking replication health on 192.168.97.119..
Fri Oct 21 11:43:42 2016 - [info] ok.
Fri Oct 21 11:43:42 2016 - [info] Checking replication health on 192.168.97.120..
Fri Oct 21 11:43:42 2016 - [info] ok.
Fri Oct 21 11:43:42 2016 - [warning] master_ip_failover_script is not defined.
Fri Oct 21 11:43:42 2016 - [warning] shutdown_script is not defined.
Fri Oct 21 11:43:42 2016 - [info] Got exit code 0 (Not master dead).
MySQL Replication Health is OK.
8,管理MHA
启动manager
nohup masterha_manager --conf=/etc/masterha/app1.cnf < /dev/null >/logs/mha/app1/manager.log 2>&1 &
查看状态:
masterha_check_status--conf=/etc/masterha/app1.cnf
停止manager:
masterha_stop --conf=/etc/masterha/app1.cnf
9,实现VIP切换
背景:MHA能够在10~30秒内实现自动故障检测和故障转移,这适用于对高可用性,数据完整性要求较高的场合。要做到无缝切换,
还需要依赖于VIP漂移。VIP漂移比较常用的方式为使用keepalived或者使用脚本直接实现。脚本方式无须安装及复杂配置,相对简单
这里采用脚本。
9.1 在master节点上添加一虚拟ip,有几种方式
第一:vim /etc/sysconfig/network-scripts/ifcfg-eno16777736 在文件中添加一ip
IPADDR1=192.168.97.124
重启网卡
systemctl restart network或者nmcli conn up "eno16777736"
第二种,
[root@m1 ~]# nmcli conn modify "eno16777736" +ipv4.address "192.168.97.129/24 "
[root@m1 ~]# nmcli conn up "eno16777736"
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/3)
centos7之前的临时方式,不推荐:
/sbin/ifconfig eno16777736:0 192.168.97.126 netmask 255.255.255.0 up
9.2,在管理节点上重新配置文件app1.cnf
master_ip_failover_script=/etc/masterha/master_ip_failover
master_ip_online_change_script=/etc/masterha/master_ip_online_change
report_script= /etc/masterha/app1/send_report #master failover时执行,有邮件通知需求是部署
9.3,部署脚本
脚本master_ip_failover,内容如下:
#!/usr/bin/env perl
use strict;
use warnings FATAL =>'all';
use Getopt::Long;
my (
$command, $ssh_user, $orig_master_host, $orig_master_ip,
$orig_master_port, $new_master_host, $new_master_ip, $new_master_port
);
my $vip = '192.168.97.124/24'; # Virtual IP
my $key = "1";
my $ssh_start_vip = "/sbin/ifconfig eno16777736:$key $vip";
my $ssh_stop_vip = "/sbin/ifconfig eno16777736:$key down";
my $exit_code = 0;
GetOptions(
'command=s' => \$command,
'ssh_user=s' => \$ssh_user,
'orig_master_host=s' => \$orig_master_host,
'orig_master_ip=s' => \$orig_master_ip,
'orig_master_port=i' => \$orig_master_port,
'new_master_host=s' => \$new_master_host,
'new_master_ip=s' => \$new_master_ip,
'new_master_port=i' => \$new_master_port,
);
exit &main();
sub main {
#print "\n\nIN SCRIPT TEST====$ssh_stop_vip==$ssh_start_vip===\n\n";
if ( $command eq "stop" || $command eq "stopssh" ) {
# $orig_master_host, $orig_master_ip, $orig_master_port are passed.
# If you manage master ip address at global catalog database,
# invalidate orig_master_ip here.
my $exit_code = 1;
eval {
print "\n\n\n***************************************************************\n";
print "Disabling the VIP - $vip on old master: $orig_master_host\n";
print "***************************************************************\n\n\n\n";
&stop_vip();
$exit_code = 0;
};
if ($@) {
warn "Got Error: $@\n";
exit $exit_code;
}
exit $exit_code;
}
elsif ( $command eq "start" ) {
# all arguments are passed.
# If you manage master ip address at global catalog database,
# activate new_master_ip here.
# You can also grant write access (create user, set read_only=0, etc) here.
my $exit_code = 10;
eval {
print "\n\n\n***************************************************************\n";
print "Enabling the VIP - $vip on new master: $new_master_host \n";
print "***************************************************************\n\n\n\n";
&start_vip();
$exit_code = 0;
};
if ($@) {
warn $@;
exit $exit_code;
}
exit $exit_code;
}
elsif ( $command eq "status" ) {
print "Checking the Status of the script.. OK \n";
`ssh $ssh_user\@$orig_master_host \" $ssh_start_vip \"`;
exit 0;
}
else {
&usage();
exit 1;
}
}
# A simple system call that enable the VIP on the new master
sub start_vip() {
`ssh $ssh_user\@$new_master_host \" $ssh_start_vip \"`;
}
# A simple system call that disable the VIP on the old_master
sub stop_vip() {
`ssh $ssh_user\@$orig_master_host \" $ssh_stop_vip \"`;
}
sub usage {
print
"Usage: master_ip_failover –command=start|stop|stopssh|status –orig_master_host=host –orig_master_ip=ip –orig_master_port=po
rt –new_master_host=host –new_master_ip=ip –new_master_port=port\n";
}
脚本master_ip_online_change(shell),内容:
#/bin/bash
source /root/.bash_profile
vip=`echo '192.168.97.124/24'` # Virtual IP
key=`echo '1'`
command=`echo "$1" | awk -F = '{print $2}'`
orig_master_host=`echo "$2" | awk -F = '{print $2}'`
new_master_host=`echo "$7" | awk -F = '{print $2}'`
orig_master_ssh_user=`echo "${12}" | awk -F = '{print $2}'`
new_master_ssh_user=`echo "${13}" | awk -F = '{print $2}'`
stop_vip=`echo "ssh root@$orig_master_host /sbin/ifconfig eno16777736:$key down"`
start_vip=`echo "ssh root@$new_master_host /sbin/ifconfig eno16777736:$key $vip"`
if [ $command = 'stop' ]
then
echo -e "\n\n\n***************************************************************\n"
echo -e "Disabling the VIP - $vip on old master: $orig_master_host\n"
$stop_vip
if [ $? -eq 0 ]
then
echo "Disabled the VIP successfully"
else
echo "Disabled the VIP failed"
fi
echo -e "***************************************************************\n\n\n\n"
fi
if [ $command = 'start' -o $command = 'status' ]
then
echo -e "\n\n\n***************************************************************\n"
echo -e "Enabling the VIP - $vip on new master: $new_master_host \n"
$start_vip
if [ $? -eq 0 ]
then
echo "Enabled the VIP successfully"
else
echo "Enabled the VIP failed"
fi
echo -e "***************************************************************\n\n\n\n"
fi
注意,这个脚本也可以使用perl版本
#!/usr/bin/env perl
use strict;
use warnings FATAL =>'all';
use Getopt::Long;
my $vip = '10.1.5.21/24'; # Virtual IP
my $key = "1";
my $ssh_start_vip = "/sbin/ifconfig eth0:$key $vip";
my $ssh_stop_vip = "/sbin/ifconfig eth0:$key down";
my $exit_code = 0;
my (
$command, $orig_master_is_new_slave, $orig_master_host,
$orig_master_ip, $orig_master_port, $orig_master_user,
$orig_master_password, $orig_master_ssh_user, $new_master_host,
$new_master_ip, $new_master_port, $new_master_user,
$new_master_password, $new_master_ssh_user,
);
GetOptions(
'command=s' => \$command,
'orig_master_is_new_slave' => \$orig_master_is_new_slave,
'orig_master_host=s' => \$orig_master_host,
'orig_master_ip=s' => \$orig_master_ip,
'orig_master_port=i' => \$orig_master_port,
'orig_master_user=s' => \$orig_master_user,
'orig_master_password=s' => \$orig_master_password,
'orig_master_ssh_user=s' => \$orig_master_ssh_user,
'new_master_host=s' => \$new_master_host,
'new_master_ip=s' => \$new_master_ip,
'new_master_port=i' => \$new_master_port,
'new_master_user=s' => \$new_master_user,
'new_master_password=s' => \$new_master_password,
'new_master_ssh_user=s' => \$new_master_ssh_user,
);
exit &main();
sub main {
#print "\n\nIN SCRIPT TEST====$ssh_stop_vip==$ssh_start_vip===\n\n";
if ( $command eq "stop" || $command eq "stopssh" ) {
# $orig_master_host, $orig_master_ip, $orig_master_port are passed.
# If you manage master ip address at global catalog database,
# invalidate orig_master_ip here.
my $exit_code = 1;
eval {
print "\n\n\n***************************************************************\n";
print "Disabling the VIP - $vip on old master: $orig_master_host\n";
print "***************************************************************\n\n\n\n";
&stop_vip();
$exit_code = 0;
};
if ($@) {
warn "Got Error: $@\n";
exit $exit_code;
}
exit $exit_code;
}
elsif ( $command eq "start" ) {
# all arguments are passed.
# If you manage master ip address at global catalog database,
# activate new_master_ip here.
# You can also grant write access (create user, set read_only=0, etc) here.
my $exit_code = 10;
eval {
print "\n\n\n***************************************************************\n";
print "Enabling the VIP - $vip on new master: $new_master_host \n";
print "***************************************************************\n\n\n\n";
&start_vip();
$exit_code = 0;
};
if ($@) {
warn $@;
exit $exit_code;
}
exit $exit_code;
}
elsif ( $command eq "status" ) {
print "Checking the Status of the script.. OK \n";
`ssh $orig_master_ssh_user\@$orig_master_host \" $ssh_start_vip \"`;
exit 0;
}
else {
&usage();
exit 1;
}
}
# A simple system call that enable the VIP on the new master
sub start_vip() {
`ssh $new_master_ssh_user\@$new_master_host \" $ssh_start_vip \"`;
}
# A simple system call that disable the VIP on the old_master
sub stop_vip() {
`ssh $orig_master_ssh_user\@$orig_master_host \" $ssh_stop_vip \"`;
}
sub usage {
print
"Usage: master_ip_failover –command=start|stop|stopssh|status –orig_master_host=host –orig_master_ip=ip –orig_master_port=po
rt –new_master_host=host –new_master_ip=ip –new_master_port=port\n";
}
脚本send_report(shell),内容:
#/bin/bash
source /root/.bash_profile
orig_master_host=`echo "$1" | awk -F = '{print $2}'`
new_master_host=`echo "$2" | awk -F = '{print $2}'`
new_slave_hosts=`echo "$3" | awk -F = '{print $2}'`
subject=`echo "$4" | awk -F = '{print $2}'`
body=`echo "$5" | awk -F = '{print $2}'`
#判断日志结尾是否有successfully,有则表示切换成功,成功与否都发邮件。
tac /etc/masterha/app1/manager.log | sed -n 2p | grep 'successfully' > /dev/null
if [ $? -eq 0 ]
then
echo -e "MHA $subject 主从切换成功\n master:$orig_master_host --> $new_master_host \n $body \n 当前从库:$new_slave_hosts" | mutt
-s "MySQL实例宕掉,MHA $subject 切换成功" -- xxx@qq.com
else
echo -e "MHA $subject 主从切换失败\n master:$orig_master_host --> $new_master_host \n $body" | mutt -s "MySQL实例宕掉,MHA $subje
ct 切换失败" -- xxx@qq.com
fi
9,测试
9.1在原主节点97.118上执行停止操作,删除虚拟ip
systemctl stop mysqld_safe
9.2,在管理节点上查看状态
[root@s2 masterha]# tail -10f /var/log/masterha/app1/manager.log
Started automated(non-interactive) failover.
The latest slave 192.168.97.119(192.168.97.119:3306) has all relay logs for recovery.
Selected 192.168.97.120(192.168.97.120:3306) as a new master.
192.168.97.120(192.168.97.120:3306): OK: Applying all logs succeeded.
192.168.97.120(192.168.97.120:3306): OK: Activated master IP address.
192.168.97.119(192.168.97.119:3306): This host has the latest relay log events.
Generating relay diff files from the latest slave succeeded.
192.168.97.119(192.168.97.119:3306): OK: Applying all logs succeeded. Slave started, replicating from 192.168.97.120(192.168.97.120:3306)
192.168.97.120(192.168.97.120:3306): Resetting slave info succeeded.
Master failover to 192.168.97.120(192.168.97.120:3306) completed successfully.
可以看到,master正常切换到了97.120
9.3,使用虚拟地址连接mysql验证切换正常
mysql -uroot -p123456 -h 192.168.97.124
OK 正常
9.4,连接s1节点查看
mysql> show slave status \G;
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.97.120
Master_User: repl
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000007
Read_Master_Log_Pos: 154
Relay_Log_File: relay-bin.000002
Relay_Log_Pos: 320
Relay_Master_Log_File: mysql-bin.000007
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 154
Relay_Log_Space: 521
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 137
Master_UUID: 49bb8923-8eb2-11e6-b687-000c29b86833
Master_Info_File: /usr/local/mysql/data/master.info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set:
Executed_Gtid_Set:
Auto_Position: 0
Replicate_Rewrite_DB:
Channel_Name:
Master_TLS_Version:
1 row in set (0.00 sec)
看到心得master节点为192.168.97.120
10,一些错误
10.1,执行:masterha_check_repl --conf=/etc/masterha/app1.cnf出现
Can't exec "mysqlbinlog": No suchfile or directory at /usr/local/share/perl5/MHA/BinlogManager.pm line 106.
mysqlbinlog version command failed with rc1:0, please verify PATH, LD_LIBRARY_PATH, and client options
解决:
type mysqlbinlog
然后:ln -s /usr/local/mysql/bin/mysqlbinlog /usr/bin/mysqlbinlog
注意/usr/local/mysql/bin/mysqlbinlog是type mysqlbinlog的结果
10.2,执行:masterha_check_repl --conf=/etc/masterha/app1.cnf出现
Testing mysql connection and privileges..sh: mysql: command not found
mysql command failed with rc 127:0!
解决:ln -s /usr/local/mysql/bin/mysql /usr/bin/mysql
11,
1,简介
1.1mha简介
MHA,即MasterHigh Availability Manager and Tools for MySQL,是日本的一位MySQL专家采用Perl语言编写的一个脚本管理工具,该工具仅适用于MySQLReplication(二层)环境,目的在于维持Master主库的高可用性。
MHA(Master High Availability)是自动的master故障转移和Slave提升的软件包.它是基于标准的MySQL复制(异步/半同步).
MHA有两部分组成:MHA Manager(管理节点)和MHA Node(数据节点).
MHA Manager可以单独部署在一台独立机器上管理多个master-slave集群,也可以部署在一台slave上.MHA Manager探测集群的node节点,当发现master出现故障的时候,它可以自动将具有最新数据的slave提升为新的master,然后将所有其它的slave导向新的master上.整个故障转移过程对应用程序是透明的。
MHA node运行在每台MySQL服务器上(master/slave/manager),它通过监控具备解析和清理logs功能的脚本来加快故障转移的。
1.2,mha特点
(1). 10-30s实现master failover(9-12s可以检测到主机故障,7-10s可以关闭主机避免SB,在用很短的时间应用差异日志)
(2) 部署简单,无需对现有M-S结构做任何改动(至少3台,保证切换后仍保持M-S结构)
(3). 支持手动在线切换(主机硬件维护),downtime几乎很短0.5-2s
(4). 保证故障切换后多从库数据的一致性
(5). 完全自动化的failover及快速复制架构恢复方案(一主多从)
(6). 恢复过程包括:选择新主库、确认从库间relaylog差异、新主库应用必要语句、其他从库同步差异语句、重新建立复制连接
1.3,工作原理
相较于其它HA软件,MHA的目的在于维持MySQL Replication中Master库的高可用性,其最大特点是可以修复多个Slave之间的差异日志,最终使所有Slave保持数据一致,然后从中选择一个充当新的Master,并将其它Slave指向它。
-从宕机崩溃的master保存二进制日志事件(binlogevents)。
-识别含有最新更新的slave。
-应用差异的中继日志(relay log)到其它slave。
-应用从master保存的二进制日志事件(binlogevents)。
-提升一个slave为新master。
-使其它的slave连接新的master进行复制。
说明:本文参考互联网
(1)、 Manager工具:
- masterha_check_ssh : 检查MHA的SSH配置。
- masterha_check_repl : 检查MySQL复制。
- masterha_manager : 启动MHA。
- masterha_check_status : 检测当前MHA运行状态。
- masterha_master_monitor : 监测master是否宕机。
- masterha_master_switch : 控制故障转移(自动或手动)。
- masterha_conf_host : 添加或删除配置的server信息。
- masterha_stop : 停止MHA
(2)、 Node工具(这些工具通常由MHAManager的脚本触发,无需人手操作)。
- save_binary_logs : 保存和复制master的二进制日志。
- apply_diff_relay_logs : 识别差异的中继日志事件并应用于其它slave。
- filter_mysqlbinlog : 去除不必要的ROLLBACK事件(MHA已不再使用这个工具)。
- purge_relay_logs : 清除中继日志(不会阻塞SQL线程)
2,主机:
ip MHA Mysql
192.168.97.118 Node Master
192.168.97.119 Node Slave
192.168.97.120 Manager Slave 软件node也安装
3,ssh
3.1 各个节点上产生密钥
cd ~/.ssh
ssh-keygen -t dsa -P '' -f id_dsa
3.2 在118上操作,将各个节点上的公钥copy到本机,再操作
scp 192.168.97.119:/root/.ssh/id_dsa.pub ./id_dsa.pub.119
scp 192.168.97.120:/root/.ssh/id_dsa.pub ./id_dsa.pub.120
cat ./id_dsa.pub.119 >> authorized_keys
cat ./id_dsa.pub.120 >> authorized_keys
3.3 查看生成的通用密钥,并copy其他节点
cat authorized_keys
scp authorized_keys 192.168.97.119:/root/.ssh/
scp authorized_keys 192.168.97.120:/root/.ssh/
3.4,验证任意两台节点无密钥登陆
ssh 192.168.118
ssh 192.168.119
ssh 192.168.120
3.5,主机名登陆
vim /etc/hosts 编辑追加
192.168.97.118 m1
192.168.97.119 s1
192.168.97.120 s2
验证主机名无密钥登陆:
ssh m1
ssh s1
ssh s2
4,搭建主从环境、配置账户
复制环境参考:http://blog.csdn.net/jc_benben/article/details/52836135
创建用户mha管理的账号,在所有mysql服务器上都需要执行:
GRANT SUPER,RELOAD,REPLICATION CLIENT,SELECT ON *.* TO manager@'192.168.97.%' IDENTIFIED BY 'manager_1234';
GRANT CREATE,INSERT,UPDATE,DELETE,DROP ON *.* TO manager@'192.168.97.%';
创建主从账号,在所有mysql服务器上都需要执行,注意如果主从上执行了,这里就不用了,这里也可以执行ip限制:
GRANT RELOAD, SUPER, REPLICATION SLAVE ON *.* TO 'repl'@'%' IDENTIFIED BY 'repl_1234';
5,安装MHA
mha包括manager节点和data节点,
data节点:包括原有的MySQL复制结构中的主机,至少3台,即1主2从,当masterfailover后,还能保证主从结构;只需安装node包。
manager server:运行监控脚本,负责monitoring 和 auto-failover;需要安装node包和manager包。
为了节省机器,可以从现有复制架构中选一台“闲置”从库作为manager server,比如:某台从库不对外提供读的服务,
只是作为候选主库,或是专门用于备份
5.1,mha4mysql-node 所有节点
tar -zxvpf mha4mysql-node-0.56.tar.gz
perl Makefile.PL
make && make install
需要首先安装一些依赖包,yum install perl-DBI perl-DBD-mysql perl-ExtUtils-CBuilder perl-ExtUtils-MakeMaker cpan -y
5.2,mha4mysql-manager (只管理节点)
yum install -y perl-DBD-MySQL
yum install -y perl-Config-Tiny
yum install -y perl-Log-Dispatch
yum install -y perl-Parallel-ForkManager
yum install -y perl-Config-IniFiles
这里有一个地址可以查找rpm包很不错的:https://pkgs.org/
安装:
tar -xvf mha4mysql-manager-0.56.tar.gz
cd mha4mysql-manager-0.56
perl Makefile.PL
make && make install
如果最后还是安装不好,可以尝试一下perl CPAN的方式:
perl -MCPAN -e shell
cpan[1]> install Log::Dispatch
6,编辑配置
mkdir /etc/masterha/
mkdir /var/log/masterha/app1
vim /etc/masterha/app1.cnf 添加内容
[server default]
user=manager #ssh免密钥登录的帐号名
password=manager_1234
manager_workdir=/var/log/masterha/app1
manager_log=/var/log/masterha/app1/manager.log
ssh_user=root
repl_user=repl #mysql复制帐号,用来在主从机之间同步二进制日志等
repl_password=123456
ping_interval=1 #间隔,用来检测master是否正常
[server1]
hostname=192.168.97.118
candidate_master=1 #master机宕掉后,优先启用这台作为新master
master_binlog_dir=/usr/local/mysql/data/
[server2]
hostname=192.168.97.119
#candidate_master=1
master_binlog_dir=/usr/local/mysql/data/
[server3]
hostname=192.168.97.120
candidate_master=1
master_binlog_dir=/usr/local/mysql/data/
#[server4]
#hostname=host4
#no_master=1 #永远不作为master
#以下是vip切换使用内容,可以暂不考虑
#master_ip_failover_script=/etc/masterha/master_ip_failover
#master_ip_online_change_script=/etc/masterha/master_ip_online_change
7,检测(2个就够了)
检查节点连接性:
[root@s2 .ssh]# masterha_check_ssh --conf=/etc/masterha/app1.cnf
检查repl环境:
[root@s2 ~]# masterha_check_repl --conf=/etc/masterha/app1.cnf
Fri Oct 21 11:43:37 2016 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Fri Oct 21 11:43:37 2016 - [info] Reading application default configuration from /etc/masterha/app1.cnf..
Fri Oct 21 11:43:37 2016 - [info] Reading server configuration from /etc/masterha/app1.cnf..
Fri Oct 21 11:43:37 2016 - [info] MHA::MasterMonitor version 0.56.
Fri Oct 21 11:43:37 2016 - [info] GTID failover mode = 0
Fri Oct 21 11:43:37 2016 - [info] Dead Servers:
Fri Oct 21 11:43:37 2016 - [info] Alive Servers:
Fri Oct 21 11:43:37 2016 - [info] 192.168.97.118(192.168.97.118:3306)
Fri Oct 21 11:43:37 2016 - [info] 192.168.97.119(192.168.97.119:3306)
Fri Oct 21 11:43:37 2016 - [info] 192.168.97.120(192.168.97.120:3306)
Fri Oct 21 11:43:37 2016 - [info] Alive Slaves:
Fri Oct 21 11:43:37 2016 - [info] 192.168.97.119(192.168.97.119:3306) Version=5.7.14-log (oldest major version between slaves) log-bin:enabled
Fri Oct 21 11:43:37 2016 - [info] Replicating from 192.168.97.118(192.168.97.118:3306)
Fri Oct 21 11:43:37 2016 - [info] 192.168.97.120(192.168.97.120:3306) Version=5.7.14-log (oldest major version between slaves) log-bin:enabled
Fri Oct 21 11:43:37 2016 - [info] Replicating from 192.168.97.118(192.168.97.118:3306)
Fri Oct 21 11:43:37 2016 - [info] Primary candidate for the new Master (candidate_master is set)
Fri Oct 21 11:43:37 2016 - [info] Current Alive Master: 192.168.97.118(192.168.97.118:3306)
Fri Oct 21 11:43:37 2016 - [info] Checking slave configurations..
Fri Oct 21 11:43:37 2016 - [info] Checking replication filtering settings..
Fri Oct 21 11:43:37 2016 - [info] binlog_do_db= t1, binlog_ignore_db= information_schema,mysql,performance_schema,t,test
Fri Oct 21 11:43:37 2016 - [info] Replication filtering check ok.
Fri Oct 21 11:43:37 2016 - [info] GTID (with auto-pos) is not supported
Fri Oct 21 11:43:37 2016 - [info] Starting SSH connection tests..
Fri Oct 21 11:43:39 2016 - [info] All SSH connection tests passed successfully.
Fri Oct 21 11:43:39 2016 - [info] Checking MHA Node version..
Fri Oct 21 11:43:40 2016 - [info] Version check ok.
Fri Oct 21 11:43:40 2016 - [info] Checking SSH publickey authentication settings on the current master..
Fri Oct 21 11:43:40 2016 - [info] HealthCheck: SSH to 192.168.97.118 is reachable.
Fri Oct 21 11:43:41 2016 - [info] Master MHA Node version is 0.56.
Fri Oct 21 11:43:41 2016 - [info] Checking recovery script configurations on 192.168.97.118(192.168.97.118:3306)..
Fri Oct 21 11:43:41 2016 - [info] Executing command: save_binary_logs --command=test --start_pos=4 --binlog_dir=/usr/local/mysql/data/ --output_file=/var/tmp/save_binary_logs_test --manager_version=0.56 --start_file=mysql-bin.000006
Fri Oct 21 11:43:41 2016 - [info] Connecting to root@192.168.97.118(192.168.97.118:22)..
Creating /var/tmp if not exists.. ok.
Checking output directory is accessible or not..
ok.
Binlog found at /usr/local/mysql/data/, up to mysql-bin.000006
Fri Oct 21 11:43:41 2016 - [info] Binlog setting check done.
Fri Oct 21 11:43:41 2016 - [info] Checking SSH publickey authentication and checking recovery script configurations on all alive slave servers..
Fri Oct 21 11:43:41 2016 - [info] Executing command : apply_diff_relay_logs --command=test --slave_user='manager' --slave_host=192.168.97.119 --slave_ip=192.168.97.119 --slave_port=3306 --workdir=/var/tmp --target_version=5.7.14-log --manager_version=0.56 --relay_log_info=/usr/local/mysql/data/relay-log.info --relay_dir=/usr/local/mysql/data/ --slave_pass=xxx
Fri Oct 21 11:43:41 2016 - [info] Connecting to root@192.168.97.119(192.168.97.119:22)..
Checking slave recovery environment settings..
Opening /usr/local/mysql/data/relay-log.info ... ok.
Relay log found at /usr/local/mysql/data, up to relay-bin.000034
Temporary relay log file is /usr/local/mysql/data/relay-bin.000034
Testing mysql connection and privileges..mysql: [Warning] Using a password on the command line interface can be insecure.
done.
Testing mysqlbinlog output.. done.
Cleaning up test file(s).. done.
Fri Oct 21 11:43:41 2016 - [info] Executing command : apply_diff_relay_logs --command=test --slave_user='manager' --slave_host=192.168.97.120 --slave_ip=192.168.97.120 --slave_port=3306 --workdir=/var/tmp --target_version=5.7.14-log --manager_version=0.56 --relay_log_info=/usr/local/mysql/data/relay-log.info --relay_dir=/usr/local/mysql/data/ --slave_pass=xxx
Fri Oct 21 11:43:41 2016 - [info] Connecting to root@192.168.97.120(192.168.97.120:22)..
Checking slave recovery environment settings..
Opening /usr/local/mysql/data/relay-log.info ... ok.
Relay log found at /usr/local/mysql/data, up to relay-bin.000038
Temporary relay log file is /usr/local/mysql/data/relay-bin.000038
Testing mysql connection and privileges..mysql: [Warning] Using a password on the command line interface can be insecure.
done.
Testing mysqlbinlog output.. done.
Cleaning up test file(s).. done.
Fri Oct 21 11:43:42 2016 - [info] Slaves settings check done.
Fri Oct 21 11:43:42 2016 - [info]
192.168.97.118(192.168.97.118:3306) (current master)
+--192.168.97.119(192.168.97.119:3306)
+--192.168.97.120(192.168.97.120:3306)
Fri Oct 21 11:43:42 2016 - [info] Checking replication health on 192.168.97.119..
Fri Oct 21 11:43:42 2016 - [info] ok.
Fri Oct 21 11:43:42 2016 - [info] Checking replication health on 192.168.97.120..
Fri Oct 21 11:43:42 2016 - [info] ok.
Fri Oct 21 11:43:42 2016 - [warning] master_ip_failover_script is not defined.
Fri Oct 21 11:43:42 2016 - [warning] shutdown_script is not defined.
Fri Oct 21 11:43:42 2016 - [info] Got exit code 0 (Not master dead).
MySQL Replication Health is OK.
8,管理MHA
启动manager
nohup masterha_manager --conf=/etc/masterha/app1.cnf < /dev/null >/logs/mha/app1/manager.log 2>&1 &
查看状态:
masterha_check_status--conf=/etc/masterha/app1.cnf
停止manager:
masterha_stop --conf=/etc/masterha/app1.cnf
9,实现VIP切换
背景:MHA能够在10~30秒内实现自动故障检测和故障转移,这适用于对高可用性,数据完整性要求较高的场合。要做到无缝切换,
还需要依赖于VIP漂移。VIP漂移比较常用的方式为使用keepalived或者使用脚本直接实现。脚本方式无须安装及复杂配置,相对简单
这里采用脚本。
9.1 在master节点上添加一虚拟ip,有几种方式
第一:vim /etc/sysconfig/network-scripts/ifcfg-eno16777736 在文件中添加一ip
IPADDR1=192.168.97.124
重启网卡
systemctl restart network或者nmcli conn up "eno16777736"
第二种,
[root@m1 ~]# nmcli conn modify "eno16777736" +ipv4.address "192.168.97.129/24 "
[root@m1 ~]# nmcli conn up "eno16777736"
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/3)
centos7之前的临时方式,不推荐:
/sbin/ifconfig eno16777736:0 192.168.97.126 netmask 255.255.255.0 up
9.2,在管理节点上重新配置文件app1.cnf
master_ip_failover_script=/etc/masterha/master_ip_failover
master_ip_online_change_script=/etc/masterha/master_ip_online_change
report_script= /etc/masterha/app1/send_report #master failover时执行,有邮件通知需求是部署
9.3,部署脚本
脚本master_ip_failover,内容如下:
#!/usr/bin/env perl
use strict;
use warnings FATAL =>'all';
use Getopt::Long;
my (
$command, $ssh_user, $orig_master_host, $orig_master_ip,
$orig_master_port, $new_master_host, $new_master_ip, $new_master_port
);
my $vip = '192.168.97.124/24'; # Virtual IP
my $key = "1";
my $ssh_start_vip = "/sbin/ifconfig eno16777736:$key $vip";
my $ssh_stop_vip = "/sbin/ifconfig eno16777736:$key down";
my $exit_code = 0;
GetOptions(
'command=s' => \$command,
'ssh_user=s' => \$ssh_user,
'orig_master_host=s' => \$orig_master_host,
'orig_master_ip=s' => \$orig_master_ip,
'orig_master_port=i' => \$orig_master_port,
'new_master_host=s' => \$new_master_host,
'new_master_ip=s' => \$new_master_ip,
'new_master_port=i' => \$new_master_port,
);
exit &main();
sub main {
#print "\n\nIN SCRIPT TEST====$ssh_stop_vip==$ssh_start_vip===\n\n";
if ( $command eq "stop" || $command eq "stopssh" ) {
# $orig_master_host, $orig_master_ip, $orig_master_port are passed.
# If you manage master ip address at global catalog database,
# invalidate orig_master_ip here.
my $exit_code = 1;
eval {
print "\n\n\n***************************************************************\n";
print "Disabling the VIP - $vip on old master: $orig_master_host\n";
print "***************************************************************\n\n\n\n";
&stop_vip();
$exit_code = 0;
};
if ($@) {
warn "Got Error: $@\n";
exit $exit_code;
}
exit $exit_code;
}
elsif ( $command eq "start" ) {
# all arguments are passed.
# If you manage master ip address at global catalog database,
# activate new_master_ip here.
# You can also grant write access (create user, set read_only=0, etc) here.
my $exit_code = 10;
eval {
print "\n\n\n***************************************************************\n";
print "Enabling the VIP - $vip on new master: $new_master_host \n";
print "***************************************************************\n\n\n\n";
&start_vip();
$exit_code = 0;
};
if ($@) {
warn $@;
exit $exit_code;
}
exit $exit_code;
}
elsif ( $command eq "status" ) {
print "Checking the Status of the script.. OK \n";
`ssh $ssh_user\@$orig_master_host \" $ssh_start_vip \"`;
exit 0;
}
else {
&usage();
exit 1;
}
}
# A simple system call that enable the VIP on the new master
sub start_vip() {
`ssh $ssh_user\@$new_master_host \" $ssh_start_vip \"`;
}
# A simple system call that disable the VIP on the old_master
sub stop_vip() {
`ssh $ssh_user\@$orig_master_host \" $ssh_stop_vip \"`;
}
sub usage {
"Usage: master_ip_failover –command=start|stop|stopssh|status –orig_master_host=host –orig_master_ip=ip –orig_master_port=po
rt –new_master_host=host –new_master_ip=ip –new_master_port=port\n";
}
脚本master_ip_online_change(shell),内容:
#/bin/bash
source /root/.bash_profile
vip=`echo '192.168.97.124/24'` # Virtual IP
key=`echo '1'`
command=`echo "$1" | awk -F = '{print $2}'`
orig_master_host=`echo "$2" | awk -F = '{print $2}'`
new_master_host=`echo "$7" | awk -F = '{print $2}'`
orig_master_ssh_user=`echo "${12}" | awk -F = '{print $2}'`
new_master_ssh_user=`echo "${13}" | awk -F = '{print $2}'`
stop_vip=`echo "ssh root@$orig_master_host /sbin/ifconfig eno16777736:$key down"`
start_vip=`echo "ssh root@$new_master_host /sbin/ifconfig eno16777736:$key $vip"`
if [ $command = 'stop' ]
then
echo -e "\n\n\n***************************************************************\n"
echo -e "Disabling the VIP - $vip on old master: $orig_master_host\n"
$stop_vip
if [ $? -eq 0 ]
then
echo "Disabled the VIP successfully"
else
echo "Disabled the VIP failed"
fi
echo -e "***************************************************************\n\n\n\n"
fi
if [ $command = 'start' -o $command = 'status' ]
then
echo -e "\n\n\n***************************************************************\n"
echo -e "Enabling the VIP - $vip on new master: $new_master_host \n"
$start_vip
if [ $? -eq 0 ]
then
echo "Enabled the VIP successfully"
else
echo "Enabled the VIP failed"
fi
echo -e "***************************************************************\n\n\n\n"
fi
注意,这个脚本也可以使用perl版本
#!/usr/bin/env perl
use strict;
use warnings FATAL =>'all';
use Getopt::Long;
my $vip = '10.1.5.21/24'; # Virtual IP
my $key = "1";
my $ssh_start_vip = "/sbin/ifconfig eth0:$key $vip";
my $ssh_stop_vip = "/sbin/ifconfig eth0:$key down";
my $exit_code = 0;
my (
$command, $orig_master_is_new_slave, $orig_master_host,
$orig_master_ip, $orig_master_port, $orig_master_user,
$orig_master_password, $orig_master_ssh_user, $new_master_host,
$new_master_ip, $new_master_port, $new_master_user,
$new_master_password, $new_master_ssh_user,
);
GetOptions(
'command=s' => \$command,
'orig_master_is_new_slave' => \$orig_master_is_new_slave,
'orig_master_host=s' => \$orig_master_host,
'orig_master_ip=s' => \$orig_master_ip,
'orig_master_port=i' => \$orig_master_port,
'orig_master_user=s' => \$orig_master_user,
'orig_master_password=s' => \$orig_master_password,
'orig_master_ssh_user=s' => \$orig_master_ssh_user,
'new_master_host=s' => \$new_master_host,
'new_master_ip=s' => \$new_master_ip,
'new_master_port=i' => \$new_master_port,
'new_master_user=s' => \$new_master_user,
'new_master_password=s' => \$new_master_password,
'new_master_ssh_user=s' => \$new_master_ssh_user,
);
exit &main();
sub main {
#print "\n\nIN SCRIPT TEST====$ssh_stop_vip==$ssh_start_vip===\n\n";
if ( $command eq "stop" || $command eq "stopssh" ) {
# $orig_master_host, $orig_master_ip, $orig_master_port are passed.
# If you manage master ip address at global catalog database,
# invalidate orig_master_ip here.
my $exit_code = 1;
eval {
print "\n\n\n***************************************************************\n";
print "Disabling the VIP - $vip on old master: $orig_master_host\n";
print "***************************************************************\n\n\n\n";
&stop_vip();
$exit_code = 0;
};
if ($@) {
warn "Got Error: $@\n";
exit $exit_code;
}
exit $exit_code;
}
elsif ( $command eq "start" ) {
# all arguments are passed.
# If you manage master ip address at global catalog database,
# activate new_master_ip here.
# You can also grant write access (create user, set read_only=0, etc) here.
my $exit_code = 10;
eval {
print "\n\n\n***************************************************************\n";
print "Enabling the VIP - $vip on new master: $new_master_host \n";
print "***************************************************************\n\n\n\n";
&start_vip();
$exit_code = 0;
};
if ($@) {
warn $@;
exit $exit_code;
}
exit $exit_code;
}
elsif ( $command eq "status" ) {
print "Checking the Status of the script.. OK \n";
`ssh $orig_master_ssh_user\@$orig_master_host \" $ssh_start_vip \"`;
exit 0;
}
else {
&usage();
exit 1;
}
}
# A simple system call that enable the VIP on the new master
sub start_vip() {
`ssh $new_master_ssh_user\@$new_master_host \" $ssh_start_vip \"`;
}
# A simple system call that disable the VIP on the old_master
sub stop_vip() {
`ssh $orig_master_ssh_user\@$orig_master_host \" $ssh_stop_vip \"`;
}
sub usage {
"Usage: master_ip_failover –command=start|stop|stopssh|status –orig_master_host=host –orig_master_ip=ip –orig_master_port=po
rt –new_master_host=host –new_master_ip=ip –new_master_port=port\n";
}
脚本send_report(shell),内容:
#/bin/bash
source /root/.bash_profile
orig_master_host=`echo "$1" | awk -F = '{print $2}'`
new_master_host=`echo "$2" | awk -F = '{print $2}'`
new_slave_hosts=`echo "$3" | awk -F = '{print $2}'`
subject=`echo "$4" | awk -F = '{print $2}'`
body=`echo "$5" | awk -F = '{print $2}'`
#判断日志结尾是否有successfully,有则表示切换成功,成功与否都发邮件。
tac /etc/masterha/app1/manager.log | sed -n 2p | grep 'successfully' > /dev/null
if [ $? -eq 0 ]
then
echo -e "MHA $subject 主从切换成功\n master:$orig_master_host --> $new_master_host \n $body \n 当前从库:$new_slave_hosts" | mutt
-s "MySQL实例宕掉,MHA $subject 切换成功" -- xxx@qq.com
else
echo -e "MHA $subject 主从切换失败\n master:$orig_master_host --> $new_master_host \n $body" | mutt -s "MySQL实例宕掉,MHA $subje
ct 切换失败" -- xxx@qq.com
fi
9,测试
9.1在原主节点97.118上执行停止操作,删除虚拟ip
systemctl stop mysqld_safe
9.2,在管理节点上查看状态
[root@s2 masterha]# tail -10f /var/log/masterha/app1/manager.log
Started automated(non-interactive) failover.
The latest slave 192.168.97.119(192.168.97.119:3306) has all relay logs for recovery.
Selected 192.168.97.120(192.168.97.120:3306) as a new master.
192.168.97.120(192.168.97.120:3306): OK: Applying all logs succeeded.
192.168.97.120(192.168.97.120:3306): OK: Activated master IP address.
192.168.97.119(192.168.97.119:3306): This host has the latest relay log events.
Generating relay diff files from the latest slave succeeded.
192.168.97.119(192.168.97.119:3306): OK: Applying all logs succeeded. Slave started, replicating from 192.168.97.120(192.168.97.120:3306)
192.168.97.120(192.168.97.120:3306): Resetting slave info succeeded.
Master failover to 192.168.97.120(192.168.97.120:3306) completed successfully.
可以看到,master正常切换到了97.120
9.3,使用虚拟地址连接mysql验证切换正常
mysql -uroot -p123456 -h 192.168.97.124
OK 正常
9.4,连接s1节点查看
mysql> show slave status \G;
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.97.120
Master_User: repl
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000007
Read_Master_Log_Pos: 154
Relay_Log_File: relay-bin.000002
Relay_Log_Pos: 320
Relay_Master_Log_File: mysql-bin.000007
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 154
Relay_Log_Space: 521
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 137
Master_UUID: 49bb8923-8eb2-11e6-b687-000c29b86833
Master_Info_File: /usr/local/mysql/data/master.info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set:
Executed_Gtid_Set:
Auto_Position: 0
Replicate_Rewrite_DB:
Channel_Name:
Master_TLS_Version:
1 row in set (0.00 sec)
看到心得master节点为192.168.97.120
10,一些错误
10.1,执行:masterha_check_repl --conf=/etc/masterha/app1.cnf出现
Can't exec "mysqlbinlog": No suchfile or directory at /usr/local/share/perl5/MHA/BinlogManager.pm line 106.
mysqlbinlog version command failed with rc1:0, please verify PATH, LD_LIBRARY_PATH, and client options
解决:
type mysqlbinlog
然后:ln -s /usr/local/mysql/bin/mysqlbinlog /usr/bin/mysqlbinlog
注意/usr/local/mysql/bin/mysqlbinlog是type mysqlbinlog的结果
10.2,执行:masterha_check_repl --conf=/etc/masterha/app1.cnf出现
Testing mysql connection and privileges..sh: mysql: command not found
mysql command failed with rc 127:0!
解决:ln -s /usr/local/mysql/bin/mysql /usr/bin/mysql
11,
1,简介
1.1mha简介
MHA,即MasterHigh Availability Manager and Tools for MySQL,是日本的一位MySQL专家采用Perl语言编写的一个脚本管理工具,该工具仅适用于MySQLReplication(二层)环境,目的在于维持Master主库的高可用性。
MHA(Master High Availability)是自动的master故障转移和Slave提升的软件包.它是基于标准的MySQL复制(异步/半同步).
MHA有两部分组成:MHA Manager(管理节点)和MHA Node(数据节点).
MHA Manager可以单独部署在一台独立机器上管理多个master-slave集群,也可以部署在一台slave上.MHA Manager探测集群的node节点,当发现master出现故障的时候,它可以自动将具有最新数据的slave提升为新的master,然后将所有其它的slave导向新的master上.整个故障转移过程对应用程序是透明的。
MHA node运行在每台MySQL服务器上(master/slave/manager),它通过监控具备解析和清理logs功能的脚本来加快故障转移的。
1.2,mha特点
(1). 10-30s实现master failover(9-12s可以检测到主机故障,7-10s可以关闭主机避免SB,在用很短的时间应用差异日志)
(2) 部署简单,无需对现有M-S结构做任何改动(至少3台,保证切换后仍保持M-S结构)
(3). 支持手动在线切换(主机硬件维护),downtime几乎很短0.5-2s
(4). 保证故障切换后多从库数据的一致性
(5). 完全自动化的failover及快速复制架构恢复方案(一主多从)
(6). 恢复过程包括:选择新主库、确认从库间relaylog差异、新主库应用必要语句、其他从库同步差异语句、重新建立复制连接
1.3,工作原理
相较于其它HA软件,MHA的目的在于维持MySQL Replication中Master库的高可用性,其最大特点是可以修复多个Slave之间的差异日志,最终使所有Slave保持数据一致,然后从中选择一个充当新的Master,并将其它Slave指向它。
-从宕机崩溃的master保存二进制日志事件(binlogevents)。
-识别含有最新更新的slave。
-应用差异的中继日志(relay log)到其它slave。
-应用从master保存的二进制日志事件(binlogevents)。
-提升一个slave为新master。
-使其它的slave连接新的master进行复制。
说明:本文参考互联网
0 0
- mysql MHA
- mysql-mha
- mysql-mha
- mysql+mha+keepalived实现MHA架构
- mysql HA方案: MHA
- MySQL MHA+Keepalived
- MySQL MHA相关测试
- MySQL MHA配置常见问题
- mha+mysql自动切换
- 再谈Mysql MHA
- 再谈Mysql MHA
- Mysql GTID Mha配置
- mysql高可用--mha
- MySQL:MHA安装配置
- MySQL:MHA测试
- mysql-mha架构
- MySQL 5.6 GTID+MHA
- mysql mha+keepalived
- json于java之间的转化
- python split() 方法
- 按键驱动:platform架构
- vlc-android 编译报错
- 微信订阅号测试自动回复开发
- mysql MHA
- utf8汉字编码16进制对照
- 互联网产品经理知识条目
- 自定义仿QQ侧滑删除
- 【fiddler】使用过程中总是弹出提示框“Protocol violation Report”
- C运算符优先级
- BIO与NIO、AIO的区别(这个容易理解)
- cookie session
- Android 计算日期差