heartbeat+drbd实现磁盘共享自动挂载切换

来源:互联网 发布:剑三正太脸型数据网盘 编辑:程序博客网 时间:2024/06/06 01:40

heartbeat 实现 DRBD 主从自动切换

介绍 :heartbeat 和 drbd

  • 如果主服务器宕机,造成的损失是不可估量的。要保证主服务器不间断服务,就需要对服务器实现冗余。在众多的实现服务器冗余的解决方案中,heartbeat 为我们提供了廉价的、可伸缩的高可用集群方案。我们通过 heartbeat+drbd 在 Linux下创建一个高可用(HA)的集群服务器。
  • DRBD 是一种块设备,可以被用于高可用(HA)之中。它类似于一个网络 RAID-1 功能。当你将数据写入本地文件系统时,数据还将会被发送到网络中另一 台主机上。以相同的形式记录在一个文件系统中。本地(主节点)与远程主机(备节点)的数据可以保证实时同步。当本地系统出现故障时,远程主机上还会保留有 一份相同的数据,可以继续使用。在高可用(HA)中使用 DRBD 功能,可以代替使用一个共享盘阵。因为数据同时存在于本地主机和远程主机上。切换时,远程 主机只要使用它上面的那份备份数据,就可以继续进行服务了。

操作环境 redhat6.5 ,iptables and selinux disabled

服务机:server1 172.25.32.1

server2 172.25.32.2

drdb安装

serever2:

[root@server2 ~]# lsanaconda-ks.cfg  drbd-8.4.3.tar.gz  nginx  server2[root@server2 ~]# tar zxf drbd-8.4.3.tar.gz [root@server2 ~]# lsanaconda-ks.cfg  drbd-8.4.3  drbd-8.4.3.tar.gz  nginx  server2[root@server2 ~]# cd drbd-8.4.3[root@server2 drbd-8.4.3]# lsautogen.sh    documentation        filelist-redhat  preamble-sles11benchmark     drbd                 filelist-suse    READMEChangeLog     drbd_config.h        Makefile.in      rpm-macro-fixesconfigure     drbd-kernel.spec.in  preamble         scriptsconfigure.ac  drbd-km.spec.in      preamble-rhel5   userCOPYING       drbd.spec.in         preamble-sles10[root@server2 drbd-8.4.3]# ./configure --help |grep enable  #过滤编译需要的参数  --disable-option-checking  ignore unrecognized --enable/--with options  --disable-FEATURE       do not include FEATURE (same as --enable-FEATURE=no)  --enable-FEATURE[=ARG]  include FEATURE [ARG=yes]  --enable-spec           Rather than creating Makefiles, create an RPM spec                          architecture (makes sense only with --enable-spec,[root@server2 drbd-8.4.3]# alias grep='grep --color'  [root@server2 drbd-8.4.3]# ./configure --help |grep km  #过滤编译需要的参数  --with-km               Enable kernel module[root@server2 drbd-8.4.3]# ./configure --enable-spec   --with-km^C[root@server2 drbd-8.4.3]# yum install rpm-build -y   #安转rpm-buid,可以通过生成rpm包的方式安装drbd[root@server2 drbd-8.4.3]# ./configure --enable-spec   --with-km...configure: error: no acceptable C compiler found in $PATH  #提示需要gcc编译器[root@server2 drbd-8.4.3]# yum install gcc -y   #安装gcc[root@server2 drbd-8.4.3]# ./configure --enable-spec   --with-km  #编译...config.status: creating drbd.specconfig.status: creating drbd-km.specconfig.status: creating drbd-kernel.spec[root@server2 drbd-8.4.3]# rpmbuild --help |grep bb  #过滤编译需要的参数  -bb                           build binary package only from <specfile>[root@server2 drbd-8.4.3]# rpmbuild -bb drbd.spec[root@server2 drbd-8.4.3]# cd ..[root@server2 ~]# lsanaconda-ks.cfg  drbd-8.4.3  drbd-8.4.3.tar.gz  nginx  rpmbuild  server2[root@server2 ~]# cp drbd-8.4.3.tar.gz /root/rpmbuild/SOURCES/[root@server2 ~]# cd -/root/drbd-8.4.3[root@server2 drbd-8.4.3]# rpmbuild -bb drbd.spec  #生成rpm包...Wrote: /root/rpmbuild/RPMS/x86_64/drbd-8.4.3-2.el6.x86_64.rpm   #生成的rpm包存放位置Wrote: /root/rpmbuild/RPMS/x86_64/drbd-utils-8.4.3-2.el6.x86_64.rpmWrote: /root/rpmbuild/RPMS/x86_64/drbd-xen-8.4.3-2.el6.x86_64.rpmWrote: /root/rpmbuild/RPMS/x86_64/drbd-udev-8.4.3-2.el6.x86_64.rpmWrote: /root/rpmbuild/RPMS/x86_64/drbd-pacemaker-8.4.3-2.el6.x86_64.rpmWrote: /root/rpmbuild/RPMS/x86_64/drbd-heartbeat-8.4.3-2.el6.x86_64.rpmWrote: /root/rpmbuild/RPMS/x86_64/drbd-bash-completion-8.4.3-2.el6.x86_64.rpm...[root@server2 drbd-8.4.3]# rpmbuild -bb drbd-km.specerror: Failed build dependencies:    kernel-devel is needed by drbd-km-8.4.3-2.el6.x86_64    #缺少kernel-devel依赖包[root@server2 drbd-8.4.3]# yum install kernel-devel -y  #安装依赖包[root@server2 drbd-8.4.3]# rpmbuild -bb drbd-km.spec...Wrote: /root/rpmbuild/RPMS/x86_64/drbd-km-2.6.32_431.el6.x86_64-8.4.3-2.el6.x86_64.rpm  #生成的和内核相关的rpm包Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.SW52f3...[root@server2 drbd-8.4.3]# cd  /root/rpmbuild/RPMS/x86_64/[root@server2 x86_64]# lsdrbd-8.4.3-2.el6.x86_64.rpm                  drbd-km-2.6.32_431.el6.x86_64-8.4.3-2.el6.x86_64.rpm  drbd-utils-8.4.3-2.el6.x86_64.rpmdrbd-bash-completion-8.4.3-2.el6.x86_64.rpm  drbd-pacemaker-8.4.3-2.el6.x86_64.rpm                 drbd-xen-8.4.3-2.el6.x86_64.rpmdrbd-heartbeat-8.4.3-2.el6.x86_64.rpm        drbd-udev-8.4.3-2.el6.x86_64.rpm[root@server2 x86_64]# rpm -ivh drbd-*   #安装生成的rpm包 Preparing...                ########################################### [100%]   1:drbd-utils             ########################################### [ 13%]   2:drbd-bash-completion   ########################################### [ 25%]   3:drbd-heartbeat         ########################################### [ 38%]   4:drbd-pacemaker         ########################################### [ 50%]   5:drbd-udev              ########################################### [ 63%]   6:drbd-xen               ########################################### [ 75%]   7:drbd                   ########################################### [ 88%]   8:drbd-km-2.6.32_431.el6.########################################### [100%][root@server2 x86_64]# scp .* root@server1:/mnt/  #将生成的rpm包发给server1并在server1上安装rpm包

drbd的配置

  • 在server1和server2各添加一块磁盘
[root@server2 x86_64]# fdisk -lDisk /dev/vdc: 5368 MB, 5368709120 bytes   #server2添加的5g的磁盘16 heads, 63 sectors/track, 10402 cylindersUnits = cylinders of 1008 * 512 = 516096 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x00000000[root@server1 drbd-8.4.3]# fdisk -lDisk /dev/vda: 5368 MB, 5368709120 bytes  #server1添加的5g的磁盘16 heads, 63 sectors/track, 10402 cylindersUnits = cylinders of 1008 * 512 = 516096 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x00000000

server2

[root@server2 x86_64]# cat /etc/drbd.d/drdb.res #创建drbd的资源文件resource example {meta-disk internal;device /dev/drbd1;syncer {verify-alg sha1;}net {allow-two-primaries;}on server1 {disk /dev/vda;address 172.25.32.1:7789;}on server2 {disk /dev/vdc;address 172.25.32.2:7789;}}[root@server2 x86_64]#  scp /etc/drbd.d/drdb.res root@server1:/etc/drbd.d/ #将配置文件发给server1一份
  • 在两台主机上分别执行以下命令
[root@server2 x86_64]#  drbdadm create-md example[root@server2 x86_64]# /etc/init.d/drbd startStarting DRBD resources: [     create res: example   prepare disk: example    adjust disk: example     adjust net: example]..........    #此时如果server2上没有没有启动drbd,会等待server启动drbd并给与提示[root@server1 x86_64]#  drbdadm create-md exampleWriting meta data...initializing activity logNOT initializing bitmapNew drbd meta data block successfully created.[root@server1 drbd-8.4.3]# /etc/init.d/drbd startStarting DRBD resources: [     create res: example   prepare disk: example    adjust disk: example     adjust net: example].[root@server1 drbd-8.4.3]# drbdsetup /dev/drbd1 primary --force  #强制将server1设为primary
  • server1设定为primary时,server自动论为sencondary,并且primary将数据同步给secondary
[root@server2 x86_64]# cat /proc/drbd version: 8.4.3 (api:1/proto:86-101)GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root@server2, 2017-09-21 14:42:19 1: cs:SyncTarget ro:Secondary/Primary ds:Inconsistent/UpToDate C r-----    ns:0 nr:4903936 dw:4903936 dr:0 al:0 bm:299 lo:1 pe:4 ua:0 ap:0 ep:1 wo:f oos:338748    [=================>..] sync'ed: 93.6% (328/5116)M    #同步中.....    finish: 0:00:04 speed: 76,584 (76,624) want: 57,840 K/sec[root@server1 drbd-8.4.3]#  cat /proc/drbd version: 8.4.3 (api:1/proto:86-101)GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root@server1, 2017-09-21 14:08:20 1: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r-----  #同步完成    ns:5242684 nr:0 dw:0 dr:5243348 al:0 bm:320 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
  • 同步完成了,就可以正常的使用drbd下的磁盘了,格式化和挂在以及写入数据,格式化,挂载,写入,只能在一边进行,不支持两边同时挂载写入。
[root@server1 drbd-8.4.3]# mkfs.ext4 /dev/drbd1mke2fs 1.41.12 (17-May-2010)Filesystem label=OS type: LinuxBlock size=4096 (log=2)Fragment size=4096 (log=2)Stride=0 blocks, Stripe width=0 blocks327680 inodes, 1310671 blocks65533 blocks (5.00%) reserved for the super userFirst data block=0Maximum filesystem blocks=134217728040 block groups32768 blocks per group, 32768 fragments per group8192 inodes per groupSuperblock backups stored on blocks:     32768, 98304, 163840, 229376, 294912, 819200, 884736Writing inode tables: done                            Creating journal (32768 blocks): doneWriting superblocks and filesystem accounting information: doneThis filesystem will be automatically checked every 32 mounts or180 days, whichever comes first.  Use tune2fs -c or -i to override.[root@server1 drbd-8.4.3]# mount /dev/drbd1 /var/lib/mysql/[root@server1 drbd-8.4.3]# df -hFilesystem                    Size  Used Avail Use% Mounted on/dev/mapper/VolGroup-lv_root   18G  1.9G   15G  12% /tmpfs                         499M   23M  477M   5% /dev/shm/dev/sda1                     485M   33M  427M   8% /boot/dev/drbd1                    5.0G  159M  4.6G   4% /var/lib/mysql[root@server1 x86_64]# mysqlERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)  #数据库服务没有开启[root@server1 x86_64]# /etc/init.d/mysqld startStarting mysqld:                                           [  OK  ][root@server1 drbd-8.4.3]# mysqlWelcome to the MySQL monitor.  Commands end with ; or \g.Your MySQL connection id is 3Server version: 5.1.71 Source distributionCopyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.Oracle is a registered trademark of Oracle Corporation and/or itsaffiliates. Other names may be trademarks of their respectiveowners.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.mysql> show databases;+---------------------+| Database            |+---------------------+| information_schema  || #mysql50#lost+found || mysql               || test                |+---------------------+4 rows in set (0.00 sec)mysql> create database hh;Query OK, 1 row affected (0.00 sec)mysql> show databases;+---------------------+| Database            |+---------------------+| information_schema  || hh                  || #mysql50#lost+found || mysql               || test                |+---------------------+5 rows in set (0.00 sec)mysql> Ctrl-C -- exit!Aborted[root@server1 drbd-8.4.3]# df -hFilesystem                    Size  Used Avail Use% Mounted on/dev/mapper/VolGroup-lv_root   18G  1.9G   15G  12% /tmpfs                         499M   23M  477M   5% /dev/shm/dev/sda1                     485M   33M  427M   8% /boot/dev/drbd1                    5.0G  159M  4.6G   4% /var/lib/mysql[root@server1 drbd-8.4.3]# /etc/init.d/mysqld stopStopping mysqld:                                           [  OK  ][root@server1 drbd-8.4.3]# umount   /var/lib/mysql[root@server1 drbd-8.4.3]# drbdadm secondary example  #将server1变为secondary,server2变为primary,server2上才能挂载[root@server1 drbd-8.4.3]# cat /proc/drbd #查看是否更新过,以及是主还是辅version: 8.4.3 (api:1/proto:86-101)GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root@server1, 2017-09-21 14:08:20 1: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r-----    ns:5483972 nr:0 dw:241288 dr:5248344 al:51 bm:320 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0[root@server2 x86_64]# drbdadm primary example --force[root@server2 x86_64]# cat /proc/drbd version: 8.4.3 (api:1/proto:86-101)GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root@server2, 2017-09-21 14:42:19 1: cs:Connected ro:Primary/Primary ds:UpToDate/UpToDate C r-----    ns:0 nr:5483972 dw:5483972 dr:664 al:0 bm:320 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0  [root@server2 x86_64]# mount /dev/drbd1 /var/lib/mysql/[root@server2 x86_64]# /etc/init.d/mysqld startStarting mysqld:                                           [  OK  ][root@server2 x86_64]# mysqlERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)[root@server2 x86_64]# ls /var/lib/mysql/hh  ibdata1  ib_logfile0  ib_logfile1  lost+found  mysql  passwd  test[root@server2 x86_64]# /etc/init.d/mysqld restartStopping mysqld:                                           [  OK  ]Starting mysqld:                                           [  OK  ][root@server2 x86_64]# mysqlWelcome to the MySQL monitor.  Commands end with ; or \g.Your MySQL connection id is 2Server version: 5.1.71 Source distributionCopyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.Oracle is a registered trademark of Oracle Corporation and/or itsaffiliates. Other names may be trademarks of their respectiveowners.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.mysql> show databases;+---------------------+| Database            |+---------------------+| information_schema  || hh                  | #可以看到server1上新建的数据库hh| #mysql50#lost+found || mysql               || test                |+---------------------+5 rows in set (0.01 sec)mysql> Ctrl-C -- exit!Aborted

注意:两台主机上的/dev/drbd1 不能同时挂载,只有状态为 primary 时,才能被挂载使
用,而此时另一方的状态为 secondary。

原创粉丝点击