Redis集群方案redis-twemproxy-keepalived

来源：互联网发布：淘宝大学一飞老师编辑：程序博客网时间：2024/05/01 16:07

转载原文自：http://ju.outofmemory.cn/entry/60425

redis现在的版本还不支持分布式，这会在3.0版本加上。Redis目前只支持“主从同步”和“从从同步”，所以需要根据需求应用自己来实现。

20131231150905

tp ------ 192.168.10.15

tp1------ 192.168.10.18

tp tp1 虚拟vip: 192.168.10.243

v1 ------ 192.168.10.224

v2 ------ 192.168.10.14

v1 v2 虚拟vip : 192.168.10.241

v3 ------ 192.168.10.16

v4 ------ 192.168.10.17

v3 v4 虚拟vip: 192.168.10.242

1. 先做redis 热备节点 2组

R1 与 R2 为第一组节点

V1 (192.168.10.224) V2 (192.168.10.14) 通过keepalived

主(master)： V1

备份(backup)：V2

虚拟ip 192.168.10.241

搭建步骤：主备相同

1. 安装redis

　　　　wget http://redis.googlecode.com/files/redis-2.0.0-rc4.tar.gz

tar zxvf redis-2.0.0-rc4.tar.gz

cd redis-2.0.0-rc4

make

cp redis.conf /etc/ 这个文件是redis启动的配置文件

cp redis-benchmark redis-cli redis-server /usr/bin/

redis-server /etc/redis.conf

redis-cli

注意修改配置：daemonize yes

2. 安装高可用软件 keepalived

wget http://www.keepalived.org/software/keepalived-1.1.20.tar.gz

tar zxvf keepalived-1.1.20.tar.gz

cd keepalived-1.1.20

./configure --with-kernel-dir=/usr/src/kernels/2.6.18-164.el5-x86_64/

make && make install

cp /usr/local/etc/rc.d/init.d/keepalived /etc/rc.d/init.d/

cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/

mkdir /etc/keepalived

cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/

cp /usr/local/sbin/keepalived /usr/sbin

/etc/init.d/keepalived start

报错：

!!! OpenSSL is not properly installed on your system. !!!
!!! Can not include OpenSSL headers files. !!!

解决

yum install openssl-devel

报错

configure: error: Popt libraries is required

解决

yum install popt-devel

配置文件修改注意所监控网卡硬件名称

V1配置文件

Keepalived:

state MASTER

virtual_ipaddress {

192.168.10.241 ###VIP

}

redis_backup.sh :

SLAVEOF 192.168.10.14 6379

redis_master.sh:

SLAVEOF 192.168.10.14 6379

V2 配置文件

Keepalived:

state BACKUP

virtual_ipaddress {

192.168.10.241 ###VIP

}

redis_backup.sh :

SLAVEOF 192.168.10.224 6379

redis_master.sh:

SLAVEOF 192.168.10.224 6379

测试此互备节点：

1.启动Master上的Redis

redis-server /etc/redis.conf

2.启动Slave上的Redis

redis-server /etc/redis.conf

3.启动Master上的Keepalived

/etc/init.d/keepalived start

4.启动Slave上的Keepalived

/etc/init.d/keepalived start

5.尝试通过VIP连接Redis:

redis-cli -h 192.168.10.241 INFO

连接成功，Slave也连接上来了。

role:master

6.尝试插入一些数据：

redis-cli -h 192.168.10.241 SET Hellor1 Redis

从VIP读取数据

redis-cli -h 192.168.10.241 GET Hellor1

"Redis"

从Master读取数据

redis-cli -h 192.168.10.224 GET Hellor1

"Redis"

从Slave读取数据

redis-cli -h 192.168.10.14 GET Hellor1

"Redis"

下面，模拟故障产生：

将Master上的Redis进程杀死：

killall -9 redis-server

查看Master上的Keepalived日志

tailf /var/log/keepalived-redis-state.log

[fault]

Mon Dec 30 10:06:13 CST 2013

同时Slave上的日志显示：

tailf /var/log/keepalived-redis-state.log

[master]

Mon Dec 30 10:06:17 CST 2013

Being master....

Run SLAVEOF cmd ...

Run SLAVEOF NO ONE cmd ...

然后我们可以发现，Slave已经接管服务，并且担任Master的角色了。

redis-cli -h 192.168.10.241 INFO

redis-cli -h 192.168.10.14 INFO

role:master

然后我们恢复Master的Redis进程

redis-server /etc/redis.conf

查看Master上的Keepalived日志

tailf /var/log/keepalived-redis-state.log

[master]

Mon Dec 30 10:10:13 CST 2013

Being master....

Run SLAVEOF cmd ...

Run SLAVEOF NO ONE cmd ...

同时Slave上的日志显示：

tailf /var/log/keepalived-redis-state.log

[backup]

Mon Dec 30 10:10:12 CST 2013

Being slave....

Run SLAVEOF cmd ...

可以发现目前的Master已经再次恢复了Master的角色，故障切换以及自动恢复都成功了。

R3 与 R4 为第二组节点

参考第一组即可

V3(192.168.10.16) V4 (192.168.10.17) 通过keepalived

主(master)： V3

备份(backup)：V4

虚拟ip 192.168.10.242

virtual_router_id 不同

2. 做twemproxy 代理互备

Tp (192.168.10.15)

Tp1(192.168.10.18)

搭建步骤：

1.安装 twemproxy

yum install automake

yum install libtool

git clone git://github.com/twitter/twemproxy.git

cd twemproxy

CFLAGS="-ggdb3 -O0" autoreconf -fvi && ./configure --prefix=/usr/local/twemproxy --enable-debug=log

编译的时候报错

autoreconf: Entering directory `.'

autoreconf: configure.ac: not using Gettext

autoreconf: running: aclocal --force -I m4

configure.ac:8: error: Autoconf version 2.64 or higher is required

configure.ac:8: the top level

autom4te: /usr/bin/m4 failed with exit status: 63

aclocal: autom4te failed with exit status: 63

autoreconf: aclocal failed with exit status: 63

原因是autoconf版本过低

升级

查看当前版本

#rpm -qf /usr/bin/autoconf

autoconf-2.63-5.1.el6.noarch

卸载当前版本

rpm -e --nodeps autoconf-2.63

下载新版本

wget http://ftp.gnu.org/gnu/autoconf/autoconf-2.64.tar.gz

解压安装

tar zxvf autoconf-2.64.tar.gz

cd autoconf-2.64

./configure --prefix=/usr

make && make install

查看是否安装成功

/usr/bin/autoconf -V

CFLAGS="-ggdb3 -O0" autoreconf -fvi && ./configure --prefix=/usr/local/twemproxy --enable-debug=log

make && make install

cp conf/nutcracker.yml /usr/local/twemproxy/conf/

测试

/usr/local/twemproxy/sbin/nutcracker -t

nutcracker: configuration file 'conf/nutcracker.yml' syntax is ok

--------------------------------------------------------------------------

启动命令

调试启动

/usr/local/twemproxy/sbin/nutcracker -c

以守护进程启动

/usr/local/twemproxy/sbin/nutcracker -d -c

2.安装 keepalived

参考上面安装步骤即可

修改配置文件

Tp 配置文件

Keepalived：

vrrp_script chk_t {

script "/etc/keepalived/scripts/t_check.sh" ###监控脚本

interval 2 ###监控时间

}

vrrp_instance VI_1 {

state MASTER ###设置为MASTER

interface eth1 ###监控网卡

virtual_router_id 53

priority 101 ###权重值

authentication {

auth_type PASS ###加密

auth_pass redis ###密码

}

track_interface {

eth1

}

virtual_ipaddress {

192.168.10.243 ###VIP

}

notify_master /etc/keepalived/scripts/t_master.sh

notify_backup /etc/keepalived/scripts/t_backup.sh

notify_fault /etc/keepalived/scripts/t_fault.sh

notify_stop /etc/keepalived/scripts/t_stop.sh

}

/usr/local/twemproxy/conf/nutcracker.yml

redis:

listen: 192.168.10.243:6379

hash: fnv1a_64

distribution: ketama

auto_eject_hosts: true

redis: true

server_retry_timeout: 2000

server_failure_limit: 1

servers:

- 192.168.10.241:6379:1 node1

- 192.168.10.242:6379:1 node2

Tp1 配置文件

Keepalived：

vrrp_script chk_t {

script "/etc/keepalived/scripts/t_check.sh" ###监控脚本

interval 2 ###监控时间

}

vrrp_instance VI_1 {

state BACKUP ###设置为BACKUP

interface eth1 ###监控网卡

virtual_router_id 53

priority 99 ###权重值

authentication {

auth_type PASS ###加密

auth_pass redis ###密码

}

track_interface {

eth1

}

virtual_ipaddress {

192.168.10.243 ###VIP

}

notify_master /etc/keepalived/scripts/t_master.sh

notify_backup /etc/keepalived/scripts/t_backup.sh

notify_fault /etc/keepalived/scripts/t_fault.sh

notify_stop /etc/keepalived/scripts/t_stop.sh

}

/usr/local/twemproxy/conf/nutcracker.yml

redis:

listen: 192.168.10.243:6379

hash: fnv1a_64

distribution: ketama

auto_eject_hosts: true

redis: true

server_retry_timeout: 2000

server_failure_limit: 1

servers:

- 192.168.10.241:6379:1 node1

- 192.168.10.242:6379:1 node2

测试

redis-cli -h 192.168.10.243

redis> set pp pp

redis> get pp

"pp"

关掉tp 查看tp1 日志

tailf /var/log/keepalived-t-state.log

[backup]

Tue Dec 31 14:37:20 CST 2013

Being slave....

[master]

Tue Dec 31 14:47:28 CST 2013

Being master....

Run twemproxy ...

Tp2 接管 vip 继续提供服务

redis-cli -h 192.168.10.243

redis> get pp

"pp"

重启tp 查看tp1 日志

tailf /var/log/keepalived-t-state.log

[backup]

Tue Dec 31 14:54:37 CST 2013

Being slave....

redis-cli -h 192.168.10.243

redis> get pp

"pp"

Tp接管主服务 tp2切换备份服务

3.可能遇到相关问题和处理办法：
1. 如果两机器不能连通，请注意关掉防火墙：　
service iptables stop #临时关闭
chkconfig iptables off #永久关闭
2. windows上编写的shell脚本，要注意通过dos2unix 命令转换，否则shell脚本可能有不可见符号不能正常执行。

0 0